Skip to content

Add telemetry for producer, producer_consumer & consumer #29

Description

@vasuadari

Producer

  • When producer starts(flume.producer.init = system_time)
  • No. of events fetched(flume.producer.events.fetch = 10)
  • Time taken to fetch events(flume.producer.events.fetch.stop)
  • metadata for all events
    • :pipeline_name - name of the pipeline for which producer is running
    • :queue_name - name of the queue in the redis
    • :rate_limit_count - no. of events that can be processed by the consumer
    • :rate_limit_scale - duration for which no. of events as per rate_limit_count can be processed
    • :rate_limit_key - unique key for which rate limit is applied
    • :max_demand - Maximum number of jobs to pull from the queue

ProducerConsumer

  • When producer consumer starts(flume.producer_consumer.init = system_time)
  • No. of events fetched(flume.producer_consumer.events.received = 10)

Consumer

  • When consumer starts(flume.consumer.init = system_time)

cc @nitinstp23

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions