multiple lines are coming as single event

[roopeshetty]

Hi Team,

I am collecting metrics using API calls for every 5 minutes , but all the metrics are coming as a single event as below for every 5 minutes.

I have attadhed the screen shot here.

confluent_kafka_server_request_bytes{kafka_id="tythtyt",principal_id="sa-r29997",type="Fetch",} 2092668.0 1683872880000

confluent_kafka_server_memory{kafka_id="yyyy",topic="host002.json.cs.tt.gg",} 0.0 1683872880000

I need to break these events as individuals (which ever events starting from text “confluent_kafka_”) . I have edited my props.conf as below but its not coming as expected still its coming as a single event. Can some one please guide me how to do it.

[source::kafka_metrics://kafka_metrics]

LINE_BREAKER = (confluent_kafka_)(\s)

SHOULD_LINEMERGE = false

[danspav]

Hi @roopeshetty,

Try this props:

[ confluent_kafka_api ]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n])+confluent_kafka_
CHARSET=UTF-8
PREAMBLE_REGEX =^#.+$

The preamble_regex removes the comments from the data

The line_breaker is set to a new line (in brackets) followed by the words "confluent_kafka_" (not in brackets)

There's no time field in the data, so it's set to CURRENT.

See if that helps.

Cheers,
Daniel