topic Re: line break to different entries in Getting Data In

line break to different entries

arunkuriakose — Fri, 06 Sep 2024 08:24:39 GMT

I have logs indexed like this. How to break entries based on each lines . i need each line as a seperate entry.

I tried to do this via line breaker but didnt succeed. Any method to do it via search after indexing

Re: line break to different entries

ITWhisperer — Fri, 06 Sep 2024 08:35:49 GMT

Please share your raw events and the configurations you have tried

Re: line break to different entries

PickleRick — Fri, 06 Sep 2024 21:32:39 GMT

There are several different issues touched here.

As you have already indexed data, you cannot break the events again and re-index them. You can, however manipulate your data during searching. But you will have to "break" the data into separate results on each search explicitly using search commands.

If you want newly ingested data properly broken and indexed as separate events you need to configure your ingestion settings properly. But that will only work on newly ingested data. Old data will stay as it was.

Re: line break to different entries

arunkuriakose — Fri, 06 Sep 2024 22:06:19 GMT

thanks for the response. If i can reindex the data how to apply line breaking settings effficiently to achieve this

Re: line break to different entries

PickleRick — Sat, 07 Sep 2024 08:16:34 GMT

As @ITWhisperer said - show us your raw events and what have you tried so far because maybe your idea was OK but applied in a wrong place.

Re: line break to different entries

arunkuriakose — Mon, 09 Sep 2024 14:10:33 GMT

thanks for the response .

sample logs: (these are coming as a single event as mentioned in screenshot)

zowin.exposed. 3600 in ns ns1.dyna-ns.net.

zowin.exposed. 3600 in ns ns2.dyna-ns.net.

zuckerberg.exposed. 3600 in ns ns1.afternic.com.

zuckerberg.exposed. 3600 in ns ns2.afternic.com.

zwiebeltvde.exposed. 3600 in ns docks13.rzone.de.

zwiebeltvde.exposed. 3600 in ns shades01.rzone.de

I am applying this on UF config . (/etc/system/local/propes.conf

[zone_files]

LINE_BREAKER= ([\r\n]+) SHOULD_LINEMERGE = false

Re: line break to different entries

PickleRick — Mon, 09 Sep 2024 11:15:55 GMT

OK. Let's back up a little.

1. How are the events ingested? Read from files with a monitor input or any other way? (like HEC input or a modular input). You mention UF so I suspect monitor input(s) but I want to be sure.

2. I assume you meant props.conf, not propes.conf - that was just a typo here, right?

3. Line breaking is _not_ happening on the UF. You need to have your LINE_BREAKER defined on the first heavy component that the event passes through (if you're sending from UF directly to indexers, you need this setting on the indexers).

Re: line break to different entries

arunkuriakose — Tue, 10 Sep 2024 05:40:13 GMT

@PickleRick Your comments helped. I was applying this on the UF level and changing to indexers made it work. Thanks