topic Re: How to merge all lines into one single event? in Getting Data In

How to merge all lines into one single event?

danielearangiom — Fri, 31 Aug 2018 11:06:26 GMT

Hi,
How can I merge all lines of a config file into one single event?
My inputs.conf is:

[monitor:D:\CatTools3\Config.Current.Running.ASA-CLI.txt]
sourcetype = CatTools:Firewall:ASA-CLI
host_regex = Config.Current.Running.(?P.+)
index = eli

And my props.conf is:
[CatTools:Firewall:ASA-CLI]
SHOULD_LINEMERGE = TRUE

But on Splunk search, each line is a single event, so now I have more than 5000 events instead of one event of 5000 lines.

Re: How to merge all lines into one single event?

richgalloway — Fri, 31 Aug 2018 11:58:05 GMT

After modifying props.conf you must restart your indexer(s) for the change to take effect.
Changes to props.conf only apply to NEW data. Events already indexed will not change.

Re: How to merge all lines into one single event?

danielearangiom — Fri, 31 Aug 2018 13:30:10 GMT

Done, same as before

Re: How to merge all lines into one single event?

richgalloway — Fri, 31 Aug 2018 15:41:58 GMT

How are you breaking events? The default setting is by line, even with line merging turned on.

Re: How to merge all lines into one single event?

danielearangiom — Fri, 31 Aug 2018 15:59:17 GMT

What do you mean? Where can i see this setting?

Re: How to merge all lines into one single event?

richgalloway — Fri, 31 Aug 2018 20:13:32 GMT

It will be in props.conf. If you have access to the CLI, run this command. It will show all of the properties for that sourcetype. Look for LINE_BREAKER, which is probably set to the default value of "([\r\n]+)". You should change the value in parentheses to something that separates one event from another.

splunk btool --debug props list CatTools:Firewall:ASA-CLI | more

Re: How to merge all lines into one single event?

Lazarix — Sun, 02 Sep 2018 22:33:53 GMT

Try this in props.conf:

[CatTools:Firewall:ASA-CLI]
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE=gobbldegook
MAX_EVENTS=200000
LINE_BREAKER = ((*FAIL))
TRUNCATE=99999999

The break only before and line breaker above are set to something that should not match in the source file.
There is also a 500 line default for max events which we are bypassing above.
The result should be that the entire ASA config is imported as a single event.

Re: How to merge all lines into one single event?

danielearangiom — Mon, 03 Sep 2018 08:32:29 GMT

Same as before, nothing change.

Re: How to merge all lines into one single event?

danielearangiom — Tue, 29 Sep 2020 21:08:22 GMT

Finally I solved by putting the following rows of my props.conf into the deployment server and also the index server:

[CatTools:Firewall:ASA-CLI]
SHOULD_LINEMERGE = TRUE
LINE_BREAKER = (ALL_ROWS_INTO_ONE_EVENT)
DATETIME_CONFIG = CURRENT
TRUNCATE = 0
MAX_EVENTS = 100000

Re: How to merge all lines into one single event?

richgalloway — Tue, 04 Sep 2018 14:18:59 GMT

@danielearangiomazza If your problem is resolved, please accept an answer to help future readers.