I have a requirement to merge multiple lines that are by default broken into multiple events by indexer, and make one event before a specific line comes. The sample log file data is given below:-
2018-03-05 15:23:03.123 [ISC.0038.0002D] <-- abcd-efgh-ijkl
2018-03-05 15:23:03.234 [ISC.0038.0002D] <-- Length=550
2018-03-05 15:23:03.345 [ISC.0038.0002D] <-- Host: localhost:7001
2018-03-05 15:23:03.456 [ISC.0038.0002D] <-- Connection: 1
2018-03-05 15:23:03.567 [ISC.0038.0002D] <-- mnop-qrst-uvw-xyz
2018-03-05 15:23:03.678 [ISC.0038.0002D] <-- Length=600
2018-03-05 15:23:03.789 [ISC.0038.0002D] <-- Host: localhost:7001
2018-03-05 15:23:03.890 [ISC.0038.0002D] <-- Connection: 2
2018-03-05 15:23:04.321 [ISC.0038.0002D] <-- mnop-qrst-uvw-xyz
2018-03-05 15:23:05.432 [ISC.0038.0002D] <-- Length=600
2018-03-05 15:23:06.542 [ISC.0038.0002D] <-- Host: localhost:7001
props.conf:-
[source::/opt/saggw101/IntegrationServer/instances/default/logs/server.log*]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}\s[\w{3}.\d{4}.\d{4}\w{1}]\s\W{3}\s\w{4}:\s\w{9}:\d{4}
The above config is not working for me and the events that are created are multiple for each line. What I want is something like
Event 1:-
2018-03-05 15:23:03.345 [ISC.0038.0002D] <-- Host: localhost:7001
2018-03-05 15:23:03.456 [ISC.0038.0002D] <-- Connection: 1
2018-03-05 15:23:03.567 [ISC.0038.0002D] <-- mnop-qrst-uvw-xyz
2018-03-05 15:23:03.678 [ISC.0038.0002D] <-- Length=600
Event 2:-
2018-03-05 15:23:03.789 [ISC.0038.0002D] <-- Host: localhost:7001
2018-03-05 15:23:03.890 [ISC.0038.0002D] <-- Connection: 2
2018-03-05 15:23:04.321 [ISC.0038.0002D] <-- mnop-qrst-uvw-xyz
2018-03-05 15:23:05.432 [ISC.0038.0002D] <-- Length=600
Could anyone help me out with this?
Hi @AdsicSplunk
Please try below config in props.conf on Indexer or Heavy Forwarder whichever comes first and then restart splunk
[source::/opt/saggw101/IntegrationServer/instances/default/logs/server.log*]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\.\d{3}\s+\[\w+\.\d+\.\w+\]\s+\W{3}\s+Host:\s+\w+:\d+
BREAK_ONLY_BEFORE_DATE=false
Hey AdsicSplunk,
You can try this:
[source::/opt/saggw101/IntegrationServer/instances/default/logs/server.log*]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
MUST_BREAK_AFTER = (Length)=\d+
Let me know if this helps!!
Thank you deepashri_123, I tried the answer by harsmarvania57 and it worked for me.
Hi @AdsicSplunk
Please try below config in props.conf on Indexer or Heavy Forwarder whichever comes first and then restart splunk
[source::/opt/saggw101/IntegrationServer/instances/default/logs/server.log*]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\.\d{3}\s+\[\w+\.\d+\.\w+\]\s+\W{3}\s+Host:\s+\w+:\d+
BREAK_ONLY_BEFORE_DATE=false
Thank you @harsmarvania57, This worked pretty well.