topic Re: How can I break events in my search? in Splunk Search

How can I break events in my search?

patouellet — Tue, 29 Sep 2020 20:22:44 GMT

Hi,

Trying to break events and can't figure this one out. I receive a bunch of events in a single line, I want to break them using a pattern but it's not working for me. I'm using the Add data screen. Events should break when encountering <162>

I've tried BREAK_ONLY_BEFORE, LINE_BREAKER - nothing makes the event break. What am I doing wrong?

Sample of the log below:

<162>Mar 11 21:45:03 MACHINE CEF:0|PowerTech|Interact|3.1|TCA0001|The bytestream file *N/*N /Help Systems/Robot SCHEDULE ENTERPRISE/tmp/s7472_6859175.sz authority has been changed for user profile *PUBLIC.|2|src=X.X.X.X dst=0.0.0.0 msg=TYPE:JRN CLS:AUD JJOB:ENTSERVER1 JUSER:RBTENTUSR JNBR:171392 PGM:QLESPI OBJECT: LIBRARY: MEMBER: DETAIL:A *N *N *STMF *PUBLIC    Y   Y Y Y Y     RPL        0000 00000 * * *NA *NA<162>Mar 11 21:45:03 MACHINE CEF:0|PowerTech|Interact|3.1|TCA0001|The bytestream file *N/*N /Help Systems/Robot SCHEDULE ENTERPRISE/tmp/s7472_6859175.sz authority has been changed for user profile RBTENTUSR.|2|src=X.X.X.X dst=0.0.0.0 msg=TYPE:JRN CLS:AUD JJOB:ENTSERVER1 JUSER:RBTENTUSR JNBR:171392 PGM:QLESPI OBJECT: LIBRARY: MEMBER: DETAIL:A *N *N *STMF RBTENTUSR  Y Y Y   Y Y Y Y   Y Y RPL        0000 00000 * * *NA *NA

Re: How can I break events in my search?

MuS — Tue, 10 Jul 2018 21:18:32 GMT

Hi patouellet,

try this props.conf on the parsing Splunk instance, and restart Splunk after the change:

[YourSourcetypeNameHere]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
CHARSET=UTF-8
LINE_BREAKER=([\r\n]+)\<162\>|\s\*NA\s\*NA(.*)\<
TIME_PREFIX=\<162\>

Hope that helps ...

cheers, MuS

UPDATE:

Took this to slack and got more details, like it is a TCP input and the events actually do not contain *NA. After some tries this line breaker worked just fine:

 LINE_BREAKER=[\*\.\r\n\)\d]+()\<162\>|^()\<162\>

The response of the OP was awesome, and I want to share it:

Re: How can I break events in my search?

patouellet — Wed, 11 Jul 2018 16:03:58 GMT

I appreciate the help. But it's not working for me. I still get most events wrapped in Splunk as a single event. I've done exactly what you suggested - no luck.

Re: How can I break events in my search?

MuS — Wed, 11 Jul 2018 19:53:14 GMT

Hi there,

take a file that contains the events, use the Add Data page http://docs.splunk.com/Documentation/Splunk/latest/Data/Howdoyouwanttoadddata and add the file. On the next screen use the advanced settings and add all the options from the above props.conf click apply and you see it works 😉
Reasons why it does not work for you:

You did not apply the props.conf on the parsing Splunk instance, that is either a heavy weight forwarder or an indexer
You did not restart Splunk after applying the props.conf
the sourcetype in the the props.conf does not match your sourcetype, eq typo? what for Cases in the sourcetype!
the props.conf will only work on new events

Hope this helps ...

cheers, MuS

Re: How can I break events in my search?

patouellet — Wed, 11 Jul 2018 20:12:46 GMT

Tried all of that - not working for me. It just doesn't split all the events like I thought it would. I still see multiple <162> tag inside a single Splunk event.

It's the first time I'm stuck like this. I'm usually pretty good at this and been using the tool for 2 years.

Have you tried with Add Data page with the sample data in my first post? Is it working for you?

Thank you.

Re: How can I break events in my search?

MuS — Wed, 11 Jul 2018 20:17:57 GMT

Yep, used your provided examples, copied multiple lines into a file and used the Add Data page to create the props.conf options.

Re: How can I break events in my search?

patouellet — Wed, 11 Jul 2018 20:22:12 GMT

Ok good. You mentionned multiple lines - make sure there's no LF or CR anywhere - what if all these multiple "lines" are just one big mess of characters, just one big line with multiple <162> - is it working then?

Re: How can I break events in my search?

dpanych — Wed, 11 Jul 2018 20:39:02 GMT

Make sure you're setting the correct conf in the right location:
http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F