How can I break events in my search?

[patouellet]

Hi,

Trying to break events and can't figure this one out. I receive a bunch of events in a single line, I want to break them using a pattern but it's not working for me. I'm using the Add data screen. Events should break when encountering <162>

I've tried BREAK_ONLY_BEFORE, LINE_BREAKER - nothing makes the event break. What am I doing wrong?

Sample of the log below:

<162>Mar 11 21:45:03 MACHINE CEF:0|PowerTech|Interact|3.1|TCA0001|The bytestream file *N/*N /Help Systems/Robot SCHEDULE ENTERPRISE/tmp/s7472_6859175.sz authority has been changed for user profile *PUBLIC.|2|src=X.X.X.X dst=0.0.0.0 msg=TYPE:JRN CLS:AUD JJOB:ENTSERVER1 JUSER:RBTENTUSR JNBR:171392 PGM:QLESPI OBJECT: LIBRARY: MEMBER: DETAIL:A *N *N *STMF *PUBLIC    Y   Y Y Y Y     RPL        0000 00000 * * *NA *NA<162>Mar 11 21:45:03 MACHINE CEF:0|PowerTech|Interact|3.1|TCA0001|The bytestream file *N/*N /Help Systems/Robot SCHEDULE ENTERPRISE/tmp/s7472_6859175.sz authority has been changed for user profile RBTENTUSR.|2|src=X.X.X.X dst=0.0.0.0 msg=TYPE:JRN CLS:AUD JJOB:ENTSERVER1 JUSER:RBTENTUSR JNBR:171392 PGM:QLESPI OBJECT: LIBRARY: MEMBER: DETAIL:A *N *N *STMF RBTENTUSR  Y Y Y   Y Y Y Y   Y Y RPL        0000 00000 * * *NA *NA

[dpanych]

Make sure you're setting the correct conf in the right location:
http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

[MuS · Accepted solution]

Hi patouellet,

try this props.conf on the parsing Splunk instance, and restart Splunk after the change:

[YourSourcetypeNameHere]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
CHARSET=UTF-8
LINE_BREAKER=([\r\n]+)\<162\>|\s\*NA\s\*NA(.*)\<
TIME_PREFIX=\<162\>

Hope that helps ...

cheers, MuS

UPDATE:

Took this to slack and got more details, like it is a TCP input and the events actually do not contain *NA. After some tries this line breaker worked just fine:

 LINE_BREAKER=[\*\.\r\n\)\d]+()\<162\>|^()\<162\> 

The response of the OP was awesome, and I want to share it:

[patouellet]

I appreciate the help. But it's not working for me. I still get most events wrapped in Splunk as a single event. I've done exactly what you suggested - no luck.

[MuS]

Hi there,

take a file that contains the events, use the Add Data page http://docs.splunk.com/Documentation/Splunk/latest/Data/Howdoyouwanttoadddata and add the file. On the next screen use the advanced settings and add all the options from the above props.conf click apply and you see it works 😉
Reasons why it does not work for you:

• You did not apply the props.conf on the parsing Splunk instance, that is either a heavy weight forwarder or an indexer
• You did not restart Splunk after applying the props.conf
• the sourcetype in the the props.conf does not match your sourcetype, eq typo? what for Cases in the sourcetype!
• the props.conf will only work on new events

Hope this helps ...

cheers, MuS

[patouellet]

Tried all of that - not working for me. It just doesn't split all the events like I thought it would. I still see multiple <162> tag inside a single Splunk event.

It's the first time I'm stuck like this. I'm usually pretty good at this and been using the tool for 2 years.

Have you tried with Add Data page with the sample data in my first post? Is it working for you?

Thank you.

[MuS]

Yep, used your provided examples, copied multiple lines into a file and used the Add Data page to create the props.conf options.

[patouellet]

Ok good. You mentionned multiple lines - make sure there's no LF or CR anywhere - what if all these multiple "lines" are just one big mess of characters, just one big line with multiple <162> - is it working then?