topic Re: Unable to break Multi line event into single event in Getting Data In

Unable to break Multi line event into single event

srimukundant — Thu, 21 Nov 2019 14:22:10 GMT

Im trying to break multiline events into single event for applying logics , but not able to to tried multiple options

How log looks as below:

Tried the config changes in props.config as below

Can some one help me...

Re: Unable to break Multi line event into single event

woodcock — Thu, 21 Nov 2019 15:12:43 GMT

Try this (I am not sure if multiple capture groups is supported for LINE_BREAKER😞

[Your Sourcetype Here]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+\s*\<UKGHS\>[,\s]+)

Re: Unable to break Multi line event into single event

oscar84x — Thu, 21 Nov 2019 15:17:10 GMT

Try removing ([/r/n]+) from your line breaker and leaving only (< UKGHS>,).
If that doesn't work please provide an actual sample of events.

Re: Unable to break Multi line event into single event

srimukundant — Wed, 30 Sep 2020 03:07:09 GMT

Sample Data

<UKGHS>,2019-11-17,8909,5759,3117
<UKGHS>,2019-11-17,9217,6001,3217
<UKGHS>,2019-11-17,9330,5984,3217
<UKGHS>,2019-11-17,1729,4176,3117
<UKGHS>,2019-11-17,5042,4171,3117
<UKGHS>,2019-11-18,9351,6048,3217
<UKGHS>,2019-11-19,9320,6067,3217
<UKGHS>,2019-11-20,8960,5824,3117

above is the one sample event which gets captured at a particular time. Wanted this complete single event to be Splitted into multiple separate events (i.e into 8 separate event lines)

Props
[UK_Stats]
SHOULD_LINEMERGE = false
LINE_BREAKER = (< UKGHS>,).

Hi thanks for your reply , but it didnt work..Can you help me

Re: Unable to break Multi line event into single event

srimukundant — Thu, 21 Nov 2019 15:48:13 GMT

Hi,

Thanks for your reply , but Still no luck ,

11/21/19
3:40:14.000 PM  
<UKGHS>,2019-11-17,8909,5759,3117
<UKGHS>,2019-11-17,9217,6001,3217
<UKGHS>,2019-11-17,9330,5984,3217
<UKGHS>,2019-11-17,1729,4176,3117
<UKGHS>,2019-11-17,5042,4171,3117
<UKGHS>,2019-11-18,9351,6048,3217
<UKGHS>,2019-11-19,9320,6067,3217
<UKGHS>,2019-11-20,8960,5824,3117
Collapse

11/21/19
3:36:46.000 PM  
<UKGHS> ,2019-11-16,8909,5759,3116
<UKGHS> ,2019-11-16,9216,6001,3216
<UKGHS>,2019-11-16,9330,5984,3216
<UKGHS>,2019-11-16,1629,4166,3116
<UKGHS>,2019-11-17,5042,4171,3116
Show all 8 lines

11/21/19
3:35:52.000 PM  
<UKGHS> ,2019-11-15,8909,5759,3115
<UKGHS> ,2019-11-15,9215,6001,3215
<UKGHS>,2019-11-16,9330,5984,3215
<UKGHS>,2019-11-16,1529,4156,3115
<UKGHS>,2019-11-17,5042,4171,3115
Show all 8 lines

Re: Unable to break Multi line event into single event

srimukundant — Thu, 21 Nov 2019 15:52:27 GMT

Sample data

Blockquote

,2019-11-17,8909,5759,3117
,2019-11-17,9217,6001,3217
,2019-11-17,9330,5984,3217
,2019-11-17,1729,4176,3117
,2019-11-17,5042,4171,3117
,2019-11-18,9351,6048,3217
,2019-11-19,9320,6067,3217
,2019-11-20,8960,5824,3117

Re: Unable to break Multi line event into single event

oscar84x — Thu, 21 Nov 2019 16:06:19 GMT

@srimukundant What you provided doesn't include < UKGHS>
Can you copy and paste exactly what you're trying to ingest?

Re: Unable to break Multi line event into single event

srimukundant — Thu, 21 Nov 2019 16:09:05 GMT

Hi, This the ingest data

 <UKGHS>,2019-11-17,8909,5759,3117
 <UKGHS>,2019-11-17,9217,6001,3217
 <UKGHS>,2019-11-17,9330,5984,3217
 <UKGHS>,2019-11-17,1729,4176,3117
 <UKGHS>,2019-11-17,5042,4171,3117
 <UKGHS>,2019-11-18,9351,6048,3217
 <UKGHS>,2019-11-19,9320,6067,3217
 <UKGHS>,2019-11-20,8960,5824,3117

Re: Unable to break Multi line event into single event

oscar84x — Thu, 21 Nov 2019 16:18:22 GMT

Thanks. For what you provided specifically, the settings below work for me. Note that in the sample you provided there are 2 or 3 empty spaces at the beginning of each line. I'm not sure if that's accurate or not:

[ <your_sourcetype> ]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\s+<UKGHS>,)
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%d
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=11

Re: Unable to break Multi line event into single event

srimukundant — Wed, 30 Sep 2020 03:07:12 GMT

Thanks,

But it didnt work for me . Not sure if i'm missing any settings

[IIS]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\s+,)
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%d
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=11

Re: Unable to break Multi line event into single event

woodcock — Wed, 30 Sep 2020 03:03:17 GMT

I am sure that these settings are correct so it must be something else. If you are doing a sourcetype override/overwrite, you must use the ORIGINAL value, NOT the new value. You must deploy your settings to the first full instance(s) of Splunk that handle the events (usually either the HF tier if you use one, or else your Indexer tier) UNLESS you are using HEC's JSON endpoint (it gets pre-cooked) or INDEXED_EXTRACTIONS (configs go on the UF in that case), then restart all Splunk instances there. When (re)evaluating, you must send in new events (old events will stay broken), then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly indexed events.