Event breaking not working properly with the regex ([\r\n]+)

[soumdey]

Event breaking not working properly with the below regex...

props.conf
LINE_BREAKER=([\r\n]+)

My Log data :

('Hi-Spec JDA','Stuck Pick Work', 'No Pick Work - R/C','0','Stuck Pick Work','2019-07-17 09:10:01');
('Hi-Spec JDA','Locking Sessions', 'None','0','Locking Sessions','2019-07-17 09:10:01');
('Hi-Spec JDA','Database Locks', 'None','0','Database Locks','2019-07-17 09:10:01');
('Hi-Spec JDA','Long Running Waves', 'None','0','Long Running Waves','2019-07-17 09:10:01');
('Hi-Spec JDA','Broken Pipe Error', 'No Errors','0','Broken Pipe Error','2019-07-17 09:10:01');
('Hi-Spec JDA','Invmov with Inspect Inventory', 'Attention: Movement with Inspect Inventory Found','1','Invmov with Inspect Inventory','2019-07-17 09:10:01');
('Hi-Spec JDA','Movement Pick to VLM', 'No Movement going to VLM from Pick','0','Movement Pick to VLM','2019-07-17 09:10:01');

Issue here is, couple of lines are coming under the same event.

From the above example... there are 7 new lines of logs and it should break into 7 events...
But it breaks into 6 events as shown below..

EVENT 1 : ('Hi-Spec JDA','Stuck Pick Work', 'No Pick Work - R/C','0','Stuck Pick Work','2019-07-17 09:10:01');

EVENT 2 : ('Hi-Spec JDA','Locking Sessions', 'None','0','Locking Sessions','2019-07-17 09:10:01');

EVENT 3 : ('Hi-Spec JDA','Database Locks', 'None','0','Database Locks','2019-07-17 09:10:01');

EVENT 4 : ('Hi-Spec JDA','Long Running Waves', 'None','0','Long Running Waves','2019-07-17 09:10:01');

EVENT 5 (The one having issue) : ('Hi-Spec JDA','Broken Pipe Error', 'No Errors','0','Broken Pipe Error','2019-07-17 09:10:01');
('Hi-Spec JDA','Invmov with Inspect Inventory', 'Attention: Movement with Inspect Inventory Found','1','Invmov with Inspect Inventory','2019-07-17 09:10:01');

EVENT 6 : ('Hi-Spec JDA','Movement Pick to VLM', 'No Movement going to VLM from Pick','0','Movement Pick to VLM','2019-07-17 09:10:01');

What I believe the issue in EVENT 6 has something to do with the length of the second line in EVENT 6 which is the longest.

Can somebody please help me out here on how to rectify the issue.....????

[soumdey]

The real problem was that my event breaking which is contained in props.conf was configured in the splunk cloud.
Now, since we are talking about HF here, so the HF was parsing and event breaking the data by-passing the configuration that I did in splunk cloud which was causing the issue.

To fix the issue, I copied the props.conf file from the splunk cloud and put it inside the HF which resolved the issue.

NOTE : The inputs.conf file was already inside the HF.

[woodcock]

You should also set these:

SHOULD_LINEMERGE = false
EVENT_BREAKER = ([\r\n]+)
EVENT_BREAKER_ENABLE = true

[soumdey]

just as FYI... the regex ([\r\n]+) works like a charm when I manually add the data file under the sourcetype... It fails when data starts coming in from the forwarder..

Also my inputs.conf file is inside the heavy forwarder and NOT in Splunk cloud and props.conf file is in splunk cloud .
Can this be an issue??

[woodcock]

Why did you accept the answer that you did? What was the real problem? I am not saying that mine was right/better but it seems like you discovered something completely different as the resolution to the problem and it helps us all learn when you share.

[soumdey]

I accepted the answer because of the comment from @richgalloway
"Something must be different about the data when it comes in from the forwarder.

Inputs.conf should be on the HF and NOT in Splunk Cloud."

The real problem was that my event breaking which is contained in props.conf was configured in the splunk cloud.
Now, since we are talking about HF here, so the HF was parsing and event breaking the data by-passing the configuration that I did in splunk cloud which was causing the issue.

To fix the issue, I copied the props.conf file from the splunk cloud and put it inside the HF which resolved the issue.

NOTE : The inputs.conf file was already inside the HF.

Hope this resolves your query.

[richgalloway · Accepted solution]

The existing line breaker should work. See if this works any better: LINE_BREAKER = ;([\r\n]+)\(.

[soumdey]

Thanks for the response richagalloway, but unfortunately the regex provided by you further complicates things.

[soumdey]

just as FYI... the regex ([\r\n]+) works like a charm when I manualy add the data file under the sourcetype... It fails when data starts coming in from the forwarder..

Also my inputs.conf file is inside the heavy forwarder and NOT in Splunk cloud. can this be an issue??

[richgalloway]

Something must be different about the data when it comes in from the forwarder.

Inputs.conf should be on the HF and NOT in Splunk Cloud.