Splunk Search

How to match the regex for LINE_BREAKER with the end of my event? Yet another one!

hvaithia
Path Finder

My event ends like this,

, "estimatedDuration": 2505189}

The no of digits in the estimated time can be a variable. How do I match this in props.conf using REGEX. I need to add it as a LINE_BREAKER.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

why are you asking about the 'end of the event'?

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

hvaithia
Path Finder

cos I had used a combination of

SHOULD_LINEMERGE=false
TRUNCATE=200000 ----------- cos my event is greater than 10k bytes
LINE_BREAKER=\"estimatedDuration\":\s\d+}
which didn't work for the event which was above 10k bytes

Also tried the other variant as described in http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/Indexmulti-lineevents

SHOULD_LINEMERGE=true
TRUNCATE=200000 ----------- cos my event is greater than 10k bytes
LINE_BREAKER=\"estimatedDuration\":\s\d+}
BREAK_ONLY_BEFORE={\"building\":
which didn't work for the event which was above 10k bytes as well

I see from the regex you gave that LINE_BREAKER should also include the place where we break/carriage return. I tried your regex as well with no luck. I have also tried the above combination with/without
MAX_EVENTS=2000
with no effect. What am I doing wrong?
PS: I am not able to write "\" (slash) marks in this comment. No idea why. Everything before 's' and 'd' in the LINE_BREAKER and BREAK_ONLY_BEFORE has a slash.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

you need to mark the code as cold... that's why the escape slashes are being hidden...

LINE_BREAKER and BREAK_ONLY_BEFORE would not be seen together as SHOULD_LINEMERGE must be 'false' for LINE_BREAKER and 'true' for BREAK_ONLY_BEFORE

for more detail you might want to look here.
Less is More here... but without question you need to understand how LINE_BREAKER works.
So I suggest you take a look at this:
http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/indexmulti-lineevents

There are instructions to help you determine exactly how your events are broken up...
You can try TRUNCATE=0 to basically turn it off while you're testing.
Leave MAX_EVENTS where it is and then check the error log to see how many lines there really are so you know where to set it... Splunk will complain and give you facts...

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

LINE_BREAKER = ([\r\n]+)\{"building":\s

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

LINE_BREAKER needs the beginning of the event. Show us the whole thing...

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

hvaithia
Path Finder

The event starts with {"building":
Sample event below
{"building": false, "changeSet": {"items": [], "kind": null}, "builtOn": "rhel6", "description": null, "artifacts": [], "timestamp": 1430241584496, "number": 13, "actions": [{"causes": [{"upstreamBuild": 14, "shortDescription": "Started by upstream project "answers" build number 14", "upstreamProject": "answers", "upstreamUrl": "job/answers/"}]}, {}, {}, {}, {}, {"highlightsData": "[{"Previous Job":"answers#14"},{"Previous Job":"answers_se"},{"Build host":"rhel6"}]", "highlightsTable": "<h4>Global Patterns</h4><b>Previous Job:</b><b>Started by <a href="/hudson/job/answers/14/">answers</a></b><b>Previous Job:</b><b>Started by <a href="/hudson/job/answers/8/">answers #8</a></b><b>Build host:</b><b>Built on <a href="/hudson/computer/rhel6/">rhel6</a></b>"}], "id": "2015-04-28_17-19-44", "keepLog": false, "url": "http://thefactory.xyz.com:9999/jenkins/job/answers/13/", "culprits": [], "result": "SUCCESS", "executor": null, "duration": 377658, "fullDisplayName": "answers", "estimatedDuration": 298415}

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...