Splunk Search

How do I define event line-breaking in a search?

driekhof
Path Finder

Our Splunk forwarder is sending events that looks something like this:

{"consumerTstamp":1488853092650,"metric":"EvTot.byDomain","types":{"events":{}}}
{"consumerTstamp":1488853093650,"metric":"EvTot.byDomain","types":{"events":{}}}
{"consumerTstamp":1488853094650,"metric":"EvTot.byDomain","types":{"events":{}}}

This is obviously 3 events, but Splunk sees it as one. I've been looking at how to get Splunk to separate on newline in a search, but have only found things about setting some property in the configuration. How would I do this in a search?

0 Karma

somesoni2
Revered Legend

The recommended method here would to be fix your sourcetype definition (props.conf on indexer/heavy forwarder) to have proper line breaking and timestamp recognition of your events. (if possible delete and re-ingest the data). If you still want to do it from search try something like this

your base search with field _raw | rex mode=sed "s/([\r\n]+)/##LF##/g" | makemv _raw delim="##LF##"
| mvexpand _raw | spath | eval _time=round(consumerTstamp/1000,3)
0 Karma

pradeepkumarg
Influencer

While the best practice is to fix the line breaking in props.conf, you can extract these three lines into a multivalued field using rex and then use mvexpand to seperate the events.

0 Karma

driekhof
Path Finder

I did find a solution (I think) from searching other answers, it looks something like this:

source="/var/log/mylog/my.log.0" | rex max_match=0 field=_raw "(?[^\n]+)" | mvexpand lineData | eval _raw=lineData | sort consumerTstamp

That works but seems a little verbose for something so basic. Is there a better way?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...