Splunk Search
Highlighted

How to combine separate events with a unique identifier?

Path Finder

Hi,

I get send-mail logs in separated events\lines (5 events). With a unique identifier that appears in any event.

Is there a way to make the whole event to one complete event, and to do it as an automated process?

Thanks

0 Karma
Highlighted

Re: How to combine separate events with a unique identifier?

Legend

Please refer to various event correlation techniques that Splunk offers: http://docs.splunk.com/Documentation/Splunk/6.5.0/Search/Abouteventcorrelation

I feel stats command with by parameter as your unique identifier should suffice the need. You might have to provide search and data example for further assistance.




| eval message="Happy Splunking!!!"


0 Karma
Highlighted

Re: How to combine separate events with a unique identifier?

Legend

Refer to another one of my answers on how Stats can be used to correlate events.
https://answers.splunk.com/answers/474449/how-to-write-a-search-to-display-the-end-date-of-f.html#an...

Send some sample data/field names and I would be able to assist.




| eval message="Happy Splunking!!!"


0 Karma
Highlighted

Re: How to combine separate events with a unique identifier?

Path Finder

Stats calculates aggregate statistics over the results set, such as average, count, and sum - Not good for me.
Thanks

0 Karma
Highlighted

Re: How to combine separate events with a unique identifier?

Legend

Stats can perform event correlation which you need kindly give some data sample and I can help. Were you able to go though documentation on how stats can perform correlation?




| eval message="Happy Splunking!!!"


0 Karma
Highlighted

Re: How to combine separate events with a unique identifier?

SplunkTrust
SplunkTrust

Looks like you need to setup event parsing configuration (props.conf on Indexer/Heavy Forwarder) to use proper line-breaking to merge related lines (based on log data format) into one event. See this for more information on the line breaking part of event processing. http://docs.splunk.com/Documentation/Splunk/6.5.0/Data/Configureeventlinebreaking

For better suggestions, please share your sanitized sample data.

0 Karma
Highlighted

Re: How to combine separate events with a unique identifier?

Esteemed Legend

There is no easy way to do this but happily, it would not be worth the effort to do it even if there was. It makes the most sense to (as much as possible) keep the raw events in their native format. Assuming that the "unique identifier" that you describe is called mid, you can easily do things like this in search to aggregate the raw events into a conglomerated event:

... | stats (*) AS * BY mid

OR:

... | stats range(_time) AS duration list(_raw) AS events BY mid
0 Karma