Splunk Search

How to combine messages in a log file that match an ID into a single event ?

ricotries
Communicator

I am currently testing forwarding logs from a file I am monitoring, but the software that generates those logs creates a line per type event related to a process, while keeping an ID of the process that generated all the individual messages. Is there a way to combine/merge all messages with the same ID into an event when they reach the indexer (the IDs are unique, even when the process runs multiple times, it generates a new ID for every occurrence)? I'm looking for a solution that does not involve setting up a search that will generate the same result, I want all the messages stored as one event that then can be searched for more details.

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

There aren't any settings that will merge events at index time based on a field.
Consider writing a scripted input that reads the file and does the required processing before writing the results to stdout for indexing. You'll need a heavy forwarder to run the script.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

There aren't any settings that will merge events at index time based on a field.
Consider writing a scripted input that reads the file and does the required processing before writing the results to stdout for indexing. You'll need a heavy forwarder to run the script.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...