Splunk Search

How to combine messages in a log file that match an ID into a single event ?

ricotries
Communicator

I am currently testing forwarding logs from a file I am monitoring, but the software that generates those logs creates a line per type event related to a process, while keeping an ID of the process that generated all the individual messages. Is there a way to combine/merge all messages with the same ID into an event when they reach the indexer (the IDs are unique, even when the process runs multiple times, it generates a new ID for every occurrence)? I'm looking for a solution that does not involve setting up a search that will generate the same result, I want all the messages stored as one event that then can be searched for more details.

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

There aren't any settings that will merge events at index time based on a field.
Consider writing a scripted input that reads the file and does the required processing before writing the results to stdout for indexing. You'll need a heavy forwarder to run the script.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

There aren't any settings that will merge events at index time based on a field.
Consider writing a scripted input that reads the file and does the required processing before writing the results to stdout for indexing. You'll need a heavy forwarder to run the script.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...

Secure Your Future: Mastering Upgrade Readiness for Splunk 10

Spotlight: The Splunk Health Assistant Add-On  The Splunk Health Assistant Add-On is your ultimate companion ...

Observability Unlocked: Kubernetes & Cloud Monitoring with Splunk IM

Ready to master Kubernetes and cloud monitoring like the pros? Join Splunk’s Growth Engineering team on ...