Splunk Search

How to parse a log file with multiple types of records?

vtsco
New Member

I have a log file with multiple line patterns. Something like this:

[name] [surname] [address] [phone] [birthdate] [lastlogin]
[key] [value]
[timestamp] [username] [url] [errcode] [srcip] [dstip]
[field1] [field2] [field3] [field4]

There are multiple types of records in this file and each record type must be dealt with differently. Each record has its own fields and they rarely overlap.

I need to extract each field in each record and then process them. How could I do that? Should I write a regex for each record type? Where should I put these regexes?

I've been searching for a solution, but didn't find anything.

Thanks.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

You can use props and transforms to extract the fields you need using regex. Its not a problem if you build a single regex for each distinct field.
Are the lines related to each other? meaning "[key] [value]" can belong to the same event as "[timestamp] [username] [url] [errcode] [srcip] [dstip]". If so, you can just the way splunk looks at your data and build a multi-line event. If they are not related then you might have an issue, since splunk indexes data based on the time stamp and for the looks of it, if they don't related, only one event has timestamp.
It easier to help you if you post a sanitized example of diferent events of you raw data.

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...