Getting Data In

Extracting time zone and other info from header of a log and apply them to events in same log

blee_i365
Explorer

Hello Splunk experts, my log files are structured in the following way:


09032011 12:23:34.567 App name: TestApp01, App version: 2.34, Server time zone: -0700
09042011 01:12:32.800 event 01
09052011 04:11:31.123 event 02
09052011 05:54:20.482 event 03

The first line contains information pertaining to all events that are recorded in the same log file, and the logs can grow to thousands of lines.

I am having two issues:

1. Can I extract the -0700 time zone info from the first line of the log, then have Splunk index all events within this log with this time zone? Right now these events are assigned with the Splunk server time zone which gets messy as I have logs coming from various time zones.

2. Can I extract the App name and App version fields, then add these fields to the subsequent events within the same log, so that a query like [search AppName="TestApp01" AND AppVersion="2.34"] would return all events from the same log file? (multi-line event or | transaction can do the trick but only for smaller log files. Unfortunately most are large files, making these methods unrealistic)

Thanks in advance!

Tags (3)
0 Karma

woodcock
Esteemed Legend

Regarding #1: No. The best you can do is to have each instance of the file generator (which presumably is always inside the same TZ) write to his own directory instead of to his own directory and then map source to TZ in props.conf like this:

[source::/blah/commondir/instance1dir/*]
TZ = US/Central
[source::/blah/commondir/instance2dir/*]
TZ = US/Mountain

Regarding #2: Not directly but you can do something like this:

... | rex "App name: (?<App_name>[^,]*), App version: (?<App_version>[^,]*)" |search App_name="X" App_version="Y" | map search="source=$source$"
0 Karma
Get Updates on the Splunk Community!

What’s New & Next in Splunk SOAR

Security teams today are dealing with more alerts, more tools, and more pressure than ever.  Join us on ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

September Community Champions: A Shoutout to Our Contributors!

As we close the books on another fantastic month, we want to take a moment to celebrate the people who are the ...