All Apps and Add-ons
Highlighted

How do add a field to each indexed event with a value from the source file header?

Explorer

Hi Guys,

I tried to figure this out from the documentation on index-time field extractions but couldn't see an obvious way of doing what I need. I am indexing a file that contains a header section that is 1-line and is the first line in the file. That line tells me the version of software that generated the log file. The header looks like this:

2014/07/24 14:00:02.438 0 GeneralLogHeader Version 1.8 (April 23, 2014), System ID=Beta_1

And then on the lines following the header I have the actual events. I would like to add a field to each event that contains the software version value. Then for example I could search for all events that were generated by a particular software version. E.g.

search SoftwareVersion="1.8"

or say,

[search criteria] | stats count BY SoftwareVersion

This way I can map events/results to a specific software version. Is there a way to extract this data and add it as a field to each event?

Highlighted

Re: How do add a field to each indexed event with a value from the source file header?

Influencer

Try this

transforms.conf

[include_version]  
REGEX = (?i)^(?:[^ ]*( {1,2})){5}(\d+\.\d+)  
FORMAT=Software_Version::$1  
WRITE_META = true  

props.conf

[<Your stanza name>]
TRANSFORMS-include = include_version

fields.conf

[Software_Version]
INDEXED=true

I am not good at writing regex. You may have to check it once.

Highlighted

Re: How do add a field to each indexed event with a value from the source file header?

Explorer

Thanks for trying. But this will only add the field to an event that matches the regular expression (which is fine btw). I want to add the field for every event that is indexed in that file. To put it another way:

The first line in the file is a header event and I want to put information from that first line into each subsequent event in that file. Similar to how the file name gets added to each event.

0 Karma
Highlighted

Re: How do add a field to each indexed event with a value from the source file header?

Explorer

it's almost like I need a lookup table, the key being a file name and the value being the software version. So that I can discriminate events based on their source file and consequently, software version.

0 Karma
Highlighted

Re: How do add a field to each indexed event with a value from the source file header?

Influencer

OOPS i missed that.
What is the naming convention followed for your log files? logfilesequence.log something like this?
Are you stripping the header lines and sending them to separate index?
Depending on your answers i can suggest an approach for your answers?

Highlighted

Re: How do add a field to each indexed event with a value from the source file header?

Path Finder

there is a way to do this plz ?

0 Karma