topic Re: How do I map these fields from my sample log file at index or search-time? in Splunk Search

How do I map these fields from my sample log file at index or search-time?

brent_weaver — Wed, 06 Jul 2016 14:40:15 GMT

Hello. I have the following log file:

2016-06-28T10:08:08.152Z: pass proto tcp from 10.60.13.19:33099 to 10.193.44.12:443
2016-06-28T10:08:08.221Z: pass proto udp from 10.194.125.121:35984 to 10.60.13.18:161
2016-06-28T10:08:08.152Z: pass proto tcp from 10.60.13.19:33099 to 10.193.44.12:443
2016-06-28T10:08:08.221Z: pass proto icmp from 10.194.125.121 to 10.60.13.18
2016-06-28T10:08:08.491Z: pass proto tcp from 10.60.13.18:40070 to 10.193.44.11:443
2016-06-28T10:08:08.491Z: pass proto tcp from 10.60.13.18:40070 to 10.193.44.11:443

I would like to map protocol ( tcp/udp ), the From address and port #, the to address and port #.

Is it best to do this at search-time or at index-time? If at index time how would I do this in my props and transforms?

THANK YOU as always!

Re: How do I map these fields from my sample log file at index or search-time?

somesoni2 — Wed, 06 Jul 2016 15:39:27 GMT

In general, we recommend search-time extractions rather than index-time extractions. There are relatively few cases where index-time extractions are better, and they come at the cost of brittleness of configuration and an increase in index size (which in turn makes searches slower).

You can still save your search time field extraction in props.conf (see this http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles#Add_an_EXTRACT_field_extraction_stanza_to_props.conf)

YOu can use the following regex in props.conf (showing inline search field extraction)

your base search | rex "pass proto (?<protocol>\S+) from (?<src_ip>\d+\.\d+\.\d+\.\d+):(?<src_port>\d+) to (?<dest_ip>\d+\.\d+\.\d+\.\d+):(?<dest_port>\d+)"

Re: How do I map these fields from my sample log file at index or search-time?

woodcock — Wed, 06 Jul 2016 18:05:39 GMT

I would do search-time like this:

Your Base Search Here | rex "\d+-\d+-\d+T\d+:\d+:\d+.\d+[^:]*:.*?(?<protocol>\w+) from (?<src>\d+\.\d+\.\d+\.\d+(?::\d+)?) to (?<dest>\d+\.\d+\.\d+\.\d+(?::\d+)?)"