topic Help in field extraction in Splunk Search

Help in field extraction

pr_blr — Mon, 28 Sep 2020 17:58:07 GMT

I need help in extracting the following fields from a log file:

log file has the following fixed format:

2013-08-11 04:00:01:  EPS2_DATA=  2600  EPS2_INDEX=  3100  EPS2_LOB_DATA=  1000  DB= 30100
2014-04-13 04:00:00:  EPS2_DATA=33267.984375  EPS2_INDEX= 11000  EPS2_LOB_DATA= 11200  DB=67078.984375

In the log,

EPS2_DATA is field name and its field value is 2600 and 33267.984375
EPS2_INDEX is field name and its value is 3100 and 11000
EPS2_LOB_DATA is field name and its value is 1000 and 11200
DB is field name and its value is 30100 and 67078.984375

I tried the following regular expression to extract the fields:

EXTRACT-EPS2_DATA = [^=\n]*=(?P[^ ]+)
EXTRACT-EPS2_INDEX = (?:[^=\n]*=){2}\s+(?P[^ ]+)
EXTRACT-EPS2_LOB_DATA = (?:[^=\n]*=){3}\s+(?P[^ ]+)
EXTRACT-DB = (?:[^=\n]*=){4}(?P.+)
EXTRACT-Start_Time = ^(?P\d+\-\d+\-\d+\s+\d+:\d+:\d+:),

But, I am unable to extract the EPS2_DATA properly because it contains an integer value and decimal value.

Re: Help in field extraction

kml_uvce — Wed, 29 Oct 2014 13:31:28 GMT

use this

EXTRACT-EPS2_DATA = [^=\n]*=\s*(?P<EPS2_DATA>[^ ]+)

Re: Help in field extraction

somesoni2 — Wed, 29 Oct 2014 14:11:14 GMT

These seems like classic kv pair log file, isn't splunk automatically extracting these fields for you?

Re: Help in field extraction

somesoni2 — Wed, 29 Oct 2014 14:21:03 GMT

Try this in your props.conf

EXTRACT-allfields = (?<Timestamp>(\d{4}-\d{2}-\d{2}\s\d{2}\:\d{2}\:\d{2})+)\:\s*EPS2_DATA=\s*(?<EPS2_DATA>[^ ]+)\s*EPS2_INDEX=\s*(?<EPS2_INDEX>[^ ]+)\s*EPS2_LOB_DATA= \s*(?<EPS2_LOB_DATA>[^ ]+)\s*DB=\s*(?<DB>[^ ]+)

Re: Help in field extraction

pr_blr — Wed, 29 Oct 2014 15:02:35 GMT

thanks , this is working..

Re: Help in field extraction

bcarnot — Sat, 28 Feb 2015 18:08:41 GMT

I am facing a similar issue, when using the "Interactive field extractor"

The data I want to capture is this:

DBWaitTime.avg: 1.273037542662116   msecs
DBWaitTime.completed:   293 ops
DBWaitTime.maxActive:   1   threads
DBWaitTime.maxTime: 23  msecs
DBWaitTime.minTime: 0   msecs
DBWaitTime.time:    373 msecs
JDBC_Connection_Url.value:  jdbc:   
JDBC_Connection_Username.value: PORTLET 
LogicalConnection.value:    null    
/JDBC/Driver/CONNECTION_5/Statement [type=JDBC_Statement]
 Execute.active:    0   threads
 Execute.avg:   1.3652482269503545  msecs
 Execute.completed: 282 ops
 Execute.maxActive: 1   threads
 Execute.maxTime:   10  msecs

I have tried as example values one at a time (msec or threads, or : 10, or 1.3265, but can never get the generated pattern (regex) to identify all the fields. As an end result I want to report on the named values and their measurements.