Splunk Search

Extraction failure

Path Finder

This seems like a straight forward config can someone spot where it's going wrong. I am unable to extract the "aaa" field. The regex and extraction works correctly with the following search.
sourcetype=alerts | rex field=_raw "(?<aaa>.*\d{4}),"

Raw data (sourcetype alerts):
Wed Nov 21 09:47:41 EST 2012,CAM,Outer Door,Door State,Closed

Props.conf(/opt/splunk/etc/apps/myapp/local/):

[alerts]
KV_MODE=none
EXTRACT-door = (?<aaa>.*\d{4}),

Search:
sourcetype=alerts | extract reload=true

Thanks,
Thomas

0 Karma

Path Finder
0 Karma

SplunkTrust
SplunkTrust

What exactly are you trying to capture? Your regex (.*\d{4}) doesn't match anything except the timestamp.

Based on your props config, I will assume you are looking for the Door name.

rex field=_raw "\d{4},(?<whatever_cam_is>[^,]*),(?<door_name>[^,]*),"

should give you whatever field the CAM refers to, and the door_name of "Outer Door".

Have you considered using a transform?

props.conf

[alerts]
REPORT-doorcontrol = doorcontrolcsv

transforms.conf

[doorcontrolcsv]
DELIM = ","
FIELDS = "timestamp", "whatever", "door_name", "alert_type", "alert_value"

0 Karma

SplunkTrust
SplunkTrust

whoops. Thanks tprzelom. If this answers your question (albeit misspelled) please accept it. Thanks!

0 Karma

Path Finder

[doorcontrolcsv]
DELIM = ","
FIELDS = "timestamp", "whatever", "doorname", "alerttype", "alert_value"

There should be an S at the end of DELIMS, for anyone who comes across this

Path Finder

I removed the pipe to extract reload=T and am receiving the same results

0 Karma

SplunkTrust
SplunkTrust

Your search should just be sourcetype=alerts. I believe the extract doesn't need to be there to pull searchtime extraction changes any more.

Path Finder

props.conf:
[alerts]
REPORT-doorcontrol = doorcontrolcsv

transforms.conf:
[doorcontrolcsv]
DELIM = ","
FIELDS = "timestamp", "whatever", "doorname", "alerttype", "alert_value"

Search:
sourcetype=alerts | extract reload=T

I'm still not getting any field extractions.
I was just trying to get the extraction to work. I was going to build out the regex once I confirmed I could extract fields.

0 Karma

Builder

Try removing KV_MODE=none

and issue

| extract reload=T

From the flashtimeline.

0 Karma

Path Finder

MUSTBREAKAFTER =
MUSTNOTBREAKAFTER =
MUST
NOTBREAKBEFORE =
REPORT-doorcontrol = doorcontrolcsv
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
maxDist = 100

There was a bunch of lookups associated with ES in the output too, but I cut them out to save space/characters.

0 Karma

Path Finder

[alerts]
ANNOTATEPUNCT = True
BREAK
ONLYBEFORE =
BREAK
ONLYBEFOREDATE = True
CHARSET = UTF-8
DATETIMECONFIG = /etc/datetime.xml
HEADER
MODE =
LEARNSOURCETYPE = true
LINE
BREAKERLOOKBEHIND = 100
MAX
DAYSAGO = 2000
MAX
DAYSHENCE = 2
MAX
DIFFSECSAGO = 3600
MAXDIFFSECSHENCE = 604800
MAX
EVENTS = 256
MAXTIMESTAMPLOOKAHEAD = 128

0 Karma

Builder

You should search through the output for the [alerts] stanza and see what configs it has.

0 Karma

Path Finder

Single SH/indexer deployment, that outputs a 3MB file because I have the Enterprise Security app installed.

0 Karma

Builder

Can we get a brief description on your architecture? Are you running a search head(s) with configured distributed peers? is there search head pooling involved? or is this just a one sh/indexer deployment? Also, if you could attach the output of the following command

./splunk cmd btool props list

That will help.

Additionally, make sure the field discovery button is turned to the on position.

0 Karma

Path Finder

Still no extraction happening

0 Karma