topic Re: Extraction failure in Splunk Search

Extraction failure

tprzelom — Wed, 21 Nov 2012 15:40:40 GMT

This seems like a straight forward config can someone spot where it's going wrong. I am unable to extract the "aaa" field. The regex and extraction works correctly with the following search.
sourcetype=alerts | rex field=_raw "(?<aaa>.*\d{4}),"

Raw data (sourcetype alerts):
Wed Nov 21 09:47:41 EST 2012,CAM,Outer Door,Door State,Closed

Props.conf(/opt/splunk/etc/apps/myapp/local/):

[alerts] KV_MODE=none EXTRACT-door = (?<aaa>.*\d{4}),

Search:
sourcetype=alerts | extract reload=true

Thanks,
Thomas

Re: Extraction failure

RicoSuave — Wed, 21 Nov 2012 15:47:21 GMT

Try removing KV_MODE=none

and issue

| extract reload=T

From the flashtimeline.

Re: Extraction failure

tprzelom — Wed, 21 Nov 2012 15:50:34 GMT

Still no extraction happening

Re: Extraction failure

RicoSuave — Wed, 21 Nov 2012 15:58:38 GMT

Can we get a brief description on your architecture? Are you running a search head(s) with configured distributed peers? is there search head pooling involved? or is this just a one sh/indexer deployment? Also, if you could attach the output of the following command

./splunk cmd btool props list

That will help.

Additionally, make sure the field discovery button is turned to the on position.

Re: Extraction failure

tprzelom — Wed, 21 Nov 2012 16:04:51 GMT

Single SH/indexer deployment, that outputs a 3MB file because I have the Enterprise Security app installed.

Re: Extraction failure

alacercogitatus — Wed, 21 Nov 2012 16:05:49 GMT

What exactly are you trying to capture? Your regex (.*\d{4}) doesn't match anything except the timestamp.

Based on your props config, I will assume you are looking for the Door name.

rex field=_raw "\d{4},(?<whatever_cam_is>[^,]*),(?<door_name>[^,]*),"

should give you whatever field the CAM refers to, and the door_name of "Outer Door".

Have you considered using a transform?

props.conf

[alerts] REPORT-doorcontrol = doorcontrolcsv

transforms.conf

[doorcontrolcsv] DELIM = "," FIELDS = "timestamp", "whatever", "door_name", "alert_type", "alert_value"

Re: Extraction failure

RicoSuave — Wed, 21 Nov 2012 16:14:57 GMT

You should search through the output for the [alerts] stanza and see what configs it has.

Re: Extraction failure

tprzelom — Mon, 28 Sep 2020 12:50:43 GMT

props.conf:
[alerts]
REPORT-doorcontrol = doorcontrolcsv

transforms.conf:
[doorcontrolcsv]
DELIM = ","
FIELDS = "timestamp", "whatever", "door_name", "alert_type", "alert_value"

Search:
sourcetype=alerts | extract reload=T

I'm still not getting any field extractions.
I was just trying to get the extraction to work. I was going to build out the regex once I confirmed I could extract fields.

Re: Extraction failure

alacercogitatus — Wed, 21 Nov 2012 16:22:08 GMT

Your search should just be sourcetype=alerts. I believe the extract doesn't need to be there to pull searchtime extraction changes any more.

Re: Extraction failure

tprzelom — Mon, 28 Sep 2020 12:50:46 GMT

[alerts]
ANNOTATE_PUNCT = True
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128

Re: Extraction failure

tprzelom — Mon, 28 Sep 2020 12:50:49 GMT

MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
REPORT-doorcontrol = doorcontrolcsv
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
maxDist = 100

There was a bunch of lookups associated with ES in the output too, but I cut them out to save space/characters.

Re: Extraction failure

tprzelom — Wed, 21 Nov 2012 18:19:36 GMT

I removed the pipe to extract reload=T and am receiving the same results

Re: Extraction failure

tprzelom — Mon, 28 Sep 2020 12:51:57 GMT

[doorcontrolcsv]
DELIM = ","
FIELDS = "timestamp", "whatever", "door_name", "alert_type", "alert_value"

There should be an S at the end of DELIMS, for anyone who comes across this

Re: Extraction failure

alacercogitatus — Mon, 26 Nov 2012 18:51:50 GMT

whoops. Thanks tprzelom. If this answers your question (albeit misspelled) please accept it. Thanks!

Re: Extraction failure

tprzelom — Mon, 26 Nov 2012 19:57:18 GMT

Found it.

App permission problem

http://splunk-base.splunk.com/answers/61063/simple-field-extractions-not-working-in-props