topic Re: regex field extraction; same source, variably different fields in Splunk Search

regex field extraction; same source, variably different fields

gsawyer1 — Mon, 30 Dec 2013 20:38:15 GMT

I've got input from a syslog source, that looks like this:

2012-10-10 04:04:52[connection-5] AUTH: User xxx authenticated.

2012-10-10 04:04:52[Scan Thread: document1.doc -> /127.0.0.1] MODULES: File analysis

I want to extract the user (xxx) and document1.doc fields at search time, if possible, for inclusion in a table, and charts, etc, but am having trouble devising a rex statement in the search that can extract both values as different fields, yet from the same source. A conditional regex is what I am looking for. This would also be part of a transaction (the next step to undertake). Any suggestions would be great!....

Re: regex field extraction; same source, variably different fields

lguinn2 — Mon, 30 Dec 2013 21:23:05 GMT

You can have multiple extract statements for a source or sourcetype...

[source::/yoursourcehere]
EXTRACT-e1=AUTH: User\s+(?<user>\S+)\s+authenticated.
EXTRACT-e2=Scan Thread:\s+(<file>\S+)\s

Re: regex field extraction; same source, variably different fields

gsawyer1 — Thu, 02 Jan 2014 21:02:58 GMT

I attempted the following in props.conf only:

[sourcetype::mysource] (because I already have multiple transforms applied to this source) EXTRACT-e1=AUTH: User\s+(?<submitter>\S+)\s+authenticated. EXTRACT-e2=Scan Thread:\s+(<file>\S+)\s\-\>\s

....and nothing happened. No fields "submitter" or "file" were extracted when I ran a search for that sourcetype, looking for results that contain this data.

Re: regex field extraction; same source, variably different fields

gsawyer1 — Thu, 02 Jan 2014 21:05:22 GMT

Was there some reason why you did not use the following syntax:

[source::/yoursourcehere]
EXTRACT-e1=AUTH:\s+User\s+(?\S+)\s+authenticated.
EXTRACT-e2=Scan\sThread:\s+(\S+)\s\-\>\s

Re: regex field extraction; same source, variably different fields

lguinn2 — Thu, 02 Jan 2014 21:18:37 GMT

No reason. It's probably even better to generalize the whitespace with \s as you did.

Re: regex field extraction; same source, variably different fields

lguinn2 — Thu, 02 Jan 2014 21:21:25 GMT

[sourcetype::mysource] is not valid. You need to specify either

[mysourcetypename]

[source::/pathto/mysource]