topic Re: help with field extraction in Splunk Search

help with field extraction

a212830 — Mon, 08 Feb 2016 17:26:50 GMT

Hi,

I have a field extraction that I'm hoping someone can help me with.

Here's the example:

2016-02-08T12:17:34.219722-05:00 myhost  02/08/2016:12:16:12  mhost 0-PPE-0 : SSLVPN LOGIN 6457080 0 : Context z123456@1.2.3.4 - SessionId: 10109- User z123456 - Client_ip 1.2.3.4

I need to extract the second z123456 field, but I also want to put the SessionId and Client_ip as validators for part of the regex.

Re: help with field extraction

somesoni2 — Mon, 08 Feb 2016 17:48:05 GMT

Try something like this

Your base search | rex  "SessionId:\s+\d+-\s+User\s+(?<User>\S+)\s+-\s+Client_ip"

Re: help with field extraction

Richfez — Mon, 08 Feb 2016 17:48:25 GMT

Sure,

 ... | rex "Context\s+(?<context>[^\s]+)\s+-\s+SessionId:\s+(?<SessionID>\d+)\s?-\s+User\s+(?<User>[^\s]+)\s+-\s+Client_ip\s+(?<client_ip>[^\s]+)"

Try that as your rex. I don't know if the lack of a space beween the session id 10109 and he dash after it is legit or a mistake, so I put that "space" as optional (hence that one is \s? instead of \s+ as the others are)

Re: help with field extraction

a212830 — Mon, 08 Feb 2016 18:24:41 GMT

Thanks. This works great. How would I plug this into the field extractor?

Re: help with field extraction

Richfez — Tue, 29 Sep 2020 08:41:42 GMT

I suspect the real question is "How do I make this 'automatic' and have it work on all appropriate events so I have fields to use in my reports?" Is that right?

The field extractor is one way to have built the rex above. I did it manually, so there's no real need for the field extractor itself.

To make it run automatically, the field extractor puts the entries that are created in props.conf as an EXTRACT line against your sourcetype (note a). You can do the same by editing your apps' $splunkhome/etc/-appname-/local/props.conf (making it if required) or using the $splunkhome/etc/system/local/props.conf one if you aren't creating or in an app (note b).

Your entry will be something like

[mysourcetype]
EXTRACT-SSLVPN_info = Context\s+(?<context>[^\s]+)\s+-\s+SessionId:\s+(?<SessionID>\d+)\s?-\s+User\s+(?<User>[^\s]+)\s+-\s+Client_ip\s+(?<client_ip>[^\s]+)

Where "mysourcetype" is the sourcetype of this data. You will need to restart splunk or at least on a blank search do | extract reload=t (I promise, beginning pipe won't break anything!).

After that, those extractions should show up in your event searches that return the data that should be extracted.

Please post back if you have difficulties!

Note, the field extractor is very useful and I use it from time to time myself, but it isn't always the best regex-writer. I usually prefer creating my own regexes because, even as bad as they are, they're better (note c) than the ones the field extractor usually writes. Besides, regex is FUN!

Oh, lastly, if you DO want to use the field extractor, I think you can tell it you want the "regex" variety of extractions (it's one of the wizard's options early on), then select a row to work on, select the piece you want extracted (let's say in this case it's the client_ip), then there's a button to "view regex" or something - show that. Once you see that, paste in your own regex which in the case of client IP would be... Client_ip\s+(?[^\s]+ then save it. I've used a process liek that before, but there have been changes to the wizard in the last few versions so you might have to make some adjustments to my instructions.

a) I may have lied, I don't know how the field extractor does it though I'm sure it's a variant of what I described.
b) I recommend an app, but hey, as long as you aren't trying to stick this in the default folder I'm OK with it. 🙂
c) For suitably loose values of "better", but often more generic yet more precise.

Re: help with field extraction

sloshburch — Fri, 12 Feb 2016 22:21:38 GMT

Remember to do like @rich7177 mentioned and DO NOT include the quotes when pasting into Field Extractor or manually adding a new extraction (through UI or conf file). That tends to nail everyone.