topic Re: Field Extraction from Regex in Splunk Search

Field Extraction from Regex

tmarlette — Mon, 28 Sep 2020 11:47:15 GMT

So I am relatively new to extracting fields in Splunk, but I have some knowledge of regex, and I'm attempting to apply it in Splunk.

I have a pattern I am attempting to extract and put into a field. The pattern looks like this:

USER@TEST

I am using this expression to match the pattern:

(\w+@\w+)

I would like to extract this into a field called "user_domain", and I'm having some difficulty renaming (\w+@\w+) as "user_domain".

PS... this forum doesn't show forward slashes, however they are there. 😃

Re: Field Extraction from Regex

Ayn — Tue, 08 May 2012 21:09:42 GMT

What ways did you try? You could make use of the rex command, like this:

... | rex "(?<user_domain>\w+@\w+)"

Or you could make this kind of extraction permanent by using the interactive field extractor (http://docs.splunk.com/Documentation/Splunk/latest/User/InteractiveFieldExtractionExample ).

Re: Field Extraction from Regex

tmarlette — Tue, 08 May 2012 22:04:11 GMT

Actually... I didn't try that at all.

<--- Shamed

| rex "(?\w+@\w+)"
that worked splendidly, thank you!

I did try the interactive extractor though, but it won't extract everything I needed it to.

Re: Field Extraction from Regex

yannK — Tue, 08 May 2012 22:08:02 GMT

By curiosity, are you trying to extract apache logs or IIS logs ?

Existing sourcetypes provide automatic extraction :

access_combined for apache logs
iis for iis logs, you have several options, use this guide to define your columns http://splunk-base.splunk.com/answers/1902/iis-and-exchange-log-header-extraction

Re: Field Extraction from Regex

tmarlette — Tue, 08 May 2012 22:11:43 GMT

Negative yannK, These are proprietary log messages that I'm attempting to scrub.

But to make sure I understand you correctly, because we do have apache and iis logs here as well, if I name my sourcetype "access_combined" and send my iss / apache logs there, Splunk will extract a set of fields auto-magically?

Re: Field Extraction from Regex

yannK — Mon, 28 Sep 2020 11:47:23 GMT

yes, some sourcetypes are defined and provide automatic field extractions. look for : syslog, access_combined and apache_errors ...

see http://docs.splunk.com/Documentation/Splunk/4.3.2/Data/Listofpretrainedsourcetypes

Re: Field Extraction from Regex

tmarlette — Thu, 29 Nov 2012 15:08:17 GMT

It does happen automagically, but you can make any sourcetype extract the same fields with the transform. start taking a look at props.conf, and transforms.conf for general iis field extractions.