topic Re: Field Extraction Question in Splunk Search

Field Extraction Question

Ant1D — Tue, 07 Sep 2010 21:50:40 GMT

Hey,

I am having a look at transforms.conf and props.conf configuration files and wondering about the following question:

How do you do a field extraction from the source field (i.e. field=source) at search time? (An example of this would be great to see)

Thanks in advance for your help.

To add to what I said above:

The source field displays a file path when you run a search in Splunk. There is a word within that file path which I want Splunk to extract and place in a field called TheWord. How can I achieve this using props.conf and/or transforms.conf?

Re: Field Extraction Question

Brian_Osburn — Tue, 07 Sep 2010 22:15:48 GMT

You can set up the just the props.conf to extract fields at search time.

I'm going to use my environment as an example. Our Apache logs are pipe | delimited. So I have the following in my props.conf:

[prod_apache_logs]

EXTRACT- = (?P[\d]+.[\d]+.[\d]+.[\d]+)\|[(?P[\d]{2}\/[\w]{3}\/[\d]{4}):(?P[\d]{2}:[\d]{2}:[\d]{2})\s-[\d]+]\|(?P[\d]+)\|\"(?P[\S]+)\s(?P\/[\S]+)\s(?P.)\"\|(?P[\d]+)\|(?P[\W\d]+)\|(?P.)\|(?P.*)

prod_apache_logs = sourcetype of my apache logs.

Easiest way to test this is to use the "rex" command in the search. IE, I would use something like this:

sourcetype="prod_apache_logs" | rex "(?P[\d]+.[\d]+.[\d]+.[\d]+)\|[(?P[\d]{2}\/[\w]{3}\/[\d]{4}):(?P[\d]{2}:[\d]{2}:[\d]{2})\s-[\d]+]\|(?P[\d]+)\|\"(?P[\S]+)\s(?P\/[\S]+)\s(?P.)\"\|(?P[\d]+)\|(?P[\W\d]+)\|(?P.)\|(?P.*)"

Hope this helps!

Re: Field Extraction Question

Ant1D — Tue, 07 Sep 2010 22:47:48 GMT

The data that I want to extract is not in _raw but it is in source. Therefore I don't believe this will work. I tried this and it did not work as expected.

Re: Field Extraction Question

Ant1D — Tue, 07 Sep 2010 22:57:28 GMT

I forgot to say thanks for your help. I added more to my question above which will hopefully confirm what I am seeking

Re: Field Extraction Question

Brian_Osburn — Tue, 07 Sep 2010 23:00:01 GMT

Okay, I just read your update. I'm not sure how you would do it in in props.conf or transforms.conf.. I know you can do it via the standard search using eval and ltrim / rtrim..

Re: Field Extraction Question

Ant1D — Tue, 07 Sep 2010 23:10:56 GMT

How would I do it using eval and ltrim/rtrim?

Re: Field Extraction Question

Brian_Osburn — Mon, 28 Sep 2020 09:17:25 GMT

If you provide an example of your source, I'll write up the exact command.

In my case, my source is something like this: /opt/http/logs/www.mysite.com-access.log.

Using ltrim / rtrim, I extract out the /opt/http/logs and the -access.log to leave me with www.mysite.com:
..| eval web_site=ltrim(source,"/opt/httpd/logs") | eval web_site=rtrim(web_site,"-access.log")

Re: Field Extraction Question

Ant1D — Tue, 07 Sep 2010 23:50:45 GMT

Have a look at http://answers.splunk.com/questions/6511/field-extraction-mystery for an example source

Re: Field Extraction Question

wollinet — Wed, 09 Mar 2011 22:54:13 GMT

You just need to use SOURCE_KEY in the transform.

props.conf:

[source-extract]
REPORT-sourcefield = source-transform

transform.conf:

[source-transform]
SOURCE_KEY = source
REGEX = .......
FORMAT = fieldname=$1