Splunk Search
Highlighted

Field Extraction Question

Motivator

Hey,

I am having a look at transforms.conf and props.conf configuration files and wondering about the following question:

How do you do a field extraction from the source field (i.e. field=source) at search time? (An example of this would be great to see)

Thanks in advance for your help.


To add to what I said above:

The source field displays a file path when you run a search in Splunk. There is a word within that file path which I want Splunk to extract and place in a field called TheWord. How can I achieve this using props.conf and/or transforms.conf?

0 Karma
Highlighted

Re: Field Extraction Question

Builder

You can set up the just the props.conf to extract fields at search time.

I'm going to use my environment as an example. Our Apache logs are pipe | delimited. So I have the following in my props.conf:

[prod_apache_logs]

EXTRACT- = (?P[\d]+.[\d]+.[\d]+.[\d]+)\|[(?P[\d]{2}\/[\w]{3}\/[\d]{4}):(?P[\d]{2}:[\d]{2}:[\d]{2})\s-[\d]+]\|(?P[\d]+)\|\"(?P[\S]+)\s(?P\/[\S]+)\s(?P.)\"\|(?P[\d]+)\|(?P[\W\d]+)\|(?P.)\|(?P.*)

prod_apache_logs = sourcetype of my apache logs.

Easiest way to test this is to use the "rex" command in the search. IE, I would use something like this:

sourcetype="prod_apache_logs" | rex "(?P[\d]+.[\d]+.[\d]+.[\d]+)\|[(?P[\d]{2}\/[\w]{3}\/[\d]{4}):(?P[\d]{2}:[\d]{2}:[\d]{2})\s-[\d]+]\|(?P[\d]+)\|\"(?P[\S]+)\s(?P\/[\S]+)\s(?P.)\"\|(?P[\d]+)\|(?P[\W\d]+)\|(?P.)\|(?P.*)"

Hope this helps!

0 Karma
Highlighted

Re: Field Extraction Question

Motivator

The data that I want to extract is not in _raw but it is in source. Therefore I don't believe this will work. I tried this and it did not work as expected.

0 Karma
Highlighted

Re: Field Extraction Question

Motivator

I forgot to say thanks for your help. I added more to my question above which will hopefully confirm what I am seeking

0 Karma
Highlighted

Re: Field Extraction Question

Builder

Okay, I just read your update. I'm not sure how you would do it in in props.conf or transforms.conf.. I know you can do it via the standard search using eval and ltrim / rtrim..

0 Karma
Highlighted

Re: Field Extraction Question

Motivator

How would I do it using eval and ltrim/rtrim?

0 Karma
Highlighted

Re: Field Extraction Question

Builder

If you provide an example of your source, I'll write up the exact command.

In my case, my source is something like this: /opt/http/logs/www.mysite.com-access.log.

Using ltrim / rtrim, I extract out the /opt/http/logs and the -access.log to leave me with www.mysite.com:
..| eval website=ltrim(source,"/opt/httpd/logs") | eval website=rtrim(web_site,"-access.log")

0 Karma
Highlighted

Re: Field Extraction Question

Motivator
0 Karma
Highlighted

Re: Field Extraction Question

Path Finder

You just need to use SOURCE_KEY in the transform.

props.conf:

[source-extract]
REPORT-sourcefield = source-transform

transform.conf:

[source-transform]
SOURCE_KEY = source
REGEX = .......
FORMAT = fieldname=$1
0 Karma