topic Re: Testing and debugging an Inline Field Extraction in Splunk Search

Testing and debugging an Inline Field Extraction

adamb0mb — Tue, 20 Nov 2012 00:33:11 GMT

I'm trying to setup Splunk to understand CloudFront log files.

I've setup a field extraction like this:

^(?<DATE>[\\S]+)\\t(?<TIME>[\\S]+)\\t(?<CF_SERVER>[\\S]+)\\t(?<BYTES>[\\S]+)\\t(?<IP>[\\S]+)\\t(?<ACTION>[\\S]+)\\t(?<CF_DIST>[\\S]+)\\t(?<REQUEST>[\\S]+)\\t(?<STATUS>[\\S]+)\\t(?<REFERRER>[\\S]+)\\t(?<BROWSER>[\\S]+)\\t(?<QUERY>[\\S]+)

I'm not seeing any of these fields showing up in the Search UI, and it I can't search for them. I've tested the regex with a rex command, and it works just fine.

A sidenote:
I'm a new user. A lot of the documentation on this subject refers to transforms.conf and props.conf, but in the locations specified in the documentation, those files don't exist.

Re: Testing and debugging an Inline Field Extraction

Rob — Mon, 28 Sep 2020 12:50:03 GMT

If you are looking to set this regex up in a props or transforms config file, then you may need to create those files in the $SPLUNK_HOME/etc/system/local/ directory or in a custom app directory depending on how you are looking to do your field extractions. Also, if that regex works with the 'rex' command then you will want to un-escape your slashes before using it within props or transforms files.

This means a line in the config file would look like:

^(?<DATE>[\S]+)\t(?<TIME>[\S]+)\t(?<CF_SERVER>[\S]+)\t(?<BYTES>[\S]+)\t(?<IP>[\S]+)\t(?<ACTION>[\S]+)\t(?<CF_DIST>[\S]+)\t(?<REQUEST>[\S]+)\t(?<STATUS>[\S]+)\t(?<REFERRER>[\S]+)\t(?<BROWSER>[\S]+)\t(?<QUERY>[\S]+)

Re: Testing and debugging an Inline Field Extraction

adamb0mb — Tue, 20 Nov 2012 02:57:47 GMT

I don't necessarily want to put this in the props.conf or transforms.conf, I just want it the extraction to work 🙂

The regex your'e showing looks really wonky... no backslashes at all? If the backslashes don't need to be escaped, shouldn't it look like:
^(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)\t(?[\S]+)

Edit: looks like the wonkiness might be splunkbase display issues.

Re: Testing and debugging an Inline Field Extraction

adamb0mb — Tue, 20 Nov 2012 03:25:53 GMT

The issue was that rex was having me double-escape the special characters. When I changed to a single backslash, it started working.