Splunk Search

Testing and debugging an Inline Field Extraction

adamb0mb
Explorer

I'm trying to setup Splunk to understand CloudFront log files.

I've setup a field extraction like this:

^(?<DATE>[\\S]+)\\t(?<TIME>[\\S]+)\\t(?<CF_SERVER>[\\S]+)\\t(?<BYTES>[\\S]+)\\t(?<IP>[\\S]+)\\t(?<ACTION>[\\S]+)\\t(?<CF_DIST>[\\S]+)\\t(?<REQUEST>[\\S]+)\\t(?<STATUS>[\\S]+)\\t(?<REFERRER>[\\S]+)\\t(?<BROWSER>[\\S]+)\\t(?<QUERY>[\\S]+)

I'm not seeing any of these fields showing up in the Search UI, and it I can't search for them. I've tested the regex with a rex command, and it works just fine.

A sidenote:
I'm a new user. A lot of the documentation on this subject refers to transforms.conf and props.conf, but in the locations specified in the documentation, those files don't exist.

Tags (2)
0 Karma
1 Solution

adamb0mb
Explorer

The issue was that rex was having me double-escape the special characters. When I changed to a single backslash, it started working.

View solution in original post

0 Karma

adamb0mb
Explorer

The issue was that rex was having me double-escape the special characters. When I changed to a single backslash, it started working.

0 Karma

Rob
Splunk Employee
Splunk Employee

If you are looking to set this regex up in a props or transforms config file, then you may need to create those files in the $SPLUNK_HOME/etc/system/local/ directory or in a custom app directory depending on how you are looking to do your field extractions. Also, if that regex works with the 'rex' command then you will want to un-escape your slashes before using it within props or transforms files.

This means a line in the config file would look like:


^(?<DATE>[\S]+)\t(?<TIME>[\S]+)\t(?<CF_SERVER>[\S]+)\t(?<BYTES>[\S]+)\t(?<IP>[\S]+)\t(?<ACTION>[\S]+)\t(?<CF_DIST>[\S]+)\t(?<REQUEST>[\S]+)\t(?<STATUS>[\S]+)\t(?<REFERRER>[\S]+)\t(?<BROWSER>[\S]+)\t(?<QUERY>[\S]+)

0 Karma

adamb0mb
Explorer

I don't necessarily want to put this in the props.conf or transforms.conf, I just want it the extraction to work 🙂

The regex your'e showing looks really wonky... no backslashes at all? If the backslashes don't need to be escaped, shouldn't it look like:
^(?[\S]+)\t(?

Edit: looks like the wonkiness might be splunkbase display issues.

0 Karma
Get Updates on the Splunk Community!

Celebrate CX Day with Splunk: Take our interactive quiz, join our LinkedIn Live ...

Today and every day, Splunk celebrates the importance of customer experience throughout our product, ...

How to Get Started with Splunk Data Management Pipeline Builders (Edge Processor & ...

If you want to gain full control over your growing data volumes, check out Splunk’s Data Management pipeline ...

Out of the Box to Up And Running - Streamlined Observability for Your Cloud ...

  Tech Talk Streamlined Observability for Your Cloud Environment Register    Out of the Box to Up And Running ...