Splunk Search

Editing logs at index time

mawwx3
Explorer

I have splunk indexing a local file that is being continuously written to and I need the first word in each event to not be indexed so that bluecoat will extract the proper fields.

Here is an example log:

Something 2010-06-25 22:09:45 194 123.123.123.123 - - - OBSERVED "none" 0 TUNNELED unknown - ssl 122.122.122.122 443 - - 123.12.12.123 8676 833 - none - - medium *.fubar.com "Sweet Deals"

I need the "Something" to not be indexed or not show up because it is offsetting the field extractions for the bluecoat app.

Any comments are appreciated.

Thanks,

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

You can use the SEDCMD rules in props.conf to do this, something like SEDCMD-something = s/^\S+ //, although you might consider instead setting up a slightly different search-time field extraction for these events to offset/ignore the first field.

http://www.splunk.com/base/Documentation/4.1.3/Admin/Anonymizedatawithsed http://www.splunk.com/base/Documentation/4.1.3/Admin/Propsconf

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

You can use the SEDCMD rules in props.conf to do this, something like SEDCMD-something = s/^\S+ //, although you might consider instead setting up a slightly different search-time field extraction for these events to offset/ignore the first field.

http://www.splunk.com/base/Documentation/4.1.3/Admin/Anonymizedatawithsed http://www.splunk.com/base/Documentation/4.1.3/Admin/Propsconf

Lowell
Super Champion

You would have to find which field extractions were broken and then simply add something like "^\S+ " to the front of them. These will be in props.conf, transforms.conf and possibly in searches in the form of rex commands. (So how did you come to the conclusion that your problem was because of and extra field in the front of your data?)

0 Karma

mawwx3
Explorer

How would one go about setting up a search-time field extraction to offset or ignore the first field?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I would recommend the search-time extraction change over using SEDCMD in general.

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...