Splunk Search

Editing logs at index time

mawwx3
Explorer

I have splunk indexing a local file that is being continuously written to and I need the first word in each event to not be indexed so that bluecoat will extract the proper fields.

Here is an example log:

Something 2010-06-25 22:09:45 194 123.123.123.123 - - - OBSERVED "none" 0 TUNNELED unknown - ssl 122.122.122.122 443 - - 123.12.12.123 8676 833 - none - - medium *.fubar.com "Sweet Deals"

I need the "Something" to not be indexed or not show up because it is offsetting the field extractions for the bluecoat app.

Any comments are appreciated.

Thanks,

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

You can use the SEDCMD rules in props.conf to do this, something like SEDCMD-something = s/^\S+ //, although you might consider instead setting up a slightly different search-time field extraction for these events to offset/ignore the first field.

http://www.splunk.com/base/Documentation/4.1.3/Admin/Anonymizedatawithsed http://www.splunk.com/base/Documentation/4.1.3/Admin/Propsconf

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

You can use the SEDCMD rules in props.conf to do this, something like SEDCMD-something = s/^\S+ //, although you might consider instead setting up a slightly different search-time field extraction for these events to offset/ignore the first field.

http://www.splunk.com/base/Documentation/4.1.3/Admin/Anonymizedatawithsed http://www.splunk.com/base/Documentation/4.1.3/Admin/Propsconf

Lowell
Super Champion

You would have to find which field extractions were broken and then simply add something like "^\S+ " to the front of them. These will be in props.conf, transforms.conf and possibly in searches in the form of rex commands. (So how did you come to the conclusion that your problem was because of and extra field in the front of your data?)

0 Karma

mawwx3
Explorer

How would one go about setting up a search-time field extraction to offset or ignore the first field?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I would recommend the search-time extraction change over using SEDCMD in general.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...