Splunk Search

Is Pre-processing logs necessary?

monicato
Path Finder

Hello,

I'm having trouble getting Splunk to identify one of the fields in my logs because this field contains a single binary number (so either 0 or 1). Splunk only identifies other zeros and ones in my other fields... My question is, does this mean I have to pre process my logs before inputting them into splunk? Anyone have the same problem? know a fix? MUCH APPRECIATED!!

alt text

Splunk is identifying the zeros in the timestamp, which I do not want. I want the column circled in green. Open image in new tab to see enlarged image.

Here's a raw log for example:

***note*: I'm having problems with the last column: it's always either 1 or 0

2011-11-25    09:31:20      198.190.172.123      HEAD      /e.download.com/49/63/4863/64bit.part2.rar       200    266    0      "-"    "tSession Interface"     "DState=""3+0+01KI+0.00+Start+0+0+14645+TransId=2ea4bd87_Guid=8e73b4564dfaace20490_IP=198.190.172.123"""       1468045    -      0
0 Karma
1 Solution

lguinn2
Legend

No, preprocessing is not necessary! But the Interactive Field Extractor is ultimately a mechanical regular expression generator - it will never be as intelligent as a human who understands the data.

One solution would be to create the field extraction manually in props.conf. If you are the Splunk admin, that might be your best choice.

But you can also put the regular expression directly into the Interactive Field Extractor. Click the Edit button and Splunk will give you a window to edit the regular expression. The following regular expression picks up the last digit on the line:

(?P<FIELDNAME>\d)\s*$

I looked at your screenshot, and I think this is what you want. Let us know if it doesn't work...

View solution in original post

lguinn2
Legend

No, preprocessing is not necessary! But the Interactive Field Extractor is ultimately a mechanical regular expression generator - it will never be as intelligent as a human who understands the data.

One solution would be to create the field extraction manually in props.conf. If you are the Splunk admin, that might be your best choice.

But you can also put the regular expression directly into the Interactive Field Extractor. Click the Edit button and Splunk will give you a window to edit the regular expression. The following regular expression picks up the last digit on the line:

(?P<FIELDNAME>\d)\s*$

I looked at your screenshot, and I think this is what you want. Let us know if it doesn't work...

monicato
Path Finder

YES that does it! THANK YOU!!! 😄

0 Karma

monicato
Path Finder

I just updated post with a raw log

0 Karma

sdaniels
Splunk Employee
Splunk Employee

We need to some raw events to help you in this case and generate the right regex for you.

0 Karma

monicato
Path Finder

clicking on an "x" in yellow in the field extractor does not revise the regex correctly... : (

0 Karma

sdaniels
Splunk Employee
Splunk Employee

When you click on an 'X' in yellow in the field extractor does it revise the regex for you and find what you need? I would try that first. If that doesn't work, if you post a raw event example i'm sure someone can give you the regex that will work.

0 Karma

monicato
Path Finder

I just updated my question post with a screenshot

0 Karma

sideview
SplunkTrust
SplunkTrust

Can you post an example event from the raw file, so we can picture it better?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...