topic Re: Extracted fields issues in Splunk Search

Extracted fields issues

wleroy — Tue, 10 Aug 2010 20:02:37 GMT

I'm experiencing weird issues with extracted fields : I have a custom field that basically get the hostname (in bold text), which is the 4th item of each log line :

Aug 10 09:42:54 172.31.55.1 **sables-garnier** monitord: RPC call failed: INTERFACE_get_link_state, aborting current process pid 164 : monitord
LABEL=monitord:

Aug  9 19:35:19 172.31.14.1 **talmont-port** monitord: RPC call failed: INTERFACE_get_link_state, aborting current process pid 158 : monitord 
LABEL=talmont-port  

Aug  9 16:25:04 172.31.38.1 **sables-olona** monitord: RPC call failed: INTERFACE_get_link_state, aborting current process pid 158 : monitord 
LABEL=sables-olona

I'm using this regexp : (?i)^(?:[^ ]* ){5}(?P[^ ]+)

Now why in the above extract Splunk shows the fifth item (process name) as a label ? Using Splunk 4.1.4 (82143) by the way

Any help appreciated

Thanks

Re: Extracted fields issues

simuvid — Tue, 10 Aug 2010 21:02:32 GMT

It's because of your date format. If you have a look at your date format the month and the day are seperated in the string by an whitespace.

Splunk counts different values in the string as differnt items. So in your case: 1st: Aug 2nd: 10 3rd: 09:42:54 4th: 172.31.55.1 5th: sables-garnier

By the way is the date normalistaion working for you in this case?

Cheers,

Christian

Re: Extracted fields issues

wleroy — Tue, 10 Aug 2010 22:20:57 GMT

Thanks for you reply

Actually I messed up explaining the issue. I agree I'm looking for the 5th item - my problem is that Splunk sometime picks the 6th item instead of the 5th, for no apparent reason (Note the second line I pasted, saying LABEL=monitord: which is the process name, not the hostname) the others lines in my example are fine.

Re: Extracted fields issues

simuvid — Wed, 11 Aug 2010 14:46:03 GMT

Hi, sorry my fault misunderstood you here!

Why dont you try another regex to match your host pattern.
Like: *[a-z]{0,}-[a-z]{0,}*

Once gain I think it is also caused by your date format.

Give it a try.

Hope that helps!

Cheers,

Christian

Re: Extracted fields issues

simuvid — Wed, 11 Aug 2010 14:46:46 GMT

\*[a-z]{0,}-[a-z]{0,}\*
Sorry HTML changed the slashes.

Re: Extracted fields issues

wleroy — Wed, 11 Aug 2010 18:27:06 GMT

Ok no prob - I'd like to give your regexp a try, but the asterisks were put there by me to highlight the hostname. I'm absolutely no expert regarding regexp and unfortunately some of the hosts don't have a dash in their hostname so I'm kinda stuck here - it seems that the only way to extract the hostname would be to filter put the 5th item of the line, no matter how many digits the day number has just like "awk '{ print $5 }'" would do in a shell

Thanks anyway for your time, I'm going to see how I can modify the date format and/or my regexp

Re: Extracted fields issues

wleroy — Wed, 11 Aug 2010 19:01:57 GMT

If I'm not mistaken the regexp Splunk has generated based on my input means "extract the word located after the 4th space of the whole string", while I need "extract the 5th word of the whole string", or "extract the word located after the 4th group of 1 or more consecutive spaces" - But after many tries using online regexp tools I still can't translate that in regexp syntax banging his head on the keyboard

Re: Extracted fields issues

Stephen_Sorkin — Wed, 25 Aug 2010 00:01:18 GMT

Your regex is designed to match the 6th non-whitespace item in the string, but the first five items are allowed to be degenerate (because of the * after the [^ ]). Occasionally it will find what appears to be the fifth word, because it will accept as the second item the empty string between the two spaces between the month and the day.

You want a regex like:

^(?:\S+\s+){4}(?<process>\S+)