Splunk Search

Using a lookup in subsearch to derive new field

Path Finder
index=xyz [|inputlookup error_strings  | table string | rename string as search | format]

In the lookup I have a list of string "error", "nullvalue",... etc

In the index xyz I have multiline events and have multiple strings, but not in a proper format to extract fields.

My requirement is whenever there is a string that matches to the lookup file in the event, I want the distinct count of the string. (Disclaimer: I do not have those strings in field as it cannot be extracted as they are uneven and each event has 240 lines.. these are java exceptions and errors)

0 Karma
1 Solution

Esteemed Legend

Let us assume that your lookup file has more than 1 field and that one of the other unique fields is called error_code. You need to make your lookup a WILDCARD lookup on field string and add an asterisk ( * ) as both the first and last character of every string. Then do this:

index=xyz [|inputlookup error_strings | table string | rename string AS query] | lookup error_strings string AS _raw OUTPUT error_code

And then either this:

| stats count BY error_code

Or this:

| stats dc(error_code) values(error_code)

Then, if you like, you can invert the lookup call to convert the error_code back to string.

View solution in original post

Esteemed Legend

Let us assume that your lookup file has more than 1 field and that one of the other unique fields is called error_code. You need to make your lookup a WILDCARD lookup on field string and add an asterisk ( * ) as both the first and last character of every string. Then do this:

index=xyz [|inputlookup error_strings | table string | rename string AS query] | lookup error_strings string AS _raw OUTPUT error_code

And then either this:

| stats count BY error_code

Or this:

| stats dc(error_code) values(error_code)

Then, if you like, you can invert the lookup call to convert the error_code back to string.

View solution in original post

Path Finder

Apologies for the delay. Awesome solution. Worked like a gem, THANKS

0 Karma

Path Finder

i created another column in lookup as error_code and what value do you want me to insert here, is it the string value and convert the string value to asterik error asterik ?

Is this correct

0 Karma

Esteemed Legend

You can put any unique string there. The simplest thing is to copy the string value, (without the asterisks).

0 Karma

Esteemed Legend

Did you add the WILDCARD setting? This can only be done in CLI (not the GUI)?

0 Karma

Influencer

Great solution

0 Karma

Esteemed Legend

Be sure to upvote!

0 Karma

Influencer

You have plenty of karma as it is 🙂

0 Karma

Esteemed Legend

But I am almost passing @duckfez (maybe today)!

Influencer

lol so close!

Just keep me ahead of @Jeremiah

0 Karma

Legend

Try this

| inputlookup error_strings | table string  | map maxsearches=20 search="search index=xyz  $string$ | stats count"
0 Karma

Path Finder

this gives some count only error !!

0 Karma

Influencer

Is there any whitespace in your list of strings ? Are your strings single words? eg just "error" or "nullvalue". Or are there strings like "threw exception" ?

0 Karma

Path Finder

Yes it has white spaces

0 Karma

Influencer

How many strings are we talking about? If its not too many you could basically do a separate search for each string.

0 Karma

Path Finder

for now there are 10 strings but client might want to modify the lookup as and when required, so that s going to be hard

0 Karma

Revered Legend

Can a value from lookup appear in the event multiple times? If yes, should they be counted as one ?

0 Karma

Influencer

I don't think Splunk is really the tool for this - you might be better off with some python or R package against the raw data if you want to do this in a scalable way

0 Karma

Path Finder

Yes it has spaces Below are the strings:

java.lang.OutOfMemoryError
Unable to send alert because queue is full
ContainerConfigPollingService Exception
java.lang.NullPointerException
com.digev.fw.exception.GException: Internal system error.
com.digev.fw.exception.GException: Unexpected error
com.digev.fw.exception.GException: Error occured while validating token null
com.digev.fw.exception.GException: Unrecognized echange pattern

The query that i mentioned on top gives me the events with these string however i am not able to get a count of those distinct strings, Thanks

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!