Splunk Search

How to remove some words from multivalued field for display

Explorer

I have a field with search terms in the log that I want to break into individual words. But I do not want stop words (like A, AN, THE) etc to show up in the result.

eg: my log file has

searchTerm=lost+cat
searchTerm=lost+a+cat

searchTerm=lost+the_dog

And I want to get a list of all the words ever used in the searchTerm field.
lost
cat
dog

But I do not want "a" , "an", "the" to appear in the list.

I use makemv delim=" "

searchTerm | eval n = urldecode(searchTerm) | makemv delim=" " setsv=true n| top limit=10000 n

But I get all the words including "a", "the" etc. How do I get to NOT display these stop words.

0 Karma
1 Solution

Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

View solution in original post

Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

View solution in original post

Explorer

Thank You , It worked !

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!