Splunk Search

How to remove some words from multivalued field for display

manjushan
Explorer

I have a field with search terms in the log that I want to break into individual words. But I do not want stop words (like A, AN, THE) etc to show up in the result.

eg: my log file has

searchTerm=lost+cat
searchTerm=lost+a+cat

searchTerm=lost+the_dog

And I want to get a list of all the words ever used in the searchTerm field.
lost
cat
dog

But I do not want "a" , "an", "the" to appear in the list.

I use makemv delim=" "

searchTerm | eval n = urldecode(searchTerm) | makemv delim=" " setsv=true n| top limit=10000 n

But I get all the words including "a", "the" etc. How do I get to NOT display these stop words.

0 Karma
1 Solution

lguinn2
Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

View solution in original post

lguinn2
Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

View solution in original post

manjushan
Explorer

Thank You , It worked !

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!