Splunk Search

How to remove some words from multivalued field for display

manjushan
Explorer

I have a field with search terms in the log that I want to break into individual words. But I do not want stop words (like A, AN, THE) etc to show up in the result.

eg: my log file has

searchTerm=lost+cat
searchTerm=lost+a+cat

searchTerm=lost+the_dog

And I want to get a list of all the words ever used in the searchTerm field.
lost
cat
dog

But I do not want "a" , "an", "the" to appear in the list.

I use makemv delim=" "

searchTerm | eval n = urldecode(searchTerm) | makemv delim=" " setsv=true n| top limit=10000 n

But I get all the words including "a", "the" etc. How do I get to NOT display these stop words.

0 Karma
1 Solution

lguinn2
Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

View solution in original post

lguinn2
Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

manjushan
Explorer

Thank You , It worked !

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...