How to remove some words from multivalued field for display

[manjushan]

I have a field with search terms in the log that I want to break into individual words. But I do not want stop words (like A, AN, THE) etc to show up in the result.

eg: my log file has

searchTerm=lost+cat
searchTerm=lost+a+cat

searchTerm=lost+the_dog

And I want to get a list of all the words ever used in the searchTerm field.
lost
cat
dog

But I do not want "a" , "an", "the" to appear in the list.

I use makemv delim=" "

searchTerm | eval n = urldecode(searchTerm) | makemv delim=" " setsv=true n| top limit=10000 n

But I get all the words including "a", "the" etc. How do I get to NOT display these stop words.

[lguinn2 · Accepted solution]

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

[manjushan]

Thank You , It worked !