Splunk Search

Search all fields except ____

smileyge
Path Finder

I have a log with say 50 fields.Is there a way to search all fields except one of them? Something like

... |search MyTerm donotMatch(ThisField=MyTerm)

Note, I am familiar with "NOT ThisField=MyTerm". That excludes a row that has MyTerm both in ThisField and SomeOtherField, but I'm interested in finding the ones in SomeOtherField. Looking for something more elegant than a really long search string of this form:

... | field_A="MyTerm" OR field_B="MyTerm" OR field_C="MyTerm"

Tags (2)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

This worked for a test to detect when the string "404" exists in a web log, but not when it only exists in the status field.

sourcetype=access_combined 404 | rex "(?<myterm>\b404\b)" max_match=0 | eval mycount=mvcount(myterm) | where mycount==1 AND status!=404 OR mycount>1

Adjust as needed.

/k

View solution in original post

kristian_kolb
Ultra Champion

This worked for a test to detect when the string "404" exists in a web log, but not when it only exists in the status field.

sourcetype=access_combined 404 | rex "(?<myterm>\b404\b)" max_match=0 | eval mycount=mvcount(myterm) | where mycount==1 AND status!=404 OR mycount>1

Adjust as needed.

/k

smileyge
Path Finder

This is very close, I think this will meet the need.

0 Karma

lukejadamec
Super Champion

I'm confused. Earlier you said you DID want other fields in the event that contained ThisField, which is what the fields - command did. But now your saying you don't want other fields in the same event that contain ThisField.

0 Karma

somesoni2
Revered Legend

If you can put all the field names you want to compare (excluding ThisField), then your can do something like this to just match all other fields (and don't care about content of ThisField)

   your base search [|inputlookup FieldNamesToMatch.csv | eval query="searchTerm=".fieldName| table query | format ]

where FieldNamesToMatch.csv is lookup file containing 49 entries with field as 'fieldName'. your search will expand like

your base search ((SearchTerm=fieldName1) OR SearchTerm=fieldName2.....)
0 Karma

smileyge
Path Finder

I like this solution except for having to make a CSV file and would have accepted it had it not been for the one I did accept.

0 Karma

linu1988
Champion

okay then if your fields are extracted then can you do the below?

search |eval ThisField=if(ThisField="MyTerm","dummy",ThisField)|table ...|where condition..

0 Karma

smileyge
Path Finder

I tried using the fields - command. The trouble is in the order. If you search MyTerm on all fields, and then eliminate that field, it still returns events that had MyTerm in ThisField, but ThisField is no longer in the set. If I flip the order, and pipe it back to search, it seems to still search the raw events or possibly the _raw field (which I tried also minusing out) and teh search still wants to match ThisField somehow. I even tried an experiment to do Table on just one field I was interested in, then piped back out to search and it searched _raw still apparently.

0 Karma

lukejadamec
Super Champion

If you're trying to get rid of ThisField regardless of the value (what you said to start your question), then you can use the fields - ThisField command.

|search MyTerm |fields - ThisField| ...

0 Karma

smileyge
Path Finder

I believe where has the same issue as the plain search. Can you give an example of how you would do this with where?

0 Karma

smileyge
Path Finder

addressed in the question. It will exclude all events that have MyTerm in that field, regardless of whether the term is in another field or not. I'm looking for MyTerm in any of the other fields and it could be contained in both.

0 Karma

linu1988
Champion

will where work? how a filter for some column is filter other column data?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

What's wrong with this?

MyTerm NOT ThisField=MyTerm
0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...