Splunk Search

A NOT Filter against a REGEX

asarolkar
Builder

I have a search that filters out the value of account number from a log entry USING A REGEX extraction -->

sourcetype="SysLog" | rex field=_raw "To BOA-(?<accountno>\d{1,11})" | dedup accountno

This works as expected which is great. Anywhere where this REGEX is met, this works like a charm and churns out the account no





Now, I have been asked to ONLY APPLY this search to a set of 10 account numbers.

So I change the search to this:

sourcetype="SysLog" | rex field=_raw "To BOA-(?<accountno>\d{1,11})" | search 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 | dedup accountno

Which works as expected as well.





Assuming that CURRENTLY this search only gives me accountno = 4,5,6,7 [ We are assuming that ONLY for those 4 accounts, did the REGEX churn out an account no] ---->

----> How do I then get any account number THAT WAS NOT FILTERED by that regex ?

Meaning, i want to modify the SECOND search above so that it gives me failed accounts -> 1,2,3,8,9,10
How do I apply a filter using a regex - and then NOT on that operation ?

Tags (3)
0 Karma

kristian_kolb
Ultra Champion

If the list of 500 is static, I'd say a lookup table/subsearch combo would be an option. The search below finds http status codes that have NOT occurred in my access_combined logs. I think you can modify that to suit your needs.

|inputlookup http_status.csv | search NOT [search sourcetype=access_combined | fields status]

UPDATE:
If you can make do with the same approach that the Deployment Monitor uses to find 'missing' forwarders, i.e. what's missing today that was here yesterday, you can always look at:

http://blogs.splunk.com/2012/02/19/compare-two-time-ranges-in-one-report/

Hope this helps,

Kristian

0 Karma

kristian_kolb
Ultra Champion

see update above. /k

0 Karma

tfletcher_splun
Splunk Employee
Splunk Employee

Couple of things. You should run the search against the field you just extracted to make sure it is working properly so:

sourcetype="SysLog" | rex field=_raw "To BOA-(?<accountno>\d{1,11})" | search accountno=1 OR accountno=2 OR accountno=3 OR accountno=4 OR accountno=5 OR accountno=6 OR accountno=7 OR accountno=8 OR accountno=9 OR accountno=10 | dedup accountno

Try that first to see if the accounts you wanted show up. Next you mention failed accounts. The search should include all of those accounts, 1,2,3,8,9,10. Think of your search as filtering down events that don't match your criteria, the way it is structured you told it to KEEP those records, so if they are not there they are not in the data, or the field extraction for them failed and your dedup filtered them out. Try running this to see if they are in your data at all:

sourcetype="SysLog" "To BOA-XX"

Where XX is the accountno you want to see. My guess is that the field extraction failed, try loosening it up a bit to make it work for all accounts.

asarolkar
Builder

I think you may have misunderstood my question (although I do see your point generally).

The search that you framed in the first part of your response works. It gives me all accounts that match the filter - I can VISUALLY substract these from the total 10 and voila ! the balance are accounts that FAILED.




However, my problem is that the total number of accounts is actually 500.

I dont want to do a visual inspection (there's 250 successful accounts in the ACTUAL search) but rather find a way to NOT that regex.

Make sense ?

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...