Splunk Search

A NOT Filter against a REGEX

asarolkar
Builder

I have a search that filters out the value of account number from a log entry USING A REGEX extraction -->

sourcetype="SysLog" | rex field=_raw "To BOA-(?<accountno>\d{1,11})" | dedup accountno

This works as expected which is great. Anywhere where this REGEX is met, this works like a charm and churns out the account no





Now, I have been asked to ONLY APPLY this search to a set of 10 account numbers.

So I change the search to this:

sourcetype="SysLog" | rex field=_raw "To BOA-(?<accountno>\d{1,11})" | search 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 | dedup accountno

Which works as expected as well.





Assuming that CURRENTLY this search only gives me accountno = 4,5,6,7 [ We are assuming that ONLY for those 4 accounts, did the REGEX churn out an account no] ---->

----> How do I then get any account number THAT WAS NOT FILTERED by that regex ?

Meaning, i want to modify the SECOND search above so that it gives me failed accounts -> 1,2,3,8,9,10
How do I apply a filter using a regex - and then NOT on that operation ?

Tags (3)
0 Karma

kristian_kolb
Ultra Champion

If the list of 500 is static, I'd say a lookup table/subsearch combo would be an option. The search below finds http status codes that have NOT occurred in my access_combined logs. I think you can modify that to suit your needs.

|inputlookup http_status.csv | search NOT [search sourcetype=access_combined | fields status]

UPDATE:
If you can make do with the same approach that the Deployment Monitor uses to find 'missing' forwarders, i.e. what's missing today that was here yesterday, you can always look at:

http://blogs.splunk.com/2012/02/19/compare-two-time-ranges-in-one-report/

Hope this helps,

Kristian

0 Karma

kristian_kolb
Ultra Champion

see update above. /k

0 Karma

tfletcher_splun
Splunk Employee
Splunk Employee

Couple of things. You should run the search against the field you just extracted to make sure it is working properly so:

sourcetype="SysLog" | rex field=_raw "To BOA-(?<accountno>\d{1,11})" | search accountno=1 OR accountno=2 OR accountno=3 OR accountno=4 OR accountno=5 OR accountno=6 OR accountno=7 OR accountno=8 OR accountno=9 OR accountno=10 | dedup accountno

Try that first to see if the accounts you wanted show up. Next you mention failed accounts. The search should include all of those accounts, 1,2,3,8,9,10. Think of your search as filtering down events that don't match your criteria, the way it is structured you told it to KEEP those records, so if they are not there they are not in the data, or the field extraction for them failed and your dedup filtered them out. Try running this to see if they are in your data at all:

sourcetype="SysLog" "To BOA-XX"

Where XX is the accountno you want to see. My guess is that the field extraction failed, try loosening it up a bit to make it work for all accounts.

asarolkar
Builder

I think you may have misunderstood my question (although I do see your point generally).

The search that you framed in the first part of your response works. It gives me all accounts that match the filter - I can VISUALLY substract these from the total 10 and voila ! the balance are accounts that FAILED.




However, my problem is that the total number of accounts is actually 500.

I dont want to do a visual inspection (there's 250 successful accounts in the ACTUAL search) but rather find a way to NOT that regex.

Make sense ?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...