So I'm trying to create a metrics search using the following query:
index="test" identities="ident_*" src=10.11.40.0/22 OR src=10.11.48.0/22 OR src=10.11.56.0/22 OR src=10.11.64.0/22 OR src=10.11.72.0/22 OR src=10.120.40.0/22 OR src=10.120.48.0/22 OR src=10.120.56.0/22 OR src=10.120.64.0/22 OR src=10.15.8.0/22 OR src=10.15.40.0/22 OR src=10.15.48.0/22 OR src=10.15.56.0/22 OR src=10.15.72.0/22 OR src=10.15.76.0/22 OR src=10.15.80.0/22 | top src | outputlookup test-excludes-no-dedup.csv
I then take the CSV and use it here:
index="test" identities="ident_*" NOT [ inputlookup test-excludes-no-dedup.csv ] | top src
Is this the correct way to [exclude] the CIDR ranges contained within the lookup CSV? I get some results doing this but here it is, almost 1AM and I'm starting to question whether OR is correct. Maybe I should be using AND?
I want to find all the 'src' items that are not in those CIDR ranges in the CSV.. am I going about it correctly?
You could try using a where command post search
index="test" identities="ident_*" | where NOT [ inputlookup test-excludes-no-dedup.csv | rename src as query | fields query | format "(" "cidrmatch(" "" ",src)" "AND" ")"] | top src
Not sure if the AND should be OR though
Hmm, same result. Well now it's 10 src's with a 50 per page setting. Trying like earlier, adding top limit=100 src returns 100.
So for w/e reason it's still not wanting to dump all the results w/o limit.
top defaults to 10 results - try
| top 0 src
Can I ask you something else related? If I perform everything you outlined above exactly, I get a nice healthy Events list. If I add
| top src
I get a list of 9 src's, with their counts. I have the page row size at 50, but only get 9.
Now if I add:
| top limit=100 src
I get two pages worth of src's with counts. I was assuming that by not including 'limit' that the search would return all results, but that doesn't seem to be happening here.
Any thoughts on how I can correctly return every SRC that's not in that lookup without setting a limit=?
Oh wow, I was way off! This is awesome.. I'm new to the more advanced query so definitely appreciate you. I do get a good long list of src's now. When I try to add a | top src to this to get a nice simple metric list I'm back to less than 10. So I'm pretty sure my metrics attempt is breaking things since this short list looks super similar to the one I'd get last night.
Maybe I should just keep it as Events..
Firstly, if your 'src' field contains IP addresses, then you cannot use src=CIDR_RANGE in your search, unless the 'src' field itself is actually that string.
If your intention is to have a lookup that contains a list of CIDR ranges, then simply create the lookup, e.g. like this
| makeresults | eval src=split("10.11.40.0/22,10.11.48.0/22,10.11.56.0/22,10.11.64.0/22,10.11.72.0/22,10.120.40.0/22,10.120.48.0/22,10.120.56.0/22,10.120.64.0/22,10.15.8.0/22,10.15.40.0/22,10.15.48.0/22,10.15.56.0/22,10.15.72.0/22,10.15.76.0/22, 10.15.80.0/22", ",") | mvexpand src | table src | outputlookup test-excludes.csv
Then create a lookup definition and in the advanced settings, set the match type field to CIRT(src)
Then in your code you can do
search... | lookup test-excludes src OUTPUT src as found | where isnull(found)
which will then have 'src' IP addresses that are NOT in the ranges of the lookup