I've constructed a lookup table containing some key data sources that I expect to see events from on a daily basis.
The lookup table, expected_datasources.csv looks like this:
sourcetype,source
st1,source1
st2,source2a;source2b;source2c
st3,/path/to/source3*
The 'globbed' source is there because the source name will change every day as the filename contains date/time information.
The search I use to confirm that I've received events within the last 24 hours:
(index=firstIndex) OR (index=secondIndex) earliest=-24h
[|inputlookup expected_datasources.csv
| makemv delim=";" source | fields source,sourcetype ]
| append [ inputlookup expected_datasources.csv
| makemv delim=";" source | fields source,sourcetype ]
| stats count by sourcetype source
| eval count=count-1 | eval count=count-1 | where count<1 AND (!source LIKE "%*")
The problem with this search, is that it works fine for all of the lookup rows EXCEPT the one where source=/path/to/source3*
If the search returns events for a source named: /path/to/source3_Mar02-2016-03:00.csv then everything works correctly. The obvious flaw here is that if nothing gets returned matching the pattern, then I've already provided the condition to ignore that data source with my (!source LIKE "%*") ...
I'm kind of stuck on the logic here of actually how to complete this thing.
As always, thanks for any suggestions.
