How to use a daily updated lookup table to search for new sourcetypes added in the past 24 hours?

[krishanpatel]

I want to create a search that displays any newly added sourcetypes in the past 24 hrs. I've created a report that outputs a lookup table and is scheduled to run/update once a day. My report search string is below:

... | inputlookup lookup.csv append=t | stats count by sourcetype | table sourcetype | outputlookup lookup.csv

I am struggling to figure out a way to search the lookup table for newly added sourcetypes. Does anyone know how I could do that? Or is there a better method in trying to accomplish this?

[lguinn2 · Accepted solution]

I hesitate to say "you are doing it wrong" but stats is really an inefficient way to get a list of sourcetypes. The metadata command is extremely fast and gives you everything you need:

Your daily scheduled search could look like this:

| metadata type=sourcetypes | outputlookup lookup.csv

You can also figure out which sourcetypes are new by looking at the firstTime field:

| metadata type=sourcetypes | where firstTime > now()-86400

Not really sure why you even need the lookup table...

[krishanpatel]

Thank you. This is what I was looking for and is a much easier approach.

[MuS]

Hi krishanpatel,

try something like this:

sourcetype=* | search NOT [ | outputlookup lookup.csv ]

hope this helps to get you started...

cheers, MuS