Alert on new... anything

[djbyler]

I'm looking for a way to alert or report when new data shows up in Splunk. For example, when a new device starts sending data to Splunk, or when a new incoming IP address shows up in my firewall logs.

I suppose I'd need to search for what did exist, what exists now, and then compare the two lists... but I'm not quite sure how to get that done. (And... how to optimize it-- probably using summary indexes??) I can build searches easily enough that show what did exist and what exists now for various things (hosts, IP addresses, etc.) but I'm not sure how to compare the two lists. Any guidance?

(I tried using a subsearch, but I got an error message about a 10,000 result limit on subsearches. When I tried to limit the subsearch to just unique IP address combinations using stats or uniq, the search crashed.)

[yannK]

update a lookup or a summary result with all your existing IPS, and then a new ip comes, run the search agains the summary data or the lookup.
If the IP is not found, it's a new one.

example : http://splunk-base.splunk.com/answers/23628/how-to-compare-a-list-of-hosts-from-a-week-to-another

[Ayn]

Perhaps you could use the metadata command? It will - per host, source or sourcetype, list metadata about each item such as when it was first seen, when it was last seen, how many events have been seen from this item, among other things. You could use this and compare when the item was first seen with the current time. For instance, a search could run once a day and compare the time with what time it was 24 hours ago, thus giving you items that are new since then.

http://docs.splunk.com/Documentation/Splunk/5.0/SearchReference/Metadata

[yannK]

Hi Djbyler

If Ayn answered your question, please let know your fellow community that this question is resolved.
It's easy, just checking the accept mark on the left side of his post.

[djbyler]

Thanks. Looks like metadata should easily tell me about new hosts reporting and I should have no trouble building an alert based on this.