Splunk Search

How to verify data for a specific sourcetype was indexed yesterday for a list of hosts from a master list?

napomokoetle
Communicator

Hi Everyone,

Every night just after midnight, I need to verify that data for a specific sourcetype has been indexed during the course of yesterday for a list of hosts that is provided as a master list. This is so that admins can take remedial measures for those hosts that for some reason stopped sending data from the sourcetype before too much data is lost.

The master list consists of two hundred hosts.

Please help with how to structure the search for this type of problem. Thanks in advance for your kind help.

0 Karma
1 Solution

somesoni2
Revered Legend

Here is what you can do

1) Save your master host list as lookup table in Splunk, say host_master.csv containing field host. See this http://docs.splunk.com/Documentation/Splunk/6.2.0/Knowledge/Addfieldsfromexternaldatasources

2) Create an alert search with following specifications http://www.splunk.com/view/SP-CAAAGYG
Alert search:

| inputlookup host_master.csv | table host | eval From="lookup" | append [ | tstats count WHERE index=YourIndex(es) AND sourcetype=YourSourceType by host | table host | eval From="Yesterday" ] | stats values(From) as From by host | where mvcount(From)=1 AND From="lookup"

Time Range: -1d@d to @d (Start time/ Finish time)
ALert condition: When number of rows greater than 0

View solution in original post

0 Karma

somesoni2
Revered Legend

Here is what you can do

1) Save your master host list as lookup table in Splunk, say host_master.csv containing field host. See this http://docs.splunk.com/Documentation/Splunk/6.2.0/Knowledge/Addfieldsfromexternaldatasources

2) Create an alert search with following specifications http://www.splunk.com/view/SP-CAAAGYG
Alert search:

| inputlookup host_master.csv | table host | eval From="lookup" | append [ | tstats count WHERE index=YourIndex(es) AND sourcetype=YourSourceType by host | table host | eval From="Yesterday" ] | stats values(From) as From by host | where mvcount(From)=1 AND From="lookup"

Time Range: -1d@d to @d (Start time/ Finish time)
ALert condition: When number of rows greater than 0

View solution in original post

0 Karma

napomokoetle
Communicator

Hi somesoni2,

Thanks for your swift response. when I execute the first part of the search

| lookup host_master.csv | table host

I get a message as follows...

Error in 'lookup' command: Must specify one or more lookup fields.

0 Karma

somesoni2
Revered Legend

Ohh wrong command.. Try the updated answer...

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.