Getting Data In

How to create an alert when a new forwarder is added to deployment server?

merrelr
Path Finder

I'm looking for a way to report/alert anytime a new forwarder is added to my deployment server. I've tried searching on internal with the following search, but this isn't unique to when a forwarder is initially added to our environment.

index=_internal sourcetype=splunkd component=PubSubSvr host=<deploymentServer>

jacobpevans
Motivator

Greetings @merrelr,

The current provided answers would work. This is how I would do it though:

| metadata type=hosts
| eval daysSinceFirstTime = round((now() - firstTime)/86400, 2)
| eval hoursSinceLastTime = round((now() - lastTime )/3600 , 2)
| convert ctime(firstTime) as firstTime
| convert ctime(lastTime)  as lastTime
| search daysSinceFirstTime < 1 OR (hoursSinceLastTime>24 AND hoursSinceLastTime<48)

In this example, you could run the alert search once a day to get all hosts that sent their first event to your indexer(s) in the past day and hosts that sent their last event to your indexer(s) between 24 and 48 hours ago. Run the search for all time as this is an extremely efficient command.

Refer here for the metadata command documentation.

Cheers,
Jacob

If you feel this response answered your question, please do not forget to mark it as such. If it did not, but you do have the answer, feel free to answer your own post and accept that as the answer.
0 Karma

buckiboy
New Member
| tstats count where index = _* by host | search NOT [| inputlookup forwarders.csv | fields+ host]
      | stats values(host) AS new_hosts

There should be a pipe before inputlookup.

0 Karma

adonio
SplunkTrust
SplunkTrust

Hello merrelr,
you can create a lookup table of all forwarders:
| tstats count where index = _* by host | table host | outputlookup forwarders.csv
and let it run as a schedule search. now you can search and compare results to lookup to check who is new on your forwarders list:

   | tstats count where index = _* by host | search NOT [inputlookup forwarders.csv | fields+ host]
     | stats values(host) AS new_hosts

you can also use the | metadata command and search for newly received data leveraging the firstTime field
more in this answer:
https://answers.splunk.com/answers/321024/compare-search-to-lookup-table-and-return-results.html

0 Karma

jlh3dz
New Member

I found the answer. "search" is missing. Use
| tstats count where index = _* by host | search NOT [inputlookup forwarders.csv | fields+ host]
| stats values(host) AS new_hosts

0 Karma

bbraun
New Member

This worked! Thank you!

0 Karma

adonio
SplunkTrust
SplunkTrust

@bbraun
modified the answer to reflect, indeed forgot to put the search after pie. this is the reason for your error, there is no NOT command

0 Karma

bbraun
New Member

This is very helpful!

Although when i enter the query to compare the results, I get the below error.

"Search Factory: Unknown search command 'not'."

I tried playing around with the query but did not have any luck.

0 Karma

jlh3dz
New Member

I get the same error, "Search Factory: Unknown search command 'not'."

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!