Ping Monitor Results

[davidirvine]

Hi,

I am using the Network Toolkit to ping site routers on our MPLS network.
We currently have 3 Runs (pings) every 1m to the routers IP address.
We then have a real-time alert as follows:
sourcetype="ping_input" packet_loss=100
| stats max(dest) as Host max(packet_loss) as "Packet Loss"

We are getting lots of results back for 100% packet loss, however if we run a continual ping from the Splunk Server (centos) we dont get any packet loss?
Looking for some assistance with how this should be setup?

thanks
David

[nickhills · Accepted solution]

The odd packet can always get a bit lost along the way, so you may want to use stats avg(packet_loss) and set a threshold |where "Packet Loss">75 to avoid some of the false positives.

You also probably want to avoid real time scheduled searches (see: https://answers.splunk.com/answers/734767/why-are-realtime-searches-disliked-in-the-splunk-w.html )

instead try using a historic search earliest=-2m@m latest=-1m@m

sourcetype="ping_input" earliest=-2m@m latest=-1m@m
| stats max(dest) as Host avg(packet_loss) as "Packet Loss"
|where "Packet Loss">75

[davidirvine]

Thanks Nick.
I have updated the Search and will monitor it.
Appreciate the link too.

[davidirvine]

Hi Nick,
Apologies for the delayed response.
That search brings up the desired results however the Alert I configured from it doesn't seem to trigger?
Have set the Alert with the same time frame as the search.
Any ideas?

Alert Type = Scheduled
Run Cron = earliest 2m latest 1m
Number of results > 0

thanks David

[nickhills]

Are you using -2m or just 2m?
You need the -

[davidirvine]

yes I have -2m and -1m

[nickhills]

If you run a historical search in Splunk over the last hour or so - do you still see results for packet loss?
If so, can you post some of the events?

[davidirvine]

Hi Nick,

Thanks for responding.
I searched for the last hour and got one result which was True as the host was restarted.
I expanded the search for Today and got some results which was a False Positive, but no others?
Looking through the amount of Alerts I received today I had 19.
Here are some events:
25/03/2019
13:30:04.000

sent=1 received=0 packet_loss=100 min_ping=NA avg_ping=NA max_ping=NA jitter=NA return_code=1 dest=router01.megt.com.au
host = splunk source = ping sourcetype = ping_input
25/03/2019
13:01:03.000

sent=1 received=0 packet_loss=100 min_ping=NA avg_ping=NA max_ping=NA jitter=NA return_code=1 dest=router02.megt.com.au
host = splunk source = ping sourcetype = ping_input

Thanks