Reporting

Need "events list" & "count by" in email AND when run on UI

Communicator

I have the following command that does exactly what I need it to do....search for "eth_port_channel" and build a Regex that defines "port", and when "port is found count the number of times and if more than 100 times notify me.

Search: sourcetype=syslog_info ETH_PORT_CHANNEL AND down OR up | rex "port-channel(?\d+):" | stats count by port | sort - count | where count>=100

What I also need when receiving the email is the actual (events list) from the syslog search, not the actual count. Is there a way to ask for the "events list" and not get the "results table" in the output? Currently I am only getting the "Results Table" in the email notification.

pstein

1 Solution

SplunkTrust
SplunkTrust

What you might prefer is to break this up differently.

If you have the base search be everything before the stats clause:

sourcetype=syslog_info ETH_PORT_CHANNEL AND down OR up | rex "port-channel(?\d+):"

and in the alerting UI, if you set 'if custom condition is met', that allows you to basically run a postProcess search against the results and only trigger if that second search returns results...

So to follow your example, the "custom condition search" would be:

| stats count by port | sort - count | where count>=100

IF the number of events in the search is up in the tens-of-thousands, you might hit postProcess restrictions on accuracy, (where the second search only knows about the first N events and therefore it can get it wrong) Can anyone confirm or refute this potential problem?

In any event you might consider this split instead and it would make the base search a lot smaller, getting around the potential problem and making the email potentially more useful.

base search:

`sourcetype=syslog_info ETH_PORT_CHANNEL AND down OR up | rex "port-channel(?\d+):" | stats count by port | sort - count

custom condition search:

| where count>=100

View solution in original post

Splunk Employee
Splunk Employee

how about this:

sourcetype=syslog_info ETH_PORT_CHANNEL AND down OR up | rex "port-channel(?\d+):" | eventstats count by port | sort - count | where count>=100

Hope this helps,

.gz

Splunk Employee
Splunk Employee

modified for a much simpler query

0 Karma

SplunkTrust
SplunkTrust

What you might prefer is to break this up differently.

If you have the base search be everything before the stats clause:

sourcetype=syslog_info ETH_PORT_CHANNEL AND down OR up | rex "port-channel(?\d+):"

and in the alerting UI, if you set 'if custom condition is met', that allows you to basically run a postProcess search against the results and only trigger if that second search returns results...

So to follow your example, the "custom condition search" would be:

| stats count by port | sort - count | where count>=100

IF the number of events in the search is up in the tens-of-thousands, you might hit postProcess restrictions on accuracy, (where the second search only knows about the first N events and therefore it can get it wrong) Can anyone confirm or refute this potential problem?

In any event you might consider this split instead and it would make the base search a lot smaller, getting around the potential problem and making the email potentially more useful.

base search:

`sourcetype=syslog_info ETH_PORT_CHANNEL AND down OR up | rex "port-channel(?\d+):" | stats count by port | sort - count

custom condition search:

| where count>=100

View solution in original post

Communicator

I found this same resolution (custom conditional is met) while resolving a different issue. Once it was tested it worked perfectly.
I found a new favorite tool.
Thanks for the heads up.

MasterOogway

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!