delta report for multiple hosts

[mcafeesecure]

I have the following log messages coming from syslog-ng

Jun 14 10:32:04 sc4-cron.mcafeesecure.com syslog-ng[2775]: Log statistics; dropped='tcp(AF_INET(10.98.202.126:514))=1613614', processed='center(queued)=175194', processed='center(received)=109152', processed='destination(d_resin)=51369', processed='destination(d_boot)=0', processed='destination(d_auth)=439', processed='destination(d_cron)=362', processed='destination(d_mlal)=0', processed='destination(loghost)=109152', processed='destination(d_mesg)=93', processed='destination(d_cons)=0', processed='destination(d_snmp)=6782', processed='destination(d_spol)=0', processed='destination(d_mail)=6997', processed='source(s_sys)=66165', processed='source(s_tail)=42187', processed='source(s_raccess)=800', processed='source(s_tail_2)=0'

Jun 14 10:22:04 sc4-cron.mcafeesecure.com syslog-ng[2775]: Log statistics; dropped='tcp(AF_INET(10.98.202.126:514))=1613614', processed='center(queued)=173017', processed='center(received)=107903', processed='destination(d_resin)=50855', processed='destination(d_boot)=0', processed='destination(d_auth)=430', processed='destination(d_cron)=359', processed='destination(d_mlal)=0', processed='destination(loghost)=107903', processed='destination(d_mesg)=92', processed='destination(d_cons)=0', processed='destination(d_snmp)=6610', processed='destination(d_spol)=0', processed='destination(d_mail)=6768', processed='source(s_sys)=65231', processed='source(s_tail)=41872', processed='source(s_raccess)=800', processed='source(s_tail_2)=0'

Jun 14 10:12:04 sc4-cron.mcafeesecure.com syslog-ng[2775]: Log statistics; dropped='tcp(AF_INET(10.98.202.126:514))=1613614', processed='center(queued)=171101', processed='center(received)=106828', processed='destination(d_resin)=50405', processed='destination(d_boot)=0', processed='destination(d_auth)=416', processed='destination(d_cron)=338', processed='destination(d_mlal)=0', processed='destination(loghost)=106828', processed='destination(d_mesg)=87', processed='destination(d_cons)=0', processed='destination(d_snmp)=6428', processed='destination(d_spol)=0', processed='destination(d_mail)=6599', processed='source(s_sys)=64382', processed='source(s_tail)=41646', processed='source(s_raccess)=800', processed='source(s_tail_2)=0'

I am using the following query to generate a diff of the field in this message I want (in this case the dropped field)

syslog-ng "Log statistics" host="sc4-cron*" | rex "(?i)^(?:[^=]=){2}(?P[^'])(?=')" | search DROPPED="" | delta DROPPED as DROPPEDDIFF | search DROPPEDDIFF=""

this works fine on a single host, however, I would like to run this across all of my hosts. The problem I then encounter, is that the previous log message is not from the same host as the latest message, therefore the diffs that are created are not accurate.

I have hundreds of hosts, and would like to build a report on this data, however, I can't really go and do them all individually. In addition getting all of this into one report would be ideal.

[gkanapathy · Accepted solution]

You can use the streamstats command, something like:

... | streamstats current=f global=f window=1 first(DROPPED) as NEXT_DROPPED by host | eval DROPPEDDIFF = NEXT_DROPPED-DROPPED

Something like that. You may have to reverse the subtraction, I'm not sure which way around it goes, and well, it might be off-by-one, i.e. the diff might be on the previous vs the current result, but you get the idea.

[mcafeesecure]

Thanks.
For the record, the final query looks like this:

syslog-ng "Log statistics" host!="qa2-cron*" | rex "(?i)^(?:[^=]=){2}(?P[^'])(?=')" | search DROPPED="*" | streamstats current=f global=f window=1 first(DROPPED) as NEXT_DROPPED by host | eval DROPPEDDIFF = NEXT_DROPPED-DROPPED | search DROPPEDDIFF>0

[morian]

Thank you, you helped me solve my problem.