searching events for "was down for" and averaging the time

[jyates76]

I have events like the below that are saying when a particular pool member was out of rotation for a particular period of time.  What would be an ideal search would be to match all events that have the "was down for" and then the length of time and simply average that, and take the 95th percentile of that duration.   Probably more difficult than it seems and I'm not sure how to approach it.

<133>Feb 13 13:01:33 slot2/US66666-CORE-LTM1.company.COM notice mcpd[8701]: 01070727:5: Pool /Common/pool-generic member /Common/servernamew006:8080 monitor status up. [ /Common/mon-xxx-prod-xxx-liveness: up ] [ was down for 0hr:0min:15sec ]

• host = US66666-core-ltm1.company.com
• source = /var/log/HOSTS/US66666-core-ltm1.company.com/xxx.xxx.com-syslog.log
• sourcetype = syslog_alb

[gcusello]

Hi @jyates76,

you have to extract the down duratio and then run a simple search:

index=your_index "was down for"
| rex "was\s+down\s+for\s+(?<hours>\d+)hr:(?<minutes>\d+)min:(?<seconds>\d+)sec"
| eval duration=hours*3600+minutes*60+seconds
| timechart perc90(duration) BY host

You can test the regex at https://regex101.com/r/75pRcf/1

then you can use other functions or aggregations.

Ciao.

Giuseppe