How do you create a dashboard by event correlation?

[saadi381]

Hi

Apologies if this has been asked before.

So here is what i am trying to achieve

1. Catch the log and create an event like when bgp goes down (easy can search and filter them out)
2. Catch the log and create if above event is clear , will get a log message with status Up
3. Correlate both events
4. Display in a dashboard when event happened and keep it there unless the clear event is not received

So basically a dashboard where i can see when bgp goes down and if it is down , if restored the event to disappear from dashboard.

Not sure if possible in Splunk and how to do it

Here are sample logs which i need to correlate

Nov 11 15:39:05 hostanme1 -  RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 192.168.200.146 (External AS 12345) changed state from Established to Idle (event HoldTime) (instance vrf-1234)

After above log i need to display an event in dashboard

Nov 11 15:43:00 hostanme1 -  RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 192.168.200.146 (External AS 12345) changed state from OpenConfirm to Established (event RecvKeepAlive) (instance vrf-1234)

After above log i want to clear the event

[renjith_nair · Accepted solution]

@saadi381,

Assuming IP can be used to correlate the events, give this a try

"your base search"
|rex field=_raw "BGP peer (?<IP>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})"
|rex field=_raw "from \w+\sto\s(?<status>\w+)"
|stats count,latest(_raw) as _raw,latest(status) as Status by IP
|where count<2 AND Status!="Established"

This should leave with only idle IPs. Test it by removing the last where clause.

[saadi381]

i think its working , thanks a heap for this , i will test it for next 24hours to verify

[renjith_nair]

@saadi381,

Its possible. Do you have a unique field in both events to correlate them (router id or something)? Would be helpful if you could provide some sample events (mask any confidential data). Also how do you want to represent in the dashboard? Just as events or some status indicator?

[saadi381]

added sample logs