Getting Data In

How can I get the time difference between two events with particular ID?

dabroma5
Explorer

I was trying to filter event ID in subsearch and then use it in the main search to find other events with related ID and compare time from subsearch with last event time from the main search.
The initial line when ID appears is: 2020-04-29 16:14:08,637 backend_7.2.15: INFO services/ConnectionManagerService(backend): \ncreations: 1262172\nupdates: \ncancellations: 1261482-1

one of the problem is that above event ID's can appear after decimal, like below:
2020-04-29 16:14:08,791 backend_7.2.15: INFO services/ConnectionManagerService(backend): \ncreations: 1262174,1262175,1262176\nupdates: \ncancellations: 1261438-1,1261436-1,1261440-1

confirmation line - last:
10.21.160.144.SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).

My query:
....... sourcetype=main ConfigurationManagerService
|append [search ................sourcetype=main "ConnectionManagerService(backend)" "\ncreations:"
| multikv noheader=t
| rex "(?:ions: )(?\d{7})"
| where ID != 0
| rename _time as start_time
| table ID start_time]
| stats earliest(start_time), latest(_time) as stop by ID

How to make it more efficient or just working?

Part of the log:

2020-04-29 16:19:13,082 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262180\nupdates:       \ncancellations: 1258780-1
2020-04-29 16:14:10,479 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/rpfPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
2020-04-29 16:14:09,498 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1....70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
2020-04-29 16:14:09,442 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.....10002/igmpPortConfig! (Config success!). New contributors: Set(book.1262176-1), removed contributors: Set().
2020-04-29 16:14:09,438 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1......70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262175-1), removed contributors: Set().
2020-04-29 16:14:09,388 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).
2020-04-29 16:14:09,314 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,308 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1..........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,306 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1)
2020-04-29 16:14:09,305 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
2020-04-29 16:14:09,303 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......10002/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,302 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1), removed contributors: Set()
2020-04-29 16:14:09,300 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
2020-04-29 16:14:08,914 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1........SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1).
2020-04-29 16:14:08,837 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,836 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,791 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262174,1262175,1262176\nupdates:       \ncancellations: 1261438-1,1261436-1,1261440-1
2020-04-29 16:14:08,637 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262172\nupdates:       \ncancellations: 1261482-1
0 Karma
1 Solution

to4kawa
Ultra Champion
| makeresults 
| eval _raw="2020-04-29 16:19:13,082 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262180\nupdates:       \ncancellations: 1258780-1
 2020-04-29 16:14:10,479 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/rpfPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,498 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1....70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,442 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.....10002/igmpPortConfig! (Config success!). New contributors: Set(book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,438 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1......70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262175-1), removed contributors: Set().
 2020-04-29 16:14:09,388 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).
 2020-04-29 16:14:09,314 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
 2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,308 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1..........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,306 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1)
 2020-04-29 16:14:09,305 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
 2020-04-29 16:14:09,303 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......10002/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,302 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1), removed contributors: Set()
 2020-04-29 16:14:09,300 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:08,914 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1........SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1).
 2020-04-29 16:14:08,837 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,836 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,791 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262174,1262175,1262176\nupdates:       \ncancellations: 1261438-1,1261436-1,1261440-1
 2020-04-29 16:14:08,637 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262172\nupdates:       \ncancellations: 1261482-1" 
| rex mode=sed "s/(?m)^\s+//g" 
| multikv noheader=t
| stats count by _raw
| rex "(?<time>\S+ \S+)"
| eval _time=strptime(time,"%F %T,%3Q")
| sort - _time
| table _time _raw
| rename COMMENT as "this is sample, from here, the logic"
| rex mode=sed "s/\\\n/
/g"
| rex max_match=0 "(?ms)(?<id>\d{7})(?!-)"
| rex max_match=0 "(?ms)(?<sub_id>\d{7}-\d)"
| mvexpand sub_id
| eval id=coalesce(id,substr(sub_id,1,7))
| mvexpand id
| stats range(eval(if(searchmatch("creations:") OR searchmatch("Config success!"),_time,NULL))) as duration by id

View solution in original post

to4kawa
Ultra Champion
| makeresults 
| eval _raw="2020-04-29 16:19:13,082 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262180\nupdates:       \ncancellations: 1258780-1
 2020-04-29 16:14:10,479 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/rpfPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,498 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1....70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,442 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.....10002/igmpPortConfig! (Config success!). New contributors: Set(book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,438 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1......70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262175-1), removed contributors: Set().
 2020-04-29 16:14:09,388 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).
 2020-04-29 16:14:09,314 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
 2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,308 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1..........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,306 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1)
 2020-04-29 16:14:09,305 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
 2020-04-29 16:14:09,303 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......10002/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,302 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1), removed contributors: Set()
 2020-04-29 16:14:09,300 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:08,914 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1........SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1).
 2020-04-29 16:14:08,837 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,836 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,791 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262174,1262175,1262176\nupdates:       \ncancellations: 1261438-1,1261436-1,1261440-1
 2020-04-29 16:14:08,637 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262172\nupdates:       \ncancellations: 1261482-1" 
| rex mode=sed "s/(?m)^\s+//g" 
| multikv noheader=t
| stats count by _raw
| rex "(?<time>\S+ \S+)"
| eval _time=strptime(time,"%F %T,%3Q")
| sort - _time
| table _time _raw
| rename COMMENT as "this is sample, from here, the logic"
| rex mode=sed "s/\\\n/
/g"
| rex max_match=0 "(?ms)(?<id>\d{7})(?!-)"
| rex max_match=0 "(?ms)(?<sub_id>\d{7}-\d)"
| mvexpand sub_id
| eval id=coalesce(id,substr(sub_id,1,7))
| mvexpand id
| stats range(eval(if(searchmatch("creations:") OR searchmatch("Config success!"),_time,NULL))) as duration by id

View solution in original post

dabroma5
Explorer

Hi 

Situation change a little bit. I am stuck because I am receiving not the correct results.

 

(ConfigurationManagerService OR PathManagerService) 
| rex max_match=0 " (?ms)Update service\[(?<id>\d{6}\-\d)(\])"
| rex max_match=0 "(?ms)(?<sub_id>\d{6}\-\d)(\))"
| mvexpand sub_id
| eval id=coalesce(id,substr(sub_id,1,8))
| mvexpand id
| stats range(eval(if(searchmatch("Update service") OR searchmatch("New: Set"),_time,NULL))) as duration by id

 Below is the perfectly measured  part of the log :

2020-10-14 17:26:46,039 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device52.3.5000000/ipRxPortConfig - New: Set(), removed: Set(book.145542-1).
2020-10-14 17:26:45,887 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device70.3.5500001/ipTxPortConfig - New: Set(), removed: Set(book.145542-1, book.145542-2).
2020-10-14 17:26:45,742 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Cancel service[145542-1]
2020-10-14 17:26:03,832 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device52.3.5000000/ipRxPortConfig - New: Set(book.145542-1), removed: Set().
2020-10-14 17:26:03,748 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device70.3.5500001/ipTxPortConfig - New: Set(book.145542-1, book.145542-2), removed: Set().
2020-10-14 17:26:03,573 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145542-1(p2mp_join): virtual.114.1.34 -> device52.3.5000000 Success
2020-10-14 17:26:03,561 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Update service[145542-1] (p2mp_join): <virtual.114.1.34> => <device52.3.5000000>  2020-10-14T17:26:03.559+03:00 -> 2255-03-14T17:59:59.000+02:00

Here is the example which makes trouble:

2020-10-15 00:03:13,222 backend_8.20.2: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.5.5500001/ipTxPortConfig - New: Set(book.145374-1), removed: Set().
2020-10-15 00:03:13,067 backend_8.20.2: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.3.5500001/ipTxPortConfig - New: Set(), removed: Set(book.145374-1).
2020-10-15 00:03:13,055 backend_8.20.2: INFO   vip.service.PathManagerService \"Attempting reroute of\" services(s) \"145374-1 LineCloud\" - alarmweighting = true"
2020-10-15 00:03:13,055 backend_8.20.2: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success
2020-10-15 00:03:13,050 backend_8.20.2: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success
2020-10-15 00:03:13,038 backend_8.20.2: INFO   vip.service.PathManagerService Reroute multiple connections: \"145374-1\" with collaterals \"\"
2020-10-14 23:05:45,842 backend_8.20.2: INFO   vip.service.PathManagerService Performing meta-only update of connection 145374-1...
2020-10-14 23:02:09,611 backend_8.20.2: INFO   vip.service.PathManagerService Performing meta-only update of connection 145374-1...
2020-10-14 22:56:43,288 backend_8.20.2: INFO   vip.service.PathManagerService Performing meta-only update of connection 145374-1...
2020-10-14 22:56:39,911 backend_8.20.2: INFO   vip.service.PathManagerService Performing meta-only update of connection 145374-1...
2020-10-14 22:45:08,156 backend_8.20.2: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.3.5500001/ipTxPortConfig - New: Set(book.145374-2, book.145374-1), removed: Set().
2020-10-14 22:45:08,066 backend_8.20.2: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device27.3.5000000/ipRxPortConfig - New: Set(book.145374-1), removed: Set().
2020-10-14 22:04:05,623 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.5.5500001/ipTxPortConfig - New: Set(), removed: Set(book.145374-2, book.145374-1).
2020-10-14 22:04:05,535 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.3.5500001/ipTxPortConfig - New: Set(book.145374-2, book.145374-1), removed: Set().
2020-10-14 22:04:05,427 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService \"Attempting reroute of\" services(s) \"145374-1 LineCloud, 145374-2 LineCloud\" - alarmweighting = true
2020-10-14 22:04:05,427 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success
2020-10-14 22:04:05,420 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success
2020-10-14 22:04:05,396 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Reroute multiple connections: \"145374-1, 145374-2\" with collaterals \"\"
2020-10-14 22:02:03,599 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.3.5500001/ipTxPortConfig - New: Set(), removed: Set(book.145374-2, book.145374-1).
2020-10-14 22:02:03,511 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.5.5500001/ipTxPortConfig - New: Set(book.145374-2, book.145374-1), removed: Set().
2020-10-14 22:02:03,339 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService \"Attempting reroute of\" services(s) \"145374-1 LineCloud, 145374-2 LineCloud\" - alarmweighting = true
2020-10-14 22:02:03,339 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success
2020-10-14 22:02:03,332 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success
2020-10-14 22:02:03,305 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Reroute multiple connections: \"145374-1, 145374-2\" with collaterals \"\"
2020-10-14 17:01:21,149 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device27.3.5000000/ipRxPortConfig - New: Set(book.145374-1), removed: Set().
2020-10-14 17:01:21,008 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.3.5500001/ipTxPortConfig - New: Set(book.145374-2, book.145374-1), removed: Set().
2020-10-14 17:01:20,863 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success"2020-10-14 17:01:20,807 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Update service[145374-1] (p2mp_join): <virtual.56.1.20> => <device27.3.5000000>  2020-10-14T17:01:20.805+03:00 -> 2255-03-14T17:59:59.000+02:00

 

I would like to start:

2020-10-14 17:01:20,863 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Booking 145374-1(p2mp_join): virtual.56.1.20 -> device27.3.5000000 Success"2020-10-14 17:01:20,807 backend_8.19.10-ALPHA: INFO   vip.service.PathManagerService Update service[145374-1] (p2mp_join): <virtual.56.1.20> => <device27.3.5000000>  2020-10-14T17:01:20.805+03:00 -> 2255-03-14T17:59:59.000+02:00

 and finish:

2020-10-14 17:01:21,149 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device27.3.5000000/ipRxPortConfig - New: Set(book.145374-1), removed: Set().
2020-10-14 17:01:21,008 backend_8.19.10-ALPHA: INFO   vip.service.ConfigurationManagerService.configStatusHandler New config status for device54.3.5500001/ipTxPortConfig - New: Set(book.145374-2, book.145374-1), removed: Set().
0 Karma

to4kawa
Ultra Champion

What will be the right result?

0 Karma

dabroma5
Explorer

It doesn't work perfectly, but works:) thanks. Could you please explain your query?

0 Karma

to4kawa
Ultra Champion

It's a pretty simple query.
What's the hard part?

0 Karma

dabroma5
Explorer

🙂 Below part is new for me

| rex mode=sed "s/(?m)^\s+//g"
...
| rex "(?\S+ \S+)"
| eval _time=strptime(time,"%F %T,%3Q")

0 Karma

to4kawa
Ultra Champion

https://www.pcre.org/current/doc/html/pcre2syntax.html

| rex mode=sed "s/(?m)^\s+//g" is ,In Splunk>Answers. code sample add extra spaces.
so, when you do copy and paste them, it can't work.
this is aims to avoid this bug.

(?m) is multi line.

next rex and eval extract timestamp value.
\S is not spaces. reference: https://www.rexegg.com/
the raw 2020-04-29 16:14:08,835 backend_7.2.15:...
_words space words space
.

time format is following:

https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Commontimeformatvariables

0 Karma

dabroma5
Explorer

got it 🙂
below part was only to format log lines :
| rex mode=sed "s/(?m)^\s+//g"
| multikv noheader=t
| stats count by _raw
| rex "(?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!