Getting Data In
Highlighted

How can I get the time difference between two events with particular ID?

Explorer

I was trying to filter event ID in subsearch and then use it in the main search to find other events with related ID and compare time from subsearch with last event time from the main search.
The initial line when ID appears is: 2020-04-29 16:14:08,637 backend7.2.15: INFO services/ConnectionManagerService(backend): _\ncreations: 1262172__\nupdates: \ncancellations: 1261482-1

one of the problem is that above event ID's can appear after decimal, like below:
2020-04-29 16:14:08,791 backend7.2.15: INFO services/ConnectionManagerService(backend): _\ncreations: 1262174,1262175,1262176__\nupdates: \ncancellations: 1261438-1,1261436-1,1261440-1

confirmation line - last:
10.21.160.144.SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).

My query:
....... sourcetype=main ConfigurationManagerService
|append [search ................sourcetype=main "ConnectionManagerService(backend)" "\ncreations:"
| multikv noheader=t
| rex "(?:ions: )(?\d{7})"
| where ID != 0
| rename time as starttime
| table ID starttime]
| stats earliest(start
time), latest(_time) as stop by ID

How to make it more efficient or just working?

Part of the log:

2020-04-29 16:19:13,082 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262180\nupdates:       \ncancellations: 1258780-1
2020-04-29 16:14:10,479 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/rpfPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
2020-04-29 16:14:09,498 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1....70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
2020-04-29 16:14:09,442 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.....10002/igmpPortConfig! (Config success!). New contributors: Set(book.1262176-1), removed contributors: Set().
2020-04-29 16:14:09,438 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1......70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262175-1), removed contributors: Set().
2020-04-29 16:14:09,388 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).
2020-04-29 16:14:09,314 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,308 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1..........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,306 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1)
2020-04-29 16:14:09,305 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
2020-04-29 16:14:09,303 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......10002/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
2020-04-29 16:14:09,302 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1), removed contributors: Set()
2020-04-29 16:14:09,300 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
2020-04-29 16:14:08,914 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1........SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1).
2020-04-29 16:14:08,837 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,836 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
2020-04-29 16:14:08,791 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262174,1262175,1262176\nupdates:       \ncancellations: 1261438-1,1261436-1,1261440-1
2020-04-29 16:14:08,637 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262172\nupdates:       \ncancellations: 1261482-1
0 Karma
Highlighted

Re: How can I get the time difference between two events with particular ID?

Ultra Champion
| makeresults 
| eval _raw="2020-04-29 16:19:13,082 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262180\nupdates:       \ncancellations: 1258780-1
 2020-04-29 16:14:10,479 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/rpfPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,498 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1....70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,442 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.....10002/igmpPortConfig! (Config success!). New contributors: Set(book.1262176-1), removed contributors: Set().
 2020-04-29 16:14:09,438 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1......70000/igmpPortConfig! (Config success!). New contributors: Set(book.1262175-1), removed contributors: Set().
 2020-04-29 16:14:09,388 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1.......SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1).
 2020-04-29 16:14:09,314 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
 2020-04-29 16:14:09,313 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1......SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,308 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1..........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262174-1, book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,306 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1, book.1262176-1), removed contributors: Set(book.1261438-1, book.1261440-1, book.1261436-1)
 2020-04-29 16:14:09,305 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1), removed contributors: Set()
 2020-04-29 16:14:09,303 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......10002/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:09,302 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262175-1, book.1262174-1), removed contributors: Set()
 2020-04-29 16:14:09,300 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262176-1), removed contributors: Set()
 2020-04-29 16:14:08,914 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): Successfully applied config for 1........SwitchingCore/openflowConfig! (Config success!). New contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1).
 2020-04-29 16:14:08,837 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,836 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/openflowConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1.......70000/igmpPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,835 backend_7.2.15: INFO     services/ConfigurationManagerService(backend): ControlledVertexFSM@1........SwitchingCore/rpfPortConfig: New config retrieved by Root state with delay None, new contributors: Set(book.1262172-1), removed contributors: Set(book.1261482-1)
 2020-04-29 16:14:08,791 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262174,1262175,1262176\nupdates:       \ncancellations: 1261438-1,1261436-1,1261440-1
 2020-04-29 16:14:08,637 backend_7.2.15: INFO     services/ConnectionManagerService(backend): \ncreations:     1262172\nupdates:       \ncancellations: 1261482-1" 
| rex mode=sed "s/(?m)^\s+//g" 
| multikv noheader=t
| stats count by _raw
| rex "(?<time>\S+ \S+)"
| eval _time=strptime(time,"%F %T,%3Q")
| sort - _time
| table _time _raw
| rename COMMENT as "this is sample, from here, the logic"
| rex mode=sed "s/\\\n/
/g"
| rex max_match=0 "(?ms)(?<id>\d{7})(?!-)"
| rex max_match=0 "(?ms)(?<sub_id>\d{7}-\d)"
| mvexpand sub_id
| eval id=coalesce(id,substr(sub_id,1,7))
| mvexpand id
| stats range(eval(if(searchmatch("creations:") OR searchmatch("Config success!"),_time,NULL))) as duration by id

View solution in original post

Highlighted

Re: How can I get the time difference between two events with particular ID?

Explorer

It doesn't work perfectly, but works:) thanks. Could you please explain your query?

0 Karma
Highlighted

Re: How can I get the time difference between two events with particular ID?

Ultra Champion

It's a pretty simple query.
What's the hard part?

0 Karma
Highlighted

Re: How can I get the time difference between two events with particular ID?

Explorer

🙂 Below part is new for me

| rex mode=sed "s/(?m)^\s+//g"
...
| rex "(?\S+ \S+)"
| eval _time=strptime(time,"%F %T,%3Q")

0 Karma
Highlighted

Re: How can I get the time difference between two events with particular ID?

Ultra Champion

https://www.pcre.org/current/doc/html/pcre2syntax.html

| rex mode=sed "s/(?m)^\s+//g" is ,In Splunk>Answers. code sample add extra spaces.
so, when you do copy and paste them, it can't work.
this is aims to avoid this bug.

(?m) is multi line.

next rex and eval extract timestamp value.
\S is not spaces. reference: https://www.rexegg.com/
the raw `2020-04-29 16:14:08,835 backend7.2.15:...`
words space words space .

time format is following:

https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Commontimeformatvariables

0 Karma
Highlighted

Re: How can I get the time difference between two events with particular ID?

Explorer

got it 🙂
below part was only to format log lines :
| rex mode=sed "s/(?m)^\s+//g"
| multikv noheader=t
| stats count by _raw
| rex "(?

0 Karma