Deployment Architecture

Can you nest buckets, and then determine time for a single occurence of these nested buckets?

stcrispan
Communicator

I'm trying to detect times when my devices are having trouble roaming from AP point to AP point. The application running on the devices is logging times when it loses connection (usually when it wakes up after having gone to sleep, or someone presses the button to put it to sleep).

The pattern is: Try 1, Try 2, Try 3...then repeat those three tries until success. I'm trying to find the times when the retries go five or more times, or those instances where the total time spent in the 3x retry ends up taking more than 15 minutes.

I'm thinking it can be done with nested buckets. Follow: first bucket will look for "tryCounter:1", "tryCounter:2", and "tryCounter:3", all happening as a single item. The trick is to catch this bucket as all coming from the same host, without specifying which host...but it has to match all three tries to the same host name, so it knows they are all related, and all happening within the same retry set.

Then, a second bucket will "catch" these first bucket trios, and count up to 4 sets before triggering and counting the 5th set of three as a single Occurence. Again, the trick here is to make sure that all 5 are from the same host.

Originally I had counted them all by host, just looking for 15 or more events in the same time period, but I've been told that search is not what they're looking for. Now, my final product of the search must detect when a single host has experienced 5 or more 3xRetry conditions. Eventually, I'd like to know how long the entire experience took, and pick out the ones which take 10 or more minutes to complete the entire retry sequence.

So - anyone feel like taking a crack at this? Dataset is below...

1/21/19  11:30:00.000 AM    2019-01-21 11:32:10.6999 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:10.7624 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:10.7845 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:18.9323 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:18.9792 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:19.0170 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:23.7838 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:23.8373 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:23.8685 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:30.3701 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:30.4079 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:30.4391 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:32.9580 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:32.9958 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:33.0426 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:37.7179 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:37.7713 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:32:37.7869 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1783C-P3G  
    index = DeviceLogs  
    source =    C:\Users\slate1783c\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:37:17.3361 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414SP-SJC  
    index = DeviceLogs  
    source =    C:\Users\slate414sp\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:37:17.3906 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414SP-SJC  
    index = DeviceLogs  
    source =    C:\Users\slate414sp\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:37:17.4363 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414SP-SJC  
    index = DeviceLogs  
    source =    C:\Users\slate414sp\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:41:59.1037 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:41:59.1037 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:41:59.1378 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:02.0996 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:02.1151 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:02.1307 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:16.8706 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:16.8987 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM 2019-01-21 11:42:17.5561 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:17.5759 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:17.5903 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:19.4224 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:19.4602 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:40.2953 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:40.2953 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:40.3264 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:48.8086 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:48.8243 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:51.0919 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:51.1232 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:53.3145 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:53.3457 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:59.5192 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:42:59.5414 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE1610R-S7J  
    index = DeviceLogs  
    source =    C:\Users\slate1610r\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt   
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:44:08.9076 exception catched - tryCounter:1 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:44:08.9214 exception catched - tryCounter:2 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:44:08.9381 exception catched - tryCounter:3 - No IPEndpoints were found for host DeviceServer.constoso.msft. 

    host =  SLATE414F-QS3   
    index = DeviceLogs  
    source =    C:\Users\slate414f\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:44:45.6185 No IPEndpoints were found for host DeviceServer.constoso.msft.-System.ServiceModel.EndpointNotFoundException System.ServiceModel.EndpointNotFoundException: No IPEndpoints were found for host DeviceServer.constoso.msft.
   at Void System.ServiceModel.Channels.SocketConnectionInitiator+<ConnectAsync>d__9.MoveNext() + 0x2ed
--- End of stack trace from previous location where exception was thrown ---
   at Void System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x5d
   at Void System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task) + 0x33
Show all 189 lines

    host =  SLATE216H-LN7   
    index = DeviceLogs  
    source =    C:\Users\slate216h\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet   

1/21/19 11:30:00.000 AM     2019-01-21 11:44:45.6585 No IPEndpoints were found for host DeviceServer.constoso.msft.-2nd catch -System.ServiceModel.EndpointNotFoundException System.ServiceModel.EndpointNotFoundException: No IPEndpoints were found for host DeviceServer.constoso.msft.
   at Void System.ServiceModel.Channels.SocketConnectionInitiator+<ConnectAsync>d__9.MoveNext() + 0x2ed
--- End of stack trace from previous location where exception was thrown ---
   at Void System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x5d
   at Void System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task) + 0x33
Show all 189 lines

    host =  SLATE216H-LN7   
    index = DeviceLogs  
    source =    C:\Users\slate216h\AppData\Local\Packages\DeviceLogs\LocalState\nLog.txt    
    sourcetype =DeviceLogs_dotnet
0 Karma

stcrispan
Communicator

Nobody wants to take a crack at this?

0 Karma

stcrispan
Communicator

Bump for the day crew...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...