Dashboards & Visualizations

Stats Calculate Time Taken to Copy the Indexes

praneethlekkala
Explorer

Our Application does a nightly re-index on node 1, once thats complete, the index build is copied to 6 other nodes,  Each of the other nodes then restore the files. These entries are noted as "Index restore started." and "Index restore complete." in the application logs. I would like to have a dashboard panel that shows how long it takes from "started" to "complete" on each host to be able to see trends for this over time. How to go about this?

Labels (1)
Tags (1)
0 Karma
1 Solution

ITWhisperer
Ultra Champion
--- search
| eval starttime=if(match(_raw,"Index restore started"),_time,null)
| eval completetime=if(match(_raw,"Index restore complete"),_time,null)
| stats latest(starttime) as starttime latest(completetime) as completetime by host

View solution in original post

praneethlekkala
Explorer

@ITWhisperer 

thanks, can we calculate the number of minutes as a column? Rather than giving the result in hours?

0 Karma

ITWhisperer
Ultra Champion

Can you share anonymised sample log entries so we can see what you are dealing with?

0 Karma

praneethlekkala
Explorer

@ITWhisperer 

Thanks for the reply.

Here is the Index Start Event:

5/4/21
2:43:26.502 AM
2021-05-04 02:43:26,502-0400 NodeReindexServiceThread:thread-0 INFO [c.a.j.index.ha.DefaultIndexCopyService] Index restore started. Total 3561438 issues on instance before loading Snapshot file: IndexSnapshot_39801.tar.sz
host = hostname

source = application.log

The Following is the Index Completed Event , inned the duration from the time it started to when its complete by the hostname please.

5/3/21
1:08:10.843 AM
2021-05-03 01:08:10,843-0400 NodeReindexServiceThread:thread-0 INFO [c.a.j.index.ha.DefaultIndexCopyService] Index restore complete. Total 3653227 issues on instance
host = hostname

source = application.log

Tags (1)
0 Karma

ITWhisperer
Ultra Champion
--- search
| eval starttime=if(match(_raw,"Index restore started"),_time,null)
| eval completetime=if(match(_raw,"Index restore complete"),_time,null)
| stats latest(starttime) as starttime latest(completetime) as completetime by host

View solution in original post

praneethlekkala
Explorer

@ITWhisperer 

Thanks i tried the query and got the following, how to make it readable?

 

host                                                           starttime                                                                   completetime

Hostname1620017696.1691620018391.676
Tags (1)
0 Karma

ITWhisperer
Ultra Champion
| fieldformat starttime=strftime(starttime,"%Y-%m-%d %H:%M:%S")
| fieldformat completetime=strftime(completetime,"%Y-%m-%d %H:%M:%S")

praneethlekkala
Explorer

@ITWhisperer 

Thanks

thanks, can we calculate the number of minutes as a column? 

0 Karma

ITWhisperer
Ultra Champion
| eval duration=tostring(completetime-starttime,"duration")
| eval completeminutes=floor((completetime-starttime)/60)
0 Karma

praneethlekkala
Explorer
 
Tags (1)
0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!