long running job

[surekhasplunk]

Hi,

I have written a query to generate lookup file for last 30days, which is taking  a lot of time like almost 4 hours which is high on cpu. So can is there a option to run query everyday but run only for last 24 hours and append to the same lookup file generated yesterday, so that the dashboard populates quickly with all the 30days data post comparison

[Nisha18789]

Hello @surekhasplunk , yes that possible, like below

<your query to generate the data for last 24 hour>| outputlookup <lookup name.csv> append=true

Also, you can use summary index fir storing this data in case the lookup has a chance to get very bulky with time.

[surekhasplunk]

thanks @Nisha18789 

So in case i use summary index, i have to schedule it to run everyday for last 24hrs ?

or once in a month with last 30 days ?

Also it will impact the license usage right where as when we write to lookup file it wont affect the license usage

Please explain

[Nisha18789]

Hi @surekhasplunk , running after midnight , for previous day will be good.

Also, logging to summary index does not add to license usage as this data is already ingested in your original index.