How to set earliest time based on current time

[JoshuaJohn]

I am ingesting data at 6AM, 2PM, 7PM, 10PM (CST)
Is there anyway I could have my query check the time and set earliest to the time that it past last?

ie:
It is 8AM CST time, the search would set earliest to 6AM
It is 9PM CST time, the search would set earliest to 7PM

Thanks

[jacobpevans · Accepted solution]

I thought about it a little more and something like this should work. It is certainly not perfect though - you would want to check month and year crossovers also to guarantee accuracy. Change log_level to WARN or INFO if you're environment is perfect and doesn't have errors.

index=_internal log_level=ERROR
| eval curr_date_hour = strftime(now(), "%H"),
       curr_date_mday = strftime(now(), "%d")
| eval search_mday    = case(curr_date_hour <= 6, curr_date_mday - 1,
                             curr_date_hour >  6, curr_date_mday)
| eval search_hour    = case(curr_date_hour <  6, 22,
                             curr_date_hour >  22, 22,
                             curr_date_hour >  19, 19,
                             curr_date_hour >  14, 14,
                             curr_date_hour >   6,  6)
| where (search_hour != 22 AND search_mday     = date_mday AND date_hour >= search_hour) OR
        (search_hour  = 22 AND search_mday     = date_mday AND date_hour >= search_hour) OR
        (search_hour  = 22 AND search_mday + 1 = date_mday)

[mayurr98]

try earliest=-2h in your search query?

[jacobpevans]

I'm sure it's possible, but the easier way to do it would be to:

1. Change the ingestion to run once every 6 hours (still 4 times per day), and set your search to earliest=-6h, or
2. Set the search to last 24 hours (or something greater than 8 hours), and perform a dedup to ensure your results only contain the latest data.