I need to get the last day of a set of events

[flck]

Hello everyone, I am getting several events reported from Trusted Advisor (AWS), the events are generated daily but at different times.
I mean, today, December 30, 20 items can be generated at different times, then, tomorrow, December 31, only 10 events can be reported. I need a filter that always gets only the events reported on the current day regardless of the difference in hours but ignoring the previous days.

Any idea how to achieve it?

Thank you very much in advance.

[flck]

Hello Friend @aberkow ,

I am doing tests and I cannot filter the result I want, the exact date of the last events I have at the moment is December 11.
I have "played" with the span modifying this to 1 month and so I can list all the events but I have not managed to filter the last available day. 😞

[aberkow]

index=yourIndex sourcetype=yourSourcetype earliest=-month
| bin _time span=d as day
| eventstats max(day) as mostRecentDay
| where day=mostRecentDay

Doesn't work? The earliest token is kind of irrelevant

[flck]

Hello Friend ,

I just converted the variable day and mostRecentDay to the format "% Y /% m /% d" and the value it is taking is 2019/12/30 so it does not filter the events of December 11.

[aberkow]

It works for me either way, you just want to make sure you're running the eventstats command before you convert, since you can't take the max() value of a string.

[flck]

Hello my friend,

You are absolutely right, the eventstats must be executed immediately after the Index search, I was able to get the result I wanted in my filter.

I thank you very much for all your help and patience.

Thank you :).

[aberkow · Accepted solution]

That makes sense - in which case you can combine the tokens (perhaps expanding it to -w@w and @w) and another concept of grouping (or binning) _time, like so:

| makeresults count=1
| bin _time span=d as day
| eventstats max(day) as mostRecentDay
| where day=mostRecentDay

What I'm doing here is:
On line 1 - creating a fake result to play around with
On line 2 - binning time to discrete buckets of day (for all logs on December 27th, they will all have the same value, etc)
On line 3 - telling all logs to show the max value of day (and since it's epoch time, that will be the most recent day)
On line 4 - filtering to just the logset that matches the most recent day.

bin is a really useful command as well https://docs.splunk.com/Documentation/Splunk/8.0.1/SearchReference/Bin, as it can allow you to manipulate the _time field in interesting ways. For your case, you want to use lines 2, 3 and 4, along with your search which must include the _time field. Does this answer your question?

[flck]

Hi friend @aberkow , thanks for your answer, it works fine.

But I have another scenario, there may be a situation in which an X number of days pass without generating new information, by default Trusted Advisor only generates reports at the end of each week, in this situation you would need to be able to filter the last most recent day.
I want to say something like this example:

On Friday, December 27, 5 events were generated at different times, and one day before Thursday 26, 10 events were also generated at different times. In this situation, I would only need to obtain the events of day 27 and ignore day 26.

I've searched a lot in the documentation and can't find how to do it.

Thank you 🙂

[aberkow]

You can use time tokens in a search to specify exact boundaries of time by snapping to relative times. Example:

index=AWS sourcetype=AWS otherInfo=Value earliest=-d@d latest=@d

This will snap the search to "at earliest, minus 1 day and snapped to the beginning of the day, and at latest, snap to the beginning of this day". You can do similar things with minutes, weeks, months, etc. In your case, you would want to run this at any time on December 31st to capture December 30th's info, and so on. By default, this configuration will always capture yesterday's full info set.

Really powerful stuff! Take a look at this doc sheet: https://docs.splunk.com/Documentation/Splunk/8.0.1/SearchReference/SearchTimeModifiers#Define_the_ti....

Hope this helps 🙂