All Apps and Add-ons

I need to get the last day of a set of events

flck
Path Finder

Hello everyone, I am getting several events reported from Trusted Advisor (AWS), the events are generated daily but at different times.
I mean, today, December 30, 20 items can be generated at different times, then, tomorrow, December 31, only 10 events can be reported. I need a filter that always gets only the events reported on the current day regardless of the difference in hours but ignoring the previous days.

Any idea how to achieve it?

Thank you very much in advance.

0 Karma
1 Solution

aberkow
Builder

That makes sense - in which case you can combine the tokens (perhaps expanding it to -w@w and @w) and another concept of grouping (or binning) _time, like so:

| makeresults count=1
| bin _time span=d as day
| eventstats max(day) as mostRecentDay
| where day=mostRecentDay

What I'm doing here is:
On line 1 - creating a fake result to play around with
On line 2 - binning time to discrete buckets of day (for all logs on December 27th, they will all have the same value, etc)
On line 3 - telling all logs to show the max value of day (and since it's epoch time, that will be the most recent day)
On line 4 - filtering to just the logset that matches the most recent day.

bin is a really useful command as well https://docs.splunk.com/Documentation/Splunk/8.0.1/SearchReference/Bin, as it can allow you to manipulate the _time field in interesting ways. For your case, you want to use lines 2, 3 and 4, along with your search which must include the _time field. Does this answer your question?

View solution in original post

0 Karma

flck
Path Finder

Hello Friend @aberkow ,

I am doing tests and I cannot filter the result I want, the exact date of the last events I have at the moment is December 11.
I have "played" with the span modifying this to 1 month and so I can list all the events but I have not managed to filter the last available day. 😞

0 Karma

aberkow
Builder
index=yourIndex sourcetype=yourSourcetype earliest=-month
| bin _time span=d as day
| eventstats max(day) as mostRecentDay
| where day=mostRecentDay

Doesn't work? The earliest token is kind of irrelevant

0 Karma

flck
Path Finder

Hello Friend ,

I just converted the variable day and mostRecentDay to the format "% Y /% m /% d" and the value it is taking is 2019/12/30 so it does not filter the events of December 11.

0 Karma

aberkow
Builder

It works for me either way, you just want to make sure you're running the eventstats command before you convert, since you can't take the max() value of a string.

0 Karma

flck
Path Finder

Hello my friend,

You are absolutely right, the eventstats must be executed immediately after the Index search, I was able to get the result I wanted in my filter.

I thank you very much for all your help and patience.

Thank you :).

0 Karma

aberkow
Builder

That makes sense - in which case you can combine the tokens (perhaps expanding it to -w@w and @w) and another concept of grouping (or binning) _time, like so:

| makeresults count=1
| bin _time span=d as day
| eventstats max(day) as mostRecentDay
| where day=mostRecentDay

What I'm doing here is:
On line 1 - creating a fake result to play around with
On line 2 - binning time to discrete buckets of day (for all logs on December 27th, they will all have the same value, etc)
On line 3 - telling all logs to show the max value of day (and since it's epoch time, that will be the most recent day)
On line 4 - filtering to just the logset that matches the most recent day.

bin is a really useful command as well https://docs.splunk.com/Documentation/Splunk/8.0.1/SearchReference/Bin, as it can allow you to manipulate the _time field in interesting ways. For your case, you want to use lines 2, 3 and 4, along with your search which must include the _time field. Does this answer your question?

0 Karma

flck
Path Finder

Hi friend @aberkow , thanks for your answer, it works fine.

But I have another scenario, there may be a situation in which an X number of days pass without generating new information, by default Trusted Advisor only generates reports at the end of each week, in this situation you would need to be able to filter the last most recent day.
I want to say something like this example:

On Friday, December 27, 5 events were generated at different times, and one day before Thursday 26, 10 events were also generated at different times. In this situation, I would only need to obtain the events of day 27 and ignore day 26.

I've searched a lot in the documentation and can't find how to do it.

Thank you 🙂

0 Karma

aberkow
Builder

You can use time tokens in a search to specify exact boundaries of time by snapping to relative times. Example:

index=AWS sourcetype=AWS otherInfo=Value earliest=-d@d latest=@d

This will snap the search to "at earliest, minus 1 day and snapped to the beginning of the day, and at latest, snap to the beginning of this day". You can do similar things with minutes, weeks, months, etc. In your case, you would want to run this at any time on December 31st to capture December 30th's info, and so on. By default, this configuration will always capture yesterday's full info set.

Really powerful stuff! Take a look at this doc sheet: https://docs.splunk.com/Documentation/Splunk/8.0.1/SearchReference/SearchTimeModifiers#Define_the_ti....

Hope this helps 🙂

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...