topic Re: Get all events/fields from latest data of an Index in Monitoring Splunk

Get all events/fields from latest data of an Index

rajeshjlnt — Wed, 11 Mar 2020 11:44:32 GMT

We have an index 'abc' to which data gets fed in non-uniform intervals. I would like to get all events of this index that were indexed recently. Could i get some guidance on how to achieve this?

Ex: Data indexed on 1st of March, 5th of March and 10th of March. I want to get all events indexed on 10th of March.

Re: Get all events/fields from latest data of an Index

manjunathmeti — Wed, 30 Sep 2020 04:32:52 GMT

You can filter data on index time also. Use index command with _index_earliest and _index_latest.

index=_internal  _index_earliest=-5m@m _index_latest=@m

Re: Get all events/fields from latest data of an Index

gcusello — Wed, 11 Mar 2020 12:16:13 GMT

Hi @rajeshjlnt
I think that you're speaking of events that are indexed with a timestamp really different with the indexing date.

To do this at first you have to find a time period where you are sure that contains all the events indexed in the monitoring period; in other words, if you index today events of last year, and you select as time frame the last month, you'll not have all the events in you results.
Then you have to use the _indextime field that are present in all the events (in epochtime).

You have to run something like this (e.g. in the last 30 days):

index=your_index earliest=-30d latest=now
| eval indextime=strftime(_indextime, "%Y-%m-%d %H:%M:%S"), diff=_time-_indextime
| table _time indextime diff

Ciao,
Giuseppe

Re: Get all events/fields from latest data of an Index

rajeshjlnt — Wed, 11 Mar 2020 13:31:48 GMT

@manjunathmeti , this is similar to time range picker. May be i am missing something, how can this help in getting the latest indexed set of events?

Re: Get all events/fields from latest data of an Index

rajeshjlnt — Wed, 11 Mar 2020 13:39:37 GMT

@gcusello , in my case _time and _indextime. I understand how timestamps work in splunk. Let me explain my requirement in more detail.
I upload a CSV file with n entries every day with a fixed timestamp. now i want to search and get events from latest uploaded file.

Re: Get all events/fields from latest data of an Index

gcusello — Wed, 30 Sep 2020 04:36:39 GMT

Hi @rajeshjlnt,
if the csv file has a different name for each file (in other words a date in its name), you can use this to filter events, something like this:
e.g. if the csv is named your_csv_2020-03-11.csv and you have a different one every day with a different name, you can run something like this:
index=your_index [ index=your_index | head 1 | fields source ]
| ...

Ciao.
Giuseppe

Re: Get all events/fields from latest data of an Index

woodcock — Wed, 11 Mar 2020 14:15:59 GMT

You can use _index_earliest=-1h _index_latest=now in your foundational search; for March 10, use this:

index="abc" earliest=0 latetst=@d+100d _index_earliest=1583816400 _index_latest=1583902800

Re: Get all events/fields from latest data of an Index

rajeshjlnt — Wed, 30 Sep 2020 04:33:03 GMT

This works for me. Great thanks @gcusello

I made a small modification to your suggestion by adding 'search', without which i got an error

index=your_index [ search index=your_index | head 1 | fields source ]

Re: Get all events/fields from latest data of an Index

gcusello — Wed, 11 Mar 2020 14:36:41 GMT

Hi @rajeshjlnt,
sorry a little missed!
If this answer solves your problem, please accept and/or upvote it for the other users of the Community.

Ciao and next time!
Giuseppe