We have an index 'abc' to which data gets fed in non-uniform intervals. I would like to get all events of this index that were indexed recently. Could i get some guidance on how to achieve this?
Ex: Data indexed on 1st of March, 5th of March and 10th of March. I want to get all events indexed on 10th of March.
Hi @rajeshjlnt
I think that you're speaking of events that are indexed with a timestamp really different with the indexing date.
To do this at first you have to find a time period where you are sure that contains all the events indexed in the monitoring period; in other words, if you index today events of last year, and you select as time frame the last month, you'll not have all the events in you results.
Then you have to use the _indextime field that are present in all the events (in epochtime).
You have to run something like this (e.g. in the last 30 days):
index=your_index earliest=-30d latest=now
| eval indextime=strftime(_indextime, "%Y-%m-%d %H:%M:%S"), diff=_time-_indextime
| table _time indextime diff
Ciao,
Giuseppe
You can use _index_earliest=-1h _index_latest=now
in your foundational search; for March 10
, use this:
index="abc" earliest=0 latetst=@d+100d _index_earliest=1583816400 _index_latest=1583902800
Hi @rajeshjlnt
I think that you're speaking of events that are indexed with a timestamp really different with the indexing date.
To do this at first you have to find a time period where you are sure that contains all the events indexed in the monitoring period; in other words, if you index today events of last year, and you select as time frame the last month, you'll not have all the events in you results.
Then you have to use the _indextime field that are present in all the events (in epochtime).
You have to run something like this (e.g. in the last 30 days):
index=your_index earliest=-30d latest=now
| eval indextime=strftime(_indextime, "%Y-%m-%d %H:%M:%S"), diff=_time-_indextime
| table _time indextime diff
Ciao,
Giuseppe
@gcusello , in my case _time and _indextime. I understand how timestamps work in splunk. Let me explain my requirement in more detail.
I upload a CSV file with n entries every day with a fixed timestamp. now i want to search and get events from latest uploaded file.
Hi @rajeshjlnt,
if the csv file has a different name for each file (in other words a date in its name), you can use this to filter events, something like this:
e.g. if the csv is named your_csv_2020-03-11.csv and you have a different one every day with a different name, you can run something like this:
index=your_index [ index=your_index | head 1 | fields source ]
| ...
Ciao.
Giuseppe
This works for me. Great thanks @gcusello
I made a small modification to your suggestion by adding 'search', without which i got an error
index=your_index [ search index=your_index | head 1 | fields source ]
Hi @rajeshjlnt,
sorry a little missed!
If this answer solves your problem, please accept and/or upvote it for the other users of the Community.
Ciao and next time!
Giuseppe
You can filter data on index time also. Use index command with _index_earliest and _index_latest.
index=_internal _index_earliest=-5m@m _index_latest=@m
@manjunathmeti , this is similar to time range picker. May be i am missing something, how can this help in getting the latest indexed set of events?