I have a lot of reports/dashboards about a particular sourcetype that receives data (from a forwarder) one time per day. Now, my requirements changed and I need to send data many times per day. My optimal solution can be "overwrite" the last data with the upcoming data, but I think that this is impossible (the data old remains inside the indexer). How can I only report on the last indexed data? Any other ideas?
Well it'll depend upon how your data is. Does all your data have same timestamp? OR if all events are ingested within a certain period like starts at 11 and finished by 11:30 ?Does the source/filename is different between different time it's received?
If you know the data will be sent at a specific frequency, you could use that in your search. For example, if the index gets data every hour, your could write your search like this
index=yourindex sourcetype=specficsourcetype earliest=-1h@h | rest of your search here
If the frequency is indeterminate, then you can use metadata to find the last time index recd data, like this
index=yourindex sourcetype=specficsourcetype [| metadata types=sourcetypes index=yourindex | eval earliest=lastTime | table earliest] | rest of your search here.
Hi. Thanks! When i ran:
[| metadata types=sourcetypes index=bucle_cm | eval earliest=lastTime | table earliest] ...
I receive an error:
Error in 'metadata': You must specify a 'type' argument to 'metadata', as in 'type=hosts'.
types=sourcetypes is ok?