How to filter out first instance of a query

[pedropiin]

Hi everyone.

I'm doing a query in which I sort it by time according to a variable and then calculate some metrics over the data. But I need to calculate these metrics without considering exactly the first instance of my data, that is, the earliest one, as it's the one associated with the server being started daily and it's not valid for my needs.

It's important to note that I don't have any information associated with this first instance before the query runs as its related to a script scheduled to run at a specific time, but it generates new values every time, and it's duration is variable, meaning that I don't know when it has finished.

I cannot share information related to the data neither the query exactly, but it's of the form

index=...
| stats ...
| eval val1=...
| eval time_val=...
| sort time_val
| eval val3=...
| stats count...

How could I do this? 

[livehybrid · Accepted solution]

Hi @pedropiin 

You could try the following:

index=...
| stats ... 
| eval val1=... 
| eval time_val=... 
| sort time_val 
| streamstats count AS row_num 
| where row_num > 1 
| eval val3=... 
| stats count...

How this works:

• `sort time_val`: Orders your events by the specified time variable.
• `streamstats count AS row_num`: Adds a sequential count to each event, starting from 1.
• `where row_num > 1`: Filters out the first event (earliest instance), allowing you to skip the unwanted data point.
• The rest of your metrics calculations then proceed on the filtered dataset.

This method allows you to dynamically exclude the first entry without needing to know its exact characteristics beforehand. Adjust the variable names and the rest of the logic specific to your context as needed.

Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards

Will