Cannot find log messages for -1d@d but -7d@d works

[baoctac]

I have a Splunk alert that has been sending false emails. The alert is sent when a string is absent from the application's log. The search itself is actually not finding the log message, which I assume is the reason for the log not being triggered.
This search in Splunk UI: "finished importing data" earliest=-1d@d
Results: no messages found.

But when I search back 7 days, the expected log message appears.
Ex: "finished importing data" earliest=7d@d
Results: messages are returned as expected

Furthermore, I am able to see other messages from the application logs from the last 24 hours in Splunk, but the specific text "finished importing data" only appears when I search for 7 days or greater.

Any ideas why 7-day search would return results but not a 1-day search? Your help is greatly appreciated.

Currently running Splunk version 7.0.1.

[baoctac]

hrm...this is interesting. Yesterday's log is somehow added to an exception from earlier this week. Let's close this one for now. I'll investigate further on my end.

Thanks all.

[Sukisen1981]

hmm intriguing indeed, from the status "Finished Importing data" I can probably guess that this is an indication of some sort of batch run. Check with the developer of the batch job, it has to be an issue with how he is logging the current day's (last 24 hours) log.
Meanwhile please accept my answer if you found it useful

[baoctac]

"Finished Importing data" | earliest=-d@d latest=@d

With the pipe, I get an error: Search Factory: Unknown search command 'earliest'.

"Finished Importing data" earliest=-d@d latest=@d

Without the pipe, I get: No results found. Try expanding the time range.

[Sukisen1981]

'Furthermore, I am able to see other messages from the application logs from the last 24 hours in Splunk, but the specific text "finished importing data" only appears when I search for 7 days or greater.'
Question is - when is the latest occurrence of this event when you search using

earliest=-7d@d?
If there are no "Finished Importing data" event in the last 24 hours searching for the same events will not yield any results

[Sukisen1981]

what happens if you try this?

<your search>|earliest=-d@d latest=@d

[baoctac]

I don't follow. Are you saying that most of my application log(at least >90% of it) is present in my data, and that single log message is not present in data? I'm just trying to understand how this would happen, and understand how to prevent it.

If my Splunk query is wrong, how would one search and/or create an alert that checks for existence of a log message for the last 24 hours?

Thanks!

[493669]

On what criteria you can say application log is present in your data?
if "finished importing data" this is contained in your data then you are considering as application log is present?

[baoctac]

I assumed that 'present in my data' means that I'm able to search and find those logs in Splunk. But perhaps it's better for me to ask for your criteria as well 🙂

Is that assumption correct? or did you mean something else?

[493669]

your assumption is correct i.e. if "finished importing data" present means you can search and find logs

[baoctac]

Thanks. I just posted this message below:
Yesterday's log is somehow added to an exception from earlier this week. Let's close this one for now. I'll investigate further on my end.

[493669]

when you search : index=<indexname> "finished importing data" earliest=-1d@d
Results: no messages found. it means "finished importing data" string is not present in your data from yesterday.
and when you search : index=<indexname>"finished importing data" earliest=-7d@d
Results: messages are returned . it means "finished importing data" string is present in data from last 7 days till day before yesterday