find empty directory that continuously index by Splunk

[indeed_2000]

Hi

I have path that every day logs copy to there

/opt/splunk/logs/$DATE

I create script that copy logs there but sometime logs not copy on that path and script just create empty directory with current date

/opt/splunk/logs/20210712

How can know when directory is empty with splunk?

FYI: this path continuously index on splunk 

Any idea?

Thanks

[venkatasri]

Hi @indeed_2000 

There is no out-of-the box approach that you can apply. Instead there could be otherways using CLI command however if i understand the requirement you want know/notify if there are no events on a particular day.

Other than Splunk you can monitor the destination dir with a custom script for empty dir check/ if copy failes then send an email to notify.

Splunk approach that i could think of is for a particular day if no events been found, then raise a Alert (this is nothing but schdulesearch in Splunk) which you can send as email. Try following query to set-up such alert.

index=<your_index_where_data_lands> sourcetype=<st> source="/opt/splunk/logs/*"
| eval today_date=strftime(now(),"%Y%m%d") 
| eval source_date=mvindex(split(source,"/"), 4) 
| where source_date=today_date 
| stats count 
| where count=0 

It compares today_date format YYYYMMDD with source path (segment 4 is date from source). This must be changed if your source path is different.

---

An upvote would be appreciated and Accept solution if this reply helps you!

[indeed_2000]

Thank you for answer,

I try command that you mention but return nothing just "0"

directory have this structure 

/opt/splunk/logs/20210712/app1

/opt/splunk/logs/20210712/app2

/opt/splunk/logs/20210712/app3

...

e.g expected result,  if app2 and app3 directory are empty:

2021:07:12 06:00:00       /opt/splunk/logs/20210712/app2      empty

2021:07:12 06:00:00       /opt/splunk/logs/20210712/app3      empty

Any idea>

Thanks

[venkatasri]

@indeed_2000 the date/app1/ directories technically does not exist in Splunk since they haven't been ingested. The query works only for current day, has to be scheduled to run every day once , earliest_time, latest_time should be with in current day. It gives you heads-up every day whether the new files being picked up or not meaning count = 0, other than that it can not show the source that you are expecting as it never exist in Splunk.