Getting Data In

How can I view the names of files that have 0 KB of data?

Motivator

Hi

I have lots of file in a directory, some with data some with no data.

If i understand correctly Splunk will forward on files only with data, however this is an issues for me as i parse the name and i use it.
So for example 535211-11233-service.log. I take out the value 11233 and use it.
In this case i also need the value 11235 on the 0KB file, but as the forwarder is not sending it i cant access it.

So how can i get the forwarder to send the file with 0KB?

535211-11233-service.log 10kb
535613-11234-service.log 100kb
535614-11235-service.log 0kb
etc...

Thanks in advance
Robert

0 Karma
1 Solution

Esteemed Legend

Splunk did monitor the file, but it never obtained any events so no events could be forwarded. What you can do is set up a cron like this on that server:

01 23 * * * find /Your/Directory/To/Files/Here -type f -name "log*.txt" -size 0 -mtime +1 | xargs -i sh -c "echo 'This file will always be empty!' >> '{}' "

Be sure to modify the cron time, the start directory, the -name details, and the -mtime value for your situation. This will cause empty files to obtain a fake event following the rules that you give it.

View solution in original post

0 Karma

Esteemed Legend

Splunk did monitor the file, but it never obtained any events so no events could be forwarded. What you can do is set up a cron like this on that server:

01 23 * * * find /Your/Directory/To/Files/Here -type f -name "log*.txt" -size 0 -mtime +1 | xargs -i sh -c "echo 'This file will always be empty!' >> '{}' "

Be sure to modify the cron time, the start directory, the -name details, and the -mtime value for your situation. This will cause empty files to obtain a fake event following the rules that you give it.

View solution in original post

0 Karma

Motivator

Hi

Thanks for this, Ideally i would love to use the forwarder (Or have the option).
However i understand why it cant, but it means i have to set this cron up on multiple environments on multiple machines etc...

My argument would be there is data in the file name, just not in the file.

I
in the end we are going to use this.

find logs/mxsession -name "*-*service.log" | xargs ls -ltr | grep "murex     0" | awk '{print $9}' > ListOfFileToFill.txt
while read file ; do
DATE=`date  +%Y-%m-%d`
TIME=`date  +%H:%M:%S`
echo "$DATE $TIME [QCST] adding any line" >> $file
done < ListOfFileToFill.txt 
0 Karma

SplunkTrust
SplunkTrust

Okay, shooting from the hip here. If there is no data to index, Splunk can't index it. Not that it doesn't see it and so forth, but it is looking for data IN the file to index. So it did index the file, but you just have no data associated with the file. The file is in the fishbucket on the forwarder. So you have a problem of how to get information about the file into Splunk.

This is one way that you can solve your dilemma. I suspect that you say you need to know about the file because you need to know about the name of the file (that it exists) in this case, and not the data that is or isn't in the file. That is my assumption for the next section.

There are a couple of approaches that you can take, The first is to run a script regularly on your host (I don't know if you are using Linux or windows) that does a listing of the directory and outputs the result to a file. Then you index that file. You then have the contents of the directory that shows the files that are in it, including your zero length ones.

The second would be to do something similar (file listing), but doing it the same way the *nix and Windows TA apps do (they just run the command and send that output regularly to the indexers).

Does either of these approaches satisfy your need?

0 Karma