can't find the file (source)

ypfbkg — Thu, 01 Dec 2011 03:21:13 GMT

i set a input (directory) and i use command "splunk list monitor"
splunk list monitor command result:
\\aaasvr\iis-pic\PM\PMLog
\\aaasvr\iis-pic\PM\PMLog\PMLog.txt
\\aaasvr\iis-pic\PM\PMLog\PMLog.txt20111126
\\aaasvr\iis-pic\PM\PMLog\PMLog.txt20111128
\\aaasvr\iis-pic\PM\PMLog\PMLog20111128.txt

but in my source data, only one file (\\aaasvr\iis-pic\PM\PMLog\PMLog.txt) be detected.

why jsut one file be detected ? hwo can i do ? anyone can help me.

this is my splunk source & command result
Link

Thanks.
Finley

Re: can't find the file (source)

Takajian — Thu, 01 Dec 2011 07:25:14 GMT

How those file are created? Did you just copy them from original one? Are those file header is same? If yes, it could be problem for splunk to index data.
Because splunk monitoring processor picks up new files and reads the first and last 256 bytes of the file, then
the data is hashed into a begin and end cyclic redundancy check (CRC).
Splunk checks new CRCs against a database that contains all the CRCs of files Splunk has seen before.
If those file header is same, CRC will be same , so Splunk can not detect them as new file.

To avoid this issue, you can use following parameter in inputs.conf.

--------------------------------------------------------------------------------------------------------
crcSalt =
* Use this setting to force Splunk to consume files that have matching CRCs (cyclic redundancy checks). (Splunk only
performs CRC checks against the first few lines of a file. This behavior prevents Splunk from indexing the same
file twice, even though you may have renamed it -- as, for example, with rolling log files. However, because the
CRC is based on only the first few lines of the file, it is possible for legitimately different files to have
matching CRCs, particularly if they have identical headers.)
* If set, is added to the CRC.
* If set to the literal string (including the angle brackets), the full directory path to the source file
is added to the CRC. This ensures that each file being monitored has a unique CRC. When crcSalt is invoked,
it is usually set to .
* Be cautious about using this attribute with rolling log files; it could lead to the log file being re-indexed
after it has rolled.
* Defaults to empty.

Re: can't find the file (source)

ypfbkg — Wed, 07 Dec 2011 06:59:55 GMT

Takajian Thanks ^^

topic Re: can't find the file (source) in Getting Data In

can't find the file (source)

Re: can't find the file (source)

Re: can't find the file (source)