Getting Data In

Source does not show up in search

chaseleechun
Explorer

I added a directory with 5 files, but the search only return events from 2 files.

Some background:

  1. Added the 5 individual files with default sourcetypes.
  2. Added the dir with the 5 files with a manual sourcetype.
  3. Use | delete to remove the earlier added files in (1).
  4. Now the search only return events from 2 files.

I used "splunk list monitor" and it shows that all 5 files are being monitored.

I used "| metadata type=sourcetypes" and the results show the "totalCount" of the 2 files only. (Note: the previously added files in (1) had 0 totalCount)

Also, when I disable or delete the Data Input (2), the results from the 2 files will still be shown.

Can anyone explain the behaviour or what I should have / not have done? How can I 're-add' the 5 files?

Tags (2)
0 Karma
1 Solution

chaseleechun
Explorer

Found the answer in the inputs.conf setting. http://www.splunk.com/base/Documentation/latest/admin/Inputsconf

crcSalt = <string>
* Use this setting to force Splunk to consume files that have matching CRCs (cyclic redundancy checks). (Splunk only 
  performs CRC checks against the first few lines of a file. This behavior prevents Splunk from indexing the same 
  file twice, even though you may have renamed it -- as, for example, with rolling log files. However, because the 
  CRC is based on only the first few lines of the file, it is possible for legitimately different files to have 
  matching CRCs, particularly if they have identical headers.)
* If set, <string> is added to the CRC.
* If set to the literal string <SOURCE> (including the angle brackets), the full directory path to the source file 
  is added to the CRC. This ensures that each file being monitored has a unique CRC.   When crcSalt is invoked, 
  it is usually set to <SOURCE>.
* Be cautious about using this attribute with rolling log files; it could lead to the log file being re-indexed 
  after it has rolled. 
* Defaults to empty. 

View solution in original post

0 Karma

chaseleechun
Explorer

Found the answer in the inputs.conf setting. http://www.splunk.com/base/Documentation/latest/admin/Inputsconf

crcSalt = <string>
* Use this setting to force Splunk to consume files that have matching CRCs (cyclic redundancy checks). (Splunk only 
  performs CRC checks against the first few lines of a file. This behavior prevents Splunk from indexing the same 
  file twice, even though you may have renamed it -- as, for example, with rolling log files. However, because the 
  CRC is based on only the first few lines of the file, it is possible for legitimately different files to have 
  matching CRCs, particularly if they have identical headers.)
* If set, <string> is added to the CRC.
* If set to the literal string <SOURCE> (including the angle brackets), the full directory path to the source file 
  is added to the CRC. This ensures that each file being monitored has a unique CRC.   When crcSalt is invoked, 
  it is usually set to <SOURCE>.
* Be cautious about using this attribute with rolling log files; it could lead to the log file being re-indexed 
  after it has rolled. 
* Defaults to empty. 
0 Karma

chaseleechun
Explorer

I read somewhere that previously indexed files may clash with the newly uploaded files if they happen to be the same files (content). Could this be the case here, even though I have deleted the earlier indexed files?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...