topic got 10k files forwarded but show show 300+ on indexer in Getting Data In

got 10k files forwarded but show show 300+ on indexer

SplunkCSIT — Wed, 09 Apr 2014 00:01:16 GMT

how come when i got 10K+ files at the forwarder but at the indexer show got 300+ files, what goes wrong and how to resolve this? Any solution would be greatly appreciated.

Re: got 10k files forwarded but show show 300+ on indexer

HiroshiSatoh — Wed, 09 Apr 2014 00:44:47 GMT

No warning?

index=_internal source=*splunkd.log component="fileclassifiermanager" message="*invalid*" host=XXX

Please change XXX.

Re: got 10k files forwarded but show show 300+ on indexer

SplunkCSIT — Thu, 17 Apr 2014 16:34:01 GMT

some of the files already indexed so it will not indexed again as the indexer will check for first 128bytes of each file whether if it indexed before. To resolve this is to put crcSalt at the inputs.conf to allow reindex if delete the index is not possible.