Thanks EmEdwards, that did the trick. ... View more

I think so, for now at least. ... View more

Ad I described, if you don't use crcSalt option Splunk doesn't read files after rotation . Bye Giuseppe ... View more

Hi Giuseppe, Thanks for the reply. The index is configured as per the default for Splunk, max size of 500Gb and don't believe there is a limit on retention. The files are on the indexer, I've done the search many days after the original import and has not changed. Probably should have tried the crSalt option. The weird thing is there is data before and after the missing sections, very weird, I initially thought it may have been something to do with the original source files maybe not being formated quite right or something but I looked at ones that worked and ones that didn't and couldn't really find a problem with them. At the moment I'm dropping one file into the monitored folder every minute and it appears to be picking up everything, just have to wait for it catch finished dropping in another 6 years worth of files to finish. Tony ... View more

Just wondering if you ever got a satisfactory answer to this? I have the same problem but with TCP forwarder: [tcpout] defaultgroup = logstash disabled = false forwardedindex.0.whitelist = .* forwardedindex.1.blacklist = _.* forwardedindex.2.blacklist = (_audit|_internal|_introspection) [tcpout:logstash] server=localhost:7777 sendCookedData = false useACK = true Seeing loads of messages like: INFO Metrics - group=thruput, name=uncooked_output, instantaneous_kbps=0.176377, instantaneous_eps=0.096773, average_kbps=0.355449, total_k_processed=44.000000, kb=5.467773, ev=3.000000 INFO Metrics - group=thruput, name=thruput, instantaneous_kbps=0.176377, instantaneous_eps=0.096773, average_kbps=0.371606, total_k_processed=46.000000, kb=5.467773, ev=3.000000, load_average=0.030000 INFO Metrics - group=tcpout_connections, name=logstash:127.0.0.1:7777:0, sourcePort=8090, destIp=127.0.0.1, destPort=7777, _tcp_Bps=186.73, _tcp_KBps=0.18, _tcp_avg_thruput=0.39, _tcp_Kprocessed=46, _tcp_eps=0.10, kb=5.47 ... View more