heavy forwarder send a data twice

[moonyoungjung]

heavy forwarder used TCP method sends a data to the third party

After processing data shown duplicate phenomenon

That phenomenon usually is shown when creating the file

I was set up the conf like this

inputs.conf

[monitor:///xxx/header01/*]
disabled = 0
index = xxx02
sourcetype = xxx002
whitelist = \d \-\d \-\d 
blacklist = .*\.swp
ignoreOlderThan = 1d
crcSalt = 
_TCP_ROUTING = pre_server_group2

outputs.conf

[tcpout:pre_server_group2]
disabled = false
sendCookedData = false
useACK = false
server = x.x.x.x:19000
maxQueueSize =100MB

and the original raw file name is:

2019-01-22-10
2019-01-22-11
2019-01-22-12
2019-01-22-13

why heavy forwarder send twice?

[nickhills]

Interesting problem - my initial thought was that perhaps you had useAck enabled, however you config suggests this is not the case.

with useAck disabled, Splunk writes the data to the network socket and then forgets about it - this (in theory) means that its impossible for Splunk to send duplicate events, unless:

1.) The source data is itself duplicated
2.) Your TCP routing matches more than one route, and it gets routed twice (albeit to the same destination)

Have you looked at the index time for the duplicated data - were they received at the same instant, and by the same server or was there some delta between each copy?

Also, whats the intent of:

crcSalt = 

Is that a typo? - (You should use the code formatter when including things which look like HTML)
You probably want this set to:

crcSalt = <SOURCE>

If you need it