Why is there event duplication via TCP port?

patriziadepaola — Mon, 22 May 2017 09:36:54 GMT

Can anyone help me and clarify why Splunk duplicates events received from TCP port? The same type of events received on a UDP port are not duplicated.

I try to post an example:
Event received on UPD port 55553

{"CALLER_INFO":"{CORPORATE#010#NET43205#null#null}","INPUT":"{COMPANY#null#null#null#03978500720#null#null#null-null}","APPLICATION_SERVICE":"MAGNETO.EXTERNALSEARCH","ESITO":"OK","OUTPUT":"0","APPLICATION":"magneto","timestamp8601":"2017-05-18T09:07:02.400389+00:00","PID":"707633604","STEP":"TOTAL","program":"journal","CLASS":"LogUtil.traceStep","message":["2017-05-18T09:07:02.400389+00:00 prod-dcos6-12102016 journal: MSG;1501718321;707633604;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;18/05/2017 11:07:02,399;SPLUNK - magneto/externalsearch/magneto_externalsearch|TOTAL|OK|1301|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#null#03978500720#null#null#null-null}|0\n","MSG;1501718321;707633604;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;18/05/2017 11:07:02,399;SPLUNK - magneto/externalsearch/magneto_externalsearch|TOTAL|OK|1301|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#null#03978500720#null#null#null-null}|0\n"],"type":"rsyslog_produzione_dcos","logsource":"prod-dcos6-12102016","tags":["journal"],"SID":"1501718321","DATE":"18/05/2017 11:07:02,399","@timestamp":"2017-05-18T09:07:03.772Z","EXECUTION_TIME":1301,"@version":"1","LABEL":"SPLUNK","SERVICE":"externalsearch/magneto_externalsearch","LOGLEVEL":"MSG"}

Event received on TCP port 55555

{"CALLER_INFO":"{CORPORATE#010#NET43205#null#null}","INPUT":"1300013","APPLICATION_SERVICE":"MAGNETO.EXTERNALSEARCH","ESITO":"OK","OUTPUT":"AUTHORIZED","APPLICATION":"magneto","timestamp8601":"2017-05-19T12:28:45.940854+00:00","PID":"1528829935","STEP":"IS_AUTHORIZED_CONSUMPTION","program":"journal","CLASS":"LogUtil.traceStep","message":["2017-05-19T12:28:45.940854+00:00 prod-dcos6-12102016 journal: MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:45,940;SPLUNK - magneto/externalsearch/magneto_externalsearch|IS_AUTHORIZED_CONSUMPTION|OK|46|{CORPORATE#010#NET43205#null#null}|1300013|AUTHORIZED\n","MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:45,940;SPLUNK - magneto/externalsearch/magneto_externalsearch|IS_AUTHORIZED_CONSUMPTION|OK|46|{CORPORATE#010#NET43205#null#null}|1300013|AUTHORIZED\n"],"type":"rsyslog_produzione_dcos","logsource":"prod-dcos6-12102016","tags":["journal"],"SID":"1937509434","DATE":"19/05/2017 14:28:45,940","@timestamp":"2017-05-19T12:28:47.080Z","EXECUTION_TIME":46,"@version":"1","LABEL":"SPLUNK","SERVICE":"externalsearch/magneto_externalsearch","LOGLEVEL":"MSG"}{"CALLER_INFO":"{CORPORATE#010#NET43205#null#null}","INPUT":"{COMPANY#null#null#01893500890#null#null#null#null-null}","APPLICATION_SERVICE":"MAGNETO.EXTERNALSEARCH","ESITO":"OK","OUTPUT":"0","APPLICATION":"magneto","timestamp8601":"2017-05-19T12:28:46.449567+00:00","PID":"1528829935","STEP":"LIMINIRIS_REQUEST","program":"journal","CLASS":"LogUtil.traceStep","message":["2017-05-19T12:28:46.449567+00:00 prod-dcos6-12102016 journal: MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:46,448;SPLUNK - magneto/externalsearch/magneto_externalsearch|LIMINIRIS_REQUEST|OK|508|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#01893500890#null#null#null#null-null}|0\n","MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:46,448;SPLUNK - magneto/externalsearch/magneto_externalsearch|LIMINIRIS_REQUEST|OK|508|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#01893500890#null#null#null#null-null}|0\n"],"type":"rsyslog_produzione_dcos","logsource":"prod-dcos6-12102016","tags":["journal"],"SID":"1937509434","DATE":"19/05/2017 14:28:46,448","@timestamp":"2017-05-19T12:28:47.080Z","EXECUTION_TIME":508,"@version":"1","LABEL":"SPLUNK","SERVICE":"externalsearch/magneto_externalsearch","LOGLEVEL":"MSG"}{"CALLER_INFO":"{CORPORATE#010#NET43205#null#null}","INPUT":"{COMPANY#null#null#01893500890#null#null#null#null-null}","APPLICATION_SERVICE":"MAGNETO.EXTERNALSEARCH","ESITO":"OK","OUTPUT":"2137352876","APPLICATION":"magneto","timestamp8601":"2017-05-19T12:28:46.540997+00:00","PID":"1528829935","STEP":"BUILD_ACCOUNT","program":"journal","CLASS":"LogUtil.traceStep","message":["2017-05-19T12:28:46.540997+00:00 prod-dcos6-12102016 journal: MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:46,540;SPLUNK - magneto/externalsearch/magneto_externalsearch|BUILD_ACCOUNT|OK|91|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#01893500890#null#null#null#null-null}|2137352876\n","MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:46,540;SPLUNK - magneto/externalsearch/magneto_externalsearch|BUILD_ACCOUNT|OK|91|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#01893500890#null#null#null#null-null}|2137352876\n"],"type":"rsyslog_produzione_dcos","logsource":"prod-dcos6-12102016","tags":["journal"],"SID":"1937509434","DATE":"19/05/2017 14:28:46,540","@timestamp":"2017-05-19T12:28:47.100Z","EXECUTION_TIME":91,"@version":"1","LABEL":"SPLUNK","SERVICE":"externalsearch/magneto_externalsearch","LOGLEVEL":"MSG"}{"CALLER_INFO":"{CORPORATE#010#NET43205#null#null}","INPUT":"{COMPANY#null#null#01893500890#null#null#null#null-null}","APPLICATION_SERVICE":"MAGNETO.EXTERNALSEARCH","ESITO":"OK","OUTPUT":"0","APPLICATION":"magneto","timestamp8601":"2017-05-19T12:28:46.541236+00:00","PID":"1528829935","STEP":"TOTAL","program":"journal","CLASS":"LogUtil.traceStep","message":["2017-05-19T12:28:46.541236+00:00 prod-dcos6-12102016 journal: MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:46,540;SPLUNK - magneto/externalsearch/magneto_externalsearch|TOTAL|OK|647|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#01893500890#null#null#null#null-null}|0\n","MSG;1937509434;1528829935;MAGNETO.EXTERNALSEARCH;LogUtil.traceStep;19/05/2017 14:28:46,540;SPLUNK - magneto/externalsearch/magneto_externalsearch|TOTAL|OK|647|{CORPORATE#010#NET43205#null#null}|{COMPANY#null#null#01893500890#null#null#null#null-null}|0\n"],"type":"rsyslog_produzione_dcos","logsource":"prod-dcos6-12102016","tags":["journal"],"SID":"1937509434","DATE":"19/05/2017 14:28:46,540","@timestamp":"2017-05-19T12:28:47.112Z","EXECUTION_TIME":647,"@version":"1","LABEL":"SPLUNK","SERVICE":"externalsearch/magneto_externalsearch","LOGLEVEL":"MSG"}

Has anyone seen anything like it before?

Re: Why is there event duplication via TCP port?

DalJeanis — Mon, 22 May 2017 16:52:13 GMT

Those are not duplicates. Each chunk of the JSON has distinct attributes that are not identical to any other chunk.

The only thing you have to look at to verify I am correct is the number after "EXECUTION_TIME":. It is different in every block.

topic Why is there event duplication via TCP port? in Getting Data In

Why is there event duplication via TCP port?

Re: Why is there event duplication via TCP port?