How can I identify which heavy forwarder or universal forwarder are sending logs?

[germancho88]

Hi, 

I have a problem in my infrastructure the logs are being duplicated, I am trying to identify from which origin (HF, UF, or Syslog) the logs are being sent, worse I have not been successful, any search ideas that can identify the origin that sent it , Thanks

[germancho88]

Thanks for your answer, at the moment of performing the search it is identified that logs are being duplicated by host, that host is the HF or UF or the source device that generated the event?, in that part I am confused

[gcusello]

Hi @germancho88,

At first you have to understand what means duplicated logs, have you:

• the same log event and the same metadata (host, sourcetype, index, source)?
• the same log but different metadata

Knowing this, you have to understand if you receive the same log from one or more hosts.

In both cases you can list the hosts with duplicated logs and check on the Deployment Server is they are OF or HF, if you have logs from syslog or HEC, probably they are HF, otherwise UF.

Anyway, if you're sure that the full logs is duplicated, you could run something like this:

index=your_index
| stats values(host) AS host values(sourcetype) As sourcetype values(source) AS source values(index) As index count BY _raw
| where count>1

In this way you can understand what's the situation and you can debug your problem.

Generally, causes of duplicated logs could be:

• clustered appliances or servers,
• errors in inputs configurations.

Ciao.

Giuseppe

[germancho88]

Thanks for your answer, at the moment of performing the search it is identified that logs are being duplicated by host, that host is the HF or UF or the source device that generated the event?, in that part I am confused

[gcusello]

Hi @germancho88,

to better understand you need:

are you speaking of logs from syslog or from servers with UF?

for this reason, the first check is to understand the metadata of your duplicated logs:

if you have more than one host, you have a cluster that send you logs twice;

Anyway, HFs only ingest own logs and syslogs and HEC, so if your duplicated logs aren't one of them, probably they come from UF, but anyway, first step is identify hosts with duplicated logs, then you can investigate to understand why.

Ciao.

Giuseppe