How to detect duplicate GUIDs on forwarders?

[davidpaper]

There are a number of posts on how to fix duplicate GUIDs on FWDs (https://answers.splunk.com/answers/32368/duplicate-guids-for-cloned-forwarders-how-to-correct.html) but there are none for how to detect them to begin with.

How do I find duplicate GUIDs on forwarders?

[johnlee2327]

I think the SPL and the concept of Davidpaper are good. But for some reason, in my environment, it will not be able to reveal all the hidden problems. My SPL is as follows, I hope people with the same problem can give it a try. It fix my problem. 

index=_internal ClientSessionsManager
| stats dc(hostname), values(hostname) by instanceId
| sort - dc(hostname)

[cbright]

Sorry I know this is an older post but I was looking to do this as well and came up with this search that will show you any clients with the same GUID and the same name and only display multiples:

index=_internal|dedup sourceIp
| search NOT [| inputlookup dmc_assets | dedup serverName | rename serverName as hostname | fields hostname]
| stats count(guid) AS dupguid, values(hostname), values(sourceIp) by guid|search dupguid > 1

[davidpaper · Accepted solution]

The DMC gets us most of the way there with the Forwarder management views. Some tweaking gets us here.

index=_internal | dedup hostname
| search NOT [| inputlookup dmc_assets | dedup serverName | rename serverName as hostname | fields hostname]
| stats count(guid), values(hostname) by guid | sort - count(guid)

[acharlieh]

Another interesting statistic along these lines from that same Metrics log may be values(sourceIp) which I wonder if the assets table would help you or not...

[dwaddle]

There's also a tell-tale message at deployment server where you see something like "GUID XXX-XXX-XX has changed attributes to " .. I'll have to find one

[thebeno]

Hi I am using something like this:

index=_internal AND sourcetype=splunkd 
AND TERM(New) 
AND TERM(Old) 
AND TERM(properties)
AND TERM(are)

| rex field=_raw "Old properties are: \{ip=(?<old_ip>[^\ ,]*)[\ ,]+dns=(?<old_dns>[^\ ,]*)[\ ,]+hostname=(?<old_hostname>[^\ ,]*)[\ ,]+deploymentClientName=(?<old_deploymentclientname>[^\ ,]*)[\ ,]+.*instanceId=(?<old_instanceid>[^\ ,]*)[\ ,]+instanceName=(?<old_instancename>[^\ ,\}]*)"
| rex field=_raw "New properties are: \{ip=(?<new_ip>[^\ ,]*)[\ ,]+dns=(?<new_dns>[^\ ,]*)[\ ,]+hostname=(?<new_hostname>[^\ ,]*)[\ ,]+deploymentClientName=(?<new_deploymentclientname>[^\ ,]*)[\ ,]+.*instanceId=(?<new_instanceid>[^\ ,]*)[\ ,]+instanceName=(?<new_instancename>[^\ ,\}]*)"

| stats latest(_time) AS ltm, count BY new_hostname, old_hostname, new_dns, old_dns, new_instancename, old_instancename, new_instanceid, old_instanceid, new_ip, old_ip
| convert timeformat="%y-%m-%d" ctime(ltm) AS LastDay
| fields - ltm