Is "RecordNumber" a unique identifier?

[michaeler]

I'm dealing with a lot of duplicate event logs at the exact same millisecond. From what I can tell, everytime this happens, the events in my search results have all of the same data. There are some events that follow with slightly different fields including a ascending "Record Number" count.

Should the record number be different for every log? 

I'm looking for a better way to identify duplicate data and stop logging it. In the past 16 hours, I've logged 550 events with different record numbers but 390,000 different events. Some record numbers repeating up to 600 times.

If you have a solution for this I'd appreciate it, but also just looking for a Unique ID to confirm this is an issue.

[richgalloway]

"Record Number" is not a built-in Splunk field so it's not a unique identifier by default.  Someone familiar with your data source would be the one to answer your questions.

Take a look at how the data is brought into Splunk.  Is there an input running on all indexers or search heads?  If so, that might explain some of the duplication.  Scripts used to ingest data must run in a single place or you risk duplicate events.