Double counts in search app

[rgeddes]

basic set up:
- splunk 4.2 on ubuntu 10.04
- rsyslog collects logs from other machines, and splunk reads and tabulates event data from /var/log/*

I noticed that a cron.hourly process which normally generates 2 events/hour in the search app, jumped up to 4 events/hour(which were duplicates), for a 24 hour period, and then returned to 2 events/hour.

grepping /var/log/syslog for that particular event does not show duplicates, and when I ask the search app to show the source, the source does not show duplicates either.

Has anyone experienced double counting before? If yes, how did you resolve?

Thanks
Richard

[rgeddes]

I just realized that the duplicate entries are tagged with different sources.

One came from /var/log/syslog, and the other came from /var/log/syslog.2.gz

I grepped though /var/log/syslogs*, and can only find one pair of events with a particular process id... which splunk search app shows as happening twice with the exact same timestamp. The duplication lasts for a 24 hour period.

I was thinking that maybe log rotation created a condition to allow this, but the duplication continues for 24 hours.... far longer than it takes to rotate the logs.

Anyway, still not resolved, but extra information.

R