Why is summary indexing creating duplicate records in the summary index?

[vganjare]

Hi,

I am have the following definition for summary indexing:

[Test_Summary_Index]
action.summary_index = 1
action.summary_index._name = main
action.summary_index.report = summary_indexing_test
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = */5 * * * *
description = Test_Summary_Index
enableSched = 1
realtime_schedule = 0
search = | gentimes start=1 end=2

Ideally, after every 5 mins, only one event should get added to the summary index, but when I search over the report, I get duplicate records in return. I have two indexers for searching.

We are using Splunk version:6.1.4 and splunk build:233537

Are there any changes I need to make in the summary indexing definition to avoid the duplicate entries?

Thanks!!!

[MuS · Accepted solution]

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

[brenthale]

This may be my problem. However, I don't have rights to the Splunk server to view the outputs.conf file. Is there a way via the UI to see the settings to determine if this is my problem or not? I've tried to look but nothing was obvious.

[MuS]

You can check it in the Splunk UI using the URI

http[s]://YourSplunkServer:YourPort/en-US/manager/launcher/data/outputs/

Obviously this will not be possible to do on an universal forwarder.

[brenthale]

Just my luck...I'm on a "universal forwarder" and I get a big "Page not found!".

We have a big farm of Splunk Servers. They are managed by another team. I just want/need to be able to write valid reports. Is there any other way to determine this through the UI?

[MuS]

You could use curl to access the universal forwarder REST API and query the config like this:

curl -k -u admin:YourUFPassword https://YourUFIP:YourUFMGMTPort/servicesNS/nobody/search/configs/conf-outputs/