Knowledge Management
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

Why is summary indexing creating duplicate records in the summary index?

vganjare
Builder

Hi,

I am have the following definition for summary indexing:

[Test_Summary_Index]
action.summary_index = 1
action.summary_index._name = main
action.summary_index.report = summary_indexing_test
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = */5 * * * *
description = Test_Summary_Index
enableSched = 1
realtime_schedule = 0
search = | gentimes start=1 end=2

Ideally, after every 5 mins, only one event should get added to the summary index, but when I search over the report, I get duplicate records in return. I have two indexers for searching.

We are using Splunk version:6.1.4 and splunk build:233537

Are there any changes I need to make in the summary indexing definition to avoid the duplicate entries?

Thanks!!!

1 Solution

MuS
SplunkTrust
SplunkTrust

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

View solution in original post

brenthale
Explorer

This may be my problem. However, I don't have rights to the Splunk server to view the outputs.conf file. Is there a way via the UI to see the settings to determine if this is my problem or not? I've tried to look but nothing was obvious.

0 Karma

MuS
SplunkTrust
SplunkTrust

You can check it in the Splunk UI using the URI

http[s]://YourSplunkServer:YourPort/en-US/manager/launcher/data/outputs/

Obviously this will not be possible to do on an universal forwarder.

0 Karma

brenthale
Explorer

Just my luck...I'm on a "universal forwarder" and I get a big "Page not found!".

We have a big farm of Splunk Servers. They are managed by another team. I just want/need to be able to write valid reports. Is there any other way to determine this through the UI?

0 Karma

MuS
SplunkTrust
SplunkTrust

You could use curl to access the universal forwarder REST API and query the config like this:

curl -k -u admin:YourUFPassword https://YourUFIP:YourUFMGMTPort/servicesNS/nobody/search/configs/conf-outputs/
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!