Knowledge Management

Why is summary indexing creating duplicate records in the summary index?

vganjare
Builder

Hi,

I am have the following definition for summary indexing:

[Test_Summary_Index]
action.summary_index = 1
action.summary_index._name = main
action.summary_index.report = summary_indexing_test
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = */5 * * * *
description = Test_Summary_Index
enableSched = 1
realtime_schedule = 0
search = | gentimes start=1 end=2

Ideally, after every 5 mins, only one event should get added to the summary index, but when I search over the report, I get duplicate records in return. I have two indexers for searching.

We are using Splunk version:6.1.4 and splunk build:233537

Are there any changes I need to make in the summary indexing definition to avoid the duplicate entries?

Thanks!!!

1 Solution

MuS
SplunkTrust
SplunkTrust

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

brenthale
Explorer

This may be my problem. However, I don't have rights to the Splunk server to view the outputs.conf file. Is there a way via the UI to see the settings to determine if this is my problem or not? I've tried to look but nothing was obvious.

0 Karma

MuS
SplunkTrust
SplunkTrust

You can check it in the Splunk UI using the URI

http[s]://YourSplunkServer:YourPort/en-US/manager/launcher/data/outputs/

Obviously this will not be possible to do on an universal forwarder.

0 Karma

brenthale
Explorer

Just my luck...I'm on a "universal forwarder" and I get a big "Page not found!".

We have a big farm of Splunk Servers. They are managed by another team. I just want/need to be able to write valid reports. Is there any other way to determine this through the UI?

0 Karma

MuS
SplunkTrust
SplunkTrust

You could use curl to access the universal forwarder REST API and query the config like this:

curl -k -u admin:YourUFPassword https://YourUFIP:YourUFMGMTPort/servicesNS/nobody/search/configs/conf-outputs/
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...