Splunk Search

Why is the configuration not applied immediately when a knowledge object is created?

schose
Builder

Hi forum,

I'm currently fighting with an installation of a Searchhead. When a Knowledge Object is created the configuration takes "a while" before it is applied. the behavior is reproducable by e.g.:

index=_internal sourcetype=splunkd |
head 20

create a setting in props.conf

[splunkd]
eval-test1="test1"

and apply the configuration by running http://localhost:8000/en-US/debug/refresh the configuration needs about 1-3min. to be applied. Rerun the search doesn't show me the field test1.

In a out-of-the-box installation the next search immediatly show me the field after applying the configuration.

/opt/splunk/bin/splunk btool server list --debug |grep -i interval
/opt/splunk/etc/system/default/server.conf                             generation_poll_interval = 5
/opt/splunk/etc/system/default/server.conf                             service_interval = 1
/opt/splunk/etc/system/default/server.conf                             report_interval = 1m
/opt/splunk/etc/system/default/server.conf                             poll.interval.check = 1m
/opt/splunk/etc/system/default/server.conf                             poll.interval.rebuild = 1m
/opt/splunk/etc/system/default/server.conf                             sampling_interval = 1s
/opt/splunk/etc/system/default/server.conf                             sampling_interval = 1s
/opt/splunk/etc/system/default/server.conf                             sampling_interval = 1s
/opt/splunk/etc/system/default/server.conf                             sampling_interval = 1s
/opt/splunk/etc/system/default/server.conf                             sampling_interval = 1s
/opt/splunk/etc/system/default/server.conf                             sampling_interval = 1s

Any hints? Using v6.3.3.

Thanks for you help in advance!

Andreas

0 Karma

dshpritz
SplunkTrust
SplunkTrust

If you are in a distributed environment, then there will usually be a delay while Splunk creates and replicates the config bundles. More info on that here:
http://docs.splunk.com/Documentation/Splunk/6.3.3/DistSearch/Whatsearchheadssend

Something that you can do to work around this is to use the extract command:

index=myindex sourcetype=mysourcetype | extract reload=true

More on extract here:
http://docs.splunk.com/Documentation/Splunk/6.3.3/SearchReference/Extract

HTH,

Dave

dshpritz
SplunkTrust
SplunkTrust

A good point brought up by @acharlieh is that in your example you have:

[splunkd]
eval-test1="test1"

But the attribute names are case-sensitive, so you should have:

[splunkd]
EVAL-test1="test1"

DMohn
Motivator

Have you checked that 'rerunning the search' actually creates a new search, or just refreshes the existing one?

Check the SID via the job inspector to make sure you are creating a new search, which will then pull the new configuration.

Already run searches will remain valid for a while, even after a refresh.

schose
Builder

Hi,

I see new increasing SID for every search. doing nothing for about 5 min solves the issue, but isn't a solution. 😕

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...