topic Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution? in All Apps and Add-ons

Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Wed, 26 Oct 2016 20:49:28 GMT

I work in a dynamic environment and often servers will get restarted for various reasons, updates for example..

Often I'll miss these events because I'm not on-call or nobody tells me. What happens is Splunk DB Connect disables all the inputs for that server and never tries to re-enable them.

So that data stops flowing into Splunk until someone manually re-enables the inputs for the servers in question.

My question is: Is there any way to have auto-retry logic? Or is there something I can use to monitor for inputs that get disabled? Right now I simply can't trust the data going into it because of this.

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

ppablo — Wed, 26 Oct 2016 21:06:12 GMT

Hi @johnpof

There's a previous Answers post similar to this topic with an answer that suggests disabling the auto_disable setting in inputs.conf. That might solve your issue:
https://answers.splunk.com/answers/438527/how-to-prevent-splunk-db-connect-2-from-disabling.html

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Wed, 26 Oct 2016 23:58:24 GMT

Nice! that's perfect. Can I update inputs.conf live and will the changes immediately be reflected?

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

ppablo — Thu, 27 Oct 2016 00:39:40 GMT

Hm, I'm not positive about that actually. I just remembered coming by that previous question before and it came to mind when I saw yours.

From looking at the inputs.conf spec for Splunk Enterprise docs, it says Splunk has to be restarted for config changes to take effect:
http://docs.splunk.com/Documentation/Splunk/6.5.0/admin/Inputsconf

I don't see that specifically in the DB Connect inputs.conf docs, but I would assume that a restart is still required after any changes:
http://docs.splunk.com/Documentation/DBX/2.3.0/DeployDBX/inputsspec

Sorry I don't have a definitive answer, but hopefully someone who does have experience with this can confirm.

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Thu, 27 Oct 2016 00:49:26 GMT

I'll probably restart it and see what happens

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Thu, 27 Oct 2016 16:37:23 GMT

For future reference, I used this handy line to add auto_disable = False right after all occurrences of disabled = 0

sed -i $'s/disabled = 0/disabled = 0\\\nauto_disable = False/g' inputs.conf

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

ppablo — Thu, 27 Oct 2016 17:26:22 GMT

Thanks for sharing that @johnpof 🙂 Cheers!

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Thu, 03 Nov 2016 02:36:30 GMT

So I came in this morning after a major maintenance, every single input was disabled.

so unfortunately this isn't working 😕 does it matter where you put the auto_disable = False?

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

ppablo — Wed, 16 Nov 2016 18:53:56 GMT

Hi @johnpof

Sorry I just saw your follow up comment. Did you happen to figure it out? I noticed the value you used for auto_disable was "False", not "false" as shown in the documentation. I'm not sure if the value is case sensitive.

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Wed, 16 Nov 2016 18:55:59 GMT

I noticed that too actually just yesterday and replaced everything 😛

I bet it'll be fine now, thanks for follow up.

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

koshyk — Fri, 09 Dec 2016 23:01:44 GMT

can u try updating the inputs.conf using REST API. if you can find it, then it doesn't require restart hopefully

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

hemendralodhi — Tue, 29 Sep 2020 13:11:48 GMT

We have SHC environment and even after setting auto_disable = false for input it is getting disabled whenever SHC gets restarted? How to prevent it from disabling then? Here is my configuration:

[mi_input://Fusion_DG_Lag]
connection = Fusion_Common_DB_PRD
description = Lag between Primary and Standby DB
enable_query_wrapping = 1
index = index
input_timestamp_column_fullname = (003) NULL.TIME_COMPUTED.VARCHAR2
input_timestamp_column_name =
interval = 120
max_rows = 10000
mode = batch
output_timestamp_format = dd-MM-yyyy HH:mm:ss
query = SELECT name,\
((SUBSTR(value,2,2)*86400)+(SUBSTR(value,5,2)*3600)+(SUBSTR(value,8,2)*60)+(SUBSTR(value,11,2))) lag_time,\
time_computed\
FROM V$DATAGUARD_STATS\
WHERE name IN ('apply lag','transport lag')
source = fusion_common_db_prd
sourcetype = fusion_db
tail_rising_column_fullname = (003) NULL.TIME_COMPUTED.VARCHAR2
tail_rising_column_name = TIME_COMPUTED
ui_query_catalog = NULL
ui_query_mode = advanced
auto_disable = false

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

johnpof — Fri, 10 Mar 2017 18:30:26 GMT

I'm pretty sure DBX 2.0 is not supported in an SHC environment, I had all sorts of problems with it and eventually moved it to a standalone search head by itself outside of my SHC.

Re: Splunk DB Connect: Inputs get disabled after server restarts. Is there a way to automatically re-enable the inputs or another solution?

hemendralodhi — Mon, 13 Mar 2017 00:17:02 GMT

I am running V 2.3.1 which support SHC but it is in documentation that running input/output is not recommended in SHC. I overlooked this and probably need to run it in standalone instance.