Created DB Input stops getting updated data from database

[kpavan]

Hi Team,

I have created db input after creation for the first time it gets the data from db, but it doesn't get the updated entries. When go and check the input status will be enabled and its not disabling as well. But still there no updated entries coming in splunk.

Splunk Version = Splunk 6.4.1
DB Connect app version = 2.2.0

Below is my db inputs.conf, let me know what need to be changed for getting updated rows from the database.

[mi_input://MYINPUT]
connection = MYDB
index = MYINDEX
interval = 600
max_rows = 10000
mode = tail
output_timestamp_format = yyyy-MM-dd HH:mm:ss
query = SELECT * FROM TASKSESSION12_5 where state=128
source = MYSOURCE
sourcetype = MYSOURCETYPE
tail_follow_only = 1
tail_rising_column_name = TASKSESSIONID
ui_query_catalog = NULL
ui_query_mode = advanced
tail_rising_column_checkpoint_value = fffce111-ebe6434d-340cb67f-1128

Thanks!
Pavan

[jcoates_splunk]

I would set tail_follow_only to 0. This option was intended to emulate the behavior of Splunk UFs with syslog files, but it's just confusing in database tailing context and has been removed from the UI in 2.3.0.

[kpavan]

Hi,

I have updated db app version from 2.2.0 to 2.3.0, after that created test input to check issue. But this time got some error as below,

2016-08-16T04:56:33+0000 [CRITICAL] [rpcstart.py], line 378: action=rpc_server_has_been_abnormally_terminated error=log4j:WARN No appenders could be found for logger (com.zaxxer.hikari.HikariConfig).

inputs.conf

[rpcstart://default]
javahome = /usr/java/jre1.8.0_74
useSSL = 0

[mi_input://CA_IDM]
connection = IDM
enable_query_wrapping = 1
index = dm_db
input_timestamp_column_fullname = (003) NULL.CREATED_TIME.DATE
input_timestamp_column_name = CREATED_TIME
interval = 300
max_rows = 10000
mode = tail
output_timestamp_format = yyyy-MM-dd HH:mm:ss
query = SELECT * FROM TASKSESSION12_5 where state=128
source = idm_logs
sourcetype = IDM
tail_rising_column_fullname = (001) NULL.TASKSESSIONID.NVARCHAR2
tail_rising_column_name = TASKSESSIONID
ui_query_catalog = NULL
ui_query_mode = advanced
tail_rising_column_checkpoint_value = fffbda8d-2cb7b0ff-676e54f1-3d005c
disabled = 0

[kpavan]

does anyone has fix for this issue, please help me out on this?

Thanks!

[davebrooking]

If I were in your situation I'd increase the logging for the DB Connect app, as described in the troubleshooting section of the documentation.

If you don't see any clues in the logs I'd open a case with Splunk support. There's a link in the app on Splunkbase to 'File a case'

Dave

[kpavan]

Hi,

I have enabled debug and found below error, not sure how to fix, I just have simple sql query like
"SELECT * FROM "TASKSESSION12_5" where (STATE=128)". Could you please help me on this where to change and how to?

2016-08-29T07:48:04+0000 [CRITICAL] [mi_base.py], line 185: action=modular_input_exited_after_maximum_failed_retries modular_input=mi_input://MYINPUT max_retries=6 error=ERROR: java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
.
Traceback (most recent call last):
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/mi_base.py", line 173, in run
checkpoint_value=checkpoint_value)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/health_logger.py", line 278, in wrapper
return get_mdc(MDC_LOGGER).do_log(func, *args, **kwargs)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/health_logger.py", line 159, in do_log
return func(*args, **kwargs)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/mi_input.py", line 201, in run
_do_tail_mode(input_name, inputws, self.db, params, self.user_name, splunk_service, output_stream)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/mi_input.py", line 57, in _do_tail_mode
inputws.doTail(db, params, user, stanza, callback=callback)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/ws.py", line 281, in doTail
self.doInput("dbinputTailIterator", database, params, user, entityName, callback)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/ws.py", line 275, in doInput
self.ws.run_forever(timeout=self.timeout)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/websocket.py", line 841, in run_forever
self._callback(self.on_error, e)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/websocket.py", line 852, in _callback
callback(self, *args)
File "/data/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/ws.py", line 328, in on_error
raise Exception ("%s" % error)
Exception: ERROR: java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended

Thanks!

[alemarzu]

Hi there,

Did you try disabling enable_query_wrapping on your inputs.conf ?

[sjohnson_splunk]

Do you see any messages in the dbx2.log file?

[kpavan]

there is no error msg in log file, any specific need to be looked at in log file?