All Apps and Add-ons

Splunk DB Connect: Error with details is empty, Slow index new data, Enscrypt/hass field before indexing

Path Finder

Hello Splunkers.

First of all, I'm sorry because my english is not good.

I am using Splunk DB Connect 2.4.1 on Splunk 7.2.6 and I got 3 problems need to help.

1. Error with detail is empty
Looking at _internal, I saw these following errors (error ="" )


2020-12-09T10:44:30+0700 [CRITICAL] [], line 61 : action=loading_input_data_failed input_mode=tail dbinput="mi_input://DATA" error=""
2020-12-09T10:44:30+0700 [CRITICAL] [], line 327: [DBInput Service] Exception encountered for entity-name = mi_input://DATA and type = input with error = .

2020-12-09T10:44:30+0700 [INFO] [], line 190: action=caught_exception_in_modular_input_with_retries modular_input=mi_input://DATA retrying="1 of 6" error=
Traceback (most recent call last):
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 183, in run
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 283, in wrapper
    return get_mdc(MDC_LOGGER).do_log(func, *args, **kwargs)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 160, in do_log
    return func(*args, **kwargs)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/", line 205, in run
    _do_tail_mode(input_name, inputws, self.db, params, self.user_name, splunk_service, output_stream)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/", line 57, in _do_tail_mode
    inputws.doTail(db, params, user, stanza, callback=callback)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 281, in doTail
    self.doInput("dbinputTailIterator", database, params, user, entityName, callback)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 275, in doInput
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 841, in run_forever
    self._callback(self.on_error, e)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 852, in _callback
    callback(self, *args)
  File "/u01/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/", line 328, in on_error
    raise Exception ("%s" % error)


Have anyone seen this before? why it happened and how to fix them.

2. Database input job works very slow.

For some time, everything was OK, but all of sudden Splunk stopped indexing new data.

Looking at _internal, I saw only this one message:


2020-12-09T10:44:33+0700 [INFO] [], line 193: action=start_executing_dbinput dbinput="mi_input://DATA"


In case job works fine, it will have many massages like below:


2020-12-09T10:44:33+0700 [INFO] [], line 193: action=start_executing_dbinput dbinput="mi_input://DATA"

2020-12-09T10:44:29+0700 [INFO] [], line 113: action=print_csv_from_jdbc_to_event_stream dbinput="mi_input://DATA" input_mode=tail events=300
2020-12-09T10:44:29+0700 [INFO] [], line 109: action=rising_column_checkpoint_updated dbinput="mi_input://DATA" checkpoint=8068170343

2020-12-09T10:45:52+0700 [INFO] [], line 193: action=complete_dbinput dbinput="mi_input://DATA"


I tried to check by query on database connect app interface, result very fast. So i think, database input job got problem.

3. Enscrypt/hass field before indexing

I am using Splunk DB Connect 2.4.1 on Splunk 7.2.6.

Some fields of data are case sentitive, ex: card_number. So i edited code in  file in the DB connect app, it will hass card_number field to new field called hass_number. It work fine.

With Splunk DB Connect 3.x version. I cant enscrypt data field by the that way because the DB connect 3.x use java and python 3. It very different than 2.4.1 version. So is there any way to encrypt a data field before splunk indexing?

Thanks in advance.

0 Karma
1 Solution

Path Finder


I resolved enscrypt/hass field before indexing by this way:

INGEST_EVAL = CARD_HASH=upper(sha256(replace(_raw, "^(.*)credit_card=\"(.*)\"$", "\2"))), _raw=replace(_raw, "^(.*credit_card)=(.*)$", "\1=".CARD_HASH)

View solution in original post

Tags (1)
0 Karma

Path Finder


I resolved enscrypt/hass field before indexing by this way:

INGEST_EVAL = CARD_HASH=upper(sha256(replace(_raw, "^(.*)credit_card=\"(.*)\"$", "\2"))), _raw=replace(_raw, "^(.*credit_card)=(.*)$", "\1=".CARD_HASH)

Tags (1)
0 Karma


@dailv1808 , I would upgrade DBConnect3 app to the latest supported version. DBConnect 3 works much faster and stable than DBConnect2. This may resolve errors and slowness. 

Regarding card numbers you can use SEDCMD on props.conf to anonymize data. Of course you are limited by Regex SED capabilities. You can find details on below doc page. 

If this reply helps you an upvote is appreciated.
0 Karma

Path Finder

Thanks your response @scelikok 

I very want to update to DBConnect 3 but I need to handle the card number encryption first.

Using SEDCMD on props.conf look like masking function, not hashing. I need hashing instead of masking at index time.

Using a hash function (like MD5 or SHA) the values instead would fully protect the original value from being discovered, and still allows for accurate grouping and/or transaction operations on the sensitive field.

0 Karma
Get Updates on the Splunk Community!

Security Highlights | November 2022 Newsletter

 November 2022 2022 Gartner Magic Quadrant for SIEM: Splunk Named a Leader for the 9th Year in a RowSplunk is ...

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...