Splunk DBConnect is not creating sourcetype

[dhirendra761]

Hi All,

I configured Splunk DB Connect app.
I tried to create new sourcetype in my db connection metadata. While checking "find events" , I am getting "0" events . Because of sourcetype was not created.

Also When I'm creating a new input for my dbconnection i can't select existing source-types. Even more, when I'm trying to create new one it doesn't show up in list of all source-types. Hence I am not able to create sourcetype for my db connect input.

Please help me on this.

[zhzh]

I have a same question here. I didn't unistall current DB connect and solve it.

The true problem is not with the Source Type or index. When you click 'save' button, the search is not generated instantly if your Execution Frequency is set at a specific point of time (for me it is 0 9 * * * (every day at 9 am)). Your Source Type will not be created unitll then.

Change the Execution Frequency to 60 (every 60 seconds) solve my question.

I'm new to splunk and just record my situation here.
Thanks.

[dhirendra761 · Accepted solution]

Hi All,

I found that the dbconnect 3.1.3 has some issue with only Winodws Server 2012 and 2016 OS.
I am getting error in dbx log files and not able to find any events from DB.
I have resolved the issue by uninstalling dbconnect 3.1.3 and install back to its previous version dbconnect 3.1.2.

And Its working fine.
Anyone having the same issue and need the previous version. Please let me know.

Thanks.

[bogdan_nicolesc]

Hi dhirendra761,

YES! Please, can you please give me bdconnect 3.1.2?

Thank you,
Bogdan.

[dhirendra761]

@bogdan_nicolescu Please share your email id,

Thanks

[harsmarvania57]

Hi @dhirendra761,

Find Events options try to search event indexed locally, if you are running DB Connect on Heavy Forwarder and sending data to Indexer, in this case when you click on Find events on Heavy Forwarder it tries to search data locally however Heavy Forwarder doesn't have any data (Because it is sending data to Indexer to store the data) so that it will give you No results.

On local splunk instance you have data available and you are running DB connect on same splunk instance so in that case you'll able to see those data when you click on Find Events.

[dhirendra761]

Hi Harsh,

Thanks for suggestion. I think my issue is with sourcetype and index. The data is not binding with sourcetype and index because it is not created.
Every time when i edit the input in DB connect it doen't show me in list (please refer above SS).

Also in "DATA SUMMARY" I am not able to locate my created sourcetype.
This problem is occurs only "Windows Server 2012 R2" and it is working fine in windows 7.
Is there is any possibility for different OS as well.

Thanks

[harsmarvania57]

As I mentioned earlier, while supplying sourcetype or index in DB Connect Input will not create those sourcetype and index configuration in Splunk.

For example: If you'll give index as test in Db Connect input and if that index is not present on Indexer then Splunk will not create that index, you need to create test index on indexer separately.

What I'll suggest is if you have test environment then first try to index data in main index and if it works then create custom index and supply that custom index in Db Connect Input.

[dhirendra761]

Hi Harsh,

I followed you appoarch. I created "dbIndex" index manually. then I create "testDB" sourcetype in dbinput setting. It doesn't still display any events. And while editing input It doesn't show in the list of sourcetype .

let me know If i missed something.

link text
https://ibb.co/YPhxHR3
https://ibb.co/tQMvR4Z

[harsmarvania57]

Does dbIndex Index exist on Indexer ? On which splunk instance DB Connect is installed (Indexer or Heavy Forwarder) ? Are you able to send data in main index for testing purpose (If it is test environment) ?

[dhirendra761]

Hi @harsmarvania57 ,

" dbIndex" is my custom index. And dbconnect is installed in virtual machine (windows 2012 R2 OS),I am using in indexer not heavy forwarder.
We are not sending data any where. Just fetching the data from oracle database.

[harsmarvania57]

I know dbIndex is custom index, what I am asking is Index is present on Indexer or not. Index must be present on Indexer to index data and it does not have any relation with DB Connect app. To check whether index exist it or not, go to Splunk Web on Indexer -> Settings -> Indexes, are you able to see dbindex on that page ?

[dhirendra761]

Yes I am able to see "dbindex" in page. and event count in dbindex is 0

[harsmarvania57]

Which means that DB Input you created is not working as intended, check DB Connect logs $SPLUNK_HOME/var/log/splunk/splunk_app_db_connect_audit_server.log and $SPLUNK_HOME/var/log/splunk/splunk_app_db_connect_job_metrics.log and check any ERROR or WARNING messages.

[dhirendra761]

Thanks for you your input. Let me check and I will get back to you by tomorrow.

Thanks for your response again.

🙂

[dhirendra761]

Hi @harshmarvania57,

I got below error in splunk_app_db_connect_server.log

2018-12-20 07:15:30.950 +0100  [QuartzScheduler_Worker-2] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch
java.io.IOException: HTTP Error 400: Bad Request
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
    at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36)
    at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203)
    at org.easybatch.core.job.BatchJob.call(BatchJob.java:79)
    at org.easybatch.extensions.quartz.Job.execute(Job.java:59)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
2018-12-20 07:15:30.950 +0100  [QuartzScheduler_Worker-2] ERROR org.easybatch.core.job.BatchJob - Unable to write records
java.io.IOException: HTTP Error 400: Bad Request
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
    at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36)
    at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203)
    at org.easybatch.core.job.BatchJob.call(BatchJob.java:79)
    at org.easybatch.extensions.quartz.Job.execute(Job.java:59)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

[harsmarvania57]

It looks like HEC (HTTP Event Collector) Error, DB Connect ingest data in Splunk using HEC. Have you disabled HEC on Indexer (Go to Settings -> Data Inputs -> HTTp Event Collector) ? Are you able to see any HEC Token in same path ?

[dhirendra761]

yes you are right harsh. The problem is seems to be related to HEC.
But i get resolved by uninstalling dbconnect 3.1.3 and install dbconnect 3.1.2.

Thanks for your support.

[harsmarvania57]

Hi @dhirendra761,

If you have Splunk server access on which DB connect is running then you can check DB Inputs configuration on $SPLUNK_HOME/etc/apps/splunk_app_db_connect/local/db_inputs.conf and check whether sourcetype = testDB2 is present with correct Input Stanza ?

When you click on sourcetype it will display only those sourcetype which is present in props.conf (For example: default sourcetypes or custom sourcetypes), if testDB2 sourcetype is not present in props.conf then it will not available in dropdown in DB Connect 3.

[dhirendra761]

Hi @harsmarvania57 ,

source type is present in db_inputs.conf.
While creating input, the proper results are shown during SQL query execution as well.
But when I click on 'Find Events' option available in created input, I don't get any results.
This problem is occuring on remote splunk instance only. On my local instance, I am able to find events without any issues.
Please suggest what could be the issue here ?

db_input.conf
[EvolynxTable]
connection = EvolynxDB
description = Table for all data from database
disabled = 0
index = main
index_time_mode = current
interval = 100
max_rows = 100
mode = batch
query = SELECT * FROM "CB80QUA2"."EVOL_UTL"
sourcetype = db_audit