All Apps and Add-ons

Splunk DBConnect is not creating sourcetype

dhirendra761
Contributor

Hi All,

I configured Splunk DB Connect app.
I tried to create new sourcetype in my db connection metadata. While checking "find events" , I am getting "0" events . Because of sourcetype was not created.
alt text
Also When I'm creating a new input for my dbconnection i can't select existing source-types. Even more, when I'm trying to create new one it doesn't show up in list of all source-types. Hence I am not able to create sourcetype for my db connect input.

Please help me on this.

0 Karma
1 Solution

dhirendra761
Contributor

Hi All,

I found that the dbconnect 3.1.3 has some issue with only Winodws Server 2012 and 2016 OS.
I am getting error in dbx log files and not able to find any events from DB.
I have resolved the issue by uninstalling dbconnect 3.1.3 and install back to its previous version dbconnect 3.1.2.

And Its working fine.
Anyone having the same issue and need the previous version. Please let me know.

Thanks.

View solution in original post

0 Karma

zhzh
Explorer

I have a same question here. I didn't unistall current DB connect and solve it.

The true problem is not with the Source Type or index. When you click 'save' button, the search is not generated instantly if your Execution Frequency is set at a specific point of time (for me it is 0 9 * * * (every day at 9 am)). Your Source Type will not be created unitll then.

Change the Execution Frequency to 60 (every 60 seconds) solve my question.

I'm new to splunk and just record my situation here.
Thanks.

0 Karma

dhirendra761
Contributor

Hi All,

I found that the dbconnect 3.1.3 has some issue with only Winodws Server 2012 and 2016 OS.
I am getting error in dbx log files and not able to find any events from DB.
I have resolved the issue by uninstalling dbconnect 3.1.3 and install back to its previous version dbconnect 3.1.2.

And Its working fine.
Anyone having the same issue and need the previous version. Please let me know.

Thanks.

View solution in original post

0 Karma

bogdan_nicolesc
Communicator

Hi dhirendra761,

YES! Please, can you please give me bdconnect 3.1.2?

Thank you,
Bogdan.

0 Karma

dhirendra761
Contributor

@bogdan_nicolescu Please share your email id,

Thanks

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @dhirendra761,

Find Events options try to search event indexed locally, if you are running DB Connect on Heavy Forwarder and sending data to Indexer, in this case when you click on Find events on Heavy Forwarder it tries to search data locally however Heavy Forwarder doesn't have any data (Because it is sending data to Indexer to store the data) so that it will give you No results.

On local splunk instance you have data available and you are running DB connect on same splunk instance so in that case you'll able to see those data when you click on Find Events.

0 Karma

dhirendra761
Contributor

Hi Harsh,

Thanks for suggestion. I think my issue is with sourcetype and index. The data is not binding with sourcetype and index because it is not created.
Every time when i edit the input in DB connect it doen't show me in list (please refer above SS).

Also in "DATA SUMMARY" I am not able to locate my created sourcetype.
This problem is occurs only "Windows Server 2012 R2" and it is working fine in windows 7.
Is there is any possibility for different OS as well.

Thanks

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

As I mentioned earlier, while supplying sourcetype or index in DB Connect Input will not create those sourcetype and index configuration in Splunk.

For example: If you'll give index as test in Db Connect input and if that index is not present on Indexer then Splunk will not create that index, you need to create test index on indexer separately.

What I'll suggest is if you have test environment then first try to index data in main index and if it works then create custom index and supply that custom index in Db Connect Input.

0 Karma

dhirendra761
Contributor

Hi Harsh,

I followed you appoarch. I created "dbIndex" index manually. then I create "testDB" sourcetype in dbinput setting. It doesn't still display any events. And while editing input It doesn't show in the list of sourcetype .

let me know If i missed something.
alt text
link text
https://ibb.co/YPhxHR3
https://ibb.co/tQMvR4Z

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Does dbIndex Index exist on Indexer ? On which splunk instance DB Connect is installed (Indexer or Heavy Forwarder) ? Are you able to send data in main index for testing purpose (If it is test environment) ?

0 Karma

dhirendra761
Contributor

Hi @harsmarvania57 ,

" dbIndex" is my custom index. And dbconnect is installed in virtual machine (windows 2012 R2 OS),I am using in indexer not heavy forwarder.
We are not sending data any where. Just fetching the data from oracle database.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

I know dbIndex is custom index, what I am asking is Index is present on Indexer or not. Index must be present on Indexer to index data and it does not have any relation with DB Connect app. To check whether index exist it or not, go to Splunk Web on Indexer -> Settings -> Indexes, are you able to see dbindex on that page ?

0 Karma

dhirendra761
Contributor

Yes I am able to see "dbindex" in page. and event count in dbindex is 0

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Which means that DB Input you created is not working as intended, check DB Connect logs $SPLUNK_HOME/var/log/splunk/splunk_app_db_connect_audit_server.log and $SPLUNK_HOME/var/log/splunk/splunk_app_db_connect_job_metrics.log and check any ERROR or WARNING messages.

0 Karma

dhirendra761
Contributor

Thanks for you your input. Let me check and I will get back to you by tomorrow.

Thanks for your response again.

🙂

0 Karma

dhirendra761
Contributor

Hi @harshmarvania57,

I got below error in splunk_app_db_connect_server.log

2018-12-20 07:15:30.950 +0100  [QuartzScheduler_Worker-2] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch
java.io.IOException: HTTP Error 400: Bad Request
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
    at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36)
    at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203)
    at org.easybatch.core.job.BatchJob.call(BatchJob.java:79)
    at org.easybatch.extensions.quartz.Job.execute(Job.java:59)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
2018-12-20 07:15:30.950 +0100  [QuartzScheduler_Worker-2] ERROR org.easybatch.core.job.BatchJob - Unable to write records
java.io.IOException: HTTP Error 400: Bad Request
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
    at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
    at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36)
    at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203)
    at org.easybatch.core.job.BatchJob.call(BatchJob.java:79)
    at org.easybatch.extensions.quartz.Job.execute(Job.java:59)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

It looks like HEC (HTTP Event Collector) Error, DB Connect ingest data in Splunk using HEC. Have you disabled HEC on Indexer (Go to Settings -> Data Inputs -> HTTp Event Collector) ? Are you able to see any HEC Token in same path ?

0 Karma

dhirendra761
Contributor

yes you are right harsh. The problem is seems to be related to HEC.
But i get resolved by uninstalling dbconnect 3.1.3 and install dbconnect 3.1.2.

Thanks for your support.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @dhirendra761,

If you have Splunk server access on which DB connect is running then you can check DB Inputs configuration on $SPLUNK_HOME/etc/apps/splunk_app_db_connect/local/db_inputs.conf and check whether sourcetype = testDB2 is present with correct Input Stanza ?

When you click on sourcetype it will display only those sourcetype which is present in props.conf (For example: default sourcetypes or custom sourcetypes), if testDB2 sourcetype is not present in props.conf then it will not available in dropdown in DB Connect 3.

0 Karma

dhirendra761
Contributor

Hi @harsmarvania57 ,

source type is present in db_inputs.conf.
While creating input, the proper results are shown during SQL query execution as well.
But when I click on 'Find Events' option available in created input, I don't get any results.
This problem is occuring on remote splunk instance only. On my local instance, I am able to find events without any issues.
Please suggest what could be the issue here ?

db_input.conf
[EvolynxTable]
connection = EvolynxDB
description = Table for all data from database
disabled = 0
index = main
index_time_mode = current
interval = 100
max_rows = 100
mode = batch
query = SELECT * FROM "CB80QUA2"."EVOL_UTL"
sourcetype = db_audit

0 Karma