topic Re: Splunk DB Connect v1: Database inputs not working in Deployment Architecture

Splunk DB Connect v1: Database inputs not working

DanielFordWA — Mon, 28 Sep 2020 20:21:11 GMT

Hi,

I have setup a Database Connection "TEST-HRi" in the Splunk Manager in DB Connect v 1.1.4 on Splunk 6.2.0

I can successfully query the Database in the DB Query section in DB Connect and pull down 1000 results using the below query.

SELECT * FROM MIS_EMPLOYEE_DATA

No matter what I do I can't seem to get the Database inputs in the Splunk Manager to pull down the entire DB.

I have setup an index I would like to pull the data into called "hri-db-test". I have confirmed with the Database owners that pulling the entire DB should not be a problem from their end.

I have the following settings..

dbmon-dump://TEST-HRi/IDM2

Name
IDM2

Input Type
Dump (Always dump the full table/query)

Database
TEST-HRi

Select SQL Query
SELECT * FROM MIS_EMPLOYEE_DATA

Sourcetype
csv

Splunk Index
hri-db-test

Output Format
CSV (with headers)

Output timestamp
MODIFYTIMESTAMP

Timestamp Format
-blank-

Interval
15 * * * *

Any help is much appreciated, even if it's pointing me towards the logs that will show me the errors. I have Splunk on Splunk installed on this instance.

Thanks,

Dan

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 11:33:28 GMT

dbx logs usually give a clue as to what happened in the poll of the db

index=_internal source=dbx

you can even throw a IDM2 in your search to filter the logs down...

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Tue, 23 Jun 2015 11:49:12 GMT

index=_internal source=dbx returns no results, even after running a successful query in DB Query.

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 11:58:14 GMT

my apologies...the text took out the star wildcards...there should be an asterisk on either side of dbx

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 12:01:00 GMT

index=_internal source=*dbx* *IDM2*

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Tue, 23 Jun 2015 12:34:22 GMT

Thanks - I tihnk I found the problem.

[CRITICAL] [rpcstart.py] RPC server has been terminated abnormally with error [No java path specified].

[CRITICAL] [rpcstart.py] No java path specified for stanza rpcstart://default

Still dont really understand why the DB Query would work and not the DB Input but at least I've found the right logs.

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Tue, 23 Jun 2015 12:35:28 GMT

Actually that is DB Connect v2 - I'll look for errors in DBv1 and uninstall DB connect v2 incase there is a conflict

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 12:48:26 GMT

Theres no conflict..they are designed to run side by side...

try this search...

index=_internal source=*dbx.log *IDM2*

You should see when the dump was executed..tells you how long it took and any results or errors

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Tue, 23 Jun 2015 13:11:43 GMT

Perfect.

see the below in the logs.

13:15:00.288 dbx6955:INFO:DumpDatabaseMonitor - Database monitor=[dbmon-dump://TEST-HRi/IDM2] finished with status=false resultCount=0 in duration=234 ms

As DB Query works fine would this be an setting owned by the Database owners causing this issue?

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 13:52:17 GMT

what happens if you go back to db query and run it without the limit 1000???

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 14:03:28 GMT

also try not specifying a custom sql query...just use the default query with the table name when u configure the dump

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Tue, 23 Jun 2015 14:10:36 GMT

command="dbquery", A database error occurred: Invalid Fetch Size

I was told by the DB owners that pulling the entire DB is not an issue, I assume it is?

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 14:22:36 GMT

ummm I would check dbx documentation for any max settings...will check in a bit...but yes have the db admins check the max rows returned for ur user

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Tue, 23 Jun 2015 14:42:41 GMT

Back to this post http://answers.splunk.com/answers/243605/splunk-db-connect-1-why-am-i-getting-error-command.html So can I index the database, or are there size limits on the Splunk side?

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 14:45:18 GMT

configure the input to not use a custom sql query...tell splunk the table and let it build the query...does that work???

Re: Splunk DB Connect v1: Database inputs not working

ppablo — Tue, 23 Jun 2015 17:56:14 GMT

Hi @n00badmin

Please be sure that when responding to someone's answer, click on "Add comment" directly below their answer or, if responding to someone's comment, type in the "Add your comment..." box directly below their comment. You keep typing all of your responses in the "Enter your answer here..." box at the very bottom of the page which, instead, posts a brand new answer each time and is confusing for other users to follow. This will help with a clean continuous flow of the conversation. I already converted your "answers" to comments, so just something to keep in mind from here on out. Thanks.

Re: Splunk DB Connect v1: Database inputs not working

n00badmin — Tue, 23 Jun 2015 18:26:53 GMT

Yah, sorry, the pitfalls of helping while on mobile.

Re: Splunk DB Connect v1: Database inputs not working

DanielFordWA — Wed, 24 Jun 2015 10:29:50 GMT

Still trying, but even without the query I get the following errors.

monsch1:INFO:Scheduler - Execution of input=[dbmon-dump://TEST-HRi/IDM2] finished in duration=375 ms with resultCount=0 success=false continueMonitoring=true

dbx6285:INFO:DumpDatabaseMonitor - Database monitor=[dbmon-dump://TEST-HRi/IDM2] finished with status=false resultCount=0 in duration=375 ms

I suspect I am making a basic error somewhere.