Splunk DB Connect v1: Database inputs not working

[DanielFordWA]

Hi,

I have setup a Database Connection "TEST-HRi" in the Splunk Manager in DB Connect v 1.1.4 on Splunk 6.2.0

I can successfully query the Database in the DB Query section in DB Connect and pull down 1000 results using the below query.

SELECT * FROM MIS_EMPLOYEE_DATA

No matter what I do I can't seem to get the Database inputs in the Splunk Manager to pull down the entire DB.

I have setup an index I would like to pull the data into called "hri-db-test". I have confirmed with the Database owners that pulling the entire DB should not be a problem from their end.

I have the following settings..

dbmon-dump://TEST-HRi/IDM2

Name
IDM2

Input Type
Dump (Always dump the full table/query)

Database
TEST-HRi

Select SQL Query
SELECT * FROM MIS_EMPLOYEE_DATA

Sourcetype
csv

Splunk Index
hri-db-test

Output Format
CSV (with headers)

Output timestamp
MODIFYTIMESTAMP

Timestamp Format
-blank-

Interval
15 * * * *

Any help is much appreciated, even if it's pointing me towards the logs that will show me the errors. I have Splunk on Splunk installed on this instance.

Thanks,

Dan

[n00badmin · Accepted solution]

ummm I would check dbx documentation for any max settings...will check in a bit...but yes have the db admins check the max rows returned for ur user

[n00badmin]

Yah, sorry, the pitfalls of helping while on mobile.

[n00badmin]

configure the input to not use a custom sql query...tell splunk the table and let it build the query...does that work???

[DanielFordWA]

Still trying, but even without the query I get the following errors.

monsch1:INFO:Scheduler - Execution of input=[dbmon-dump://TEST-HRi/IDM2] finished in duration=375 ms with resultCount=0 success=false continueMonitoring=true

dbx6285:INFO:DumpDatabaseMonitor - Database monitor=[dbmon-dump://TEST-HRi/IDM2] finished with status=false resultCount=0 in duration=375 ms

I suspect I am making a basic error somewhere.

[ppablo]

Hi @n00badmin

Please be sure that when responding to someone's answer, click on "Add comment" directly below their answer or, if responding to someone's comment, type in the "Add your comment..." box directly below their comment. You keep typing all of your responses in the "Enter your answer here..." box at the very bottom of the page which, instead, posts a brand new answer each time and is confusing for other users to follow. This will help with a clean continuous flow of the conversation. I already converted your "answers" to comments, so just something to keep in mind from here on out. Thanks.

[DanielFordWA]

Back to this post http://answers.splunk.com/answers/243605/splunk-db-connect-1-why-am-i-getting-error-command.html So can I index the database, or are there size limits on the Splunk side?

[n00badmin]

dbx logs usually give a clue as to what happened in the poll of the db

index=_internal source=dbx

you can even throw a IDM2 in your search to filter the logs down...

[n00badmin]

also try not specifying a custom sql query...just use the default query with the table name when u configure the dump

[n00badmin]

what happens if you go back to db query and run it without the limit 1000???

[DanielFordWA]

command="dbquery", A database error occurred: Invalid Fetch Size

I was told by the DB owners that pulling the entire DB is not an issue, I assume it is?

[n00badmin]

Theres no conflict..they are designed to run side by side...

try this search...

index=_internal source=*dbx.log *IDM2* 

You should see when the dump was executed..tells you how long it took and any results or errors

[n00badmin]

index=_internal source=*dbx* *IDM2*

[DanielFordWA]

Thanks - I tihnk I found the problem.

[CRITICAL] [rpcstart.py] RPC server has been terminated abnormally with error [No java path specified].

[CRITICAL] [rpcstart.py] No java path specified for stanza rpcstart://default

Still dont really understand why the DB Query would work and not the DB Input but at least I've found the right logs.

[DanielFordWA]

Actually that is DB Connect v2 - I'll look for errors in DBv1 and uninstall DB connect v2 incase there is a conflict

[DanielFordWA]

Perfect.

see the below in the logs.

13:15:00.288 dbx6955:INFO:DumpDatabaseMonitor - Database monitor=[dbmon-dump://TEST-HRi/IDM2] finished with status=false resultCount=0 in duration=234 ms

As DB Query works fine would this be an setting owned by the Database owners causing this issue?

[n00badmin]

my apologies...the text took out the star wildcards...there should be an asterisk on either side of dbx

[DanielFordWA]

index=_internal source=dbx returns no results, even after running a successful query in DB Query.