Deployment Architecture

Splunk DB Connect v1: Database inputs not working

DanielFordWA
Contributor

Hi,

I have setup a Database Connection "TEST-HRi" in the Splunk Manager in DB Connect v 1.1.4 on Splunk 6.2.0

I can successfully query the Database in the DB Query section in DB Connect and pull down 1000 results using the below query.

SELECT * FROM MIS_EMPLOYEE_DATA

No matter what I do I can't seem to get the Database inputs in the Splunk Manager to pull down the entire DB.

I have setup an index I would like to pull the data into called "hri-db-test". I have confirmed with the Database owners that pulling the entire DB should not be a problem from their end.

I have the following settings..

dbmon-dump://TEST-HRi/IDM2

Name
IDM2

Input Type
Dump (Always dump the full table/query)

Database
TEST-HRi

Select SQL Query
SELECT * FROM MIS_EMPLOYEE_DATA

Sourcetype
csv

Splunk Index
hri-db-test

Output Format
CSV (with headers)

Output timestamp
MODIFYTIMESTAMP

Timestamp Format
-blank-

Interval
15 * * * *

Any help is much appreciated, even if it's pointing me towards the logs that will show me the errors. I have Splunk on Splunk installed on this instance.

Thanks,

Dan

0 Karma
1 Solution

n00badmin
Communicator

ummm I would check dbx documentation for any max settings...will check in a bit...but yes have the db admins check the max rows returned for ur user

View solution in original post

n00badmin
Communicator

ummm I would check dbx documentation for any max settings...will check in a bit...but yes have the db admins check the max rows returned for ur user

n00badmin
Communicator

Yah, sorry, the pitfalls of helping while on mobile.

n00badmin
Communicator

configure the input to not use a custom sql query...tell splunk the table and let it build the query...does that work???

0 Karma

DanielFordWA
Contributor

Still trying, but even without the query I get the following errors.

monsch1:INFO:Scheduler - Execution of input=[dbmon-dump://TEST-HRi/IDM2] finished in duration=375 ms with resultCount=0 success=false continueMonitoring=true

dbx6285:INFO:DumpDatabaseMonitor - Database monitor=[dbmon-dump://TEST-HRi/IDM2] finished with status=false resultCount=0 in duration=375 ms

I suspect I am making a basic error somewhere.

0 Karma

ppablo
Retired

Hi @n00badmin

Please be sure that when responding to someone's answer, click on "Add comment" directly below their answer or, if responding to someone's comment, type in the "Add your comment..." box directly below their comment. You keep typing all of your responses in the "Enter your answer here..." box at the very bottom of the page which, instead, posts a brand new answer each time and is confusing for other users to follow. This will help with a clean continuous flow of the conversation. I already converted your "answers" to comments, so just something to keep in mind from here on out. Thanks.

DanielFordWA
Contributor

Back to this post http://answers.splunk.com/answers/243605/splunk-db-connect-1-why-am-i-getting-error-command.html So can I index the database, or are there size limits on the Splunk side?

0 Karma

n00badmin
Communicator

dbx logs usually give a clue as to what happened in the poll of the db

index=_internal source=dbx

you can even throw a IDM2 in your search to filter the logs down...

0 Karma

n00badmin
Communicator

also try not specifying a custom sql query...just use the default query with the table name when u configure the dump

0 Karma

n00badmin
Communicator

what happens if you go back to db query and run it without the limit 1000???

0 Karma

DanielFordWA
Contributor

command="dbquery", A database error occurred: Invalid Fetch Size

I was told by the DB owners that pulling the entire DB is not an issue, I assume it is?

0 Karma

n00badmin
Communicator

Theres no conflict..they are designed to run side by side...

try this search...

index=_internal source=*dbx.log *IDM2* 

You should see when the dump was executed..tells you how long it took and any results or errors

0 Karma

n00badmin
Communicator
index=_internal source=*dbx* *IDM2*
0 Karma

DanielFordWA
Contributor

Thanks - I tihnk I found the problem.

[CRITICAL] [rpcstart.py] RPC server has been terminated abnormally with error [No java path specified].

[CRITICAL] [rpcstart.py] No java path specified for stanza rpcstart://default

Still dont really understand why the DB Query would work and not the DB Input but at least I've found the right logs.

0 Karma

DanielFordWA
Contributor

Actually that is DB Connect v2 - I'll look for errors in DBv1 and uninstall DB connect v2 incase there is a conflict

0 Karma

DanielFordWA
Contributor

Perfect.

see the below in the logs.

13:15:00.288 dbx6955:INFO:DumpDatabaseMonitor - Database monitor=[dbmon-dump://TEST-HRi/IDM2] finished with status=false resultCount=0 in duration=234 ms

As DB Query works fine would this be an setting owned by the Database owners causing this issue?

0 Karma

n00badmin
Communicator

my apologies...the text took out the star wildcards...there should be an asterisk on either side of dbx

0 Karma

DanielFordWA
Contributor

index=_internal source=dbx returns no results, even after running a successful query in DB Query.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...