topic Re: Monitor PostgresSQL table in Dashboards & Visualizations

Monitor PostgresSQL table

jcrane — Tue, 18 Dec 2012 21:00:48 GMT

I am trying to setup a monitor to tail a table in my Postgres DB. I have a TIMESTAMP column that I am specifying as the rising column. It does not seem to be pulling the data in.

I can query the DB just fine with the exception that the timestamp is formatted in epoch.

I have a directory monitor that is working just fine. Can someone please give me a few pointers on how to get my table monitor working?

Thank you,
j

Re: Monitor PostgresSQL table

ziegfried — Wed, 19 Dec 2012 20:17:47 GMT

The timestamp being emitted in epoch format is actually intended behavior.

Regarding your input not indexing your events: Can you please provide further information?

The input configuration
Any error messages you might spot in DB Connect's debug log. There is a saved search in the app called "Recent DB Connect errors" which can help you with that.

Re: Monitor PostgresSQL table

jcrane — Wed, 19 Dec 2012 23:38:01 GMT

With your help I was able to see what looks like a memory issue. But I have no idea what to do with it. The table I have is large and I just want to start data from now forward.

2012-12-19 15:27:16.031 monsch1:ERROR:Scheduler - Caught ExecutionException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space

2012-12-19 15:27:16.031 dbx4409:INFO:ExecutionContext - Execution finished in duration=24530 ms

Re: Monitor PostgresSQL table

jcrane — Mon, 28 Sep 2020 12:59:37 GMT

cat ./etc/apps/dbx/default/inputs.conf

Copyright (C) 2005-2012 Splunk Inc. All Rights Reserved.

JBridge Server script

[script://$SPLUNK_HOME/etc/apps/dbx/bin/jbridge_server.py]
index = _internal
sourcetype = dbx_jbridge
interval = 0
disabled = true
passAuth = admin
[script://$SPLUNK_HOME\etc\apps\dbx\bin\jbridge_server.py]
index = _internal
sourcetype = dbx_jbridge
interval = 0
disabled = true
passAuth = admin

Re: Monitor PostgresSQL table

jcrane — Mon, 28 Sep 2020 12:59:39 GMT

cat ./etc/apps/dbx/local/inputs.conf
[script://$SPLUNK_HOME/etc/apps/dbx/bin/jbridge_server.py]
disabled = 0

[batch://$SPLUNK_HOME/var/spool/dbmon/*.dbmonevt]
crcSalt =
disabled = 0
move_policy = sinkhole
sourcetype = dbmon:spool

[dbmon-tail://MyType/Events]
output.format = kv
output.timestamp = 0
table = events
tail.rising.column = uploaded
host = MyHost
index = default
interval = auto
sourcetype = MyType
disabled = 0

Re: Monitor PostgresSQL table

ziegfried — Wed, 19 Dec 2012 23:56:38 GMT

You can configure the input to fetch new data from now on by specifying

tail.follow.only = true

in the [dbmon-tail://MyType/Events] stanza in inputs.conf.

Re: Monitor PostgresSQL table

ziegfried — Wed, 19 Dec 2012 23:57:18 GMT

(After you changed the file, a restart is necessary in order for DB Connect to pick up the changes)

Re: Monitor PostgresSQL table

jcrane — Thu, 20 Dec 2012 01:52:17 GMT

This still did not solve the monitor issue. Maybe I don't need it. I seem to be able to save queries and put the results in a dashboard. Should I just ignore doing a monitor on the DB tables?

Re: Monitor PostgresSQL table

jcrane — Thu, 20 Dec 2012 01:53:01 GMT

Forgot to mention I am not seeing any log issues for the monitor.

Re: Monitor PostgresSQL table

ziegfried — Wed, 16 Jan 2013 04:03:51 GMT

You might want to try the database monitor again with the latest version. There was a potential problem with the postgres JDBC driver that is now being taken care of.

Re: Monitor PostgresSQL table

jcrane — Tue, 12 Feb 2013 23:16:00 GMT

I did try to upgrade and it didn't seem to help the monitor. Actually it seems the change made the errors coming back from the DB less clear in Splunk. That is not a big deal the logs in the DB show exactly what happened to the query but it was worth noting.