Splunk DB Connect App not putting data in Splunk index

[raidercom]

Hi:
I'm trying to get SplunkDB Connect app to pull data from an Oracle database into Splunk.

Working:

Database Connection

DB Info

DB Query with the SQL statement I'm using

Not Working:
When I go to setup the query to actually pull data into a Splunk index, I cannot seem to get anything to go into the index from the query.

What I've done:

Created a new index for the data

Specify the SQL query in a new database input

Scheduled query

New index in the spunk app & new index in the dbx app

Every 1/2 hour & auto

dump (not tailed, wanted to keep it simple to start)

no source type & tried with a new sourcetype

Output formatting:

Key-Value format

Output timestamp with proper field and value & no output timestamp specified

Before restarting Splunk after specifying the new index, I've gotten this message, with different variations on the information contained within based on what I've chosen on the query:
This message has seemed to go away after rebooting Splunk

earch peer has the following message: received event for unconfigured/disabled/deleted index='dbxtest' with source='source::dbmon-dump://' host='host::' sourcetype='sourcetype::dbmon:kv' (3 missing total)

Things I have not tried:

Custom source type, don't see the point

Look-up table, again, don't see the point

Maybe I'm looking at this the wrong way, but I'm trying to test whether there is data in the index by searching for index=dbxtest range:all time from within the splunk dbx app

Thank you for any information/assistance you could provide.

[lukejadamec]

Here is some info regarding output.timestamp.parse.format http://splunk-base.splunk.com/answers/71485/splunk-db-connect-timestamp-not-working, but I did not need to use it.
More words of wisdom: always access the dbx input configuration from the dbx app. If you go from another app to the manager to the db inputs then the inputs.conf will get written somewhere else, and you will either not be able to find them, or worse you will end up with conflicting input.conf files.

[lukejadamec]

Your timestamp is being read as 2016, but don't worry about correcting for it with max days hence because once you get it right it won't matter.

[lukejadamec]

The timestamp is not being read properly. Sometimes folks have this problem, and sometimes they don't. Try using to_char(log_date,'YYYY-MM-DD HH24:MI:SS') in your query to convert the timestamp. We did this on the oracle side, but you should be able to get it to work on the dbx side. Also, make sure the timestamp column is listed first in your query.

[raidercom]

I've added MAX_DAYS_AGO = 999 to the props.conf of the dbx app $splunk/etc/apps/dbx/local/props.conf, and restarted the splunk server.

The part I don't understand, is that the dates it is referencing are a couple days ago, and should be within the default MAX_DAYS_AGO for Splunk. I don't understand why it is rejecting events that are recent.

[raidercom]

Hmm...
splunkd.log seems to be littered with these messages now.

08-06-2013 16:30:21.575 -0400 WARN DateParserVerbose - A possible timestamp match (Tue Aug 2 18:59:59 2016) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::dbmon-dump://DMART/|host::|dbmon:kv|

[lukejadamec]

Just remember to include {{WHERE $rising_column$ > ?}} at the end. - when you configure that database input that is. Don't use when you're testing the query.

[lukejadamec]

What we do is create and name the query on the Oracle side, and just call that. The DBX query interpreter can be a bit picky.
What you want to do works, but for simplicities sake keep the query simple. You can also test the query with the DBX query window.
Just remember to include {{WHERE $rising_column$ > ?}} at the end.

[raidercom]

Just testing my latest iteration of the database input with your suggestions.

When you say enterqueryname, does that have to be a saved query within Oracle?
If I have a full SQL query to paste in there, I'd imagine that I can't just paste that in there, as the query would be out of order.

The reason I wanted to get a straight dump working first was for simplicity.
-Jeff