Getting Data In

Splunk DB Connect: Why is the timestamp specified in inputs.conf not being parsed?

dimitris_vergos
Path Finder

Hello,

I am trying to import data from a MySQL database.

While the import works fine, the time field gets populated with the time that the event is being imported, but not the 'datetime' field that I have specified in the database (in my case V_Date).

inputs.conf / [$SPLUNK_HOME/var/lib/splunk/persistentstorage/dbx]

[dbmon-tail://CTM/CTM Violations]
host = CTM
index = development_index
output.format = kv
output.timestamp = 0
query = SELECT VIOLATION_ID,V_DATE,VIOLATION_TYPE_ID,V_CLIENT_ID,VIOLATION_SOURCE, VIOLATION_FREQUENCY,V_LICENCE_ID,V_MODULE_ID\r\nFROM VIOLATIONS {{WHERE $rising_column$ > ?}}
sourcetype = CTM Violations
tail.rising.column = VIOLATION_ID
interval = auto
table = CTM Violations
disabled = 0
output.timestamp.column = V_DATE
output.timestamp.format = yyyy-MM-dd HH:mm:ss

I have also tried without the

output.timestamp.column = V_DATE
output.timestamp.format = yyyy-MM-dd HH:mm:ss

Date Column is V_DATE // V_DATE datetime.

I tried creating a props.conf file at a second stage.

[host::CTM]
DATETIME_CONFIG = NONE
SHOULD_LINEMERGE = false

Any suggestions?

jcoates_splunk
Splunk Employee
Splunk Employee

In my personal opinion, time formatting is easier to do in SQL than SPL, so I prefer to do it there when working with DB Connect 1.

If you use DB Connect 2, it has a UI to help you set the right time format when you build your input.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Can you supply an example of the results of the SQL query? That may not help, but it may give us more to work with.

0 Karma

dimitris_vergos
Path Finder

Here you go,

ID V_ID C_ID C_IP L_ID V_DATE V_F V_M
90050 1 6 31.5.253.88 8 2015-03-04 14:26:56 58 1

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...