Splunk not indexing data if time contains a colon

caatplan_mike — Wed, 16 Jan 2013 14:23:52 GMT

I've having an odd issue with Splunk. I'm attempting a scripted input that outputs current users logged into an oracle database and am formatting the login date value as yyyy-mm-dd hh24:mi:ss. This seems like a reasonable time format.

Splunk seems to have a problem with the : in the time. Looking in splunkd.log, everything looks fine. eg. "Ran script: /opt/splunkforwarder/etc/apps/scripts/bin/oracle_who, took 81.59 milliseconds to run, 1825 bytes read". But if I look for the data in splunk, it's nowhere to be found.

If I change the time separator to a space, Splunk indexes the data, but I'm not sure it recognizes the values as a time value.

Here's sample of the data that is ignored by Splunk.

build,bob,152,42901,bob,terminal02,toad.exe,2013-01-14 17:22:10
build,sue,154,21447,sue,terminal01,toad.exe,2013-01-14 17:22:15
build,jim,195,34447,jim,unknown,sql developer,2013-01-14 13:50:49

Here's a sample of data that is indexed by Splunk successfully.

build,bob,152,42901,bob,terminal02,toad.exe,2013-01-14 17 22 10
build,sue,154,21447,sue,terminal01,toad.exe,2013-01-14 17 22 15
build,jim,195,34447,jim,unknown,sql developer,2013-01-14 13 50 49

Using colon as a field separator works fine too.

build:bob:152:42901:bob:terminal02:toad.exe:2013-01-14 17 22 10
build:sue:154:21447:sue:terminal01:toad.exe:2013-01-14 17 22 15
build:jim:195:34447:jim:unknown:sql developer:2013-01-14 13 50 49

I'd prefer to keep the colons in the time value since it's pretty standard, but I'm not adverse to formatting the time in a different way if it's usually done some other way.

I'm running Splunk 5.0.1 on both the forwarder and the indexer.

--UPDATE--

It's now clear splunk was using the login time as the timestamp which isn't what I'm after. I'd like Splunk to use the current time as the timestamp. I read through the props.conf.spec and have made the following configuration files, but they don't seem to be having the desired effect. All config files are located in /opt/splunkforwarder/etc/apps/scripts/default.

inputs.conf

[script:///opt/splunkforwarder/etc/apps/scripts/bin/oracle_who]
interval = 300
sourcetype = oracle_who
source = script://./bin/oracle_who

props.conf

[oracle_who]
REPORT-oracle_who-fields = extract-oracle_who-fields
DATETIME_CONFIG = NONE
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE_DATE = false

transforms.conf

[extract-oracle_who-fields]
DELIMS = ","
FIELDS = instance, username, sid, serial, osuser, host, program, login_time

I used this document for developing my scripted input: http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Setupcustominputs#Example_using_inputs.conf

Re: Splunk not indexing data if time contains a colon

kristian_kolb — Wed, 16 Jan 2013 14:32:37 GMT

I guess a good place to start is to check out the TIME_FORMAT, TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD parameters that can be set for your source/sourcetype in props.conf.

http://docs.splunk.com/Documentation/Splunk/5.0.1/Admin/Propsconf

hope this helps,

Kristian

Re: Splunk not indexing data if time contains a colon

caatplan_mike — Wed, 16 Jan 2013 15:11:14 GMT

Hi Kristian, this definitely helped. I can see now that splunk was indexing my data using the login_time as the timestamp value of the record which is not the behaviour I'm after. I've attampted to disable this without success (I'll update the OP with my config files).

topic Re: Splunk not indexing data if time contains a colon in Getting Data In

Splunk not indexing data if time contains a colon

Re: Splunk not indexing data if time contains a colon

Re: Splunk not indexing data if time contains a colon