Getting Data In

props.conf time_format appears to be ignored even though data preview works correctly

msbware
Engager

Hello, I've been banging my head against a wall trying to figure out this problem and haven't been able to make any progress. My props.conf file has the following:

[source::.../query_logs/AP-*.log]
MAX_TIMESTAMP_LOOKAHEAD = 12
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TIME_FORMAT = %y%m%d%H%M%S
TZ = America/Los_Angeles
sourcetype = appid_query_log

And the file being indexed that matches the source above is:

140903094720|crf-room-p88a92b5 |room | 10. 74.126.225|danchen |danchen
140903110402|crf-room-p88a92b5 |room | 10. 73.214. 82|        |
140903110403|crf-room-p88a92b5 |room | 10. 73.214. 82|        |

Where the number at the beginning is the date in the format YYMMDDHHMMSS. I've tested it in data preview and the above props.conf settings worked perfectly - however, when I save those settings to props.conf on the forwarder, clear the fishbowl and restart, it still indexes that number as a UNIX timestamp instead of using the format I've told it to use. I know it is using the [source::.../query_logs/AP-*.log] directive because it is correctly assigning the sourcetype to those files.

Any ideas what I'm missing?

0 Karma
1 Solution

msbware
Engager

Apparently all I needed to do to get it figured out was to post a question about it. After doing so, I found this related post ( http://answers.splunk.com/answers/59447/time_format-ignored ) and discovered that the indexer is actually doing the parsing, not the forwarder (which is where my props.conf was). I moved the configuration to the indexer and it worked!

View solution in original post

0 Karma

msbware
Engager

Apparently all I needed to do to get it figured out was to post a question about it. After doing so, I found this related post ( http://answers.splunk.com/answers/59447/time_format-ignored ) and discovered that the indexer is actually doing the parsing, not the forwarder (which is where my props.conf was). I moved the configuration to the indexer and it worked!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Catalog Is Now Generally Available on Splunk Cloud Platform

A Unified View of Your Data  Security logs, application events, business data, and historical telemetry often ...