Getting Data In

props.conf time_format appears to be ignored even though data preview works correctly

msbware
Engager

Hello, I've been banging my head against a wall trying to figure out this problem and haven't been able to make any progress. My props.conf file has the following:

[source::.../query_logs/AP-*.log]
MAX_TIMESTAMP_LOOKAHEAD = 12
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TIME_FORMAT = %y%m%d%H%M%S
TZ = America/Los_Angeles
sourcetype = appid_query_log

And the file being indexed that matches the source above is:

140903094720|crf-room-p88a92b5 |room | 10. 74.126.225|danchen |danchen
140903110402|crf-room-p88a92b5 |room | 10. 73.214. 82|        |
140903110403|crf-room-p88a92b5 |room | 10. 73.214. 82|        |

Where the number at the beginning is the date in the format YYMMDDHHMMSS. I've tested it in data preview and the above props.conf settings worked perfectly - however, when I save those settings to props.conf on the forwarder, clear the fishbowl and restart, it still indexes that number as a UNIX timestamp instead of using the format I've told it to use. I know it is using the [source::.../query_logs/AP-*.log] directive because it is correctly assigning the sourcetype to those files.

Any ideas what I'm missing?

0 Karma
1 Solution

msbware
Engager

Apparently all I needed to do to get it figured out was to post a question about it. After doing so, I found this related post ( http://answers.splunk.com/answers/59447/time_format-ignored ) and discovered that the indexer is actually doing the parsing, not the forwarder (which is where my props.conf was). I moved the configuration to the indexer and it worked!

View solution in original post

0 Karma

msbware
Engager

Apparently all I needed to do to get it figured out was to post a question about it. After doing so, I found this related post ( http://answers.splunk.com/answers/59447/time_format-ignored ) and discovered that the indexer is actually doing the parsing, not the forwarder (which is where my props.conf was). I moved the configuration to the indexer and it worked!

0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...