Getting Data In

props.conf time_format appears to be ignored even though data preview works correctly

msbware
Engager

Hello, I've been banging my head against a wall trying to figure out this problem and haven't been able to make any progress. My props.conf file has the following:

[source::.../query_logs/AP-*.log]
MAX_TIMESTAMP_LOOKAHEAD = 12
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TIME_FORMAT = %y%m%d%H%M%S
TZ = America/Los_Angeles
sourcetype = appid_query_log

And the file being indexed that matches the source above is:

140903094720|crf-room-p88a92b5 |room | 10. 74.126.225|danchen |danchen
140903110402|crf-room-p88a92b5 |room | 10. 73.214. 82|        |
140903110403|crf-room-p88a92b5 |room | 10. 73.214. 82|        |

Where the number at the beginning is the date in the format YYMMDDHHMMSS. I've tested it in data preview and the above props.conf settings worked perfectly - however, when I save those settings to props.conf on the forwarder, clear the fishbowl and restart, it still indexes that number as a UNIX timestamp instead of using the format I've told it to use. I know it is using the [source::.../query_logs/AP-*.log] directive because it is correctly assigning the sourcetype to those files.

Any ideas what I'm missing?

0 Karma
1 Solution

msbware
Engager

Apparently all I needed to do to get it figured out was to post a question about it. After doing so, I found this related post ( http://answers.splunk.com/answers/59447/time_format-ignored ) and discovered that the indexer is actually doing the parsing, not the forwarder (which is where my props.conf was). I moved the configuration to the indexer and it worked!

View solution in original post

0 Karma

msbware
Engager

Apparently all I needed to do to get it figured out was to post a question about it. After doing so, I found this related post ( http://answers.splunk.com/answers/59447/time_format-ignored ) and discovered that the indexer is actually doing the parsing, not the forwarder (which is where my props.conf was). I moved the configuration to the indexer and it worked!

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...