https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264794#M50827 <P>Looks good thanks! However Im seeing timestamps issue where in the events the leading zeros are trimmed off of the UTC_TIME field, so that midnight is represented as a single 0, 12:15 is represented as the 2 digit 15, etc</P> Tue, 31 Jan 2017 00:57:46 GMT smudge797 2017-01-31T00:57:46Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264792#M50825 <P>Have data that Splunk is struggling with and needs props.conf and transforms.conf.<BR /> The year/month/date followed by time hours minutes seconds are in the 3rd and 4th fields for each event:</P> <PRE><CODE>20170129|4928|20170128|191928|14390803651|DIMPLE|DESAITT|V123456|EMPLOYEE|058000-5440 SOMEDATA|05800001 SOMEDATA5440|3681-1-01 SOMEDATA|Open|GLOBAL|058000|3681|SOMEDATA|MUMBAI|INDIA| 20170129|4928|20170129|191928|14390803651|DIMPLE|DESAITT|E123456|EMPLOYEE|058000-5440 SOMEDATA|05800001 SOMEDATA5440|3681-1-01 SOMEDATA|Open|GLOBAL|058000|3681|SOMEDATA|CONGO|AFRICA| </CODE></PRE> <P>Any help to sort time stamps and the pipe separated fields would be much appreciated. </P> Mon, 30 Jan 2017 22:45:24 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264792#M50825 smudge797 2017-01-30T22:45:24Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264793#M50826 <P>You should be able to configure your sourcetype in props.conf like this:</P> <PRE><CODE>TIME_PREFIX=^\d+\|\d+\| TIME_FORMAT=%Y%m%d|%H%M%S </CODE></PRE> <P>TIME_PREFIX basically says that your timestamp is following two sequences of digits between pipes at the beginning of each line.<BR /> TIME_FORMAT then describes the timestamp as year/month/date, followed by a pipe symbol, followed by hour/minute/second.<BR /> While you're at it, set MAX_TIMESTAMP_LOOKAHEAD to 25.</P> <P>I haven't tested this, but it should be pretty straightforward. What timezone are these times in, can you be sure they all come in from the source in the same timezone?</P> Tue, 29 Sep 2020 12:37:53 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264793#M50826 s2_splunk 2020-09-29T12:37:53Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264794#M50827 <P>Looks good thanks! However Im seeing timestamps issue where in the events the leading zeros are trimmed off of the UTC_TIME field, so that midnight is represented as a single 0, 12:15 is represented as the 2 digit 15, etc</P> Tue, 31 Jan 2017 00:57:46 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264794#M50827 smudge797 2017-01-31T00:57:46Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264795#M50828 <P>Check out <A href="https://answers.splunk.com/answers/147950/can-i-have-different-timestamp-formats-using-the-same-sourcetype.html">this answers post</A> on how to configure datetime.xml to support multiple timestamp formats within the same log file.</P> <P>If you have any influence over how those logs are written, you may want to encourage the developer to make your life a little easier. <span class="lia-unicode-emoji" title=":winking_face:">😉</span></P> Tue, 31 Jan 2017 05:02:24 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-props-conf-and-transforms-conf-to-process-the/m-p/264795#M50828 s2_splunk 2017-01-31T05:02:24Z