topic TIME_FORMAT in props.conf in Getting Data In

TIME_FORMAT in props.conf

RobertRi — Fri, 21 Sep 2012 11:33:37 GMT

I use Splunk 4.1.4 and have difficulties to get the right timestamp from my event

I have modified the props.conf

[timetest]
TIME_FORMAT = %Y%m%d%H%M%S%3N

which successfully get the right time from this string
20120921130215999

If I would like the short year than there is a problem

props.conf
[timetest]
TIME_FORMAT = %y%m%d%H%M%S%3N

String = 120921130315999

Here I get the following date
4/26/08 2:01:43.159 PM

Any hints why this happen ?
Thanks
Rob

Re: TIME_FORMAT in props.conf

kristian_kolb — Fri, 21 Sep 2012 12:20:43 GMT

What happens is:
The string gets parsed as epoch time, save for the last to 9's.

1209211303159 = April 26, 2008 2pm (roughly)

Why this happens is more of a mystery at this point. Do you also have a TIME_PREFIX set?

Please update your question with a few sample events, and the props.conf settings for the sourcetype in question.

Hope this sheds some light,

/Kristian

Re: TIME_FORMAT in props.conf

echalex — Fri, 21 Sep 2012 12:38:24 GMT

I arrived at the same conclusion, but kristian.kolb beat me to it. Just want to point out that that's milliseconds from epoch, rather than seconds, in case someone's wondering.

Re: TIME_FORMAT in props.conf

RobertRi — Mon, 28 Sep 2020 12:29:50 GMT

originaly I would like to extract a Tivoli Timestamp which is

1120924073559999
1 = centaury
12 = year
09 = motnh
24 = day
07 = hour
35 = minutes
59 = seconds
999 = milliseconds

with the following props.conf

[timetest]
TIME_PREFIX = ^1
TIME_FORMAT = %y%m%d%H%M%S%3N

but this doesn't work and than I tried a couple of things

For my test I had a normal textfile and echoed my strings in it

STRING: 1120924070011999 test2

Any Ideas?

Re: TIME_FORMAT in props.conf

RobertRi — Mon, 24 Sep 2012 12:59:25 GMT

Now the timestamp will be set correct.

Additional I have tried to manipulate the timestring in the raw message

transforms.conf

[resortdate]
REGEX = ^1\d{2}\d{2}\d{2}\d{2}\d{2}\d{2}\d{3}
FORMAT = $2-$3-20$1 $4:$5:$6.$7
DEST_KEY = _raw

props.conf

[timetest]
TIME_PREFIX = ^1
TIME_FORMAT = %y%m%d%H%M%S%3N
MAX_TIMESTAMP_LOOKAHEAD = 16
TRASFORMS-sortdate = resortdate

This works fine but I see only the modified timestamp, not the message behind the timestamp.
Is there anything to do ?

Thanks
Rob

Re: TIME_FORMAT in props.conf

kristian_kolb — Mon, 24 Sep 2012 13:39:56 GMT

The reason you only see the timestamp is because you re-write the _raw field with ONLY the timestamp.

Are you sure that the REGEX you mention above in transforms.conf is correctly written here? I was under the impression that you could only use the $n notation if you had capture groups in the regex.

In that case you should have a final capture group

(.*)$

that contains the rest of the event,to add as $8 in your FORMAT.

Still I think this looks a bit of an awkward way of getting the desired results. Is there no way to alter the timestamps where they are written?

Re: TIME_FORMAT in props.conf

RobertRi — Tue, 25 Sep 2012 07:18:22 GMT

thanks

So I get it to work

[resortdate]
REGEX = (?i)^1(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{3})\s+(.*$)
FORMAT = $2-$3-20$1 $4:$5:$6.$7 $8
DEST_KEY = _raw