Getting Data In
Highlighted

Timestamp milliseconds not appearing

Builder

Hi there,

I am extracting a timestamp in props.. everything is working fine except for the milliseconds at the end of it.

Date format is 2017-07-11 08:54:12,815 -- my extraction is %Y-%m-%d %H-%M-%S,%3N

That extraction works for similar data with a .(period) before the milliseconds instead of a comma - changing the end of the extract to .%3N - but it never works for a comma.

Is there anything else needed when trying to extract milliseconds after a comma?

Thanks

Highlighted

Re: Timestamp milliseconds not appearing

Legend

Hi mwdbhyat,
let me understand:
you have another sourcetype with time format %Y-%m-%d %H-%M-%S.%3N that works with other data and to read these data, do you want to use the same time format or to create another one?

If you want to create another one it's easy because you insert the correct format in props.conf
TIME_FORMAT = %Y-%m-%d %H-%M-%S,%3N

If instead you want to use the same sourcetype, don't set any time format, Splunk recognize both the time format (I tried it).

Bye.
Giuseppe

0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Builder

Hi Giuseppe,

The issue im facing is that it will not extract the milliseconds from the timestamp when there is a comma in the timestamp before the milliseconds..

0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Legend

Hi mwdbhyat,
if you set the time format (with dot or comma), it's correct that Splunk doesn't recognize both timestamps but only ther one that matches your settings.
If instead you don't set the time format and leave that Splunk recognizes time format, it recognizes both the time formats.
I created a file with your two time formats and Splunk recognized both of them.
Bye.
Giuseppe

0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Legend

@cusello, slight correction to your answer, Hour Minute and Second are separated by colons(:) not hyphens (-).
@mwdbhyat, please try out colon as separator for Time. I have added a run-anywhere search to test.

| makeresults
| eval epochTime=strptime("2017-07-11 08:54:12,815","%Y-%m-%d %H:%M:%S,%3N")
| eval stringTime=strptime(epochTime,"%Y-%m-%d %H:%M:%S,%3N")
| table epochTime stringTime



| eval message="Happy Splunking!!!"


0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Builder

Hi guys,

Thanks for the help. @niketnilay - that was a typo in this question 🙂 ..I used colons not hyphens. the makeresults query only returns epochTime value?

The issue at hand is this - %Y-%m-%d %H-%M-%S,%3N should match 2017-07-11 08:54:12,815 but it doesnt take the milliseconds.. Thats the only problem..

Not setting a TIME_FORMAT will be less performant so I dont want to leave it blank.

0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Legend

makeresults is a generating command which allows you to create your own variable/data to be displayed.

strptime() converts string time to epoch and strftime() converts from epoch to string. If you compare the two values (epochTime and stringTime) you will see that 815 is present in both hence milliseconds is working as expected.

Is this extraction being done while sourcetype definition in props.conf? Can you add your current settings and some sample raw data(mock or anonymize sensitive info where ever needed?




| eval message="Happy Splunking!!!"


0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Builder

Ah right i get it now(think you had a typo in your command strp instead of strf for the stringTime.

[sourcectypename]
LINEBREAKER = ([\r\n]+)
TIME
FORMAT = as above
TIMEPREFIX = ^
MAX
TIMESTAMPLOOKAHEAD = 27 ( i overset this this, initially it matched the actual length..thought it might be cutting it off)
Truncate 999999
SHOULD
LINEMERGE = false

sample data:

2017-07-11 08:54:12,815 31 [INFO] - - 5ms textHere MoreTEXT here[652].moretextHere(): MoreTextHere

0 Karma
Highlighted

Re: Timestamp milliseconds not appearing

Legend

This seems to work for me as well. Can you try createing a new sourcetype with only TIMEFORMAT and MAXTIMESTAMP_LOOKAHEAD? Test with your data in preview mode whether it is working as expected or not.

[customSourcetypename]
DATETIME_CONFIG = 
MAX_TIMESTAMP_LOOKAHEAD = 24
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
category = Custom
pulldown_type = true

alt text




| eval message="Happy Splunking!!!"


View solution in original post

Highlighted

Re: Timestamp milliseconds not appearing

Explorer
0 Karma