Getting Data In

Timestamp milliseconds not appearing

mwdbhyat
Builder

Hi there,

I am extracting a timestamp in props.. everything is working fine except for the milliseconds at the end of it.

Date format is 2017-07-11 08:54:12,815 -- my extraction is %Y-%m-%d %H-%M-%S,%3N

That extraction works for similar data with a .(period) before the milliseconds instead of a comma - changing the end of the extract to .%3N - but it never works for a comma.

Is there anything else needed when trying to extract milliseconds after a comma?

Thanks

1 Solution

niketn
Legend

This seems to work for me as well. Can you try createing a new sourcetype with only TIME_FORMAT and MAX_TIMESTAMP_LOOKAHEAD? Test with your data in preview mode whether it is working as expected or not.

[customSourcetypename]
DATETIME_CONFIG = 
MAX_TIMESTAMP_LOOKAHEAD = 24
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
category = Custom
pulldown_type = true

alt text

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

freedomson
Explorer
0 Karma

niketn
Legend

This seems to work for me as well. Can you try createing a new sourcetype with only TIME_FORMAT and MAX_TIMESTAMP_LOOKAHEAD? Test with your data in preview mode whether it is working as expected or not.

[customSourcetypename]
DATETIME_CONFIG = 
MAX_TIMESTAMP_LOOKAHEAD = 24
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
category = Custom
pulldown_type = true

alt text

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

gcusello
SplunkTrust
SplunkTrust

Hi mwdbhyat,
let me understand:
you have another sourcetype with time format %Y-%m-%d %H-%M-%S.%3N that works with other data and to read these data, do you want to use the same time format or to create another one?

If you want to create another one it's easy because you insert the correct format in props.conf
TIME_FORMAT = %Y-%m-%d %H-%M-%S,%3N

If instead you want to use the same sourcetype, don't set any time format, Splunk recognize both the time format (I tried it).

Bye.
Giuseppe

0 Karma

mwdbhyat
Builder

Hi Giuseppe,

The issue im facing is that it will not extract the milliseconds from the timestamp when there is a comma in the timestamp before the milliseconds..

0 Karma

niketn
Legend

@cusello, slight correction to your answer, Hour Minute and Second are separated by colons(:) not hyphens (-).
@mwdbhyat, please try out colon as separator for Time. I have added a run-anywhere search to test.

| makeresults
| eval epochTime=strptime("2017-07-11 08:54:12,815","%Y-%m-%d %H:%M:%S,%3N")
| eval stringTime=strptime(epochTime,"%Y-%m-%d %H:%M:%S,%3N")
| table epochTime stringTime
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

mwdbhyat
Builder

Hi guys,

Thanks for the help. @niketnilay - that was a typo in this question 🙂 ..I used colons not hyphens. the makeresults query only returns epochTime value?

The issue at hand is this - %Y-%m-%d %H-%M-%S,%3N should match 2017-07-11 08:54:12,815 but it doesnt take the milliseconds.. Thats the only problem..

Not setting a TIME_FORMAT will be less performant so I dont want to leave it blank.

0 Karma

niketn
Legend

makeresults is a generating command which allows you to create your own variable/data to be displayed.

strptime() converts string time to epoch and strftime() converts from epoch to string. If you compare the two values (epochTime and stringTime) you will see that 815 is present in both hence milliseconds is working as expected.

Is this extraction being done while sourcetype definition in props.conf? Can you add your current settings and some sample raw data(mock or anonymize sensitive info where ever needed?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

mwdbhyat
Builder

Ah right i get it now(think you had a typo in your command strp instead of strf for the stringTime.

[sourcectypename]
LINE_BREAKER = ([\r\n]+)
TIME_FORMAT = as above
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 27 ( i overset this this, initially it matched the actual length..thought it might be cutting it off)
Truncate 999999
SHOULD_LINEMERGE = false

sample data:

2017-07-11 08:54:12,815 31 [INFO] - - 5ms textHere MoreTEXT here[652].moretextHere(): MoreTextHere

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi mwdbhyat,
if you set the time format (with dot or comma), it's correct that Splunk doesn't recognize both timestamps but only ther one that matches your settings.
If instead you don't set the time format and leave that Splunk recognizes time format, it recognizes both the time formats.
I created a file with your two time formats and Splunk recognized both of them.
Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...