Getting Data In

Why is Splunk not indexing in milli seconds?

siva_cg
Path Finder

Hi All,

I configured an input in which the timestamp field is in format 20180830112930314 (%Y%m%d%H%M%S%3N). The same has been configured in props.conf on Splunk Indexers, but still, I am seeing event time as 2018/08/30 11:29:30.000. I mean Splunk is showing 000 as milli seconds even if the timestamp field has milli seconds other than 000. Could you please help me to find out the issue? Thanks in advance.

0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

I have tested props.conf which you have provided with sample data and it is not extracting any time stamp so I guess splunk is assigning default timestamp which is on server at the time of indexing data.

When you use FIELD_DELIMITER, at that time I guess you need to provide all field name as well. So i have tested below props.conf with sample data and it is working fine.

[st]
TZ = GMT
FIELD_DELIMITER = ,
FIELD_NAMES = FIELD1,FIELD2,FIELD3,FIELD4,FIELD5,FIELD6,FIELD7,FIELD8,FIELD9,FIELD10,FIELD11,FIELD12,FIELD13,FIELD14,FIELD15,FIELD16
TIME_FORMAT = %Y%m%d%H%M%S%3N
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 17

Sample Data which I used

20180830113004270,1,45,09e3ab1a,IR1234,USA1234,,,IR3456,,989368837464,,hasdghghj,00000000,0,0.0

View solution in original post

harsmarvania57
SplunkTrust
SplunkTrust

I have tested props.conf which you have provided with sample data and it is not extracting any time stamp so I guess splunk is assigning default timestamp which is on server at the time of indexing data.

When you use FIELD_DELIMITER, at that time I guess you need to provide all field name as well. So i have tested below props.conf with sample data and it is working fine.

[st]
TZ = GMT
FIELD_DELIMITER = ,
FIELD_NAMES = FIELD1,FIELD2,FIELD3,FIELD4,FIELD5,FIELD6,FIELD7,FIELD8,FIELD9,FIELD10,FIELD11,FIELD12,FIELD13,FIELD14,FIELD15,FIELD16
TIME_FORMAT = %Y%m%d%H%M%S%3N
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 17

Sample Data which I used

20180830113004270,1,45,09e3ab1a,IR1234,USA1234,,,IR3456,,989368837464,,hasdghghj,00000000,0,0.0

mstjohn_splunk
Splunk Employee
Splunk Employee

Hey @siva_cg,

Did @harsmarvania57 's solution in the comment chain work? Let me know so that I can convert it to an answer. Then, you can approve it, and also, give him some sweet sweet upvotes

0 Karma

siva_cg
Path Finder

it worked well. Thank you @harsmarvania57

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Great, I have converted my comment to answer so you can accept & upvote it.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @siva_cg,

Can you please let us know, universal forwarder is sending data directly to Indexers or it goes to Heavy Forwarder and then Indexer ? Also it will be good to if you will provide your props.conf stanza.

0 Karma

siva_cg
Path Finder

Hi @harsmarvania57,

Data Flow is: UF -> Intermediate Forwarder -> Indexer
props.conf
[st]
TZ = GMT
FIELD_DELIMITER = ,
TIMESTAMP_FIELDS = Timestamp
TIME_FORMAT = %Y%m%d%H%M%S%3N
SHOULD_LINEMERGE = false

TIME_PREFIX = ^

LINE_BREAKER = (^\d{17})

MAX_TIMESTAMP_LOOKAHEAD = 17

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Your intermediate forwarder is Universal Forwarder or Heavy Forwarder ? If it is Heavy Forwarder, in that case you need to apply props.conf on Heavy Forwarder not on Indexer. If it is Universal forwarder then please provide sample data

siva_cg
Path Finder

It is Universal forwarder. Due to sensitivity of data, everything is masked and only event is provided. Apologies.
20180830113004270,1,45,09e3ab1a,IRxxxxxx,USAxxxxxxxx,,,IRxxxx,,989368837464,,xxxxxxxx,00000000,0,0.0

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>