Why is Splunk not indexing in milli seconds?

[siva_cg]

Hi All,

I configured an input in which the timestamp field is in format 20180830112930314 (%Y%m%d%H%M%S%3N). The same has been configured in props.conf on Splunk Indexers, but still, I am seeing event time as 2018/08/30 11:29:30.000. I mean Splunk is showing 000 as milli seconds even if the timestamp field has milli seconds other than 000. Could you please help me to find out the issue? Thanks in advance.

[harsmarvania57 · Accepted solution]

I have tested props.conf which you have provided with sample data and it is not extracting any time stamp so I guess splunk is assigning default timestamp which is on server at the time of indexing data.

When you use FIELD_DELIMITER, at that time I guess you need to provide all field name as well. So i have tested below props.conf with sample data and it is working fine.

[st]
TZ = GMT
FIELD_DELIMITER = ,
FIELD_NAMES = FIELD1,FIELD2,FIELD3,FIELD4,FIELD5,FIELD6,FIELD7,FIELD8,FIELD9,FIELD10,FIELD11,FIELD12,FIELD13,FIELD14,FIELD15,FIELD16
TIME_FORMAT = %Y%m%d%H%M%S%3N
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 17

Sample Data which I used

20180830113004270,1,45,09e3ab1a,IR1234,USA1234,,,IR3456,,989368837464,,hasdghghj,00000000,0,0.0

[mstjohn_splunk]

Hey @siva_cg,

Did @harsmarvania57 's solution in the comment chain work? Let me know so that I can convert it to an answer. Then, you can approve it, and also, give him some sweet sweet upvotes

[siva_cg]

it worked well. Thank you @harsmarvania57

[harsmarvania57]

Great, I have converted my comment to answer so you can accept & upvote it.

[harsmarvania57]

Hi @siva_cg,

Can you please let us know, universal forwarder is sending data directly to Indexers or it goes to Heavy Forwarder and then Indexer ? Also it will be good to if you will provide your props.conf stanza.

[siva_cg]

Hi @harsmarvania57,

Data Flow is: UF -> Intermediate Forwarder -> Indexer
props.conf
[st]
TZ = GMT
FIELD_DELIMITER = ,
TIMESTAMP_FIELDS = Timestamp
TIME_FORMAT = %Y%m%d%H%M%S%3N
SHOULD_LINEMERGE = false

TIME_PREFIX = ^

LINE_BREAKER = (^\d{17})

MAX_TIMESTAMP_LOOKAHEAD = 17

[harsmarvania57]

Your intermediate forwarder is Universal Forwarder or Heavy Forwarder ? If it is Heavy Forwarder, in that case you need to apply props.conf on Heavy Forwarder not on Indexer. If it is Universal forwarder then please provide sample data

[siva_cg]

It is Universal forwarder. Due to sensitivity of data, everything is masked and only event is provided. Apologies.
20180830113004270,1,45,09e3ab1a,IRxxxxxx,USAxxxxxxxx,,,IRxxxx,,989368837464,,xxxxxxxx,00000000,0,0.0