Miliseconds in timestamp are not extracted

[mvagionakis]

Hello all,

I have again something strange with my logs, the milliseconds in the _time field are not detected despite the applied props.conf parameters.

Here how it looks every line of my log:

1234 08/08/2019 15:08:56:924 123456789 0000049T6 TOTOPROCESS INF TOTO settings - process timeout set to 70 s

Here my props.conf:

 [mysourcetype] 
 TIME_PREFIX =^\d+\s\d{1,2}\/\d{1,2}\/\d{4}\s\d{1,2}\:\d{1,2}\:\d{1,2}\:\d{1,4}\s
 TIME_FORMAT = %d/%m/%Y %H:%M:%S:%3N
 MAX_TIMESTAMP_LOOKAHEAD = 29 
 TZ = GMT
 BREAK_ONLY_BEFORE =^\d+\s\d{1,2}\/\d{1,2}\/\d{4}\s\d{1,2}\:\d{1,2}\:\d{1,2}\:\d{1,4}\s

I tried all the possible solutions that I could find in the forum but nothing works.

The timestamp shows always three zeros for the milliseconds.

8/8/19 3:08:56.000 PM

I tried also by disabling the time_prefix, changing the time_format parameters,etc, but nothing helps.
At the beginning I thought that the props.conf were not being applied but I changed the "TZ" parameter (for testing purposes) and it was immediately applied so I don't think that the UF ignores my configuration.

I don't have any solution for the moment and any suggestion is welcome.

Thank you in advance.
Michael

[mayurr98]

okay I misunderstood your question.
here you go,

  [mysourcetype] 
  TIME_PREFIX = \d+\s+
  TIME_FORMAT = %d/%m/%Y %H:%M:%S%:%3N
  MAX_TIMESTAMP_LOOKAHEAD = 25 

[mvagionakis]

hey mayurr98

it's not better 😞

[gcusello]

Hi mvagionakis,
I'm not sure if there are some spaces at the beginning of each row, anyway try this:
(without spaces)

TIME_PREFIX =^\d+\s
TIME_FORMAT = %d/%m/%Y %H:%M:%S%:%3N
MAX_TIMESTAMP_LOOKAHEAD = 34 

(with spacese):

TIME_PREFIX =^\s+\d+\s
TIME_FORMAT = %d/%m/%Y %H:%M:%S%:%3N
MAX_TIMESTAMP_LOOKAHEAD = 34 

Bye.
Giuseppe

[mvagionakis]

Hey gcusello ,

thanks for your suggestion but I've already tried it but without success.

ps: no spaces at the beginning of the line.

[gcusello]

Hi mvagionakis,
where did you put this props.conf?
It must be located on Indexers.

Bye.
Giuseppe

[gcusello]

If you have heavy Forwarders, it must be appliaed on HFs.
Bye.
Giuseppe

[mvagionakis]

I applied it on indexers also....but still the same...

something strange happened....I disabled completely the props.conf and the timestamp is finally recognized by splunk without problem....

I also asked from the developer to remove the first digits so to start every line with the timestamp....I hope that it will be better once the modification done.

thank you again.

[neha898]

Hi There,
Please use TIME_FORMAT = %d/%m/%Y %H:%M:%S:%f
This works for me every time.

[neha898]

Hi mvagionakis,
This should solve your issue:
SHOULD_LINEMERGE=false
TIME_FORMAT=%d/%m/%Y %H:%M:%S:%f
TIME_PREFIX=^\d+

Your TIME_PREFIX is incorrect

[mvagionakis]

hello neha898,

once your config applied, UF stopped forwarding so I rollback to my old config.

[neha898]

These configs need to be applied on Indexer, not on UFs

[mvagionakis]

it is applied also on indexers.

[mvagionakis]

hey neha898,

it doesn't work 😞