topic Re: Splunk Forwarder - timestamp switched!!! in Deployment Architecture

Splunk Forwarder - timestamp switched!!!

verbal_666 — Tue, 05 Nov 2019 10:03:32 GMT

Splunk Forwarder 7.0.2 (Windows 64 bit)
Splunk Enterprise 7.0.0 (Linux)

The strange problem:
until 31/10, Europe TZ, logs are correctly indexed with this format (no Indexer props configured, set by default)

[31/10/2019 23:59:55] [URL] [HTTP_RC]

From 1/11, Splunk starts switching %m & %d in USA_TZ, so

[01/11/2019 00:00:00] [URL] [HTTP_RC]

become with timestamp (Europe) == 11/01/2019 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! And also today, i have 11/05/2019, but logs are

[05/11/2019 00:00:00] [URL] [HTTP_RC]

Why??????????????????? I know i should force the props in Indexers, such as,

CHARSET = UTF-8
TIME_FORMAT = %d/%m/%Y %T
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true

But why until 31/10 i have NO DATE PARSING problem at all?

Suggestions? Thanks.

Re: Splunk Forwarder - timestamp switched!!!

gcusello — Tue, 05 Nov 2019 10:14:30 GMT

Hi verbal_666,
Splunk by default use the american date format mm/dd/yyyy, so if you want to use an european format you have to forma time format in props.conf.
so in sourcetype stanza of props.conf on Indexers you have to insert:

[my_sourcetype]
TIME_FORMAT = %m/%d/%Y %H:%M:%S
TIMEPREFIX = ^\[
....

For more details see https://docs.splunk.com/Documentation/Splunk/8.0.0/Admin/Propsconf .

Ciao.
Giuseppe

Re: Splunk Forwarder - timestamp switched!!!

verbal_666 — Tue, 05 Nov 2019 11:16:08 GMT

Thanks.
As i wrote in main post, i'll (eventually) write the correct props in Indexer(s) asap.

The question now is: why i have many others logs, from other forwarders, logging like

2019/05/11

without their sourcetype stanza in props, and are still logging Europe since 01/11?

Also, on same host, same forwarder, others logs are logged randomly 05/11 and 11/05, today.
Example, 3 events are 05/11, other 3 are 11/05! Strange behaviour.....

Thanks, anyway.

Re: Splunk Forwarder - timestamp switched!!!

gcusello — Tue, 05 Nov 2019 11:55:42 GMT

Hi verbal_666,
Because the problem there's only when there could be a mistake between american and european date format, if you have yyyy-mm-dd as date format there isn't any mistake.
The problem is born because Splunk is american and Europe has a different date format.
For this reason it's always a best practice to format date in props.conf when there could be this mistake.

Ciao.
Giuseppe

Re: Splunk Forwarder - timestamp switched!!!

verbal_666 — Tue, 05 Nov 2019 12:11:08 GMT

Ok, thanks.

Is right the

TIME_FORMAT = %m/%d/%Y %H:%M:%S

you wrote up?

Shouldn't be

TIME_FORMAT = %d/%m/%Y %H:%M:%S

date +'%d/%m/%Y %H:%M:%S'
05/11/2019 13:10:24

Right?

Re: Splunk Forwarder - timestamp switched!!!

verbal_666 — Tue, 05 Nov 2019 12:19:52 GMT

Anyway, yes, this props did it,

TIME_FORMAT=[%d/%m/%Y %T]
SHOULD_LINEMERGE=false

Re: Splunk Forwarder - timestamp switched!!!

gcusello — Tue, 05 Nov 2019 13:24:30 GMT

Yes sorry, it's correct

TIME_FORMAT = %d/%m/%Y %H:%M:%S

otherwise it's unnecessary.

Ciao.
Giuseppe

Re: Splunk Forwarder - timestamp switched!!!

verbal_666 — Tue, 05 Nov 2019 14:25:14 GMT

Thanks again.