topic Re: Timestamp format in Getting Data In

Why is Timestamp format being ignored?

mala_banana — Tue, 31 May 2022 14:07:22 GMT

Problem:
Timestamp format setting is ignored when sending request

I have created SourceType "test" with settings
Timestamp format: %s,%3N
Timestamp fields: time

Created HTTP Event Collector with settings
Source Type: test
Restarted Splunk

And when making a request
http://banana:8088/services/collector/event/1.0
Body:
{
"time":"1653643363,529",
"sourcetype": "test",
"event":{
"id":"1",
"severity":"Information",
"message":"Test",
}
}

Response with status 400 is returned
"text": "Error in handling indexed fields",
"code": 15,
"invalid-event-number": 0

Why is timestamp format ignored (works with "." but not with ",")?

Re: Timestamp format

PickleRick — Tue, 31 May 2022 10:20:58 GMT

Because HEC input expects the time field to contain unix timestamp with fraction part delimited by a dot, not a comma. It's jusy how it works. The "timeformat" here is not redefineable. This is just how it works.

Oh, and by default, when you use HEC the timestamp is not extracted from the raw event but is either read from the time field supplied with the event (as you did in your example) or assumed from the system time. In order to parse timestamp from the event data you'd have to not provide the time field and add "?auto_extract_timestamp=true" to your REST endpoint url.

Notice that in your example you don't have any time-related info in the event.

Re: Timestamp format

mala_banana — Tue, 31 May 2022 11:04:59 GMT

Thank you for quick reply.

Iam using Splunk.Logging.TraceListener.Core package and setup is done like in this example for HTTP Event Collector, TraceListener. But this is using HttpEventCollectorEventInfo with constructor:

public HttpEventCollectorEventInfo(DateTime datetime, string id, string severity, string message, object data, Metadata metadata) { Timestamp = (datetime - new DateTime(1970, 1, 1)).TotalSeconds.ToString("#.000", Thread.CurrentThread.CurrentCulture); this.metadata = (metadata ?? new Metadata()); Event = new LoggerEvent(id, severity, message, data); }

Timestamp format here is dependent on my Culture and if culture has comma as delimiter, requests will always fail. Is there recommended workaround for this?

Re: Timestamp format

PickleRick — Tue, 31 May 2022 18:33:37 GMT

Sorry, can't help you here. Touched .Net only once some 15 years ago 🙂

I'd say that you need to somehow set proper format for your ToString to produce dot instead of comma but have no idea whatsoever how to do that.