Getting Data In

Why is Timestamp format being ignored?

mala_banana
Engager

Problem:
Timestamp format setting is ignored when sending request

I have created SourceType "test" with settings
Timestamp format: %s,%3N
Timestamp fields: time

Created HTTP Event Collector with settings
Source Type: test
Restarted Splunk

And when making a request
http://banana:8088/services/collector/event/1.0
Body:
{
    "time":"1653643363,529",
    "sourcetype": "test",
    "event":{
        "id":"1",
        "severity":"Information",
        "message":"Test",
    }
}

Response with status 400 is returned
"text": "Error in handling indexed fields",
"code": 15,
"invalid-event-number": 0

Why is timestamp format ignored (works with "." but not with ",")?

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Because HEC input expects the time field to contain unix timestamp with fraction part delimited by a dot, not a comma. It's jusy how it works. The "timeformat" here is not redefineable. This is just how it works.

Oh, and by default, when you use HEC the timestamp is not extracted from the raw event but is either read from the time field supplied with the event (as you did in your example) or assumed from the system time. In order to parse timestamp from the event data you'd have to not provide the time field and add "?auto_extract_timestamp=true" to your REST endpoint url.

Notice that in your example you don't have any time-related info in the event.

0 Karma

mala_banana
Engager

Thank you for quick reply.

Iam using Splunk.Logging.TraceListener.Core package and setup is done like in this example for HTTP Event Collector, TraceListener. But this is using HttpEventCollectorEventInfo with constructor:

 

public HttpEventCollectorEventInfo(DateTime datetime, string id, string severity, string message, object data, Metadata metadata)
        {
            Timestamp = (datetime - new DateTime(1970, 1, 1)).TotalSeconds.ToString("#.000", Thread.CurrentThread.CurrentCulture);
            this.metadata = (metadata ?? new Metadata());
            Event = new LoggerEvent(id, severity, message, data);
        }

 

 Timestamp format here is dependent on my Culture and if culture has comma as delimiter, requests will always fail. Is there recommended workaround for this?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Sorry, can't help you here. Touched .Net only once some 15 years ago 🙂

I'd say that you need to somehow set proper format for your ToString to produce dot instead of comma but have no idea whatsoever how to do that.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...