Getting Data In

Why is Timestamp format being ignored?

mala_banana
Engager

Problem:
Timestamp format setting is ignored when sending request

I have created SourceType "test" with settings
Timestamp format: %s,%3N
Timestamp fields: time

Created HTTP Event Collector with settings
Source Type: test
Restarted Splunk

And when making a request
http://banana:8088/services/collector/event/1.0
Body:
{
    "time":"1653643363,529",
    "sourcetype": "test",
    "event":{
        "id":"1",
        "severity":"Information",
        "message":"Test",
    }
}

Response with status 400 is returned
"text": "Error in handling indexed fields",
"code": 15,
"invalid-event-number": 0

Why is timestamp format ignored (works with "." but not with ",")?

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Because HEC input expects the time field to contain unix timestamp with fraction part delimited by a dot, not a comma. It's jusy how it works. The "timeformat" here is not redefineable. This is just how it works.

Oh, and by default, when you use HEC the timestamp is not extracted from the raw event but is either read from the time field supplied with the event (as you did in your example) or assumed from the system time. In order to parse timestamp from the event data you'd have to not provide the time field and add "?auto_extract_timestamp=true" to your REST endpoint url.

Notice that in your example you don't have any time-related info in the event.

0 Karma

mala_banana
Engager

Thank you for quick reply.

Iam using Splunk.Logging.TraceListener.Core package and setup is done like in this example for HTTP Event Collector, TraceListener. But this is using HttpEventCollectorEventInfo with constructor:

 

public HttpEventCollectorEventInfo(DateTime datetime, string id, string severity, string message, object data, Metadata metadata)
        {
            Timestamp = (datetime - new DateTime(1970, 1, 1)).TotalSeconds.ToString("#.000", Thread.CurrentThread.CurrentCulture);
            this.metadata = (metadata ?? new Metadata());
            Event = new LoggerEvent(id, severity, message, data);
        }

 

 Timestamp format here is dependent on my Culture and if culture has comma as delimiter, requests will always fail. Is there recommended workaround for this?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Sorry, can't help you here. Touched .Net only once some 15 years ago 🙂

I'd say that you need to somehow set proper format for your ToString to produce dot instead of comma but have no idea whatsoever how to do that.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Manage Edge Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Edge Processor, onboard one instance, deploy a pipeline ...

Splunk Brings Proof of Control to Digital Asset Transfers

Digital assets now sit inside the core of financial services. HSBC launched its Tokenized Deposit Service in ...

Manage Ingest Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Ingest Processor pipeline, deploy it to the ...