Getting Data In

Why is Timestamp format being ignored?

mala_banana
Engager

Problem:
Timestamp format setting is ignored when sending request

I have created SourceType "test" with settings
Timestamp format: %s,%3N
Timestamp fields: time

Created HTTP Event Collector with settings
Source Type: test
Restarted Splunk

And when making a request
http://banana:8088/services/collector/event/1.0
Body:
{
    "time":"1653643363,529",
    "sourcetype": "test",
    "event":{
        "id":"1",
        "severity":"Information",
        "message":"Test",
    }
}

Response with status 400 is returned
"text": "Error in handling indexed fields",
"code": 15,
"invalid-event-number": 0

Why is timestamp format ignored (works with "." but not with ",")?

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Because HEC input expects the time field to contain unix timestamp with fraction part delimited by a dot, not a comma. It's jusy how it works. The "timeformat" here is not redefineable. This is just how it works.

Oh, and by default, when you use HEC the timestamp is not extracted from the raw event but is either read from the time field supplied with the event (as you did in your example) or assumed from the system time. In order to parse timestamp from the event data you'd have to not provide the time field and add "?auto_extract_timestamp=true" to your REST endpoint url.

Notice that in your example you don't have any time-related info in the event.

0 Karma

mala_banana
Engager

Thank you for quick reply.

Iam using Splunk.Logging.TraceListener.Core package and setup is done like in this example for HTTP Event Collector, TraceListener. But this is using HttpEventCollectorEventInfo with constructor:

 

public HttpEventCollectorEventInfo(DateTime datetime, string id, string severity, string message, object data, Metadata metadata)
        {
            Timestamp = (datetime - new DateTime(1970, 1, 1)).TotalSeconds.ToString("#.000", Thread.CurrentThread.CurrentCulture);
            this.metadata = (metadata ?? new Metadata());
            Event = new LoggerEvent(id, severity, message, data);
        }

 

 Timestamp format here is dependent on my Culture and if culture has comma as delimiter, requests will always fail. Is there recommended workaround for this?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Sorry, can't help you here. Touched .Net only once some 15 years ago 🙂

I'd say that you need to somehow set proper format for your ToString to produce dot instead of comma but have no idea whatsoever how to do that.

Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Splunk Observability Synthetic Monitoring - Resolved Incident on Detector Alerts

We’ve discovered a bug that affected the auto-clear of Synthetic Detectors in the Splunk Synthetic Monitoring ...

Video | Tom’s Smartness Journey Continues

Remember Splunk Community member Tom Kopchak? If you caught the first episode of our Smartness interview ...