The Http Event Collector (HEC) accepts but doesn't index _json event with accented characters — is this a bug?

[joelroth]

Hi,

I've tracked down an issue we've been having where some events being sent through our HEC haven't been indexed, even though it responds with HTTP 200 and Success (0).

I've found two workarounds for this that resolve the issue, but I'm pretty sure the HEC should either have indexed the data anyway, or responded with some sort of error instead of Success.

My tests are done in PowerShell 5.1 with en-US culture.

The following snippet is an example of this, where it'll respond with 200 OK and not index the event:

Invoke-RestMethod -Method Post -Uri "https://splunk-hec.example.com:8088/services/collector/event" -Headers @{Authorization = "Splunk xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"} -ContentType "application/json" -Body @"
{
    "sourcetype":  "_json",
    "host":  "TESTHOST",
    "source":  "test:encoding",
    "event": { 
        "testtype": "charset not defined, sourcetype _json",
        "characterdata": "Têst vâlué thät has ąccents"
    }
}
"@

Also, if you specify charset=iso-8859-1 or charset=windows-1252, it also accepts and silently drops the event.

Workaround 1: Change the sourcetype to JSON (without the underscore)

# Change the sourcetype to json (no underscore)
Invoke-RestMethod -Method Post -Uri "https://splunk-hec.example.com:8088/services/collector/event" -Headers @{Authorization = "Splunk xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"} -ContentType "application/json" -Body @"
{
    "sourcetype":  "json",
    "host":  "TESTHOST",
    "source":  "test:encoding",
    "event": { 
        "testtype": "charset not defined, sourcetype json",
        "characterdata": "Têst vâlué thät has ąccents"
    }
}
"@

Workaround 2: Specify the charset with IRM

Invoke-RestMethod -Method Post -Uri "https://splunk-hec.example.com:8088/services/collector/event" -Headers @{Authorization = "Splunk xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"} -ContentType "application/json; charset=utf-8" -Body @"
{
    "sourcetype":  "_json",
    "host":  "TESTHOST",
    "source":  "test:encoding",
    "event": { 
        "testtype": "charset utf-8, sourcetype _json",
        "characterdata": "Têst vâlué thät has ąccents"
    }
}
"@

This is on Splunk Enterprise 7.1.2 using a heavy forwarder. Thanks!