topic Re: How do you send raw XML event logs to an HTTP event collector? in Getting Data In

How do you send raw XML event logs to an HTTP event collector?

cameronharris6 — Fri, 14 Dec 2018 20:03:10 GMT

Hello,

I have a source that creates raw XML event log data. I'd like to send this directly to the HTTP event collector in a raw format. I've viewed the available documentation here http://dev.splunk.com/view/event-collector/SP-CAAAE8Y#raw. However, I couldn't find the answer I'm looking for.

Is it possible to send raw XML from the EventLog directly to the HTTP event collector and have it be parsed correctly?

Thanks!

Re: How do you send raw XML event logs to an HTTP event collector?

richgalloway — Fri, 14 Dec 2018 20:22:36 GMT

The HTTP Event Collector expects events to arrive in JSON format. You can send XML, but it has to be in a JSON wrapper. The wrapper includes meta-fields Splunk needs like timestamp and sourcetype.

Re: How do you send raw XML event logs to an HTTP event collector?

cameronharris6 — Sat, 15 Dec 2018 16:47:41 GMT

Thanks for the response. Is there documentation available where I can find these meta fields required for Splunk?

Re: How do you send raw XML event logs to an HTTP event collector?

gjanders — Sat, 15 Dec 2018 21:29:58 GMT

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

Re: How do you send raw XML event logs to an HTTP event collector?

richgalloway — Thu, 20 Dec 2018 22:34:53 GMT

See the HEC docs at https://docs.splunk.com/Documentation/Splunk/7.2.1/Data/UsetheHTTPEventCollector#Send_data_to_HTTP_Event_Collector