Getting Data In

How do you send raw XML event logs to an HTTP event collector?

cameronharris6
New Member

Hello,

I have a source that creates raw XML event log data. I'd like to send this directly to the HTTP event collector in a raw format. I've viewed the available documentation here http://dev.splunk.com/view/event-collector/SP-CAAAE8Y#raw. However, I couldn't find the answer I'm looking for.

Is it possible to send raw XML from the EventLog directly to the HTTP event collector and have it be parsed correctly?

Thanks!

0 Karma
1 Solution

gjanders
SplunkTrust
SplunkTrust

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

View solution in original post

0 Karma

gjanders
SplunkTrust
SplunkTrust

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The HTTP Event Collector expects events to arrive in JSON format. You can send XML, but it has to be in a JSON wrapper. The wrapper includes meta-fields Splunk needs like timestamp and sourcetype.

---
If this reply helps you, Karma would be appreciated.
0 Karma

cameronharris6
New Member

Thanks for the response. Is there documentation available where I can find these meta fields required for Splunk?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

See the HEC docs at https://docs.splunk.com/Documentation/Splunk/7.2.1/Data/UsetheHTTPEventCollector#Send_data_to_HTTP_E...

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...