Getting Data In

How do you send raw XML event logs to an HTTP event collector?

cameronharris6
New Member

Hello,

I have a source that creates raw XML event log data. I'd like to send this directly to the HTTP event collector in a raw format. I've viewed the available documentation here http://dev.splunk.com/view/event-collector/SP-CAAAE8Y#raw. However, I couldn't find the answer I'm looking for.

Is it possible to send raw XML from the EventLog directly to the HTTP event collector and have it be parsed correctly?

Thanks!

0 Karma
1 Solution

gjanders
SplunkTrust
SplunkTrust

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

View solution in original post

0 Karma

gjanders
SplunkTrust
SplunkTrust

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The HTTP Event Collector expects events to arrive in JSON format. You can send XML, but it has to be in a JSON wrapper. The wrapper includes meta-fields Splunk needs like timestamp and sourcetype.

---
If this reply helps you, Karma would be appreciated.
0 Karma

cameronharris6
New Member

Thanks for the response. Is there documentation available where I can find these meta fields required for Splunk?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

See the HEC docs at https://docs.splunk.com/Documentation/Splunk/7.2.1/Data/UsetheHTTPEventCollector#Send_data_to_HTTP_E...

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...