Getting Data In

How do you send raw XML event logs to an HTTP event collector?

cameronharris6
New Member

Hello,

I have a source that creates raw XML event log data. I'd like to send this directly to the HTTP event collector in a raw format. I've viewed the available documentation here http://dev.splunk.com/view/event-collector/SP-CAAAE8Y#raw. However, I couldn't find the answer I'm looking for.

Is it possible to send raw XML from the EventLog directly to the HTTP event collector and have it be parsed correctly?

Thanks!

0 Karma
1 Solution

gjanders
SplunkTrust
SplunkTrust

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

View solution in original post

0 Karma

gjanders
SplunkTrust
SplunkTrust

If you use the raw endpoint (/services/collector/raw) you can send data in any format, not just JSON-wrapped.
If you use the event endpoint, send the data in a json wrapper and it should get through.

The metadata fields for the event endpoint are here on that same page you can look for services/collector/raw, although the documentation you posted does explain the raw endpoint doesn't expect metadata...

Note that if using the raw endpoint the standard Splunk data processing rules apply, you will need to use the correct sourcetype to ensure the data works as expected.
If you use the event endpoint, then the timestamp parsing (including time offset) is based on the metadata, not the raw data, and the line breaking is also done by the event endpoint, details in https://wiki.splunk.com/File:Splunk_EventProcessing_v20.0_UF_Indexer.png look for the dotted line mentioning "JSON" if you use the event endpoint.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The HTTP Event Collector expects events to arrive in JSON format. You can send XML, but it has to be in a JSON wrapper. The wrapper includes meta-fields Splunk needs like timestamp and sourcetype.

---
If this reply helps you, Karma would be appreciated.
0 Karma

cameronharris6
New Member

Thanks for the response. Is there documentation available where I can find these meta fields required for Splunk?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

See the HEC docs at https://docs.splunk.com/Documentation/Splunk/7.2.1/Data/UsetheHTTPEventCollector#Send_data_to_HTTP_E...

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...

Major Splunk Upgrade – Prepare your Environment for Splunk 10 Now!

Attention App Developers: Test Your Apps with the Splunk 10.0 Beta and Ensure Compatibility Before the ...

Stay Connected: Your Guide to June Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...