Getting Data In

Is it possible to write a lightweight custom forwarder to collect data, and not have to deploy the universal forwarder on every machine that needs monitoring?

sbroberg
Engager

We're trying to determine if Splunk is appropriate for our scenario, which is to monitor our own agent that runs on our users' PCs and Macs. We have several million customers, and it seems like it would be burdensome (based on the posted system requirements) to deploy a universal forwarder onto every user's machine (plus I'm not sure how we would integrate this into the existing installer & upgrader features of our app).

All we really need to do is to periodically upload (either daily or hourly) a .json file containing some structured data for metrics that describe the current state of the app during that interval, as well as some exception events (crashes, thrown exceptions of note, etc.). In theory, this would just be an HTTPS call to our Splunk instance with the appropriate authentication, but I can't locate any online documentation that describes this - the REST API seems to be more about controlling existing collectors and doing extraction & analysis of collected data.

0 Karma
1 Solution

xpac
SplunkTrust
SplunkTrust

Hey,

as far as my experience goes, the Universal Forwarder is not really ressource intense, however there is an option that fits so well that it feels as it had only been made for your question. 😉

Check out the Splunk HTTP Event Collector. There is a lot of documentation that allows to send data via HTTP, control authentication and other stuff.
If you're Python-literate, you could take a look at this class written by George Starcher, it's really fast and easily transmits large amounts of data to Splunk: Splunk-Class-httpevent

Hope that helps - if it does I'd be happy if you would upvote/accept this answer, so others could profit from it. 🙂

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

This sounds like the perfect case for the HTTP Event Collector (HEC). The HEC reads JSON-encoded events sent via HTTP(S). The universal forwarder is not needed. See http://docs.splunk.com/Documentation/Splunk/7.1.1/Data/HECWalkthrough.

---
If this reply helps you, Karma would be appreciated.
0 Karma

xpac
SplunkTrust
SplunkTrust

Hey,

as far as my experience goes, the Universal Forwarder is not really ressource intense, however there is an option that fits so well that it feels as it had only been made for your question. 😉

Check out the Splunk HTTP Event Collector. There is a lot of documentation that allows to send data via HTTP, control authentication and other stuff.
If you're Python-literate, you could take a look at this class written by George Starcher, it's really fast and easily transmits large amounts of data to Splunk: Splunk-Class-httpevent

Hope that helps - if it does I'd be happy if you would upvote/accept this answer, so others could profit from it. 🙂

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...