Getting Data In

How to send JSON data (sent via HTTP POST) to a heavy forwarder?

packet_hunter
Contributor

Currently I have a security appliance sending JSON data via HTTP POST to an all-in-one stand alone Splunk test instance.
Now I want to send the JSON data to an intermediate Heavy Forwarder in production (which feeds the indexers).

The test instance is receiving the json data via HTTP POST. A Splunk user account was created to pass the RESTful API data with a RESTfulAPI role and edit_tcp capabilities. The security appliance is configured with the username and password created previously, and is sending data to:

https://<SplunkAD.DR.ESS>:<PORT>/services/receivers/simple? host=<SecurityApplianceAddress>&source=wmps sourcetype=fe_json

The stand alone test instance has an enabled receiver directly on the indexer (I believe) and receives the data without a problem.

At this point I need to reconfigure the security appliance to send data to the heavy fwdr and I am not sure how to set up a receiver on the heavy forwarder so that it will act the same as the test instance. After the connection is established I would like to edit down the amount of data from the security appliance to only the desired fields by changing the .conf files.

Any advice or reference is appreciated.

Thank you

Thank you

0 Karma

woodcock
Esteemed Legend

To have your heavy forwarder send to the indexers without taking a double license hit, make sure that you set outputs.conf (not inputs.conf) like this:

[tcpout]
defaultGroup=YourIndexerGroupHere
indexAndForward=false

http://docs.splunk.com/Documentation/SplunkCloud/6.5.1612/Forwarding/Configureforwarderswithoutputs....

0 Karma

packet_hunter
Contributor

Thank you for the instructions. When I checked we had that already setup outputs.conf like that.

I am currently trying to find out where we went wrong but as I move thru the flow I will find it and post the resolution.

Currently the security appliance acts as a server to the heavy fwder, and we don't need inputs.conf because the appliance sends host, sourcetype, index, time. I think we just mis-configured where we assigned the index to... but still looking

0 Karma

hardikJsheth
Motivator

You can use same method, as you were doing with single test instance

https://answers.splunk.com/answers/226482/splunk-rest-api-data-input-receiverssimple-informa.html

In case you can modify header of the HTTP Post, you can also have a look at HTTP Event Collector.
http://dev.splunk.com/view/event-collector/SP-CAAAE73#scen1

0 Karma

packet_hunter
Contributor

Thank you for the reply, I got the security appliance to send to the heavy forwarder, but now I need a inputs.conf to send it to the indexers.

Any advice on sending this to the indexer is greatly appreciated.
Thank you

0 Karma

woodcock
Esteemed Legend

You can click on the gear icon in the upper-right of your question and re-edit it. Even with your clarification, I am certain that I do not understand what you need.

0 Karma

packet_hunter
Contributor

ok here is another attempt to explain, I hope it makes sense

0 Karma

woodcock
Esteemed Legend

MUCH better! Now I know that I am not the right guy to help but now the right guy will know that he is!

0 Karma

packet_hunter
Contributor

sorry about all my noob confusion

Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...