topic Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder? in Getting Data In

How to send JSON data (sent via HTTP POST) to a heavy forwarder?

packet_hunter — Mon, 03 Apr 2017 16:32:18 GMT

Currently I have a security appliance sending JSON data via HTTP POST to an all-in-one stand alone Splunk test instance.
Now I want to send the JSON data to an intermediate Heavy Forwarder in production (which feeds the indexers).

The test instance is receiving the json data via HTTP POST. A Splunk user account was created to pass the RESTful API data with a RESTfulAPI role and edit_tcp capabilities. The security appliance is configured with the username and password created previously, and is sending data to:

https://<SplunkAD.DR.ESS>:<PORT>/services/receivers/simple? host=<SecurityApplianceAddress>&source=wmps sourcetype=fe_json

The stand alone test instance has an enabled receiver directly on the indexer (I believe) and receives the data without a problem.

At this point I need to reconfigure the security appliance to send data to the heavy fwdr and I am not sure how to set up a receiver on the heavy forwarder so that it will act the same as the test instance. After the connection is established I would like to edit down the amount of data from the security appliance to only the desired fields by changing the .conf files.

Any advice or reference is appreciated.

Thank you

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

woodcock — Mon, 03 Apr 2017 19:20:42 GMT

You can click on the gear icon in the upper-right of your question and re-edit it. Even with your clarification, I am certain that I do not understand what you need.

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

packet_hunter — Mon, 03 Apr 2017 19:45:06 GMT

ok here is another attempt to explain, I hope it makes sense

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

woodcock — Tue, 04 Apr 2017 00:02:58 GMT

MUCH better! Now I know that I am not the right guy to help but now the right guy will know that he is!

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

packet_hunter — Tue, 04 Apr 2017 00:13:49 GMT

sorry about all my noob confusion

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

hardikJsheth — Tue, 04 Apr 2017 07:31:38 GMT

You can use same method, as you were doing with single test instance

https://answers.splunk.com/answers/226482/splunk-rest-api-data-input-receiverssimple-informa.html

In case you can modify header of the HTTP Post, you can also have a look at HTTP Event Collector.
http://dev.splunk.com/view/event-collector/SP-CAAAE73#scen1

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

packet_hunter — Thu, 06 Apr 2017 19:37:31 GMT

Thank you for the reply, I got the security appliance to send to the heavy forwarder, but now I need a inputs.conf to send it to the indexers.

Any advice on sending this to the indexer is greatly appreciated.
Thank you

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

woodcock — Sat, 08 Apr 2017 18:40:20 GMT

To have your heavy forwarder send to the indexers without taking a double license hit, make sure that you set outputs.conf (not inputs.conf) like this:

[tcpout]
defaultGroup=YourIndexerGroupHere
indexAndForward=false

http://docs.splunk.com/Documentation/SplunkCloud/6.5.1612/Forwarding/Configureforwarderswithoutputs.conf

Re: How to send JSON data (sent via HTTP POST) to a heavy forwarder?

packet_hunter — Mon, 10 Apr 2017 14:12:28 GMT

Thank you for the instructions. When I checked we had that already setup outputs.conf like that.

I am currently trying to find out where we went wrong but as I move thru the flow I will find it and post the resolution.

Currently the security appliance acts as a server to the heavy fwder, and we don't need inputs.conf because the appliance sends host, sourcetype, index, time. I think we just mis-configured where we assigned the index to... but still looking