Getting Data In

Send data via Splunk REST API to a Heavy Forwarder requires index on the HF?

xiaoweiwu
New Member

We have indexer cluster setup. We are trying to use the REST API on the Heavy Forwarder to receive data update. But i am not sure if creating index on the HF is required to make sure indexer gets the data.

I do see an earlier posting is similar to this one. Since it's an old posting, i just want to confirm If using REST API on the HF, while output.conf can point to the indexer/cluster would work, with no index stored on the HF.
https://answers.splunk.com/answers/38176/data-indexed-via-rest-api-then-forwarded-on-to-another-inde...

0 Karma

koshyk
Super Champion
  • indexes.conf is NOT required at HF.
  • But it is very important that your HF forwards (outputs.conf) to Indexers
  • props.conf/tranforms.conf is required at HF and indextime changes happen at HF though it sends data to indexers
0 Karma

xiaoweiwu
New Member

I think i've gone on a tangent. All I wanted to know is for REST API, it's pushing data to this URI.
http://[Splunk IP]:8089/services/receivers/simple?index=myindex&source=sccm.py&sourcetype=web_event
If no index is required on the HF, then it will be sent to the indexer directly.

Is that correct?

0 Karma

xiaoweiwu
New Member

i do want to mention. I have an index on the HF. I am not sure why it is not in the indexes.conf.

0 Karma

koshyk
Super Champion

it might be one or another app contain indexes.conf
if you do a find on all apps with indexes.conf or a btool then it shows which app have created it incorrectly

/opt/splunk/bin/splunk cmd btool indexes list --debug > /tmp/indexes.btool

in your HF

0 Karma

xiaoweiwu
New Member

ok, I was able to find on the HF at this location for the indexes.conf.

/opt/splunk/etc/apps/search/local

And I've found the index in that indexes.conf file.
[myindex]
coldPath = $SPLUNK_DB/myindex/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/myindex/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/myindex/thaweddb

0 Karma

xiaoweiwu
New Member

I did look at my HF.
There is no entry in the indexes.conf at this location /opt/splunk/etc/system/default.

Here is outputs.conf in /opt/splunk/etc/system/local

[tcpout]
defaultGroup = default-autolb-group
indexAndForward = 1

[tcpout:default-autolb-group]
server = xx.xx.xx.xx:9997

[tcpout-server://xx.xx.xx.xx:9997]

0 Karma

starcher
Influencer

HTTP Event collector is better choice than trying to post to the api.

0 Karma

xiaoweiwu
New Member

Not to ignore your comments. Why is the event collector is better? Just want to know your view point.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...