Can we submit events over HEC to an index other than main?

[dstromberg]

Hi people.

I'm attempting to submit an event over HEC to an index called dev_game-publishing.

This looks like:

curl -k 'https://so1:8088/services/collector/event?index=dev_game-publishing' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique"}'; echo
{"text":"Success","code":0}

Or:
curl -k 'https://so1:8088/services/collector/event' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique"}'; echo
{"text":"Success","code":0}

I've of course changed my token to nonsense.

Those "Success" strings sound to me like the submission has worked.

However, when I go to Search in the web interface and look for "index=*" (for All time), I see only a small number of my test events, and they're all on the main index.  

The token I'm using defaults to the dev_game-publishing index.

I believe I'm using a Trial license.

What do I need to do to get Splunk to accept events on the dev_game-publishing index?

Thanks!

[PickleRick]

If you use the /collector/event endpoint, unless you specify the index field in  your json, the event goes to the index specified in the input (or the default index if you haven't specify any for the input). Same goes for the other metadata fields - source, sourcetype. But you can also specify additional metadata witn your request to set them accordingly for the event, like

{ "source":"myserver",
"sourcetype":"my_sourcetype",
"index":"myindex",
"event":"blah blah whatever"}

You  have to remember though that if you have predefined set of indexes your hec input accepts, you can only send to those events using that token.

[dstromberg]

Hi @PickleRick 

I think I have my token set up to go to dev_game-publishing.

I tried:

[ansible@3e106d027298 paths]$ curl -k 'https://so1:8088/services/collector/event?index=dev_game-publishing' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique", "index": "dev_game-publishing"}'; echo
{"text":"Success","code":0}

...but I still don't see my event.

[PickleRick]

To be honest I don't recall ever posting an event without timestamp and withou specifying the time explicitly so I'm not sure "when" this one should land. Theoreticaly, judging from the typical timestamp processing rules, it should get indexed at the indexer's (or HF's if your input is on a HF) local time.

But there are two things to check.

Firstly - of course check the _internal log for any errors.

Secondly - just to be on the safe side - if it's your dev index and you don't have many events there and no "side activity" do a

| tstats count where index=dev_game-publishing earliest=0 latest=now()*2

before and after your curl and see if it increases or simply search the index over All Time and sort by _index_time

[gcusello]

Hi @dstromberg,

did you remembered to configure the destination index in the input section?

you can do this by GUi at [Setings -- Data Inputs -- HTTP Event Collector -- New Token] or in inputs.conf file.

you can find more infos at https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/UsetheHTTPEventCollector

Ciao.

Giuseppe

[dstromberg · Accepted solution]

I wound up putting the following in /opt/splunk/etc/splunk-launch.conf:

[http]
disabled = 0

[http://splunk_hec_token]
disabled = 0
token = big-token-string-goes-here-and-not-there
index = dev_game-publishing

Thanks for the tip giuseppe.

[dstromberg]

My mistake.

It wasn't /opt/splunk/etc/splunk-launch.conf

It was /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf

The file content described was correct though.

[dstromberg]

Hi @gcusello 

I think I have enabled writing to dev_game-publishing

I've attached a screenshot that may help clarify what I do and don't have configured.

Thanks.