Getting Data In

Can we submit events over HEC to an index other than main?

dstromberg
Path Finder

 

Hi people.

I'm attempting to submit an event over HEC to an index called dev_game-publishing.

This looks like:

curl -k 'https://so1:8088/services/collector/event?index=dev_game-publishing' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique"}'; echo
{"text":"Success","code":0}

Or:
curl -k 'https://so1:8088/services/collector/event' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique"}'; echo
{"text":"Success","code":0}

I've of course changed my token to nonsense.

Those "Success" strings sound to me like the submission has worked.

However, when I go to Search in the web interface and look for "index=*" (for All time), I see only a small number of my test events, and they're all on the main index.  

The token I'm using defaults to the dev_game-publishing index.

I believe I'm using a Trial license.

What do I need to do to get Splunk to accept events on the dev_game-publishing index?

Thanks!

Labels (1)
Tags (2)
0 Karma
1 Solution

dstromberg
Path Finder

 

I wound up putting the following in /opt/splunk/etc/splunk-launch.conf:

[http]
disabled = 0

[http://splunk_hec_token]
disabled = 0
token = big-token-string-goes-here-and-not-there
index = dev_game-publishing

 

Thanks for the tip giuseppe.

View solution in original post

0 Karma

PickleRick
Ultra Champion

If you use the /collector/event endpoint, unless you specify the index field in  your json, the event goes to the index specified in the input (or the default index if you haven't specify any for the input). Same goes for the other metadata fields - source, sourcetype. But you can also specify additional metadata witn your request to set them accordingly for the event, like

{ "source":"myserver",
"sourcetype":"my_sourcetype",
"index":"myindex",
"event":"blah blah whatever"}

You  have to remember though that if you have predefined set of indexes your hec input accepts, you can only send to those events using that token.

0 Karma

dstromberg
Path Finder

 

Hi @PickleRick 

I think I have my token set up to go to dev_game-publishing.

I tried:

[ansible@3e106d027298 paths]$ curl -k 'https://so1:8088/services/collector/event?index=dev_game-publishing' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique", "index": "dev_game-publishing"}'; echo
{"text":"Success","code":0}

...but I still don't see my event.

0 Karma

PickleRick
Ultra Champion

To be honest I don't recall ever posting an event without timestamp and withou specifying the time explicitly so I'm not sure "when" this one should land. Theoreticaly, judging from the typical timestamp processing rules, it should get indexed at the indexer's (or HF's if your input is on a HF) local time.

But there are two things to check.

Firstly - of course check the _internal log for any errors.

Secondly - just to be on the safe side - if it's your dev index and you don't have many events there and no "side activity" do a

| tstats count where index=dev_game-publishing earliest=0 latest=now()*2

before and after your curl and see if it increases or simply search the index over All Time and sort by _index_time

0 Karma

gcusello
Legend

Hi @dstromberg,

did you remembered to configure the destination index in the input section?

you can do this by GUi at [Setings -- Data Inputs -- HTTP Event Collector -- New Token] or in inputs.conf file.

you can find more infos at https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/UsetheHTTPEventCollector

Ciao.

Giuseppe

dstromberg
Path Finder

 

I wound up putting the following in /opt/splunk/etc/splunk-launch.conf:

[http]
disabled = 0

[http://splunk_hec_token]
disabled = 0
token = big-token-string-goes-here-and-not-there
index = dev_game-publishing

 

Thanks for the tip giuseppe.

0 Karma

dstromberg
Path Finder

 

My mistake.

It wasn't /opt/splunk/etc/splunk-launch.conf

It was /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf

The file content described was correct though.

 

0 Karma

dstromberg
Path Finder

 

Hi @gcusello 

I think I have enabled writing to dev_game-publishing

I've attached a screenshot that may help clarify what I do and don't have configured.

Thanks.

Screen Shot 2022-04-25 at 8.49.36 AM.png

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...