Getting Data In

Can we submit events over HEC to an index other than main?

dstromberg
Path Finder

 

Hi people.

I'm attempting to submit an event over HEC to an index called dev_game-publishing.

This looks like:

curl -k 'https://so1:8088/services/collector/event?index=dev_game-publishing' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique"}'; echo
{"text":"Success","code":0}

Or:
curl -k 'https://so1:8088/services/collector/event' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique"}'; echo
{"text":"Success","code":0}

I've of course changed my token to nonsense.

Those "Success" strings sound to me like the submission has worked.

However, when I go to Search in the web interface and look for "index=*" (for All time), I see only a small number of my test events, and they're all on the main index.  

The token I'm using defaults to the dev_game-publishing index.

I believe I'm using a Trial license.

What do I need to do to get Splunk to accept events on the dev_game-publishing index?

Thanks!

Labels (1)
Tags (2)
0 Karma
1 Solution

dstromberg
Path Finder

 

I wound up putting the following in /opt/splunk/etc/splunk-launch.conf:

[http]
disabled = 0

[http://splunk_hec_token]
disabled = 0
token = big-token-string-goes-here-and-not-there
index = dev_game-publishing

 

Thanks for the tip giuseppe.

View solution in original post

0 Karma

PickleRick
SplunkTrust
SplunkTrust

If you use the /collector/event endpoint, unless you specify the index field in  your json, the event goes to the index specified in the input (or the default index if you haven't specify any for the input). Same goes for the other metadata fields - source, sourcetype. But you can also specify additional metadata witn your request to set them accordingly for the event, like

{ "source":"myserver",
"sourcetype":"my_sourcetype",
"index":"myindex",
"event":"blah blah whatever"}

You  have to remember though that if you have predefined set of indexes your hec input accepts, you can only send to those events using that token.

0 Karma

dstromberg
Path Finder

 

Hi @PickleRick 

I think I have my token set up to go to dev_game-publishing.

I tried:

[ansible@3e106d027298 paths]$ curl -k 'https://so1:8088/services/collector/event?index=dev_game-publishing' -H "Authorization: Splunk 11111111-2222-3333-4444-555555555555" -d '{"event": "hello world unique", "index": "dev_game-publishing"}'; echo
{"text":"Success","code":0}

...but I still don't see my event.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

To be honest I don't recall ever posting an event without timestamp and withou specifying the time explicitly so I'm not sure "when" this one should land. Theoreticaly, judging from the typical timestamp processing rules, it should get indexed at the indexer's (or HF's if your input is on a HF) local time.

But there are two things to check.

Firstly - of course check the _internal log for any errors.

Secondly - just to be on the safe side - if it's your dev index and you don't have many events there and no "side activity" do a

| tstats count where index=dev_game-publishing earliest=0 latest=now()*2

before and after your curl and see if it increases or simply search the index over All Time and sort by _index_time

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @dstromberg,

did you remembered to configure the destination index in the input section?

you can do this by GUi at [Setings -- Data Inputs -- HTTP Event Collector -- New Token] or in inputs.conf file.

you can find more infos at https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/UsetheHTTPEventCollector

Ciao.

Giuseppe

dstromberg
Path Finder

 

I wound up putting the following in /opt/splunk/etc/splunk-launch.conf:

[http]
disabled = 0

[http://splunk_hec_token]
disabled = 0
token = big-token-string-goes-here-and-not-there
index = dev_game-publishing

 

Thanks for the tip giuseppe.

0 Karma

dstromberg
Path Finder

 

My mistake.

It wasn't /opt/splunk/etc/splunk-launch.conf

It was /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf

The file content described was correct though.

 

0 Karma

dstromberg
Path Finder

 

Hi @gcusello 

I think I have enabled writing to dev_game-publishing

I've attached a screenshot that may help clarify what I do and don't have configured.

Thanks.

Screen Shot 2022-04-25 at 8.49.36 AM.png

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...