Can’t search by sourcetype without index – data ingested via HEC, visible by index

[AJH2000]

Hi community,

I'm running into a permissions/visibility issue (I don't know) with an index created for receiving data via HTTP Event Collector (HEC) in Splunk Cloud.

Context:

• I have a custom index: rapid7

• Data is being successfully ingested via a Python script using the /services/collector/event endpoint

• The script defines index: rapid7 and sourcetype: rapid7:assets

• I can search the data using:

  index=rapid7

  and get results.

• I can also confirm the sourcetype:

  index=rapid7
  | stats count by sourcetype

Problem:

I am trying to add rapid7 to my role’s default search indexes, but when I go to:

Settings → Roles → admin → Edit → Indexes searched by default

The index rapid7 appear blank, I don't know that this is the all problem.

 What I’ve verified:

• The index exists and receives data

• The data is visible in Search & Reporting if I explicitly specify index=rapid7

• I am an admin user

• I confirmed the index is created (visible under Settings → Indexes)

My Questions:

1. What could cause an index to not appear in the "Indexes searched by default" list under role settings?

2. Could this be related to the app context of the index (e.g., if created under http_event_collector)?

3. Is there a way in Splunk Cloud to globally share an index created via HEC so it appears in role configuration menus?

I want to be able to search sourcetype="rapid7:assets" without explicitly specifying my index=rapid7, by including it in my role's default search indexes.

Any advice, experience or support links would be appreciated!

Thanks!

[PickleRick]

Ok, a word of advise - it's usually better to specify indexes explicitly than to have them as searched by default. Especially with an admkn role! It spares you unnecessary load on your environment for searches in which you haven't specified the indexes and it saves you a lot of debugging when you have different roles with different default indexes and people report mismatch in searches functionality. You have been warned.

One additional hint - it's way better to do a quick check with

| tstats count where index=rapid7 by sourcetype

than

index=rapid7 | stats count by sourcetype

The first one only checks the summarized indexed fields while yours needs to plow through all events from the index.

And there is something that doesn't add up.

On Cloud you cannot have the admin user role. You can only have sc_admin (which is a limited admin role). So if you're trying to edit the admin role you shouldn't be able to do so.

[gcusello]

Hi @AJH2000 ,

have you a stand-alone server or a distributed architecture?

if a stand alone server you should see all the indexes.

If instead, you have a distributed architecture and you are working on the Search Head, you don't see all the indexes in the Indexers.

The easiest approach is to add an empty index also on the Search Head, only to see this index in the dropdown lists.

Ciao.

Giuseppe

[livehybrid]

Hi @AJH2000 

It sounds like your HEC connection is working as expected, and you have confirmed that the data is being ingested, so I think your HEC configuration is all good.

You havent mentioned your deployment architecture however I suspect you are using a SH/SHC connecting to an indexer cluster. When you configured the index, did you also create the index on the SH/SHC ? If you didnt then it would explain why the index is not visible in the Edit Role screen.

Please make sure the index definition exists on the SH and then check again.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing