topic Re: Splunk Ingest Actions in Getting Data In

Splunk Ingest Actions

tbarn005 — Thu, 10 Jul 2025 21:12:23 GMT

Trying to filter out all perfmon data using ingest actions. so, i try and see the samples and i get this error

I checked to see if my forwarders have the same pass4SymmKey and they did. I am not sure what to do im checking now to ensure the FW isnt blocking communication but i think that is unlikely. I can see the servers in forwarder management picking up the deployment apps from the indexer. anyone have any ideas??

Re: Splunk Ingest Actions

livehybrid — Fri, 11 Jul 2025 07:47:38 GMT

Hi @tbarn005

Can you confirm which pass4SymmKey you have verified is the same across the SH and HFs?

The pass4SymmKey under deployment stanza in server.conf matches between deployment server and heavy forwarder is used for the Ingest Action preview and I believe this cannot be a default value.

For more info and diagnostic/troubleshooting check out https://splunk.my.site.com/customer/s/article/Ingest-Actions-are-not-working

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Splunk Ingest Actions

isoutamo — Mon, 14 Jul 2025 21:28:58 GMT

What is your architecture and how you have configured IA?

Re: Splunk Ingest Actions

tbarn005 — Tue, 15 Jul 2025 18:36:40 GMT

So we have a single search head here. I should mention that our deployment and indexer are on the same server. I am aware that best practices is to separate them. Do you think this could be it? As far as how i've configured Ingest actions I only have one rule now to drop all PerfmonMk:CPU > filter using regex > "^PerfmonMk:CPU$" it does not seem to be dropping the data

Re: Splunk Ingest Actions

tbarn005 — Tue, 15 Jul 2025 18:49:09 GMT

Yes, this is one of the first things i've found when searching and i reset that password on both the indexer and my forwarders and still nothing

Re: Splunk Ingest Actions

isoutamo — Tue, 15 Jul 2025 21:04:48 GMT

What you are meaning with “ that our deployment and indexer are on the same server”?
What is this deployment, is it deployment server or something else? Have you one indexer or several and/or cluster? When you are deploying with IA are targets only HFs or are you managing also UFs or other HFs without IA rulesets?

Re: Splunk Ingest Actions

tbarn005 — Wed, 16 Jul 2025 15:49:36 GMT

To clarify, we’re running a single Splunk instance where the Deployment Server, Indexer, and Search head all reside on the same server so it’s a non-distributed architecture. When I mentioned “deployment,” I was referring both to our overall Splunk setup and the fact that our Deployment Server shares the same host as the Indexer. We have only one indexer, no clustering, and no heavy forwarders (HFs) in use. However, we do have universal forwarders (UFs) installed on various servers, and they’re configured to send data directly to the indexer. Regarding Ingest Actions (IA), I’ve configured one rule locally on the indexer to drop data from the source type PerfmonMK:CPU. The rule uses a regex filter (^PerfmonMk:CPU$) with a drop action. IA rules are applied only on the indexer.

Re: Splunk Ingest Actions

isoutamo — Wed, 16 Jul 2025 17:02:25 GMT

Ok. It’s better to use terminology from Splunk SVA documentation https://docs.splunk.com/Documentation/SVA/current/Architectures/TopologyGuidance. In that way we all understand better and clearly what others have. In this case you have single server installation (S1).
When you have S1 and also DS role configured into it and you want to use IA, I’m not sure if that is valid architecture or not with IA? You cannot configure server itself with DS and when you are using IA in server with DS I’m not sure if IA part is always use DS or not in that case? Also UFs is not supported platform for IA and it could try to install also IA part to those?
Can you find anything else from internal logs which can explain what has happened?