Splunk Ingest Actions

[tbarn005]

Trying to filter out all perfmon data using ingest actions. so, i try and see the samples and i get this error 

I checked to see if my forwarders have the same pass4SymmKey and they did. I am not sure what to do im checking now to ensure the FW isnt blocking communication but i think that is unlikely. I can see the servers in forwarder management picking up the deployment apps from the indexer. anyone have any ideas??

[isoutamo]

What is your architecture and how you have configured IA?

[tbarn005]

So we have a single search head here. I should mention that our deployment and indexer are on the same server. I am aware that best practices is to separate them. Do you think this could be it?  As far as how i've configured Ingest actions I only have one rule now to drop all PerfmonMk:CPU > filter using regex >  "^PerfmonMk:CPU$" it does not seem to be dropping the data 

[isoutamo]

What you are meaning with “ that our deployment and indexer are on the same server”?
What is this deployment, is it deployment server or something else? Have you one indexer or several and/or cluster? When you are deploying with IA are targets only HFs or are you managing also UFs or other HFs without IA rulesets?

[tbarn005]

To clarify, we’re running a single Splunk instance where the Deployment Server, Indexer, and Search head all reside on the same server so it’s a non-distributed architecture. When I mentioned “deployment,” I was referring both to our overall Splunk setup and the fact that our Deployment Server shares the same host as the Indexer. We have only one indexer, no clustering, and no heavy forwarders (HFs) in use. However, we do have universal forwarders (UFs) installed on various servers, and they’re configured to send data directly to the indexer. Regarding Ingest Actions (IA), I’ve configured one rule locally on the indexer to drop data from the source type PerfmonMK:CPU. The rule uses a regex filter (^PerfmonMk:CPU$) with a drop action. IA rules are applied only on the indexer.

[isoutamo]

Ok. It’s better to use terminology from Splunk SVA documentation https://docs.splunk.com/Documentation/SVA/current/Architectures/TopologyGuidance. In that way we all understand better and clearly what others have. In this case you have single server installation (S1).
When you have S1 and also DS role configured into it and you want to use IA, I’m not sure if that is valid architecture or not with IA? You cannot configure server itself with DS and when you are using IA in server with DS I’m not sure if IA part is always use DS or not in that case? Also UFs is not supported platform for IA and it could try to install also IA part to those?
Can you find anything else from internal logs which can explain what has happened?

[livehybrid]

Hi @tbarn005 

Can you confirm which pass4SymmKey you have verified is the same across the SH and HFs?

The pass4SymmKey under deployment stanza in server.conf matches between deployment server and heavy forwarder is used for the Ingest Action preview and I believe this cannot be a default value. 

For more info and diagnostic/troubleshooting check out https://splunk.my.site.com/customer/s/article/Ingest-Actions-are-not-working

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[tbarn005]

Yes, this is one of the first things i've found when searching and i reset that password on both the indexer and my forwarders and still nothing