Hi all, We’re deploying a custom Splunk app (e.g., my_app) that includes a scripted input to pull data from an internal API (in-house application). This API requires an API token, and we want to store that token securely using Splunk's passwords.conf mechanism — i.e., the native storage/passwords feature that encrypts the credential on disk. This app needs to be deployed on a Splunk Heavy Forwarder (HF) which is: Managed entirely via Deployment Server Does not have a UI or user access for entering credentials manually But we can get temporary shell access if absolutely needed (e.g., during bootstrap)   What We Know and Have Tried (in dev system without Deployment server) Adding the credential securely via the REST API works fine: curl -k -u admin:changeme \ https://localhost:8089/servicesNS/nobody/search/storage/passwords \ -d name=my_realm:my_api_key \ -d password=actual_api_token ​ and this then stores password encrypted in 'search' app [credential::my_realm:my_api_key:] password = $1$encrypted_string_here However, if we try to deploy a plain-text password via my_app/local/passwords.conf like this: [credential::my_realm:my_api_key:] password = plaintext_token # Splunk does not encrypt if I add it via shell and restart Splunk — and the token remains in clear text on disk, which is not acceptable for production. We also know that encrypting the token on another instance and copying the encrypted config doesn’t work, because encryption depends on the local splunk.secret, which is unique per instance. (Though got a worse case workaround of getting the splunk.secret and run a docker instance, create passwords.conf and copy it back. Quite a long winded option)   What is the best practice to securely bootstrap the credential? Specifically: Should we: Add the credential once via REST API during the shell access window Then copy the resulting passwords.conf into my_app/local/ for persistence? How does other Splunk app that run in Heavy Forwarders (HF) which requires passwords store credentials? Are there any community-recommended patterns (scripts, startup hooks, init-time credential registration, etc.) for this kind of controlled environment? ... View more

hi folks, the scenario is like below - have Enterprise security (ESS) in Splunk cloud + ESCU (content updates) as part of it - if we enable a ESCU detection it works all good. - we need to modify the ESCU slightly with a standard field and also the name of the search to fit existing organisation policy - The uuid remain the same What will happen when the next ESCU update comes? Will it overwrite the custom changes? What is the actual ESCU update looking for? is it looking for 'search name' or the 'search id (uuid)?'?   What will happen when the next ESCU update comes? ... View more

We have a customer who has Splunk as main Security platform, but now they are trying to onboard other datasets for forensic/compliance/data retention purposes/application data. This doesn't need to be in Splunk as such, but any searchable tools like OpenSearch or similar. Before looking into such extra tools, wanted to understand if there is any provision with Splunk which would allow a data ingestion at cheaper cost (not counting to the main license cost or a cheaper license option?) So the scenario is (Security + compliance + application data) => Splunk Heavy Forwarder -> (A) Security data to Splunk  &&  (B) Rest of data to a log retention service Before going into this avenue, wanted to check if Splunk provide such a cheaper license option? i.e. for a log retention mode or non-important data (In future, they may have funding to move into Splunk, but not for atleast 6-8 months)   ... View more