Environment Sending from: Azure Web Apps (Kudu CLI) Target: Two Splunk instances Personal trial Splunk (working) Customer's Splunk Cloud (not receiving data) Problem Description I am attempting to send log data to two different Splunk HTTP Event Collector (HEC) endpoints from an Azure Web Apps environment. I am using identical cURL commands for both: Working Splunk Instance (Trial) curl -k "https://[TRIAL_SPLUNK_HEC_URL]/services/collector/event" \ -H "Authorization: Splunk [REDACTED_TOKEN]" \ -d '{ "event": { ... }, "sourcetype": "_json", "index": "[TRIAL_INDEX]" }' Non-Working Splunk Instance (Customer Cloud) curl -k "https://[CUSTOMER_SPLUNK_HEC_URL]/services/collector/event" \ -H "Authorization: Splunk [REDACTED_TOKEN]" \ -d '{ "event": { ... }, "sourcetype": "_json", "index": "[CUSTOMER_INDEX]" }' Observations Identical cURL command structure. For both Splunk instances, the cURL command on the CLI returns {"text":"Success","code":0} Despite the success response, data is successfully ingested into my personal trial Splunk instance, but no data appears in the customer's Splunk Cloud instance. Questions Given that the CLI returns "Success" for both, what could be preventing the data from appearing in the customer's Splunk Cloud instance? What additional troubleshooting steps can verify HEC configuration and permissions on the customer's Splunk Cloud? Could this issue be related to specific user role permissions or privileges, as I have been granted only limited access by the customer? Are there particular Splunk roles or permissions required for HEC token usage that might be missing? Attempted Troubleshooting Verified correct HEC URL for both instances. Confirmed valid HEC tokens for both instances. Checked general network connectivity from the Azure Web Apps environment. Validated JSON payload structure. Any guidance on identifying and resolving this selective data ingestion issue, especially considering the "Success" response from the CLI would be greatly appreciate ... View more

Hello Splunk Community, I am trying to trace a Python FastAPI RAG API running on my local PC to my Splunk Cloud instance using the HTTP Event Collector (HEC). I am facing persistent issues with DNS resolution for the HEC endpoint hostname. What I have done on Splunk Cloud: Enabled HTTP Event Collector (HEC) in Global Settings. Confirmed that SSL is required and enabled (cannot be unchecked). Set the HTTP Port Number to 8088 in Global Settings. Created an HEC token named "rag_tracer". Assigned the token to use _json as the Sourcetype and mgap_cit_be_dev as the Index. My local setup: My application is a Python FastAPI service. I manage environment variables via a `.env` file, which is loaded at application startup. My tracing logic is implemented in a custom `tracing_helper.py` module, which attempts to connect to the HEC endpoint using the configured environment variables. The Issue: Despite configuring the HEC token and settings in Splunk Cloud, my application and direct `curl` commands consistently fail to connect to the HEC endpoint when using the hostname, resulting in `NameResolutionError` (or `[Errno 8] nodename nor servname provided, or not known`). I have tried various `SPLUNK_HEC_URL` formats in my `.env` file, including: https://http-inputs-prd-p-gu5st.splunkcloud.com:8088/services/collector https://http-inputs-prd-p-gu5st.splunkcloud.com:443/services/collector https://inputs-prd-p-gu5st.splunkcloud.com:8088/services/collector https://prd-p-gu5st.splunkcloud.com:8088/services/collector https://prd-p-gu5st.splunkcloud.com:443/services/collector (This returned a 303 See Other, redirecting to `/en-US/services/collector`) My application's logs repeatedly show: DNS Resolution Error: [Errno 8] nodename nor servname provided, or not known Unable to resolve Splunk Cloud HEC hostname. Check network configuration. HTTPSConnectionPool(host='http-inputs-prd-p-gu5st.splunkcloud.com', port=443): Max retries exceeded with url: /services/collector (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x...>: Failed to resolve 'http-inputs-prd-p-gu5st.splunkcloud.com' ([Errno 8] nodename nor servname provided, or not known)")) I have also performed network-level troubleshooting from my local machine: `nslookup http-inputs-prd-p-gu5st.splunkcloud.com` resulted in `** server can't find http-inputs-prd-p-gu5st.splunkcloud.com: REFUSED` `ping -c 4 http-inputs-prd-p-gu5st.splunkcloud.com` resulted in `ping: cannot resolve http-inputs-prd-p-gu5st.splunkcloud.com: Unknown host` `nslookup prd-p-gu5st.splunkcloud.com` (the main Splunk Cloud domain) also showed `REFUSED` for DNS resolution. However, `curl -v https://prd-p-gu5st.splunkcloud.com` successfully connected and resolved to `52.202.25.197`. Crucially, when I configured `SPLUNK_HEC_URL` in my `.env` file to use the direct IP address `https://52.202.25.197:8088/services/collector`, a `curl` command to HEC succeeded with `{"text":"Success","code":0}`. This confirms that the HEC endpoint is reachable via IP, but hostname resolution is the core problem. My questions to the community are: What is the exact and correct HEC endpoint URL format for Splunk Cloud trial instances (e.g., `prd-p-gu5st.splunkcloud.com`)? I cannot find a clear example of the full HEC URL in my Splunk Cloud console. Can you outline the correct process to set up HEC in Splunk Cloud, step-by-step, specifically for a trial account? What are the client-side configuration steps (e.g., environment variables, code changes) required to correctly communicate with HEC in Splunk Cloud from a local Python application? Is there any specific IP allow-listing that needs to be done on the Splunk Cloud side for HEC, even though the documentation states it's open by default and I can access the Splunk Cloud web interface? Any guidance or suggestions would be greatly appreciated. Thank you! ... View more