https://community.splunk.com/t5/Getting-Data-In/Open-Telemetry-Errors-in-Windows/m-p/666834#M111872 <P>Getting a ton of these Telemetry errors in Event Log of a windows server with at UF installed. They started a few days ago. What could be causing them? No changes have been made to the UF or splunk infrastructure recently.<BR /><BR /></P><DIV>1.6987038408387303e+09 error exporterhelper/queued_retry.go:183 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "name": "signalfx", "error": "Permanent error: \"HTTP/2.0 401 Unauthorized\\r\\nContent-Length: 0\\r\\nDate: Mon, 30 Oct 2023 22:10:40 GMT\\r\\nServer: istio-envoy\\r\\nWww-Authenticate: Basic realm=\\\"Splunk\\\"\\r\\nX-Envoy-Upstream-Service-Time: 5\\r\\n\\r\\n\"", "dropped_items": 50}</DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/queued_retry.go:183</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/metrics.go:132</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/queued_retry_inmemory.go:119</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper/internal.consumerFunc.consume</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/internal/bounded_memory_queue.go:82</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func2</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/internal/bounded_memory_queue.go:69</SPAN></DIV><P>&nbsp;</P> Mon, 30 Oct 2023 22:38:14 GMT Dominic32 2023-10-30T22:38:14Z https://community.splunk.com/t5/Getting-Data-In/Open-Telemetry-Errors-in-Windows/m-p/666834#M111872 <P>Getting a ton of these Telemetry errors in Event Log of a windows server with at UF installed. They started a few days ago. What could be causing them? No changes have been made to the UF or splunk infrastructure recently.<BR /><BR /></P><DIV>1.6987038408387303e+09 error exporterhelper/queued_retry.go:183 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "name": "signalfx", "error": "Permanent error: \"HTTP/2.0 401 Unauthorized\\r\\nContent-Length: 0\\r\\nDate: Mon, 30 Oct 2023 22:10:40 GMT\\r\\nServer: istio-envoy\\r\\nWww-Authenticate: Basic realm=\\\"Splunk\\\"\\r\\nX-Envoy-Upstream-Service-Time: 5\\r\\n\\r\\n\"", "dropped_items": 50}</DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/queued_retry.go:183</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/metrics.go:132</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/queued_retry_inmemory.go:119</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper/internal.consumerFunc.consume</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/internal/bounded_memory_queue.go:82</SPAN></DIV><DIV>go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func2</DIV><DIV><SPAN>/builds/o11y-gdi/splunk-otel-collector-releaser/.go/pkg/mod/go.opentelemetry.io/collector@v0.53.0/exporter/exporterhelper/internal/bounded_memory_queue.go:69</SPAN></DIV><P>&nbsp;</P> Mon, 30 Oct 2023 22:38:14 GMT https://community.splunk.com/t5/Getting-Data-In/Open-Telemetry-Errors-in-Windows/m-p/666834#M111872 Dominic32 2023-10-30T22:38:14Z https://community.splunk.com/t5/Getting-Data-In/Open-Telemetry-Errors-in-Windows/m-p/667028#M111893 <P>Error was being caused by the OpenTelemetry Collector service, which is separate from the Splunk UF service. I stopped the service and errors are no longer being logged in Event Viewer.</P> Wed, 01 Nov 2023 15:36:52 GMT https://community.splunk.com/t5/Getting-Data-In/Open-Telemetry-Errors-in-Windows/m-p/667028#M111893 Dominic32 2023-11-01T15:36:52Z