Splunk Observability Cloud

Traces not showing up in APM with Splunk Observability Trial

AndrewBriggs
Engager

I am calling the trace endpoint https://ingest.<realm>.signalfx.com/v2/trace/signalfxv1 and sending this span in the body:

[
{
"id": "003cfb6642471ba4",
"traceId": "0025ecb5dc31498b931bce60be0784cd",
"name": "reloadoptionsmanager",
"timestamp": 1716477080494000,
"kind": "SERVER",
"remoteEndpoint": {
"serviceName": "XXXXXXX"
},
"Tags": {
"data": "LogData",
"eventId": "0",
"severity": "8"
}
}
]


The request receives a 200 response. The response body is OK. However the span does not appear in APM.

timestamp is the the number of microseconds since 1/1/1970.

 

Labels (1)
0 Karma
1 Solution

bishida
Splunk Employee
Splunk Employee

Manually sending a span using something like curl can often be trickier than instrumenting a sample application. But, that said, it is possible. I see you're using the zipkin compatible endpoint.

This page may help if you want to try other formats/endpoints:
https://docs.splunk.com/observability/en/apm/apm-spans-traces/span-formats.html

Did you replace <realm> with your realm? e.g. "us1" or "us0", etc

With a single span, you won't see anything on the service map, but you should see something on the APM home screen. Here is a sample span that worked for me:

curl -X POST https://ingest.us1.signalfx.com/v2/trace/signalfxv1 \
-H 'Content-Type: application/json' \
-H 'X-SF-TOKEN: XXXXXXXXXXXXXXXXXX' \
-d '[{"traceId": "a03ee8fff1dcd9b9","id": "2e8cfb154b59a41f","kind": "SERVER","name": "post /location/update/v4","timestamp": 1716495917000,"duration": 131848,"localEndpoint": {"serviceName": "routing"},"tags": {"ecosystem": "prod","habitat": "uswest1aprod","http.uri.client": "/location/update/v4","region": "uswest1-prod","response_status_code": "200"},"shared": true}]'

bishida_0-1716496698803.png

 

 

View solution in original post

0 Karma

bishida
Splunk Employee
Splunk Employee

Manually sending a span using something like curl can often be trickier than instrumenting a sample application. But, that said, it is possible. I see you're using the zipkin compatible endpoint.

This page may help if you want to try other formats/endpoints:
https://docs.splunk.com/observability/en/apm/apm-spans-traces/span-formats.html

Did you replace <realm> with your realm? e.g. "us1" or "us0", etc

With a single span, you won't see anything on the service map, but you should see something on the APM home screen. Here is a sample span that worked for me:

curl -X POST https://ingest.us1.signalfx.com/v2/trace/signalfxv1 \
-H 'Content-Type: application/json' \
-H 'X-SF-TOKEN: XXXXXXXXXXXXXXXXXX' \
-d '[{"traceId": "a03ee8fff1dcd9b9","id": "2e8cfb154b59a41f","kind": "SERVER","name": "post /location/update/v4","timestamp": 1716495917000,"duration": 131848,"localEndpoint": {"serviceName": "routing"},"tags": {"ecosystem": "prod","habitat": "uswest1aprod","http.uri.client": "/location/update/v4","region": "uswest1-prod","response_status_code": "200"},"shared": true}]'

bishida_0-1716496698803.png

 

 

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...