Splunk Search

timechart with multiple series

clamarkv
Explorer

Hi, 

Im trying to create a dashboard that easily presents api endpoint performance metrics 

I am generating a summary index using the following search

 

index=my_index app_name="my_app" sourcetype="aws:ecs" "line.logger"=USAGE_LOG
| fields _time line.uri_path line.execution_time line.status line.clientId
``` use a regex to figure out the endpoint from the uri path```
| lookup endpoint_regex_lookup matchstring as line.uri_path OUTPUT app endpoint match
| rename line.status as http_status, line.clientId as client_id
| fillnull value="" http_status client_id
| bin _time span=1m
| sistats count as volume p50(line.execution_time) as P50 p90(line.execution_time) as P90 p95(line.execution_time) as P95 p99(line.execution_time) as P99 by _time app endpoint http_status client_id

 


and i can use searches like this 

 

index=summary source=summary-my_app
| timechart $t_span$  p50(line.execution_time) as P50  p90(line.execution_time) as P90  p95(line.execution_time) as P95  p99(line.execution_time) as P99 by endpoint
| sort endpoint

---

index=summary source=summary-my_app
| timechart span=1m  count by endpoint

 

so i can generate a dashboard using a trellis layout that maps the performance of our endpoints without having to hard-code a bunch of panels.


Screenshot 2024-01-29 at 11.25.29.png

im trying to add a chart that displays the http_status counts over time for each endpoint (similar to the latency chart).

Ive tried a number of different things, but cant get it work.

i know i cant use the following: 

 

index=summary source=summary-my_app
| timechart count by endpoint http_status

 

so thought the following might work:

 

index=summary source=summary-my_app
| stats count by endpoint http_status _time

 


but this shows me the http_status counts on a single line rather than as seperate series.

Does anyone know how i could get this work?
 
 
 
Labels (4)
0 Karma

shonias
Explorer

I'm not entirely sure if I understand what you're asking for, but it sounds like this might be resolved by defining more fields? eg:

| eval status_{http_status}=http_status
| timechart count(status_*) as * by endpoint

Would that do the trick? 

Get Updates on the Splunk Community!

Splunk Enterprise Security: Your Command Center for PCI DSS Compliance

Every security professional knows the drill. The PCI DSS audit is approaching, and suddenly everyone's asking ...

Developer Spotlight with Guilhem Marchand

From Splunk Engineer to Founder: The Journey Behind TrackMe    After spending over 12 years working full time ...

Cisco Catalyst Center Meets Splunk ITSI: From 'Payments Are Down' to Root Cause in ...

The Problem: When Networks and Services Don't Talk Payment systems fail at a retail location. Customers are ...