Splunk Search

timechart with multiple series

clamarkv
Explorer

Hi, 

Im trying to create a dashboard that easily presents api endpoint performance metrics 

I am generating a summary index using the following search

 

index=my_index app_name="my_app" sourcetype="aws:ecs" "line.logger"=USAGE_LOG
| fields _time line.uri_path line.execution_time line.status line.clientId
``` use a regex to figure out the endpoint from the uri path```
| lookup endpoint_regex_lookup matchstring as line.uri_path OUTPUT app endpoint match
| rename line.status as http_status, line.clientId as client_id
| fillnull value="" http_status client_id
| bin _time span=1m
| sistats count as volume p50(line.execution_time) as P50 p90(line.execution_time) as P90 p95(line.execution_time) as P95 p99(line.execution_time) as P99 by _time app endpoint http_status client_id

 


and i can use searches like this 

 

index=summary source=summary-my_app
| timechart $t_span$  p50(line.execution_time) as P50  p90(line.execution_time) as P90  p95(line.execution_time) as P95  p99(line.execution_time) as P99 by endpoint
| sort endpoint

---

index=summary source=summary-my_app
| timechart span=1m  count by endpoint

 

so i can generate a dashboard using a trellis layout that maps the performance of our endpoints without having to hard-code a bunch of panels.


Screenshot 2024-01-29 at 11.25.29.png

im trying to add a chart that displays the http_status counts over time for each endpoint (similar to the latency chart).

Ive tried a number of different things, but cant get it work.

i know i cant use the following: 

 

index=summary source=summary-my_app
| timechart count by endpoint http_status

 

so thought the following might work:

 

index=summary source=summary-my_app
| stats count by endpoint http_status _time

 


but this shows me the http_status counts on a single line rather than as seperate series.

Does anyone know how i could get this work?
 
 
 
Labels (4)
0 Karma

shonias
Explorer

I'm not entirely sure if I understand what you're asking for, but it sounds like this might be resolved by defining more fields? eg:

| eval status_{http_status}=http_status
| timechart count(status_*) as * by endpoint

Would that do the trick? 

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Observability Simplified: Combining User Experience, Application Performance & ...

Tech Talk Observability Simplified: Combining User Experience, Application Performance & Network ...

Event Series May & June: From Network Visibility to Service Intelligence

Unifying the Network: Moving from Alert Noise to Service Intelligence with Splunk ITSI In today’s hybrid ...

Global Splunk User Group Events: May + June 2026

Your Splunk Community Awaits: Discover Upcoming User Group Events Worldwide    Staying ahead in the fast-paced ...