Splunk Search

timechart with multiple series

clamarkv
Explorer

Hi, 

Im trying to create a dashboard that easily presents api endpoint performance metrics 

I am generating a summary index using the following search

 

index=my_index app_name="my_app" sourcetype="aws:ecs" "line.logger"=USAGE_LOG
| fields _time line.uri_path line.execution_time line.status line.clientId
``` use a regex to figure out the endpoint from the uri path```
| lookup endpoint_regex_lookup matchstring as line.uri_path OUTPUT app endpoint match
| rename line.status as http_status, line.clientId as client_id
| fillnull value="" http_status client_id
| bin _time span=1m
| sistats count as volume p50(line.execution_time) as P50 p90(line.execution_time) as P90 p95(line.execution_time) as P95 p99(line.execution_time) as P99 by _time app endpoint http_status client_id

 


and i can use searches like this 

 

index=summary source=summary-my_app
| timechart $t_span$  p50(line.execution_time) as P50  p90(line.execution_time) as P90  p95(line.execution_time) as P95  p99(line.execution_time) as P99 by endpoint
| sort endpoint

---

index=summary source=summary-my_app
| timechart span=1m  count by endpoint

 

so i can generate a dashboard using a trellis layout that maps the performance of our endpoints without having to hard-code a bunch of panels.


Screenshot 2024-01-29 at 11.25.29.png

im trying to add a chart that displays the http_status counts over time for each endpoint (similar to the latency chart).

Ive tried a number of different things, but cant get it work.

i know i cant use the following: 

 

index=summary source=summary-my_app
| timechart count by endpoint http_status

 

so thought the following might work:

 

index=summary source=summary-my_app
| stats count by endpoint http_status _time

 


but this shows me the http_status counts on a single line rather than as seperate series.

Does anyone know how i could get this work?
 
 
 
Labels (4)
0 Karma

shonias
Explorer

I'm not entirely sure if I understand what you're asking for, but it sounds like this might be resolved by defining more fields? eg:

| eval status_{http_status}=http_status
| timechart count(status_*) as * by endpoint

Would that do the trick? 

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...