API dashboard -passed SLA/request per min/response time

[nithys]

Hi Team
I want to have a dashboard that show API stats
1.Nof request--how to get the total count for a request made based on date range selected
below is my splunk log for 

index=* source IN (*)
{
   event: { [-]
     body: null
     httpMethod: GET

path:/data/v1/name
     queryStringParameters: { 
       identifier: 106
     }
      requestContext: { 
       authorizer: { 
         integrationLatency: 0
         principalId: some@example.com
       }
       domainName: domain
       }
       domainName: domain
     }
     resource: /v1/name
   }
   msg: data:invoke

}

2.Response Time-how to get the total count for a response time  based on date range selected
below is the splunk log format
{ 
   client: Ksame@example.com
   domain: domain
   entity: name
   msg: responseTime
   queryParams: { 
     identifier: 666
   }
   requestType: GET
   responseTime: 114
}

i have only above two logs in splunk how do i get below stats count
3.Request per min(Count of requests processed by an API service per minute.)
4.Passed SLA%

(Percentage of service requests that passed service level

agreement parameters, including response time and uptime.)

[ITWhisperer]

Please share (anonymised) raw events for your two examples (not pretty-print formatted versions) preferably in a code block using the </> button.

Please explain what your desired results would look like - for example, in requirement 2, do you want the count of the number of times the response time has been 114 over the period of time of your search?

These events look like they might be JSON. Have you already extracted the JSON fields during ingestion or are you working with raw, unparsed data?

The more information you can give, the quicker you are likely to receive a useful response.

[nithys]

Hi 
I have raw event data in Splunk, where the message contains “data invoke.” Should this message be considered as a count of requests made by a user or writing a query to count an API request when the path matches a specific query string parameter. My goal is to display the total number of API requests made by any user on a dashboard, filtered by a selected date range. Is this the correct query to achieve that?

index= source IN ("") "event" | spath input=_raw output=queryStringParameters path=queryStringParameters | table queryStringParameters | stats count


No of request--how to get the total count for a request made based on date range selected
below is my splunk log for

{
   event: { [-]
     body: null
     httpMethod: GET

path:/data/v1/name
     queryStringParameters: { 
       identifier: 106
     }
      requestContext: { 
       authorizer: { 
         integrationLatency: 0
         principalId: some@example.com
       }
       domainName: domain
       }
       domainName: domain
     }
     resource: /v1/name
   }
   msg: data:invoke

}

2.Response Time-how to get the total count for a response time  based on date range selected
below is the splunk log format
I using below query
index=* source IN ("*") *responseTime* | fields responseTime | table responseTime,total | addcoltotals labelfield=total label="Total" | search total!="" | fields - total

{ 
   client: same@example.com
   domain: domain
   entity: name
   msg: responseTime
   queryParams: { 
     identifier: 666
   }
   requestType: GET
   responseTime: 114
}

Should i set SLA based on below formaula or should i also need to add response time
 passed sla =(total request -total failed request/total request)X100

[ITWhisperer]

These are formatted versions of your events, please share the raw unformatted versions of your events (in a code block just like you did with the formatted versions).

[nithys]

this how i get the events

{
   event: { [-]
     body: null
     httpMethod: GET
     path:/data/v1/name
     queryStringParameters: {
       identifier: 106
     }
     requestContext: {
       authorizer: {
         integrationLatency: 0
         principalId: some@example.com
       }
       domainName: domain
     }
     domainName: domain
   }
   resource: /v1/name
}
msg: data:invoke

{ event: { [-] body: null httpMethod: GET path:/data/v1/name queryStringParameters: { identifier: 106 } requestContext: { authorizer: { integrationLatency: 0 principalId: some@example.com } domainName: domain } domainName: domain } resource: /v1/name } msg: data:invoke


2.

{    client: same@example.com    domain: domain    entity: name    msg: responseTime    queryParams: {      identifier: 666    }    requestType: GET    responseTime: 114 }

{
   client: same@example.com
   domain: domain
   entity: name
   msg: responseTime
   queryParams: {
     identifier: 666
   }
   requestType: GET
   responseTime: 114
}

[ITWhisperer]

What you have shared are formatted events, not the raw unformatted data. Please share the unformatted _raw field from your events.

[nithys]

request

{"name":"","awsRequestId":"","hostname":"","pid":8,"level":30,"event":{"resource":"/v1/","path":"/data/v1/","httpMethod":"GET","queryStringParameters":{"identifier":"10"},"body":null,"requestContext":{"requestId":"","authorizer":{"principalId":"","integrationLatency":0},"domainName":""}},"msg":"init : data :invoke","time":"","v":0} 

response

{"name":"","awsRequestId":"","hostname":"","pid":8,"level":30,"requestType":"GET","entity":"entity","client":"","domain":"","queryParams":{"identifier":"10"},"responseTime":291,"msg":"init: data :responseTime","time":"","v":0}

[ITWhisperer]

Thanks, and have the fields already been extracted from these event?

For 1, do you just want a count of these events?

For 2, do you just want the total response time for all the events?

[nithys]

Hi @ITWhisperer @livehybrid 

I was able to get the avg response time by identifier ..
Now as next step I want to set an %Passed SLA(Percentage of service requests that passed service level

agreement parameters, including response time and uptime).How do i set the SLA

index=* source IN ("") *response*
| eval identifier=coalesce('queryParams.identifier',
'event.queryStringParameters.identifier')
| eval responseTime=coalesce(responseTime, null)
| where isnotnull(identifier) and isnotnull(responseTime)
| stats avg(responseTime) as avg_response_time by identifier
| eventstats avg(responseTime) as overall_avg_response_time

Get the totla no of request separetely by

index=* source IN ("*") *data*
| eval identifier=coalesce('queryParams.identifier',
'event.queryStringParameters.identifier')
| eval msg=coalesce(msg, null)
| where isnotnull(identifier) and isnotnull(msg)
| stats count

[ITWhisperer]

Your eventstats isn't doing anything since the responseTime field is no long available after the stats command.

Try something like this

| eval identifier=coalesce('queryParams.identifier',
'event.queryStringParameters.identifier')
| eval responseTime=coalesce(responseTime, null)
| where isnotnull(identifier) and isnotnull(responseTime)
| stats avg(responseTime) as avg_response_time by identifier
| eval SLA_response_time=300
| eval met_SLA=if(avg_response_time <= SLA_response_time, 1, 0)
| stats count sum(met_SLA) as count_within_SLA
| eval percentage_met_SLA=100 * count_within_SLA / count

This assumes that your SLA has a static value of 300.

If you want to use a different SLA value, you need to define how that is set or calculated.

[nithys]

yes it worked .Thanks

[livehybrid]

Hi @nithys 

If you want to look at count per minute then you should be able to add something like the following to your existing search:
| timechart span=1m count

Regarding the SLA - Is the SLA based on the responses taking less than a certain time? If so, what is that? 
You can do an eval to determine if SLA is met or not
| eval SLA_met=IF(responseTime>100,0,1)
| timechart span=1 count by SLA_met

(1 = Is met, 0 = is not met).

Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards

Will

[nithys]

Hi @livehybrid 
Thanks ...Let me try with above solution .Also i want to have how to get the total count for a request made based on date range selected below is my splunk log for 

is this the correct way should i consider if there is anypath=queryStringParameters ,then count that as a single API request
index=* source IN ("") "event" | spath input=_raw output=queryStringParameters path=queryStringParameters | table queryStringParameters | stats count


index=* source IN (*)
{
   event: { 
     body: null
     httpMethod: GET

path:/data/v1/name
     queryStringParameters: { 
       identifier: 106
     }
      requestContext: { 
       authorizer: { 
         integrationLatency: 0
         principalId: some@example.com
       }
       domainName: domain
       }
       domainName: domain
     }
     resource: /v1/name
   }
   msg: data:invoke

}