Knowledge Management

Extract Url from access log and count the no of times an API is hitted


This is the sample accessLog which is coming up in the Splunk ui.

  1. {"timestamp":"2021-10-17T15:03:56,763Z","level":"INFO","thread":"reactor-http-epolpl-20","message":"method=GET, uri=/api/v1/hello1, status=200, duration=1, "logger":"reactor.netty.http.server.AccessLog"}

  2. {"timestamp":"2021-10-17T15:03:56,763Z","level":"INFO","thread":"reactor-http-epolpl-20","message":"method=GET, uri=/api/v1/dummy1, status=200, duration=1, "logger":"reactor.netty.http.server.AccessLog"}

I want to extract the url and make a count for all the API's like how many times an API is hitted from the uri part(uri=/api/v1/dummy1).

I tried the below query, but it's not giving the desired result.

index=dummy OR index=dummy1  source=*dummy-service*  logger=reactor.netty.http.server.AccessLog | rex field=message "(?<url>uri.[\/api\/v1\/hello]+)" | chart count by url

Can someone help in this ? 

Labels (2)
0 Karma


Here is a regex you can use regardless if the date is included at the end of the uri.




0 Karma


Is your example JSON correct, i.e. there is no trailing quote mark for message, i.e. you have

"message":"method=GET, uri=/api/v1/hello1, status=200, duration=1, ...

therefore your JSON does not parse correctly, as the logger field is not extracted, so that may be why you get no results. If that is correct, then you could manually extract logger with the additional rex below, or fix the json

| rex field=message "uri=(?<url>uri.[^,]*)"
| rex field=_raw "logger\":\"(?<logger>[^\"]*)"
| search logger=reactor.netty.http.server.AccessLog 
| stats count by url


0 Karma


This is the proper formatted log :

"timestamp": "2021-10-17T15:03:56,763Z",
"level": "INFO",
"thread": "reactor-http-epolpl-20",
"message": "method=GET, uri=/api/v1/hello1, status=200, duration=1",
"logger": "reactor.netty.http.server.AccessLog"

Problem I am facing is like, I am having some GET requests in which values are coming in path variables like below :

1. /api/v1/dummy/D000/2021-10-14T11:00:00+00:00/2021-10-15T06:00:01+00:00
2. /api/v1/dummy/D023/2021-10-14T21:00:00+00:00/2021-10-19T06:00:25+00:00

Now for this particular url (/api/v1/dummy/{id}/{date1}/{date2}), it logs all the request with the above query,

I want a generic regex to collect all the request.

The above query works perfectly for the POST url, in which there are no incoming parameters like (/api/v1/add/user)

0 Karma


If the URL has extra parts to the URL path, as opposed to parameters, then would it be acceptable to define the url as the first 3 parts of the url, e.g.

| rex field=uri "(?<path>/[^/]*/[^/]*/[^/? ,]*)"

or you could do something like this

| rex field=uri mode=sed "s/\/\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}[^\/?$]*//g"

to remove date components in the path from the uri - this example leaves your ID in place, but you could remove that too if you wanted.

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...