topic Re: Grouping HTTP GET requests based on first directory in Splunk Search

Grouping HTTP GET requests based on first directory

JaredYong — Mon, 18 Oct 2021 10:40:21 GMT

Very new to splunk here. I would like to group each http request to each directory based on their directory, and produce a count for each and plot it in a pie chart.

GET /vendor, GET /Services, GET /config, GET /About

For example GET /vendor/vendor/auth/signin and GET /vendor/vendor/browse should be classified under /vendor in a table.

my current query is wrong and doesn't show anything, modified it based on a GIAC paper.

index="apache_logs" | stats count by request | eval request=case(
request="GET /config*", "/config",
request="GET /vendor*", "/vendor",
request="GET /Services*", "/Services",
request="GET /About*", "/About")
request="GET /about*", "/about")
| top request limit=0 useother=f
| eval request=request." (".count." events, ".round(percent,2)."%)"

I would also like to differentiate requests to /about and /About
I hope this made sense.

Re: Grouping HTTP GET requests based on first directory

venkatasri — Mon, 18 Oct 2021 10:26:41 GMT

Hi @JaredYong

Can't give you a final query, however you can try using the combination of field extraction of /directory/ and stats it.

Example:

<your_base_query> "GET" | rex field=request "\/(?<dir>\w+)\/" | stats count by dir

An upvote would be appreciated if this reply helps!

Re: Grouping HTTP GET requests based on first directory

ITWhisperer — Mon, 18 Oct 2021 10:32:16 GMT

Re: Grouping HTTP GET requests based on first directory

JaredYong — Mon, 18 Oct 2021 11:58:33 GMT

Hi I have tried your solution, but it seems like its all categorized under /backend in the result, showing 30 events, while i should have 133 events. upon closer inspection, I noticed that each of the 30 events have their own count, which adds up to 133. How should I modify the solution such that it shows 133 instead of 30?

Re: Grouping HTTP GET requests based on first directory

JaredYong — Mon, 18 Oct 2021 12:04:59 GMT

Thank you for this solution!

Re: Grouping HTTP GET requests based on first directory

JaredYong — Mon, 18 Oct 2021 12:08:35 GMT

one slight problem though, i am unable to put in the percentage for the pie chart, am i missing something here?

Re: Grouping HTTP GET requests based on first directory

ITWhisperer — Mon, 18 Oct 2021 12:15:41 GMT

Change the order of the commands so that the directory is resolved before the counting

Re: Grouping HTTP GET requests based on first directory

venkatasri — Mon, 18 Oct 2021 23:26:32 GMT

@JaredYong Are you good with pie chart?