topic Re: Need help in ignoring the ID of API endpoints in Splunk Search

How do I extract only endpoints and ignore the ID of API endpoints?

labaningombam — Thu, 11 Aug 2022 14:19:55 GMT

Hi,

I have a bunch of failure events of different api endpoints. The field is called RequestPath and some examples are:

/v1/locations/45BH-JGN
/v1/exceptions/ABS/12
/v1/exceptions/ODD/13
/v2/absence/100

Basically, I am trying to extract only the endpoints without the ids, so that I can get a count of which endpoints are failing, example

/v1/locations/ --- 1 failure
/v1/exceptions/ABS/ ----- 4 failures
/v1/exceptions/ODD/ ---- 10 failures , etc.

How can I do the same?

Re: Need help in ignoring the ID of API endpoints

ITWhisperer — Thu, 11 Aug 2022 07:33:32 GMT

| rex field=RequestPath "(?<location>.*/)"

Re: Need help in ignoring the ID of API endpoints

labaningombam — Thu, 11 Aug 2022 09:04:06 GMT

Actually,

I have my request path in log as

/v1/locations/45BH-JGN

I need to extract "/v1/locations/" from it. Similarly I have endpoint

/v1/exceptions/ABS/12

I need to extract only "/v1/exceptions/ABS/"

So I need to ignore the last string which comes after "/" and get the same.

Re: Need help in ignoring the ID of API endpoints

labaningombam — Thu, 11 Aug 2022 09:05:06 GMT

Actually,

I have my request path in log as

/v1/locations/45BH-JGN

I need to extract "/v1/locations/" from it. Similarly I have endpoint

/v1/exceptions/ABS/12

I need to extract only "/v1/exceptions/ABS/"

So I need to ignore the last string which comes after "/" and get the same.

@ITWhisperer

Re: Need help in ignoring the ID of API endpoints

ITWhisperer — Thu, 11 Aug 2022 12:06:04 GMT

So when you said you had a field called RequestPath, you meant you don't have a field called RequestPath?

Instead you meant, you have a raw event which you need to extract a field called RequestPath from, and then extract the first part (up to the last /)?

Perhaps you could share some of your actual events and identify which fields have already been extracted?

Re: Need help in ignoring the ID of API endpoints

labaningombam — Thu, 11 Aug 2022 12:43:34 GMT

Hi,

I do have a field called RequestPath. Here are 3 different event logs.

Properties: { [-] Host: MachineName: RequestId: RequestPath: /v1/locations/41b2ee1b-145es StatusCode: 404 }

Properties: { [-] Host: MachineName: RequestId: RequestPath: /v1/exceptions/ODD/123 StatusCode: 404 }

Properties: { [-] Host: MachineName: RequestId: RequestPath: /v2/timebuckets/A4GH-A StatusCode: 404 }

My need is to have a count of how many errors are there for each request path without the ID(which is the last string in the endpoint after '/')

/v1/locations/ - 1 Failure
/v1/exceptions/ODD/ - 1 Failure
/v2/timebuckets/ - 1 Failure

Something like this.

Re: Need help in ignoring the ID of API endpoints

ITWhisperer — Thu, 11 Aug 2022 13:17:12 GMT

Thanks. It looks like your events are partially JSON. Have you extract the RequestPath field already, or do you need some guidance on that? (If it has been done already, it might have a different name "...Properties.RequestPath for example.

Re: Need help in ignoring the ID of API endpoints

yuanliu — Fri, 12 Aug 2022 03:42:53 GMT

Something like

| rex field=RequestPath "^(?<endpoint>.+/)[^/]+$" | stats count by endpoint

or, more "formal"ly,

| eval RequestPath = split(RequestPath, "/") | eval endpoint = mvjoin(mvindex(RequestPath, 0, mvcount(RequestPath) - 1), "/") | stats count by endpoint