topic Re: How to extract portion of the string using Regex in Splunk Search

How to extract portion of the string using Regex

aditsss — Sun, 16 Aug 2020 20:32:14 GMT

Hi Eveyone,

Can anyone help me out in this.

I have a field name Request_URL as = https://xyz/api/groups/230df08c/registry.

I want to extarct "230df08c" portion from every Request_URL .

Can someone guide me with the regular expression of it in splunk

Thanks In advance

Re: How to extract portion of the string using Regex

to4kawa — Mon, 17 Aug 2020 08:38:39 GMT

| rex field=Request_URL "groups\/(?<id>[^\/]+)"

Re: How to extract portion of the string using Regex

aditsss — Mon, 17 Aug 2020 08:34:52 GMT

Hi,

Ultra Champion.

I tried this not working getting the below Error:

Error in 'rex' command: The regex 'Request_URL' does not extract anything. It should specify at least one named group. Format: (?<name>...).

Re: How to extract portion of the string using Regex

to4kawa — Mon, 17 Aug 2020 08:37:55 GMT

field=(space)Request_URL is my mistake. please fix it.

Re: How to extract portion of the string using Regex

aditsss — Mon, 17 Aug 2020 08:55:56 GMT

Hi I tried with space. But still getting the same Error

rex field = Request_URL "groups\/(?<id>[^\/]+)"

Error in 'rex' command: The regex 'field' does not extract anything. It should specify at least one named group. Format: (?<name>...).

Re: How to extract portion of the string using Regex

to4kawa — Mon, 17 Aug 2020 10:07:20 GMT

remove space

Re: How to extract portion of the string using Regex

aditsss — Mon, 17 Aug 2020 11:01:01 GMT

Thanks a lot!!!! It works.

Re: How to extract portion of the string using Regex

aditsss — Wed, 19 Aug 2020 16:16:52 GMT

Hi to4kawa,

Required your help one more time.

This time I have field Request_URL like this

https://xyz/api/connections/c1d30603ddf0

https://yte/api/flow/groups/314e8fead333/controller-services

https://tyu/api/services/968d06b5666b

https://hju/api/processors/b5f990b529f4/run-status

I want to extract c1d30603ddf0,314e8fead333,968d06b5666b,b5f990b529f4

portion . This time Request_Url is different. Can you guide me how can I do this?

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 04:37:36 GMT

Hi , I am not allow to temper data.

Is there any other approach I can follow.

Please guide me.

Re: How to extract portion of the string using Regex

yeahnah — Fri, 21 Aug 2020 05:45:12 GMT

Hi @aditsss

Based on your examples, the following will work for you.

...| rex field=Request_URL "([^\r\n\/]*\/){4,5}(connections|groups|services|processors)\/(?<id>[^\r\n\/]+)"

If you need to alter this further, or understand what the regex code is doing, then I suggest a site like regex101 (https://regex101.com/) as a useful place to test and learn about regex code.

Hope this helps.

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 08:37:59 GMT

Hey,

I cant hard coded the words

connections|groups|services|processors

These are some of the examples I have given of the Request_URL

https://uyz/api/flow/controller-service-types

There are certain URL's which are of these types(as shown above). If I will hardcode the word then they will not come. I want to display the whole data . The REQUEST_URL which contain the id should display id otherwise blank.

Re: How to extract portion of the string using Regex

gcusello — Fri, 21 Aug 2020 08:55:52 GMT

Hi @aditsss,

to extract a regex, you have to find a rule: position, format, near words, or a combination of them, otherwise it isn't possible to extract the word you need.

Ciao.

Giuseppe

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 14:17:46 GMT

Hi,

I will follow this approch

rex field=Request_URL "([^\r\n\/]*\/){4,5}(connections|groups|services|processors)\/(?<id>[^\r\n\/]+)"

But there are certain URL's which also have index at 3,6,7,8

https://apz/api/queues/61c458568edb/flowfiles/content /regisrtry

https://tyu/policies/read/groups/4e25daf4d5d6/var

https://com/6547890e/

Can u give me complete regex with all indexes as {3,4,5,6,7,8} to extract id.

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 16:07:26 GMT

Please guide me on that .

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 17:58:34 GMT

Hi,

Can someone please provide me the complete regex for {3,4,5,6,7,8} as well.

Below regex will going to work for me. Please provide me for {3,4,5,6,7,8} currently I am getting records only which have{4,5} from below regex.

 rex field=Request_URL "([^\r\n\/]*\/){4,5}(connections|groups|services|processors)\/(?<id>[^\r\n\/]+)"

Please provide me complete regex.

Re: How to extract portion of the string using Regex

ITWhisperer — Fri, 21 Aug 2020 17:59:24 GMT

Looks like you are trying to extract a hexadecimal string - try this:

| rex field=Request_URL "\/(?<id>[0-9a-f]+)($|\/)"

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 19:19:40 GMT

Hi,

I am not able to see any records from this.

| rex field=Request_URL "\/(?<id>[0-9a-f]+)($|\/)"

Previously after using this

| rex field=Request_URL "([^\r\n\/]*\/){4,5}(connections|groups|services|processors)\/(?<id>[^\r\n\/]+)"

I am able to fetch the ID from the Request_url which includes 4 and 5 slash like below

https://xyz/api/connections/c1d30603ddf0

https://hju/api/processors/b5f990b529f4/run-status

But I also have Reuest_Url which includes slashes as 3,6,7,8 as well like below

https://apz/api/queues/61c458568edb/flowfiles/content /regisrtry

https://tyu/policies/read/groups/4e25daf4d5d6/var

https://com/6547890e/

so basically I want this below complete regex for slashes (3,4,5,6,7,8)

rex field=Request_URL "([^\r\n\/]*\/){4,5}(connections|groups|services|processors)\/(?<id>[^
\r\n\/]+)"

Please help me out in this.

Re: How to extract portion of the string using Regex

to4kawa — Fri, 21 Aug 2020 20:11:13 GMT

(?J)(\/api\/\w+\/(?<id>\w+))|(\/groups\/(?<id>\w+))|(\/com\/(?<id>\w+))

REGEX must be made to match the exact log.
It is impossible to create a reliable one if you are not presented with the correct sample.

Re: How to extract portion of the string using Regex

ITWhisperer — Fri, 21 Aug 2020 21:29:19 GMT

@aditsss I am not sure what it is you are trying to do. Given the example URLs you have provided, the rex expression will extract the ids. Is this not what you are after

Request_URL	id
https://xyz/api/connections/c1d30603ddf0	c1d30603ddf0
https://hju/api/processors/b5f990b529f4/run-status	b5f990b529f4
https://apz/api/queues/61c458568edb/flowfiles/content/regisrtry	61c458568edb
https://tyu/policies/read/groups/4e25daf4d5d6/var	4e25daf4d5d6
https://com/6547890e/	6547890e

Re: How to extract portion of the string using Regex

aditsss — Fri, 21 Aug 2020 22:03:38 GMT

Hi WHisperer, to4kawa

My requirement is some what like this.

I have a field called Request_URL (50+ Request_URL are there)

Some of the sample Request_URL 's are

https://abc/api/flow/groups/7d0c111a-0173-1000-ffff-ffffb9f9694c

https://uip/api/groups/3fe13d52-d326-15a1-acef-ed3395edd973/variable-registry

https://yui/api/flowfile-queues/05ee3b30-d5e1-1977-9aa9-61c458568edb/flowfiles/content

https://hjk/api/connections/0a88df6f-0174-1000-0000-0000577a28e9

https://com/022adcc6-8001-3d7a-b291-3d0831458357/

I want to extract ID's from Request_URL i.e 7d0c111a-0173-1000-ffff-ffffb9f9694c,3fe13d52-d326-15a1-acef-ed3395edd973 etc. ID pattern is same in all Request_URL.

What is the exact Regex that I can use as the patterns of the URL is different.

I use below Regex but its showing only the Request_URL with {4,5} / slashes

rex field=Request_URL "([^\r\n\/]*\/){4,5}(connections|groups|services|processors)\/(?<id>[^
\r\n\/]+)"

Can someone provide me complete Regex for it.