topic Re: Rex command to Show number values between two fields in Splunk Search

Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 07:16:53 GMT

How to use Rex command to show Value in between 'Id' and `language` for example 0827ce61-e07c-4b51-a052-681dcc94fa2f to show in table.

I tried EREX to generate pattern, it worked in most cases but in some cases failed.
Is there any easy way to show the value between 2 fields?

Provided the Event sample

{"line":"c5fd03529388e8d6 INFO 2021-06-10 04:11:02,966 [reactor-http-epoll-2] com.test.content.test.external.SnsPayloadEventSender Sent payload event id=9a67e81d-5ae3-5e84-b1f7-b126c9f5c787 for payload event={\"approvalDate\":\"2015-06-10T04:11:02.128Z\",\"adenceIds\":[],\"adenceTargets\":[],\"collGroupId\":\"008be467-6c78-4079-94f0-70e2d6cc4003\",\"collections\":[\"d879aa85-698c-41a1-b066-a8ecdf6b8a9c\",\"d879aa85-698c-41a1-b066-a8ecdf6b8a9c\"],\"endSchedule\":\"3000-01-01T19:00:00.000Z\",\"id\":\"0827ce61-e07c-4b51-a052-681dcc94fa2f\",\"language\":\"ja\",\"mplace\":\"JP\",\"nodeVersion\":null,\"payloadType\":\"thread\",\"preview\":false,\"resourceSubType\":\"thread\",\"resourceType\":\"thread\",\"startSchedule\":\"2021-03-26T22:00:28.000Z\",\"type\":\"P\",\"version\":\"1622644250765\"}","source":"stdout","tag":"21c5897e605c","attrs":{"application":"test","team":"test"}}

Re: Rex command to Show number values between two fields

ITWhisperer — Thu, 10 Jun 2021 07:36:04 GMT

| rex "id\\\\\":\\\\\"(?<id>.*)\\\\\",\\\\\"language"

Re: Rex command to Show number values between two fields

gcusello — Thu, 10 Jun 2021 07:40:19 GMT

hi @rajasplunk89,

it depends on the presence of backslashes in your log.

If they are use this regex:

| rex "\"id\\\":\\\"(?<id>[^\\]+)"

that you can test at https://regex101.com/r/JNvX1g/1

if they aren't use this regex:

| rex "\"id\":\"(?<id>[^\"]+)"

that you can test at https://regex101.com/r/0nuV10/1

Ciao.

Giuseppe

Re: Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 07:52:12 GMT

got the following error

Re: Rex command to Show number values between two fields

gcusello — Thu, 10 Jun 2021 07:56:08 GMT

Hi @rajasplunk89,

regex is correct, as you can test in regex101, but sometimes in Splunk is required an additional backslash to escape the backslash in logs, please try this:

| rex "\"id\\\\":\\\\"(?<id>[^\\]+)"

Ciao.

Giuseppe

Re: Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 07:58:29 GMT

It worked in Most cases
But some cases it pulled entire event

Is it Possible to filter even that?
What is the logic behind using ////
could you please explain that too?

Re: Rex command to Show number values between two fields

gcusello — Thu, 10 Jun 2021 08:04:09 GMT

Hi @rajasplunk89,

could you share the full event that is wrong?

Ciao.

Giuseppe

Re: Rex command to Show number values between two fields

ITWhisperer — Thu, 10 Jun 2021 08:04:46 GMT

In the case you have highlighted, the language element does not appear immediately after the id element, so the extract gives what is between id and language (which is what you asked for). If you just want what is between the double quotes after the id tag try this

| rex "id\\\\\":\\\\\"(?<id>[^\\\]*)"

The backslashes are to escape the backslashes and the double quotes

Re: Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 08:27:46 GMT

Some values are wrong If I use

rex "id\\\\\":\\\\\"(?<id>[^\\\]*)"

For example 52049d5f-23b4-4226-805d-4a210879b0aa is not Pulled whereas 4282b6b8-56dd-406d-8fbc-165b82a3f4d2 is getting Pulled

{"line":"d4a1bdc6f233db7d INFO 2021-06-09 19:35:45,010 [reactor-http-epoll-4] com.test.content.test.external.SnsPayloadEventSender Sent payload event id=daae6ff7-bf8f-5b42-9975-36cca5dfd502 for payload event={\"approvalDate\":\"2021-06-09T19:35:44.670Z\",\"audienceIds\":[\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\"],\"audienceTargets\":[{\"id\":\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\",\"type\":null}],\"collectionGroupId\":\"82a74ac1-c527-4470-b7b0-fb5f3ef3c2e2\",\"collections\":[\"90eef6b9-8a55-43cd-9aa8-73d29a55494b\"],\"endSchedule\":\"3000-01-01T19:00:00.000Z\",\"id\":\"52049d5f-23b4-4226-805d-4a210879b0aa\",\"language\":\"es-419\",\"mplace\":\"US\",\"nodeVersion\":null,\"payloadType\":\"thread\",\"preview\":false,\"resourceSubType\":\"thread\",\"resourceType\":\"thread\",\"startSchedule\":\"2021-06-11T21:00:57.000Z\",\"type\":\"PUBLISH\",\"version\":\"1623267340386\"}","source":"stdout","tag":"4d9520571148","attrs":{"application":"test","team":"test"}}

Re: Rex command to Show number values between two fields

ITWhisperer — Thu, 10 Jun 2021 08:31:46 GMT

| rex "\"id\\\\\":\\\\\"(?<id>[^\\\]*)"

Re: Rex command to Show number values between two fields

gcusello — Thu, 10 Jun 2021 08:34:31 GMT

Hi @rajasplunk89,

please try this:

| rex max_match=1 "\"id\\\":\\\"(?<id>[^\\]+)"

Ciao.

Giuseppe

Re: Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 08:35:31 GMT

Still getting the wrong value

{"line":"d4a1bdc6f233db7d INFO 2021-06-09 19:35:45,010 [reactor-http-epoll-4] com.test.content.test.external.SnsPayloadEventSender Sent payload event id=daae6ff7-bf8f-5b42-9975-36cca5dfd502 for payload event={\"approvalDate\":\"2021-06-09T19:35:44.670Z\",\"audienceIds\":[\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\"],\"audienceTargets\":[{\"id\":\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\",\"type\":null}],\"collectionGroupId\":\"82a74ac1-c527-4470-b7b0-fb5f3ef3c2e2\",\"collections\":[\"90eef6b9-8a55-43cd-9aa8-73d29a55494b\"],\"endSchedule\":\"3000-01-01T19:00:00.000Z\",\"id\":\"52049d5f-23b4-4226-805d-4a210879b0aa\",\"language\":\"es-419\",\"mplace\":\"US\",\"nodeVersion\":null,\"payloadType\":\"thread\",\"preview\":false,\"resourceSubType\":\"thread\",\"resourceType\":\"thread\",\"startSchedule\":\"2021-06-11T21:00:57.000Z\",\"type\":\"PUBLISH\",\"version\":\"1623267340386\"}","source":"stdout","tag":"4d9520571148","attrs":{"application":"test","team":"test"}}

Re: Rex command to Show number values between two fields

ITWhisperer — Thu, 10 Jun 2021 08:42:37 GMT

| rex "\"id\\\\\":\\\\\"(?<id>[^\\\]*)\\\\\",\\\\\"language"

Re: Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 08:44:34 GMT

Got this error

Re: Rex command to Show number values between two fields

rajasplunk89 — Thu, 10 Jun 2021 08:48:50 GMT

Thanks a lot It worked
Could you please explain the Logic
Why so many Back slashes used ? Which backslash avoids which one

Re: Rex command to Show number values between two fields

ITWhisperer — Thu, 10 Jun 2021 09:00:12 GMT

rex takes a double-quoted string, this means if it contains a double quote, this needs to be escaped so it doesn't close the string. The escaping character is a backslash. This also means that backslashes have to be escaped with another backslash. The string you are searching also has escaped double quotes, so the match string needs to escape the escape and escape the double quote. That would get you to the point where the match string would work in regex101.com. However, is appears that the way splunk parses the spl, it requires a further 2 backslashes for each escaped original backslash. So, you can see that the backslashes keep piling up.