Rex command to Show number values between two fields

[rajasplunk89]

How to use Rex command to show Value in between 'Id' and `language` for example 0827ce61-e07c-4b51-a052-681dcc94fa2f to show in table.

I tried EREX to generate pattern, it worked in most cases but in some cases failed.
Is there any easy way to show the value between 2 fields?

Provided the Event sample

{"line":"c5fd03529388e8d6 INFO 2021-06-10 04:11:02,966 [reactor-http-epoll-2] com.test.content.test.external.SnsPayloadEventSender Sent payload event id=9a67e81d-5ae3-5e84-b1f7-b126c9f5c787 for payload event={\"approvalDate\":\"2015-06-10T04:11:02.128Z\",\"adenceIds\":[],\"adenceTargets\":[],\"collGroupId\":\"008be467-6c78-4079-94f0-70e2d6cc4003\",\"collections\":[\"d879aa85-698c-41a1-b066-a8ecdf6b8a9c\",\"d879aa85-698c-41a1-b066-a8ecdf6b8a9c\"],\"endSchedule\":\"3000-01-01T19:00:00.000Z\",\"id\":\"0827ce61-e07c-4b51-a052-681dcc94fa2f\",\"language\":\"ja\",\"mplace\":\"JP\",\"nodeVersion\":null,\"payloadType\":\"thread\",\"preview\":false,\"resourceSubType\":\"thread\",\"resourceType\":\"thread\",\"startSchedule\":\"2021-03-26T22:00:28.000Z\",\"type\":\"P\",\"version\":\"1622644250765\"}","source":"stdout","tag":"21c5897e605c","attrs":{"application":"test","team":"test"}}

[gcusello]

hi @rajasplunk89,

it depends on the presence of backslashes in your log.

If they are use this regex:

| rex "\"id\\\":\\\"(?<id>[^\\]+)"

that you can test at https://regex101.com/r/JNvX1g/1

if they aren't use this regex:

| rex "\"id\":\"(?<id>[^\"]+)"

that you can test at https://regex101.com/r/0nuV10/1

Ciao.

Giuseppe

[rajasplunk89]

got the following error

[gcusello]

Hi @rajasplunk89,

regex is correct, as you can test in regex101, but sometimes in Splunk is required an additional backslash to escape the backslash in logs, please try this:

| rex "\"id\\\\":\\\\"(?<id>[^\\]+)"

 Ciao.

Giuseppe

[ITWhisperer]

| rex "id\\\\\":\\\\\"(?<id>.*)\\\\\",\\\\\"language"

[rajasplunk89]

It worked in Most cases
But some cases it pulled entire event

Is it Possible to filter even that?
What is the logic behind using //// 
could you please explain that too?

[ITWhisperer]

In the case you have highlighted, the language element does not appear immediately after the id element, so the extract gives what is between id and language (which is what you asked for). If you just want what is between the double quotes after the id tag try this

| rex "id\\\\\":\\\\\"(?<id>[^\\\]*)"

The backslashes are to escape the backslashes and the double quotes

[gcusello]

Hi @rajasplunk89,

could you share the full event that is wrong?

Ciao.

Giuseppe

[rajasplunk89]

Some values are wrong If I use

rex "id\\\\\":\\\\\"(?<id>[^\\\]*)"


For example 52049d5f-23b4-4226-805d-4a210879b0aa is not Pulled whereas 4282b6b8-56dd-406d-8fbc-165b82a3f4d2 is getting Pulled

{"line":"d4a1bdc6f233db7d INFO 2021-06-09 19:35:45,010 [reactor-http-epoll-4] com.test.content.test.external.SnsPayloadEventSender Sent payload event id=daae6ff7-bf8f-5b42-9975-36cca5dfd502 for payload event={\"approvalDate\":\"2021-06-09T19:35:44.670Z\",\"audienceIds\":[\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\"],\"audienceTargets\":[{\"id\":\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\",\"type\":null}],\"collectionGroupId\":\"82a74ac1-c527-4470-b7b0-fb5f3ef3c2e2\",\"collections\":[\"90eef6b9-8a55-43cd-9aa8-73d29a55494b\"],\"endSchedule\":\"3000-01-01T19:00:00.000Z\",\"id\":\"52049d5f-23b4-4226-805d-4a210879b0aa\",\"language\":\"es-419\",\"mplace\":\"US\",\"nodeVersion\":null,\"payloadType\":\"thread\",\"preview\":false,\"resourceSubType\":\"thread\",\"resourceType\":\"thread\",\"startSchedule\":\"2021-06-11T21:00:57.000Z\",\"type\":\"PUBLISH\",\"version\":\"1623267340386\"}","source":"stdout","tag":"4d9520571148","attrs":{"application":"test","team":"test"}}

[gcusello]

Hi @rajasplunk89,

please try this:

| rex max_match=1 "\"id\\\":\\\"(?<id>[^\\]+)"

Ciao.

Giuseppe

[rajasplunk89]

Got this error

[ITWhisperer]

| rex "\"id\\\\\":\\\\\"(?<id>[^\\\]*)"

[rajasplunk89]

Still getting the wrong value

{"line":"d4a1bdc6f233db7d INFO 2021-06-09 19:35:45,010 [reactor-http-epoll-4] com.test.content.test.external.SnsPayloadEventSender Sent payload event id=daae6ff7-bf8f-5b42-9975-36cca5dfd502 for payload event={\"approvalDate\":\"2021-06-09T19:35:44.670Z\",\"audienceIds\":[\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\"],\"audienceTargets\":[{\"id\":\"4282b6b8-56dd-406d-8fbc-165b82a3f4d2\",\"type\":null}],\"collectionGroupId\":\"82a74ac1-c527-4470-b7b0-fb5f3ef3c2e2\",\"collections\":[\"90eef6b9-8a55-43cd-9aa8-73d29a55494b\"],\"endSchedule\":\"3000-01-01T19:00:00.000Z\",\"id\":\"52049d5f-23b4-4226-805d-4a210879b0aa\",\"language\":\"es-419\",\"mplace\":\"US\",\"nodeVersion\":null,\"payloadType\":\"thread\",\"preview\":false,\"resourceSubType\":\"thread\",\"resourceType\":\"thread\",\"startSchedule\":\"2021-06-11T21:00:57.000Z\",\"type\":\"PUBLISH\",\"version\":\"1623267340386\"}","source":"stdout","tag":"4d9520571148","attrs":{"application":"test","team":"test"}}

[ITWhisperer · Accepted solution]

| rex "\"id\\\\\":\\\\\"(?<id>[^\\\]*)\\\\\",\\\\\"language"

[rajasplunk89]

Thanks a lot It worked
Could you please explain the Logic
Why so many Back slashes used ? Which backslash avoids which one

[ITWhisperer]

rex takes a double-quoted string, this means if it contains a double quote, this needs to be escaped so it doesn't close the string. The escaping character is a backslash. This also means that backslashes have to be escaped with another backslash. The string you are searching also has escaped double quotes, so the match string needs to escape the escape and escape the double quote. That would get you to the point where the match string would work in regex101.com. However, is appears that the way splunk parses the spl, it requires a further 2 backslashes for each escaped original backslash. So, you can see that the backslashes keep piling up.