topic Re: rex and extracted fields in Splunk Search

rex and extracted fields

Mick_OBrien — Thu, 18 Nov 2021 13:03:32 GMT

I have a search string that gives me count of txns processed by a job...

....| rex field=_raw "Total txns:(?<TxnsCount>.*)#015" | table _time, TxnsCount

...but when I try to extract txns where value greater than 10...

...| rex field=_raw "Total txns:(?<TxnsCount>.*)#015" | table _time, TxnsCount | where TxnsCount > 10

...no data is returned

Any help welcome

Thanks in advance

Re: rex and extracted fields

richgalloway — Thu, 18 Nov 2021 13:59:57 GMT

Are you sure there are events where TxnsCount is greater than 10? Please share some sample (sanitized) events.

Re: rex and extracted fields

ITWhisperer — Thu, 18 Nov 2021 14:13:17 GMT

Do you have leading/trailing spaces? Try:

| rex field=_raw "Total txns:(?<TxnsCount>.*)#015" | eval TxnsCount=trim(TxnsCount) | table _time, TxnsCount | where TxnsCount > 10

Re: rex and extracted fields

Mick_OBrien — Thu, 18 Nov 2021 14:51:49 GMT

There are txns with 500 [I remove the 'where' clause and I see them] and multiple lower digit txns [not of interested]

I tried trim command but that did not help - still no dataset returned

Re: rex and extracted fields

ITWhisperer — Thu, 18 Nov 2021 14:55:48 GMT

Can you share some events, otherwise, we are blindly guessing!

Re: rex and extracted fields

Mick_OBrien — Thu, 18 Nov 2021 16:25:28 GMT

Some events [sanitised]...

2021-11-18T14:25:01.124036+00:00 <Removed>2021-11-18 14: 24:59.4485|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|897c7235-4a92-4464-8062-0f35748b582a|BOTName|19|||||||||Total TXNs: 4#015

2021-11-18T14:22:30.732755+00:00 <Removed>2021-11-18 14: 22:30.5245|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|2ce1a155-0704-49b3-9c0e-24cab1d807d4|BOTName|19|||||||||Total TXNs: 7#015

2021-11-18T14:22:08.971441+00:00 <Removed>2021-11-18 14: 22:08.8370|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|2ce1a155-0704-49b3-9c0e-24cab1d807d4|BOTName|19|||||||||Total TXNs: 1#015

2021-11-18T14:17:26.722148+00:00 <Removed>2021-11-18 14: 17:26.5446|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|b4db87da-a8f6-4fa6-a292-617c9823488d|BOTName|19|||||||||Total TXNs: 1#015

2021-11-18T14:12:55.708077+00:00 <Removed>2021-11-18 14: 12:55.4094|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|ef394e25-0274-45a7-b209-5ac1dab8c46c|BOTName|19|||||||||Total TXNs: 1#015

2021-11-18T12:58:08.374584+00:00 <Removed>2021-11-18 12: 58:08.2601|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|16ef2534-2a29-4ad4-85c2-afbcaec77a5d|BOTName|19|||||||||Total TXNs: 112#015

2021-11-18T12:48:35.557133+00:00 <Removed>2021-11-18 12: 48:35.2166|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|320327f0-aa79-4f8f-a1d8-f28649da6297|BOTName|19|||||||||Total TXNs: 113#015

2021-11-18T09:46:44.959929+00:00 <Removed>2021-11-18 09: 46:44.8054|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|f0005ca4-22b1-4cf7-a3ec-ef4dc9ee685c|BOTName|19|||||||||Total TXNs: 3#015

2021-11-18T09:46:08.117037+00:00 <Removed>2021-11-18 09: 46:07.9206|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|f0005ca4-22b1-4cf7-a3ec-ef4dc9ee685c|BOTName|19|||||||||Total TXNs: 22#015

2021-11-18T09:41:09.172225+00:00 <Removed>2021-11-18 09: 41:09.0983|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|39b311fa-e223-4e30-93b3-3774096313f3|BOTName|19|||||||||Total TXNs: 5#015

2021-11-18T09:40:17.503636+00:00 <Removed>2021-11-18 09: 40:17.1882|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|39b311fa-e223-4e30-93b3-3774096313f3|BOTName|19|||||||||Total TXNs: 4#015

2021-11-18T09:37:59.351887+00:00 <Removed>2021-11-18 09: 37:59.0219|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|15007061-2dc2-424f-8046-115b11ca2617|BOTName|19|||||||||Total TXNs: 4#015

2021-11-18T08:57:55.157899+00:00 <Removed>2021-11-18 08: 57:54.9116|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|9721848f-7963-40f4-9f5e-0832da57b55c|BOTName|19|||||||||Total TXNs: 10#015

2021-11-18T08:25:42.887812+00:00 <Removed>2021-11-18 08: 25:42.5543|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|a79e6366-a739-4c57-a4ec-26359730034a|BOTName|19|||||||||Total TXNs: 1#015

2021-11-18T08:25:07.681935+00:00 <Removed>2021-11-18 08: 25:07.4193|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|a79e6366-a739-4c57-a4ec-26359730034a|BOTName|19|||||||||Total TXNs: 1#015

2021-11-18T07:48:49.009846+00:00 <Removed>2021-11-18 07: 48:48.8747|INFO|Robot.tenantId:4.UiPath.Orchestrator.Application.Services.Logs.LogsWriter|<Removed>|ProcessName|4|e722d559-7a70-4ef8-a75b-63fa6841eb37|BOTName|19|||||||||Total TXNs: 31#015

Re: rex and extracted fields

ITWhisperer — Thu, 18 Nov 2021 16:30:33 GMT

Your rex doesn't match your events - try this

| rex field=_raw "Total TXNs:\s+(?<TxnsCount>\d+)#015"

Re: rex and extracted fields

Mick_OBrien — Thu, 18 Nov 2021 16:39:47 GMT

@ITWhisperer wrote:
Your rex doesn't match your events - try this

| rex field=_raw "Total TXNs:\s+(?<TxnsCount>\d+)#015"

Sorry - my fault when trying to sanitise data to obscure process details

Re-edited...

_raw

2021-11-18T15:47:15.675045+00:00 <Removed>2021-11-18 15: 47:15.5492|INFO|<Removed>|<Removed>|ProcessName|4|b18f74b5-6210-4294-802b-89c806f8cdcd|BOTName|19|||||||||Total txns: 1#015

2021-11-18T15:40:08.950766+00:00 <Removed>2021-11-18 15: 40:08.8161|INFO|<Removed>|<Removed>|ProcessName|4|1311ded0-b556-4aba-8381-0a1ce89064ad|BOTName|19|||||||||Total txns: 1#015

2021-11-18T15:37:11.174909+00:00 <Removed>2021-11-18 15: 37:11.0629|INFO|<Removed>|<Removed>|ProcessName|4|ca35e8e0-9ba9-46b0-9281-56d90bd337a9|BOTName|19|||||||||Total txns: 3#015

2021-11-18T15:35:53.424646+00:00 <Removed>2021-11-18 15: 35:53.2097|INFO|<Removed>|<Removed>|ProcessName|4|c671c0da-9dd4-421a-be40-845c9ed27021|BOTName|19|||||||||Total txns: 3#015

2021-11-18T14:25:01.124036+00:00 <Removed>2021-11-18 14: 24:59.4485|INFO|<Removed>|<Removed>|ProcessName|4|897c7235-4a92-4464-8062-0f35748b582a|BOTName|19|||||||||Total txns: 4#015

2021-11-18T14:22:30.732755+00:00 <Removed>2021-11-18 14: 22:30.5245|INFO|<Removed>|<Removed>|ProcessName|4|2ce1a155-0704-49b3-9c0e-24cab1d807d4|BOTName|19|||||||||Total txns: 7#015

2021-11-18T12:58:08.374584+00:00 <Removed>2021-11-18 12: 58:08.2601|INFO|<Removed>|<Removed>|ProcessName|4|16ef2534-2a29-4ad4-85c2-afbcaec77a5d|BOTName|19|||||||||Total txns: 112#015

2021-11-18T12:48:35.557133+00:00 <Removed>2021-11-18 12: 48:35.2166|INFO|<Removed>|<Removed>|ProcessName|4|320327f0-aa79-4f8f-a1d8-f28649da6297|BOTName|19|||||||||Total txns: 113#015

2021-11-18T09:46:08.117037+00:00 <Removed>2021-11-18 09: 46:07.9206|INFO|<Removed>|<Removed>|ProcessName|4|f0005ca4-22b1-4cf7-a3ec-ef4dc9ee685c|BOTName|19|||||||||Total txns: 22#015

2021-11-18T09:37:46.727116+00:00 <Removed>2021-11-18 09: 37:46.6463|INFO|<Removed>|<Removed>|ProcessName|4|15007061-2dc2-424f-8046-115b11ca2617|BOTName|19|||||||||Total txns: 31#015

2021-11-18T09:25:33.222767+00:00 <Removed>2021-11-18 09: 25:32.7040|INFO|<Removed>|<Removed>|ProcessName|4|58c03fe0-941b-41ec-841e-9b1602e1c806|BOTName|19|||||||||Total txns: 10#015

2021-11-18T08:29:17.696167+00:00 <Removed>2021-11-18 08: 29:17.3912|INFO|<Removed>|<Removed>|ProcessName|4|d29e1bb5-9ffa-4746-ae22-223bd1975a98|BOTName|19|||||||||Total txns: 10#015

2021-11-17T17:40:17.998833+00:00 <Removed>2021-11-17 17: 40:17.1217|INFO|<Removed>|<Removed>|ProcessName|4|5164ec1c-5463-476a-873b-c16529d9ad98|BOTName|19|||||||||Total txns: 66#015

2021-11-17T15:23:00.091137+00:00 <Removed>2021-11-17 15: 22:59.9499|INFO|<Removed>|<Removed>|ProcessName|4|d6a27318-89da-4815-a9bf-6de77adab50f|BOTName|19|||||||||Total txns: 3#015

2021-11-17T15:22:49.715505+00:00 <Removed>2021-11-17 15: 22:49.5033|INFO|<Removed>|<Removed>|ProcessName|4|d6a27318-89da-4815-a9bf-6de77adab50f|BOTName|19|||||||||Total txns: 2#015

2021-11-17T15:05:52.019927+00:00 <Removed>2021-11-17 15: 05:51.4867|INFO|<Removed>|<Removed>|ProcessName|4|a78efcf6-a4c1-465b-aca2-b995d62d229f|BOTName|19|||||||||Total txns: 1#015

2021-11-17T15:05:32.929245+00:00 <Removed>2021-11-17 15: 05:31.3525|INFO|<Removed>|<Removed>|ProcessName|4|a78efcf6-a4c1-465b-aca2-b995d62d229f|BOTName|19|||||||||Total txns: 1#015

2021-11-17T15:05:05.251231+00:00 <Removed>2021-11-17 15: 05:02.5109|INFO|<Removed>|<Removed>|ProcessName|4|a78efcf6-a4c1-465b-aca2-b995d62d229f|BOTName|19|||||||||Total txns: 1#015

Re: rex and extracted fields

ITWhisperer — Thu, 18 Nov 2021 16:48:45 GMT

| makeresults | eval _raw="2021-11-18T15:47:15.675045+00:00 <Removed>2021-11-18 15: 47:15.5492|INFO|<Removed>|<Removed>|ProcessName|4|b18f74b5-6210-4294-802b-89c806f8cdcd|BOTName|19|||||||||Total txns: 1#015 2021-11-18T15:40:08.950766+00:00 <Removed>2021-11-18 15: 40:08.8161|INFO|<Removed>|<Removed>|ProcessName|4|1311ded0-b556-4aba-8381-0a1ce89064ad|BOTName|19|||||||||Total txns: 1#015 2021-11-18T15:37:11.174909+00:00 <Removed>2021-11-18 15: 37:11.0629|INFO|<Removed>|<Removed>|ProcessName|4|ca35e8e0-9ba9-46b0-9281-56d90bd337a9|BOTName|19|||||||||Total txns: 3#015 2021-11-18T15:35:53.424646+00:00 <Removed>2021-11-18 15: 35:53.2097|INFO|<Removed>|<Removed>|ProcessName|4|c671c0da-9dd4-421a-be40-845c9ed27021|BOTName|19|||||||||Total txns: 3#015 2021-11-18T14:25:01.124036+00:00 <Removed>2021-11-18 14: 24:59.4485|INFO|<Removed>|<Removed>|ProcessName|4|897c7235-4a92-4464-8062-0f35748b582a|BOTName|19|||||||||Total txns: 4#015 2021-11-18T14:22:30.732755+00:00 <Removed>2021-11-18 14: 22:30.5245|INFO|<Removed>|<Removed>|ProcessName|4|2ce1a155-0704-49b3-9c0e-24cab1d807d4|BOTName|19|||||||||Total txns: 7#015 2021-11-18T12:58:08.374584+00:00 <Removed>2021-11-18 12: 58:08.2601|INFO|<Removed>|<Removed>|ProcessName|4|16ef2534-2a29-4ad4-85c2-afbcaec77a5d|BOTName|19|||||||||Total txns: 112#015 2021-11-18T12:48:35.557133+00:00 <Removed>2021-11-18 12: 48:35.2166|INFO|<Removed>|<Removed>|ProcessName|4|320327f0-aa79-4f8f-a1d8-f28649da6297|BOTName|19|||||||||Total txns: 113#015 2021-11-18T09:46:08.117037+00:00 <Removed>2021-11-18 09: 46:07.9206|INFO|<Removed>|<Removed>|ProcessName|4|f0005ca4-22b1-4cf7-a3ec-ef4dc9ee685c|BOTName|19|||||||||Total txns: 22#015 2021-11-18T09:37:46.727116+00:00 <Removed>2021-11-18 09: 37:46.6463|INFO|<Removed>|<Removed>|ProcessName|4|15007061-2dc2-424f-8046-115b11ca2617|BOTName|19|||||||||Total txns: 31#015 2021-11-18T09:25:33.222767+00:00 <Removed>2021-11-18 09: 25:32.7040|INFO|<Removed>|<Removed>|ProcessName|4|58c03fe0-941b-41ec-841e-9b1602e1c806|BOTName|19|||||||||Total txns: 10#015 2021-11-18T08:29:17.696167+00:00 <Removed>2021-11-18 08: 29:17.3912|INFO|<Removed>|<Removed>|ProcessName|4|d29e1bb5-9ffa-4746-ae22-223bd1975a98|BOTName|19|||||||||Total txns: 10#015 2021-11-17T17:40:17.998833+00:00 <Removed>2021-11-17 17: 40:17.1217|INFO|<Removed>|<Removed>|ProcessName|4|5164ec1c-5463-476a-873b-c16529d9ad98|BOTName|19|||||||||Total txns: 66#015 2021-11-17T15:23:00.091137+00:00 <Removed>2021-11-17 15: 22:59.9499|INFO|<Removed>|<Removed>|ProcessName|4|d6a27318-89da-4815-a9bf-6de77adab50f|BOTName|19|||||||||Total txns: 3#015 2021-11-17T15:22:49.715505+00:00 <Removed>2021-11-17 15: 22:49.5033|INFO|<Removed>|<Removed>|ProcessName|4|d6a27318-89da-4815-a9bf-6de77adab50f|BOTName|19|||||||||Total txns: 2#015 2021-11-17T15:05:52.019927+00:00 <Removed>2021-11-17 15: 05:51.4867|INFO|<Removed>|<Removed>|ProcessName|4|a78efcf6-a4c1-465b-aca2-b995d62d229f|BOTName|19|||||||||Total txns: 1#015 2021-11-17T15:05:32.929245+00:00 <Removed>2021-11-17 15: 05:31.3525|INFO|<Removed>|<Removed>|ProcessName|4|a78efcf6-a4c1-465b-aca2-b995d62d229f|BOTName|19|||||||||Total txns: 1#015 2021-11-17T15:05:05.251231+00:00 <Removed>2021-11-17 15: 05:02.5109|INFO|<Removed>|<Removed>|ProcessName|4|a78efcf6-a4c1-465b-aca2-b995d62d229f|BOTName|19|||||||||Total txns: 1#015" | multikv noheader=t | table _raw | rex "Total txns:\s+(?<TxnsCount>\d+)#015" | where TxnsCount > 10

Re: rex and extracted fields

Mick_OBrien — Thu, 18 Nov 2021 16:55:30 GMT

Thanks - that worked!!

What does your rex syntax do that the syntax I used did not do?

Re: rex and extracted fields

ITWhisperer — Thu, 18 Nov 2021 17:05:35 GMT

".*" is a greedy match whereas \d+ is only picking up 1 or more digits, also, the \s+ takes into account the leading spaces prior to the field so you only have digits left in the field, which makes it numeric, which can then be compared to a numeric value (10). It is usually better to be as specific as you can in the patterns.

Re: rex and extracted fields

Mick_OBrien — Thu, 18 Nov 2021 17:30:41 GMT

Thanks for the explanation - I thought issue was with SPLUNK [and could not see why] when in the end it was the monster [T] rex