topic Re: Unable to get the correct rex to extract fields from string in Splunk Search

How to get the correct rex to extract fields from string?

j2menanda — Tue, 29 Mar 2022 20:49:34 GMT

Hi, I have below string and I am trying to get StartTime, EndTime and Count to be displayed in the dashboard.

"Non-Match - Window Event not matches with events Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100\"

I tried multiple rex formats but couldn't succeed. Can I get some help with this please?

Re: Unable to get the correct rex to extract fields from string

ITWhisperer — Tue, 29 Mar 2022 19:38:40 GMT

StartTime=(?<StartTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)\sEndTime=(?<EndTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)\sCount=(?<Count>\d+)

Re: Unable to get the correct rex to extract fields from string

j2menanda — Tue, 29 Mar 2022 20:09:34 GMT

No, that did not really work for me.

I tried the below and with it, I am able to get the start & end times but not the count.

rex "StartTime=(?<startTime>.*) EndTime=(?<endTime>.*) Count=(?<Count>\d+)"

Re: Unable to get the correct rex to extract fields from string

ITWhisperer — Tue, 29 Mar 2022 20:18:17 GMT

Try them as separate rex commands

| rex "StartTime=(?<StartTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)" | rex "EndTime=(?<EndTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)" | rex "Count=(?<Count>\d+)"

Re: Unable to get the correct rex to extract fields from string

j2menanda — Tue, 29 Mar 2022 20:24:05 GMT

Thanks for that. It is as good as the below one:

| rex "StartTime=(?<startTime>.*) EndTime=(?<endTime>.*) Count=(?<Count>[^ ]+)"

except for it doesn't get the 'Count'.

Below is my log:

{"timestamp":"2022-03-25T15:16:49.066+00:00","logger":"config.SomeConfig","message":"FID=SomeConfig APPL= RQID= TEXT=\"Recon :: Non-Match - Window Event not matches with Transaction Store Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100\" STRT=1648221409","level":"INFO","application-id":"103299","application-name":"ingest"}

Re: Unable to get the correct rex to extract fields from string

yuanliu — Wed, 30 Mar 2022 07:30:31 GMT

In that case you would have a field named 'message'. Consider extract aka kv. For example,

| rename _raw AS temp, message AS _raw | kv pairdelim=" " | rename temp AS _raw ``` only if you still need original _raw ```

Your sample data gives

Count	EndTime	FID	STRT	StartTime	TEXT	application-id	application-name	level	logger	timestamp
100	2020-02-03T11:00:00.000Z	SomeConfig	1648221409	2020-02-03T11:00:00.000Z	Recon :: Non-Match - Window Event not matches with Transaction Store Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100	103299	ingest	INFO	config.SomeConfig	2022-03-25T15:16:49.066+00:00

Re: Unable to get the correct rex to extract fields from string

j2menanda — Wed, 30 Mar 2022 14:24:25 GMT

Thank you. Used the below as is.

| rename _raw AS temp, message AS _raw
| kv pairdelim=" "

The 'Text' is one single string that includes start time and end time along with the count, and the TEXT itself is part of the 'message' field.

Re: Unable to get the correct rex to extract fields from string

j2menanda — Wed, 30 Mar 2022 15:33:26 GMT

| rename _raw AS temp, message AS _raw
| extract pairdelim="?&" kvdelim="="
| table StartTime, EndTime, Count

The above query worked for me when I ran in browser. However, I am not able to use this in the dashboard. It says invalid character entity. For that matter, any other query that uses a regex is showing error in the xml for dashboard saying unsatisfied close tag or something of that kind.

Re: Unable to get the correct rex to extract fields from string

yuanliu — Wed, 30 Mar 2022 16:20:41 GMT

Not sure why you need pairdelim="?&" - the sample data reads like white space to me. But if the ampersand (&) is needed in Simple XML, you must substitute with "&" (no quotes) if you use source editor. In visual editor you must use "&".