How to get the correct rex to extract fields from string?

[j2menanda]

Hi, I have below string and I am trying to get StartTime, EndTime and Count to be displayed in the dashboard.

"Non-Match - Window Event not matches with events Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100\"

I tried multiple rex formats but couldn't succeed. Can I get some help with this please?

[ITWhisperer]

StartTime=(?<StartTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)\sEndTime=(?<EndTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)\sCount=(?<Count>\d+)

[j2menanda]

No, that did not really work for me.

I tried the below and with it, I am able to get the start & end times but not the count.

rex "StartTime=(?<startTime>.*) EndTime=(?<endTime>.*) Count=(?<Count>\d+)"

[ITWhisperer]

Try them as separate rex commands

| rex "StartTime=(?<StartTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)"
| rex "EndTime=(?<EndTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)"
| rex "Count=(?<Count>\d+)"

[j2menanda]

Thanks for that. It is as good as the below one:

| rex "StartTime=(?<startTime>.*) EndTime=(?<endTime>.*) Count=(?<Count>[^ ]+)"

except for it doesn't get the 'Count'.

Below is my log:

{"timestamp":"2022-03-25T15:16:49.066+00:00","logger":"config.SomeConfig","message":"FID=SomeConfig APPL= RQID= TEXT=\"Recon :: Non-Match - Window Event not matches with Transaction Store Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100\" STRT=1648221409","level":"INFO","application-id":"103299","application-name":"ingest"}

[yuanliu]

In that  case you would have a field named 'message'.  Consider extract aka kv.  For example,

| rename _raw AS temp, message AS _raw
| kv pairdelim=" "
| rename temp AS _raw ``` only if you still need original _raw ```

Your sample data gives

Count	EndTime	FID	STRT	StartTime	TEXT	application-id	application-name	level	logger	timestamp
100	2020-02-03T11:00:00.000Z	SomeConfig	1648221409	2020-02-03T11:00:00.000Z	Recon :: Non-Match - Window Event not matches with Transaction Store Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100	103299	ingest	INFO	config.SomeConfig	2022-03-25T15:16:49.066+00:00

[j2menanda]

| rename _raw AS temp, message AS _raw
| extract pairdelim="?&" kvdelim="="
| table StartTime, EndTime, Count

The above query worked for me when I ran in browser. However, I am not able to use this in the dashboard. It says invalid character entity. For that matter, any other query that uses a regex is showing error in the xml for dashboard saying unsatisfied close tag or something of that kind.

[yuanliu · Accepted solution]

Not sure why you need pairdelim="?&" - the sample data reads like white space to me.  But if the ampersand (&) is needed in Simple XML, you must substitute with "&amp;" (no quotes) if you use source editor.  In visual editor you must use "&".

[j2menanda]

Thank you. Used the below as is.

| rename _raw AS temp, message AS _raw
| kv pairdelim=" "

The 'Text' is one single string that includes start time and end time along with the count, and the TEXT itself is part of the 'message' field.