Splunk Search

How to get the correct rex to extract fields from string?

j2menanda
Explorer

Hi, I have below string and I am trying to get StartTime, EndTime and Count to be displayed in the dashboard.

"Non-Match - Window Event not matches with events Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100\"

 

I tried multiple rex formats but couldn't succeed. Can I get some help with this please?

Labels (1)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

Not sure why you need pairdelim="?&" - the sample data reads like white space to me.  But if the ampersand (&) is needed in Simple XML, you must substitute with "&" (no quotes) if you use source editor.  In visual editor you must use "&".

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust
StartTime=(?<StartTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)\sEndTime=(?<EndTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)\sCount=(?<Count>\d+)
0 Karma

j2menanda
Explorer

No, that did not really work for me.


splunk-rex-no-results.png

I tried the below and with it, I am able to get the start & end times but not the count.

rex "StartTime=(?<startTime>.*) EndTime=(?<endTime>.*) Count=(?<Count>\d+)"

Tags (2)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try them as separate rex commands

| rex "StartTime=(?<StartTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)"
| rex "EndTime=(?<EndTime>\d{4}\-\d\d\-\d\dT\d\d:\d\d:\d\d\.\d+Z)"
| rex "Count=(?<Count>\d+)"
0 Karma

j2menanda
Explorer

Thanks for that. It is as good as the below one:

| rex "StartTime=(?<startTime>.*) EndTime=(?<endTime>.*) Count=(?<Count>[^ ]+)"

 

except for it doesn't get the 'Count'.

 

Below is my log:

{"timestamp":"2022-03-25T15:16:49.066+00:00","logger":"config.SomeConfig","message":"FID=SomeConfig APPL= RQID= TEXT=\"Recon :: Non-Match - Window Event not matches with Transaction Store Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100\" STRT=1648221409","level":"INFO","application-id":"103299","application-name":"ingest"}

0 Karma

yuanliu
SplunkTrust
SplunkTrust

In that  case you would have a field named 'message'.  Consider extract aka kv.  For example,

 

| rename _raw AS temp, message AS _raw
| kv pairdelim=" "
| rename temp AS _raw ``` only if you still need original _raw ```

 

Your sample data gives

CountEndTimeFIDSTRTStartTimeTEXTapplication-idapplication-namelevelloggertimestamp
1002020-02-03T11:00:00.000ZSomeConfig16482214092020-02-03T11:00:00.000ZRecon :: Non-Match - Window Event not matches with Transaction Store Count with StartTime=2020-02-03T11:00:00.000Z EndTime=2020-02-03T11:00:00.000Z Count=100103299ingestINFOconfig.SomeConfig2022-03-25T15:16:49.066+00:00

 

Tags (1)

j2menanda
Explorer

| rename _raw AS temp, message AS _raw
| extract pairdelim="?&" kvdelim="="
| table StartTime, EndTime, Count

The above query worked for me when I ran in browser. However, I am not able to use this in the dashboard. It says invalid character entity. For that matter, any other query that uses a regex is showing error in the xml for dashboard saying unsatisfied close tag or something of that kind.

0 Karma

yuanliu
SplunkTrust
SplunkTrust

Not sure why you need pairdelim="?&" - the sample data reads like white space to me.  But if the ampersand (&) is needed in Simple XML, you must substitute with "&amp;" (no quotes) if you use source editor.  In visual editor you must use "&".

j2menanda
Explorer

Thank you. Used the below as is.

| rename _raw AS temp, message AS _raw
| kv pairdelim=" "

 

splunk-rex-no-results.png

The 'Text' is one single string that includes start time and end time along with the count, and the TEXT itself is part of the 'message' field.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...