topic Re: Regex Matching/Please give any solution in Splunk Search

Regex Matching/Please give any solution

rajaguru2790 — Wed, 26 Jun 2019 13:16:53 GMT

A field has multiple lines like a chat log. Below aLL DATA IS in one field of Splunk in the same way as below. Now need to match agent's initial response and capture in a separate field. Aju is the user and Rohi is the agent. Whenever chat is assigned to agent system messgae generated(In this example 1/1/2019 2:42:59 AM Rohi system message: ready to chat). Need to match the next line of agent response time after this agent system message "Rohi system message: ready to chat" in the entire log . That is called initial reponse for this chat (In this example it's timestamp/transcript is (1/1/2019 2:51:16 AM Rohi Hello Aju my name is Rohi. How can I help you today?) )Please help me on calculating Initial reponse time to a separate field using REGEX or someother way. Thanks

1/1/2019 2:42:55 AM
Aju

Hi Team

1/1/2019 2:42:56 AM
System
The data has been added:

- Customer Info

1/1/2019 2:42:59 AM
Rohi

System Message: Rohi is ready to chat.

1/1/2019 2:43:09 AM
Aju

Wish you a very happy ne year

1/1/2019 2:43:12 AM
Aju

new*

1/1/2019 2:43:25 AM
Aju

I need to KNOW ABOUT A CAR

1/1/2019 2:43:32 AM
Aju

please help me

1/1/2019 2:45:07 AM
Aju

Anyone there ?

1/1/2019 2:47:13 AM
Aju

??

1/1/2019 2:49:23 AM
Aju

?? Hi Rohi You there?

1/1/2019 2:51:16 AM
Rohi

Hello Aju my name is Rohi. How can I help you today?

1/1/2019 2:51:27 AM

Chat goes on....

Re: Regex Matching/Please give any solution

gcusello — Wed, 26 Jun 2019 14:12:34 GMT

Hi rajaguru2790,
Try something like this

| rex "System Message: \w+ is ready to chat\.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P<my_field>.+)$"

You can test it at https://regex101.com/r/fjaU3e/1

Bye.
Giuseppe

Re: Regex Matching/Please give any solution

jnudell_2 — Wed, 30 Sep 2020 01:03:54 GMT

Hi @rajaguru2790 ,

You can use the following:

| rex "^(?<chat_start_time>[^\r\n]+)[\r\n]+[\S\s]+System\s+Message:\s+(?<agent>\S+) is ready to chat[\S\s]+[\r\n]+(?<initial_response_time>[^\r\n]+)[\r\n]+\2[\r\n]+(?<initial_response>[^\r\n]+)[\r\n]+" 

| eval chat_start_time = strptime(chat_start_time, "%m/%d/%Y %I:%M:%S %p") 

| eval initial_response_time = strptime(initial_response_time, "%m/%d/%Y %I:%M:%S %p")

| eval agent_response_time = tostring(initial_response_time - chat_start_time, "duration")

This will look something like this:

Re: Regex Matching/Please give any solution

woodcock — Sat, 29 Jun 2019 19:12:07 GMT

Like this:

| makeresults | eval DATA="1/1/2019 2:42:55 AM 
Aju 
Hi Team

1/1/2019 2:42:56 AM 
System 
The data has been added: 
- Customer Info

1/1/2019 2:42:59 AM 
Rohi
System Message: Rohi is ready to chat. 

1/1/2019 2:43:09 AM 
Aju 
Wish you a very happy ne year 

1/1/2019 2:43:12 AM 
Aju 
new* 

1/1/2019 2:43:25 AM 
Aju 
I need to KNOW ABOUT A CAR

1/1/2019 2:43:32 AM 
Aju 
please help me 

1/1/2019 2:45:07 AM 
Aju 
Anyone there ? 

1/1/2019 2:47:13 AM 
Aju 
?? 

1/1/2019 2:49:23 AM 
Aju 
?? Hi Rohi You there? 

1/1/2019 2:51:16 AM 
Rohi
Hello Aju my name is Rohi. How can I help you today? 

1/1/2019 2:51:27 AM"

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| makemv tokenizer="(?ms)(.*?)\s*[\r\n]{2,}\s*" DATA
| rex field=DATA "(?<agent>\S+) is ready to chat\."
| eval DATA=mvindex(DATA, mvfind(DATA, "ready to chat") + 1, -1)
| eval agentFilterRegEx = " [AP]M[\r\n\s]+" . agent . "[\r\n\s]"
| eval agentFirstResponse = mvindex(DATA, mvfind(DATA, agentFilterRegEx))
| rex field=agentFirstResponse "(?ms)^(?<agentFirstResponseTime>[^\r\n]+)[\r\n]+[^\r\n]+[\r\n]+(?<agentFirstResponse>.*)$"
| eval agentFirstResponseTime = strptime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| fieldformat agentFirstResponseTime = strftime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")

Re: Regex Matching/Please give any solution

rajaguru2790 — Wed, 30 Sep 2020 01:05:08 GMT

Thanks for the help. I am getting this error.

Error in 'rex' command: regex="System Message: \w+ is ready to chat.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P.+)$" has exceeded configured match_limit, consider raising the value in limits.conf

I chnaged the limits.conf in local like below. But didnot work

[rex]
match_limit = 0
depth_limit = 0

Re: Regex Matching/Please give any solution

rajaguru2790 — Wed, 30 Sep 2020 01:05:12 GMT

Error in 'rex' command: regex="System Message: \w+ is ready to chat.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P.+)$" has exceeded configured match_limit, consider raising the value in limits.conf

I chnaged the limits.conf in local like below. But didnot work

[rex]
match_limit = 0
depth_limit = 0

Re: Regex Matching/Please give any solution

rajaguru2790 — Mon, 01 Jul 2019 06:26:12 GMT

Hi . Thanks for teh help. Like this there are 10000 chats with different names . How can I take this generically. Please assist

Re: Regex Matching/Please give any solution

woodcock — Mon, 01 Jul 2019 15:26:34 GMT

This should work as-is, so long as the field with your message is called DATA. If not, just change the field name everywhere. Of course, you throw out the stuff before the COMMENT.

Re: Regex Matching/Please give any solution

rajaguru2790 — Wed, 30 Sep 2020 01:05:23 GMT

Now working. One change in the log. Message for System message is: Rohi is online and Ready to chat. It its not ready to chat.

This is the one whcih I executed.

index="use_case_one"
| makemv tokenizer="(?ms)(.?)\s[\r\n]{2,}\s*" DATA
| rex field=DATA "(?\S+) is online and ready to chat."
| eval DATA=mvindex(DATA, mvfind(DATA, "online and ready to chat") + 1, -1)
| eval agentFilterRegEx = " [AP]M[\r\n\s]+" . agent . "[\r\n\s]"
| eval agentFirstResponse = mvindex(DATA, mvfind(DATA, agentFilterRegEx))
| rex field=agentFirstResponse "(?ms)^(?[^\r\n]+)[\r\n]+[^\r\n]+[\r\n]+(?.*)$"
| eval agentFirstResponseTime = strptime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| fieldformat agentFirstResponseTime = strftime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| table "Session Log" "DATA"

Re: Regex Matching/Please give any solution

FrankVl — Tue, 02 Jul 2019 10:03:22 GMT

Guess something like this is the only way, huh. Shame Splunk doesn't support using (?P=name) to match a named subpattern. That way you could have done it with a single regex: https://regex101.com/r/d8F6SN/1/

Edit: oh, wait, that is supported: | rex "(?s)System Message: (?<agent>\w+) is ready to chat.*?[\r\n]+[\d\/]+\s+[\d:]+\s+\w+\s+[\r\n]+(?P=agent)[\r\n]+(?<agentFirstResponse>.*?)[\r\n]+[\d\/]+\s+[\d:]+\s+\w+"

Or see the answer from @jnudell_2 for a similar solution using \1 instead of (?P=agent).

Re: Regex Matching/Please give any solution

FrankVl — Tue, 02 Jul 2019 10:24:20 GMT

That is not the same regex that jnudell_2 suggested. Did you actually try the correct regex? I've tested this as well and it seems to work fine.

| makeresults | eval _raw="1/1/2019 2:42:55 AM 
Aju 
Hi Team

1/1/2019 2:42:56 AM 
System 
The data has been added: 
- Customer Info

1/1/2019 2:42:59 AM 
Rohi
System Message: Rohi is ready to chat. 

1/1/2019 2:43:09 AM 
Aju 
Wish you a very happy ne year 

1/1/2019 2:43:12 AM 
Aju 
new* 

1/1/2019 2:43:25 AM 
Aju 
I need to KNOW ABOUT A CAR

1/1/2019 2:43:32 AM 
Aju 
please help me 

1/1/2019 2:45:07 AM 
Aju 
Anyone there ? 

1/1/2019 2:47:13 AM 
Aju 
?? 

1/1/2019 2:49:23 AM 
Aju 
?? Hi Rohi You there? 

1/1/2019 2:51:16 AM 
Rohi
Hello Aju my name is Rohi. How can I help you today? 

1/1/2019 2:51:27 AM

Chat goes on...."
 | rex "^(?<chat_start_time>[^\r\n]+)[\r\n]+[\S\s]+System\s+Message:\s+(?<agent>\S+) is ready to chat[\S\s]+[\r\n]+(?<initial_response_time>[^\r\n]+)[\r\n]+\2[\r\n]+(?<initial_response>[^\r\n]+)[\r\n]+"

Alternatively, this rex also works: | rex "System Message: (?<agent>\w+) is ready to chat.*?[\r\n]+[\d\/]+\s+[\d:]+\s+\w+\s+[\r\n]+(?P=agent)[\r\n]+(?<agentFirstResponse>.*?)[\r\n]+[\d\/]+\s+[\d:]+\s+\w+"