Splunk Search

Regex Matching/Please give any solution

rajaguru2790
Explorer

A field has multiple lines like a chat log. Below aLL DATA IS in one field of Splunk in the same way as below. Now need to match agent's initial response and capture in a separate field. Aju is the user and Rohi is the agent. Whenever chat is assigned to agent system messgae generated(In this example 1/1/2019 2:42:59 AM Rohi system message: ready to chat). Need to match the next line of agent response time after this agent system message "Rohi system message: ready to chat" in the entire log . That is called initial reponse for this chat (In this example it's timestamp/transcript is (1/1/2019 2:51:16 AM Rohi Hello Aju my name is Rohi. How can I help you today?) )Please help me on calculating Initial reponse time to a separate field using REGEX or someother way. Thanks

1/1/2019 2:42:55 AM
Aju

Hi Team

1/1/2019 2:42:56 AM
System
The data has been added:

- Customer Info

1/1/2019 2:42:59 AM
Rohi

System Message: Rohi is ready to chat.

1/1/2019 2:43:09 AM
Aju

Wish you a very happy ne year

1/1/2019 2:43:12 AM
Aju

new*

1/1/2019 2:43:25 AM
Aju

I need to KNOW ABOUT A CAR

1/1/2019 2:43:32 AM
Aju

please help me

1/1/2019 2:45:07 AM
Aju

Anyone there ?

1/1/2019 2:47:13 AM
Aju

??

1/1/2019 2:49:23 AM
Aju

?? Hi Rohi You there?

1/1/2019 2:51:16 AM
Rohi

Hello Aju my name is Rohi. How can I help you today?

1/1/2019 2:51:27 AM

Chat goes on....

Tags (2)
0 Karma

woodcock
Esteemed Legend

Like this:

| makeresults | eval DATA="1/1/2019 2:42:55 AM 
Aju 
Hi Team

1/1/2019 2:42:56 AM 
System 
The data has been added: 
- Customer Info

1/1/2019 2:42:59 AM 
Rohi
System Message: Rohi is ready to chat. 

1/1/2019 2:43:09 AM 
Aju 
Wish you a very happy ne year 

1/1/2019 2:43:12 AM 
Aju 
new* 

1/1/2019 2:43:25 AM 
Aju 
I need to KNOW ABOUT A CAR

1/1/2019 2:43:32 AM 
Aju 
please help me 

1/1/2019 2:45:07 AM 
Aju 
Anyone there ? 

1/1/2019 2:47:13 AM 
Aju 
?? 

1/1/2019 2:49:23 AM 
Aju 
?? Hi Rohi You there? 

1/1/2019 2:51:16 AM 
Rohi
Hello Aju my name is Rohi. How can I help you today? 

1/1/2019 2:51:27 AM"

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| makemv tokenizer="(?ms)(.*?)\s*[\r\n]{2,}\s*" DATA
| rex field=DATA "(?<agent>\S+) is ready to chat\."
| eval DATA=mvindex(DATA, mvfind(DATA, "ready to chat") + 1, -1)
| eval agentFilterRegEx = " [AP]M[\r\n\s]+" . agent . "[\r\n\s]"
| eval agentFirstResponse = mvindex(DATA, mvfind(DATA, agentFilterRegEx))
| rex field=agentFirstResponse "(?ms)^(?<agentFirstResponseTime>[^\r\n]+)[\r\n]+[^\r\n]+[\r\n]+(?<agentFirstResponse>.*)$"
| eval agentFirstResponseTime = strptime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| fieldformat agentFirstResponseTime = strftime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
0 Karma

FrankVl
Ultra Champion

Guess something like this is the only way, huh. Shame Splunk doesn't support using (?P=name) to match a named subpattern. That way you could have done it with a single regex: https://regex101.com/r/d8F6SN/1/

Edit: oh, wait, that is supported: | rex "(?s)System Message: (?<agent>\w+) is ready to chat.*?[\r\n]+[\d\/]+\s+[\d:]+\s+\w+\s+[\r\n]+(?P=agent)[\r\n]+(?<agentFirstResponse>.*?)[\r\n]+[\d\/]+\s+[\d:]+\s+\w+"

Or see the answer from @jnudell_2 for a similar solution using \1 instead of (?P=agent).

0 Karma

rajaguru2790
Explorer

Now working. One change in the log. Message for System message is: Rohi is online and Ready to chat. It its not ready to chat.

This is the one whcih I executed.

index="use_case_one"
| makemv tokenizer="(?ms)(.?)\s[\r\n]{2,}\s*" DATA
| rex field=DATA "(?\S+) is online and ready to chat."
| eval DATA=mvindex(DATA, mvfind(DATA, "online and ready to chat") + 1, -1)
| eval agentFilterRegEx = " [AP]M[\r\n\s]+" . agent . "[\r\n\s]"
| eval agentFirstResponse = mvindex(DATA, mvfind(DATA, agentFilterRegEx))
| rex field=agentFirstResponse "(?ms)^(?[^\r\n]+)[\r\n]+[^\r\n]+[\r\n]+(?.*)$"
| eval agentFirstResponseTime = strptime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| fieldformat agentFirstResponseTime = strftime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| table "Session Log" "DATA"

0 Karma

rajaguru2790
Explorer

Hi . Thanks for teh help. Like this there are 10000 chats with different names . How can I take this generically. Please assist

0 Karma

woodcock
Esteemed Legend

This should work as-is, so long as the field with your message is called DATA. If not, just change the field name everywhere. Of course, you throw out the stuff before the COMMENT.

0 Karma

jnudell_2
Builder

Hi @rajaguru2790 ,

You can use the following:

| rex "^(?<chat_start_time>[^\r\n]+)[\r\n]+[\S\s]+System\s+Message:\s+(?<agent>\S+) is ready to chat[\S\s]+[\r\n]+(?<initial_response_time>[^\r\n]+)[\r\n]+\2[\r\n]+(?<initial_response>[^\r\n]+)[\r\n]+" 
| eval chat_start_time = strptime(chat_start_time, "%m/%d/%Y %I:%M:%S %p")
| eval initial_response_time = strptime(initial_response_time, "%m/%d/%Y %I:%M:%S %p")
| eval agent_response_time = tostring(initial_response_time - chat_start_time, "duration")

This will look something like this:
alt text

0 Karma

rajaguru2790
Explorer

Error in 'rex' command: regex="System Message: \w+ is ready to chat.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P.+)$" has exceeded configured match_limit, consider raising the value in limits.conf

I chnaged the limits.conf in local like below. But didnot work

[rex]
match_limit = 0
depth_limit = 0

0 Karma

FrankVl
Ultra Champion

That is not the same regex that jnudell_2 suggested. Did you actually try the correct regex? I've tested this as well and it seems to work fine.

| makeresults | eval _raw="1/1/2019 2:42:55 AM 
Aju 
Hi Team

1/1/2019 2:42:56 AM 
System 
The data has been added: 
- Customer Info

1/1/2019 2:42:59 AM 
Rohi
System Message: Rohi is ready to chat. 

1/1/2019 2:43:09 AM 
Aju 
Wish you a very happy ne year 

1/1/2019 2:43:12 AM 
Aju 
new* 

1/1/2019 2:43:25 AM 
Aju 
I need to KNOW ABOUT A CAR

1/1/2019 2:43:32 AM 
Aju 
please help me 

1/1/2019 2:45:07 AM 
Aju 
Anyone there ? 

1/1/2019 2:47:13 AM 
Aju 
?? 

1/1/2019 2:49:23 AM 
Aju 
?? Hi Rohi You there? 

1/1/2019 2:51:16 AM 
Rohi
Hello Aju my name is Rohi. How can I help you today? 

1/1/2019 2:51:27 AM

Chat goes on...."
 | rex "^(?<chat_start_time>[^\r\n]+)[\r\n]+[\S\s]+System\s+Message:\s+(?<agent>\S+) is ready to chat[\S\s]+[\r\n]+(?<initial_response_time>[^\r\n]+)[\r\n]+\2[\r\n]+(?<initial_response>[^\r\n]+)[\r\n]+"

Alternatively, this rex also works: | rex "System Message: (?<agent>\w+) is ready to chat.*?[\r\n]+[\d\/]+\s+[\d:]+\s+\w+\s+[\r\n]+(?P=agent)[\r\n]+(?<agentFirstResponse>.*?)[\r\n]+[\d\/]+\s+[\d:]+\s+\w+"

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi rajaguru2790,
Try something like this

| rex "System Message: \w+ is ready to chat\.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P<my_field>.+)$"

You can test it at https://regex101.com/r/fjaU3e/1

Bye.
Giuseppe

0 Karma

rajaguru2790
Explorer

Thanks for the help. I am getting this error.

Error in 'rex' command: regex="System Message: \w+ is ready to chat.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P.+)$" has exceeded configured match_limit, consider raising the value in limits.conf

I chnaged the limits.conf in local like below. But didnot work

[rex]
match_limit = 0
depth_limit = 0

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...