A field has multiple lines like a chat log. Below aLL DATA IS in one field of Splunk in the same way as below. Now need to match agent's initial response and capture in a separate field. Aju is the user and Rohi is the agent. Whenever chat is assigned to agent system messgae generated(In this example 1/1/2019 2:42:59 AM Rohi system message: ready to chat). Need to match the next line of agent response time after this agent system message "Rohi system message: ready to chat" in the entire log . That is called initial reponse for this chat (In this example it's timestamp/transcript is (1/1/2019 2:51:16 AM Rohi Hello Aju my name is Rohi. How can I help you today?) )Please help me on calculating Initial reponse time to a separate field using REGEX or someother way. Thanks
1/1/2019 2:42:55 AM
Aju
1/1/2019 2:42:56 AM
System
The data has been added:
1/1/2019 2:42:59 AM
Rohi
1/1/2019 2:43:09 AM
Aju
1/1/2019 2:43:12 AM
Aju
1/1/2019 2:43:25 AM
Aju
1/1/2019 2:43:32 AM
Aju
1/1/2019 2:45:07 AM
Aju
1/1/2019 2:47:13 AM
Aju
1/1/2019 2:49:23 AM
Aju
1/1/2019 2:51:16 AM
Rohi
1/1/2019 2:51:27 AM
Chat goes on....
Like this:
| makeresults | eval DATA="1/1/2019 2:42:55 AM
Aju
Hi Team
1/1/2019 2:42:56 AM
System
The data has been added:
- Customer Info
1/1/2019 2:42:59 AM
Rohi
System Message: Rohi is ready to chat.
1/1/2019 2:43:09 AM
Aju
Wish you a very happy ne year
1/1/2019 2:43:12 AM
Aju
new*
1/1/2019 2:43:25 AM
Aju
I need to KNOW ABOUT A CAR
1/1/2019 2:43:32 AM
Aju
please help me
1/1/2019 2:45:07 AM
Aju
Anyone there ?
1/1/2019 2:47:13 AM
Aju
??
1/1/2019 2:49:23 AM
Aju
?? Hi Rohi You there?
1/1/2019 2:51:16 AM
Rohi
Hello Aju my name is Rohi. How can I help you today?
1/1/2019 2:51:27 AM"
| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"
| makemv tokenizer="(?ms)(.*?)\s*[\r\n]{2,}\s*" DATA
| rex field=DATA "(?<agent>\S+) is ready to chat\."
| eval DATA=mvindex(DATA, mvfind(DATA, "ready to chat") + 1, -1)
| eval agentFilterRegEx = " [AP]M[\r\n\s]+" . agent . "[\r\n\s]"
| eval agentFirstResponse = mvindex(DATA, mvfind(DATA, agentFilterRegEx))
| rex field=agentFirstResponse "(?ms)^(?<agentFirstResponseTime>[^\r\n]+)[\r\n]+[^\r\n]+[\r\n]+(?<agentFirstResponse>.*)$"
| eval agentFirstResponseTime = strptime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| fieldformat agentFirstResponseTime = strftime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
Guess something like this is the only way, huh. Shame Splunk doesn't support using (?P=name)
to match a named subpattern. That way you could have done it with a single regex: https://regex101.com/r/d8F6SN/1/
Edit: oh, wait, that is supported: | rex "(?s)System Message: (?<agent>\w+) is ready to chat.*?[\r\n]+[\d\/]+\s+[\d:]+\s+\w+\s+[\r\n]+(?P=agent)[\r\n]+(?<agentFirstResponse>.*?)[\r\n]+[\d\/]+\s+[\d:]+\s+\w+"
Or see the answer from @jnudell_2 for a similar solution using \1
instead of (?P=agent)
.
Now working. One change in the log. Message for System message is: Rohi is online and Ready to chat. It its not ready to chat.
This is the one whcih I executed.
index="use_case_one"
| makemv tokenizer="(?ms)(.?)\s[\r\n]{2,}\s*" DATA
| rex field=DATA "(?\S+) is online and ready to chat."
| eval DATA=mvindex(DATA, mvfind(DATA, "online and ready to chat") + 1, -1)
| eval agentFilterRegEx = " [AP]M[\r\n\s]+" . agent . "[\r\n\s]"
| eval agentFirstResponse = mvindex(DATA, mvfind(DATA, agentFilterRegEx))
| rex field=agentFirstResponse "(?ms)^(?[^\r\n]+)[\r\n]+[^\r\n]+[\r\n]+(?.*)$"
| eval agentFirstResponseTime = strptime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| fieldformat agentFirstResponseTime = strftime(agentFirstResponseTime, "%m/%d/%Y %H:%M:%S %p")
| table "Session Log" "DATA"
Hi . Thanks for teh help. Like this there are 10000 chats with different names . How can I take this generically. Please assist
This should work as-is, so long as the field with your message is called DATA
. If not, just change the field name everywhere. Of course, you throw out the stuff before the COMMENT
.
Hi @rajaguru2790 ,
You can use the following:
| rex "^(?<chat_start_time>[^\r\n]+)[\r\n]+[\S\s]+System\s+Message:\s+(?<agent>\S+) is ready to chat[\S\s]+[\r\n]+(?<initial_response_time>[^\r\n]+)[\r\n]+\2[\r\n]+(?<initial_response>[^\r\n]+)[\r\n]+"
| eval chat_start_time = strptime(chat_start_time, "%m/%d/%Y %I:%M:%S %p")
| eval initial_response_time = strptime(initial_response_time, "%m/%d/%Y %I:%M:%S %p")
| eval agent_response_time = tostring(initial_response_time - chat_start_time, "duration")
This will look something like this:
Error in 'rex' command: regex="System Message: \w+ is ready to chat.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P.+)$" has exceeded configured match_limit, consider raising the value in limits.conf
I chnaged the limits.conf in local like below. But didnot work
[rex]
match_limit = 0
depth_limit = 0
That is not the same regex that jnudell_2 suggested. Did you actually try the correct regex? I've tested this as well and it seems to work fine.
| makeresults | eval _raw="1/1/2019 2:42:55 AM
Aju
Hi Team
1/1/2019 2:42:56 AM
System
The data has been added:
- Customer Info
1/1/2019 2:42:59 AM
Rohi
System Message: Rohi is ready to chat.
1/1/2019 2:43:09 AM
Aju
Wish you a very happy ne year
1/1/2019 2:43:12 AM
Aju
new*
1/1/2019 2:43:25 AM
Aju
I need to KNOW ABOUT A CAR
1/1/2019 2:43:32 AM
Aju
please help me
1/1/2019 2:45:07 AM
Aju
Anyone there ?
1/1/2019 2:47:13 AM
Aju
??
1/1/2019 2:49:23 AM
Aju
?? Hi Rohi You there?
1/1/2019 2:51:16 AM
Rohi
Hello Aju my name is Rohi. How can I help you today?
1/1/2019 2:51:27 AM
Chat goes on...."
| rex "^(?<chat_start_time>[^\r\n]+)[\r\n]+[\S\s]+System\s+Message:\s+(?<agent>\S+) is ready to chat[\S\s]+[\r\n]+(?<initial_response_time>[^\r\n]+)[\r\n]+\2[\r\n]+(?<initial_response>[^\r\n]+)[\r\n]+"
Alternatively, this rex also works: | rex "System Message: (?<agent>\w+) is ready to chat.*?[\r\n]+[\d\/]+\s+[\d:]+\s+\w+\s+[\r\n]+(?P=agent)[\r\n]+(?<agentFirstResponse>.*?)[\r\n]+[\d\/]+\s+[\d:]+\s+\w+"
Hi rajaguru2790,
Try something like this
| rex "System Message: \w+ is ready to chat\.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P<my_field>.+)$"
You can test it at https://regex101.com/r/fjaU3e/1
Bye.
Giuseppe
Thanks for the help. I am getting this error.
Error in 'rex' command: regex="System Message: \w+ is ready to chat.\s+\d+\/\d+\/\d+\s\d+:\d+:\d+\s+\w+\s+\w+\s+(?P.+)$" has exceeded configured match_limit, consider raising the value in limits.conf
I chnaged the limits.conf in local like below. But didnot work
[rex]
match_limit = 0
depth_limit = 0