Need your help matching the next line of agent occurence timestamp. Example captured in link below
Below is the entire log which needs to be parsed. Rohi is the agent and Saj is the user. Using system message we can find the agent . Then we need to match the next line of agent in the log.Here Rohi(agent) second line of message in the log and return that line TIME in separate field.
Step1: Match the agent line using the System message "online for chatting" and return the agent name in agent field. Rohi is the agent in this log and below line is parsed and Rohi needs to be captured for this event. Like this there are many events
Step2: Match the next line of response from agent(Rohi) by Parsing the entire log. This line is called initial response as before line is the system assigning chat automatically to Rohi. This line timestamp 1/1/2019 2:42:55 AM has to be captured in the separate field Initial Response
Right Timestamp needed:
Now my regex (?s)(?\w+) is online for chatting.*!TIME!(?[^!]+)!\/TIME!.!NAME!\k!\/NAME!.!TEXT!(?!System Message) matches the timestamp which is capturing is 1/1/2019 2:53:47 AM. But it should be the response which comes immediately after "online for chatting line" and correct timestamp to be captured is 1/1/2019 2:51:16 AM as mentioned in Right Timestamp step below
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:16 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!<translateitem>Hello Saj my name is Rohi. How can I help you today?</translateitem>!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!12!_/MID_!!_UTCEPOCHTIME_!1546329076000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
Wrong Timestamp getting Captured in my current Regex:
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:53:47 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!<translateitem>Hello Saj my name is Rohi. How can I help you today?</translateitem>!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!14!_/MID_!!_UTCEPOCHTIME_!1546329227000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
Though I didn't understand your question completely, what i think is your regex seems to be good. You can achieve what you are looking for with a little different approach.
Try using this search for your dashboard/report/alert etc, "stats first(AgentInitialResponseTime) by Agent"
Alternatively try "stats last(AgentInitialResponseTime) by Agent" to see if you need the opposite results.
Can you try this?
Logs look bit wierd. My assumption here is that a new message always start with MID=1 and ends with "%To start a new chat click on%" . Below solution is based on the same assumption.
here I have dumped all your lines into an index called dummy.
If you know exact lines that can be indexed, you can use heavy forwarder and send only specific lines to your indexer.
index="dummy" |
rex field=_raw "TIME\_\!(?<Time>.*?)\s*\!\_\/TIME" |
rex field=_raw "TEXT\_\!(?<Message>.*?)\s*\!\_\/TEXT" |
rex field=_raw "MID\_\!(?<MID>.*?)\s*\!\_\/MID" |
reverse |
dedup Message|
rex field=Message "\:\s+(?<Agent>.*?)\s*\s+is\s+online" |
rex field=Message "Hello\s+(?<User>.*?)\s*\s+my\s+name"|
eval Initial_Response=if(MID=1,Time,"")|
eval Agent_Online_Time=if(Agent!="",Time,"") |
eval Closing_Time=if(like(Message,"%To start a new chat click on%"),Time,"")|
reverse|
streamstats current=f window=1 last(MID) as prev_MID |
eval id=1 |
eval id=if((prev_MID=1),(id+1),id) |
transaction id|
table Agent User Initial_Response Agent_Online_Time Closing_Time
If this solution helped you, kindly accept and/or upvote to help others with similar request.
Hi Naresh, It is not working and 5 events are coming up. Can you please help me with Regex to match the second occuring agent line and return the timestamp. If I give k now it is taking the last occuring agent line and returning that line's timestamp.
Current Regex matching Agent last response line occurence:
index=main | rex field="Session Log" "(?s)(?<agent>\w+) is online for chatting.*!_TIME_!(?<time>[^!]+)!_\/TIME_!.!_NAME_!\k<agent>!_\/NAME_!.!_TEXT_!(?!System Message)" | table agent time
Can you tell me what timestamp are you after in your example logs? 1/1/2019 2:55:12 AM ? or 1/1/2019 2:42:55 AM?
Did you run the code which I provided? below is the output from my snippet
Agent User Initial_Response Agent_Online_Time Closing_Time
Rohi Saj 1/1/2019 2:42:55 AM 1/1/2019 2:42:59 AM 1/1/2019 2:55:12 AM
Entire Log
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:55 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!Hi Team!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!1!_/MID_!!_UTCEPOCHTIME_!1546328575000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:56 AM!_/TIME_!
!_NAME_!System!_/NAME_!
!_TEXT_!The following associated data has been added:<ul><li>Customer Information</li></ul>!_SM+msg_DataAdded+Customer InformationSM_!!_/TEXT_!!_NAMEID_!system@email.com!_/NAMEID_!!_MID_!3!_/MID_!!_UTCEPOCHTIME_!1546328576000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:59 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!System Message: Rohi is online for chatting.!_SM+msg_AgentOnline+RohiSM_!!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!4!_/MID_!!_UTCEPOCHTIME_!1546328579000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:09 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!Wish you a very happy ne year!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!5!_/MID_!!_UTCEPOCHTIME_!1546328589000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:12 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!new*!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!6!_/MID_!!_UTCEPOCHTIME_!1546328592000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:25 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!I need to close this ticket 10936307!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!7!_/MID_!!_UTCEPOCHTIME_!1546328605000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:32 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!please help me in closing the same!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!8!_/MID_!!_UTCEPOCHTIME_!1546328612000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:45:07 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!Anyone there ?!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!9!_/MID_!!_UTCEPOCHTIME_!1546328719000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:47:13 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!??!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!10!_/MID_!!_UTCEPOCHTIME_!1546328833000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:49:23 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!?? Hi Rohi You there?!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!11!_/MID_!!_UTCEPOCHTIME_!1546328963000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:16 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!Hello Saj my name is Rohi. How can I help you today?!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!12!_/MID_!!_UTCEPOCHTIME_!1546329076000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:27 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!Yes!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!13!_/MID_!!_UTCEPOCHTIME_!1546329087000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:53:47 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!Hello Saj my name is Rohi. How can I help you today?!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!14!_/MID_!!_UTCEPOCHTIME_!1546329227000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:54:38 AM!_/TIME_!
!_NAME_!System!_/NAME_!
!_TEXT_!System Message: Saj G has closed the browser!_SM+msg_hasClosed+Saj GSM_!!_/TEXT_!!_NAMEID_!system@email.com!_/NAMEID_!!_MID_!15!_/MID_!!_UTCEPOCHTIME_!1546329278000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:55:12 AM!_/TIME_!
!_NAME_!System!_/NAME_!
!_TEXT_!System Message: rohi has closed and abandoned. To start a new chat click on "Chat now".!_SM+msg_UserAbandoned+rohiSM_!!_/TEXT_!!_NAMEID_!system@email.com!_/NAMEID_!!_MID_!16!_/MID_!!_UTCEPOCHTIME_!1546329312000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------