Join the Conversation
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: Need to match the right Regex
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Need to match the right Regex
Need your help matching the next line of agent occurence timestamp. Example captured in link below
Below is the entire log which needs to be parsed. Rohi is the agent and Saj is the user. Using system message we can find the agent . Then we need to match the next line of agent in the log.Here Rohi(agent) second line of message in the log and return that line TIME in separate field.
Step1: Match the agent line using the System message "online for chatting" and return the agent name in agent field. Rohi is the agent in this log and below line is parsed and Rohi needs to be captured for this event. Like this there are many events
Step2: Match the next line of response from agent(Rohi) by Parsing the entire log. This line is called initial response as before line is the system assigning chat automatically to Rohi. This line timestamp 1/1/2019 2:42:55 AM has to be captured in the separate field Initial Response
Right Timestamp needed:
Now my regex (?s)(?\w+) is online for chatting.*!TIME!(?[^!]+)!\/TIME!.!NAME!\k!\/NAME!.!TEXT!(?!System Message) matches the timestamp which is capturing is 1/1/2019 2:53:47 AM. But it should be the response which comes immediately after "online for chatting line" and correct timestamp to be captured is 1/1/2019 2:51:16 AM as mentioned in Right Timestamp step below
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:16 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!<translateitem>Hello Saj my name is Rohi. How can I help you today?</translateitem>!_/[email protected]!_/NAMEID_!!_MID_!12!_/MID_!!_UTCEPOCHTIME_!1546329076000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
Wrong Timestamp getting Captured in my current Regex:
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:53:47 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!<translateitem>Hello Saj my name is Rohi. How can I help you today?</translateitem>!_/[email protected]!_/NAMEID_!!_MID_!14!_/MID_!!_UTCEPOCHTIME_!1546329227000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Though I didn't understand your question completely, what i think is your regex seems to be good. You can achieve what you are looking for with a little different approach.
Try using this search for your dashboard/report/alert etc, "stats first(AgentInitialResponseTime) by Agent"
Alternatively try "stats last(AgentInitialResponseTime) by Agent" to see if you need the opposite results.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you try this?
Logs look bit wierd. My assumption here is that a new message always start with MID=1 and ends with "%To start a new chat click on%" . Below solution is based on the same assumption.
here I have dumped all your lines into an index called dummy.
If you know exact lines that can be indexed, you can use heavy forwarder and send only specific lines to your indexer.
index="dummy" |
rex field=_raw "TIME\_\!(?<Time>.*?)\s*\!\_\/TIME" |
rex field=_raw "TEXT\_\!(?<Message>.*?)\s*\!\_\/TEXT" |
rex field=_raw "MID\_\!(?<MID>.*?)\s*\!\_\/MID" |
reverse |
dedup Message|
rex field=Message "\:\s+(?<Agent>.*?)\s*\s+is\s+online" |
rex field=Message "Hello\s+(?<User>.*?)\s*\s+my\s+name"|
eval Initial_Response=if(MID=1,Time,"")|
eval Agent_Online_Time=if(Agent!="",Time,"") |
eval Closing_Time=if(like(Message,"%To start a new chat click on%"),Time,"")|
reverse|
streamstats current=f window=1 last(MID) as prev_MID |
eval id=1 |
eval id=if((prev_MID=1),(id+1),id) |
transaction id|
table Agent User Initial_Response Agent_Online_Time Closing_Time
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If this solution helped you, kindly accept and/or upvote to help others with similar request.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Naresh, It is not working and 5 events are coming up. Can you please help me with Regex to match the second occuring agent line and return the timestamp. If I give k now it is taking the last occuring agent line and returning that line's timestamp.
Current Regex matching Agent last response line occurence:
index=main | rex field="Session Log" "(?s)(?<agent>\w+) is online for chatting.*!_TIME_!(?<time>[^!]+)!_\/TIME_!.!_NAME_!\k<agent>!_\/NAME_!.!_TEXT_!(?!System Message)" | table agent time
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you tell me what timestamp are you after in your example logs? 1/1/2019 2:55:12 AM ? or 1/1/2019 2:42:55 AM?
Did you run the code which I provided? below is the output from my snippet
Agent User Initial_Response Agent_Online_Time Closing_Time
Rohi Saj 1/1/2019 2:42:55 AM 1/1/2019 2:42:59 AM 1/1/2019 2:55:12 AM
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Entire Log
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:55 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!Hi Team!_/[email protected]!_/NAMEID_!!_MID_!1!_/MID_!!_UTCEPOCHTIME_!1546328575000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:56 AM!_/TIME_!
!_NAME_!System!_/NAME_!
!_TEXT_!The following associated data has been added:<ul><li>Customer Information</li></ul>!_SM+msg_DataAdded+Customer InformationSM_!!_/[email protected]!_/NAMEID_!!_MID_!3!_/MID_!!_UTCEPOCHTIME_!1546328576000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:59 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!System Message: Rohi is online for chatting.!_SM+msg_AgentOnline+RohiSM_!!_/[email protected]!_/NAMEID_!!_MID_!4!_/MID_!!_UTCEPOCHTIME_!1546328579000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:09 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!Wish you a very happy ne year!_/[email protected]!_/NAMEID_!!_MID_!5!_/MID_!!_UTCEPOCHTIME_!1546328589000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:12 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!new*!_/[email protected]!_/NAMEID_!!_MID_!6!_/MID_!!_UTCEPOCHTIME_!1546328592000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:25 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!I need to close this ticket 10936307!_/[email protected]!_/NAMEID_!!_MID_!7!_/MID_!!_UTCEPOCHTIME_!1546328605000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:32 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!please help me in closing the same!_/[email protected]!_/NAMEID_!!_MID_!8!_/MID_!!_UTCEPOCHTIME_!1546328612000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:45:07 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!Anyone there ?!_/[email protected]!_/NAMEID_!!_MID_!9!_/MID_!!_UTCEPOCHTIME_!1546328719000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:47:13 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!??!_/[email protected]!_/NAMEID_!!_MID_!10!_/MID_!!_UTCEPOCHTIME_!1546328833000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:49:23 AM!_/TIME_!
!_NAME_!Saj!_/NAME_!
!_TEXT_!?? Hi Rohi You there?!_/[email protected]!_/NAMEID_!!_MID_!11!_/MID_!!_UTCEPOCHTIME_!1546328963000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:16 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!Hello Saj my name is Rohi. How can I help you today?!_/[email protected]!_/NAMEID_!!_MID_!12!_/MID_!!_UTCEPOCHTIME_!1546329076000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:27 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!Yes!_/[email protected]!_/NAMEID_!!_MID_!13!_/MID_!!_UTCEPOCHTIME_!1546329087000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:53:47 AM!_/TIME_!
!_NAME_!Rohi!_/NAME_!
!_TEXT_!Hello Saj my name is Rohi. How can I help you today?!_/[email protected]!_/NAMEID_!!_MID_!14!_/MID_!!_UTCEPOCHTIME_!1546329227000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:54:38 AM!_/TIME_!
!_NAME_!System!_/NAME_!
!_TEXT_!System Message: Saj G has closed the browser!_SM+msg_hasClosed+Saj GSM_!!_/[email protected]!_/NAMEID_!!_MID_!15!_/MID_!!_UTCEPOCHTIME_!1546329278000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
!_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:55:12 AM!_/TIME_!
!_NAME_!System!_/NAME_!
!_TEXT_!System Message: rohi has closed and abandoned. To start a new chat click on "Chat now".!_SM+msg_UserAbandoned+rohiSM_!!_/[email protected]!_/NAMEID_!!_MID_!16!_/MID_!!_UTCEPOCHTIME_!1546329312000!_/UTCEPOCHTIME_!!_/CINST_!
--------------------------------------------------------------------------------------
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
A Four-Part Event Series: Full Stack Observability For the AI Era
SOC4Kafka - New Kafka Connector Powered by OpenTelemetry
Event Series: Level up your SOC: Advancing with Splunk Enterprise Security
