Splunk Search
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question

Need to match the right Regex

rajaguru2790
Explorer
‎08-14-2019 04:58 PM

Need your help matching the next line of agent occurence timestamp. Example captured in link below

link text

Below is the entire log which needs to be parsed. Rohi is the agent and Saj is the user. Using system message we can find the agent . Then we need to match the next line of agent in the log.Here Rohi(agent) second line of message in the log and return that line TIME in separate field.
Step1: Match the agent line using the System message "online for chatting" and return the agent name in agent field. Rohi is the agent in this log and below line is parsed and Rohi needs to be captured for this event. Like this there are many events
Step2: Match the next line of response from agent(Rohi) by Parsing the entire log. This line is called initial response as before line is the system assigning chat automatically to Rohi. This line timestamp 1/1/2019 2:42:55 AM has to be captured in the separate field Initial Response

Right Timestamp needed:
Now my regex (?s)(?\w+) is online for chatting.*!TIME!(?[^!]+)!\/TIME!.!NAME!\k!\/NAME!.!TEXT!(?!System Message) matches the timestamp which is capturing is 1/1/2019 2:53:47 AM. But it should be the response which comes immediately after "online for chatting line" and correct timestamp to be captured is 1/1/2019 2:51:16 AM as mentioned in Right Timestamp step below

 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:16 AM!_/TIME_!
 !_NAME_!Rohi!_/NAME_!
 !_TEXT_!<translateitem>Hello Saj my name is Rohi. How can I help you today?</translateitem>!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!12!_/MID_!!_UTCEPOCHTIME_!1546329076000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------

Wrong Timestamp getting Captured in my current Regex:

 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:53:47 AM!_/TIME_!
 !_NAME_!Rohi!_/NAME_!
 !_TEXT_!<translateitem>Hello Saj my name is Rohi. How can I help you today?</translateitem>!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!14!_/MID_!!_UTCEPOCHTIME_!1546329227000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
Tags (1)
0 Karma
Reply

apcsplunk
Explorer
‎08-14-2019 10:40 PM

Though I didn't understand your question completely, what i think is your regex seems to be good. You can achieve what you are looking for with a little different approach.

Try using this search for your dashboard/report/alert etc, "stats first(AgentInitialResponseTime) by Agent"
Alternatively try "stats last(AgentInitialResponseTime) by Agent" to see if you need the opposite results.

0 Karma
Reply

nareshinsvu
Builder
‎08-14-2019 10:06 PM

Can you try this?

Logs look bit wierd. My assumption here is that a new message always start with MID=1 and ends with "%To start a new chat click on%" . Below solution is based on the same assumption.

here I have dumped all your lines into an index called dummy.

If you know exact lines that can be indexed, you can use heavy forwarder and send only specific lines to your indexer. 

index="dummy" |
rex field=_raw "TIME\_\!(?<Time>.*?)\s*\!\_\/TIME" |
rex field=_raw "TEXT\_\!(?<Message>.*?)\s*\!\_\/TEXT" |
rex field=_raw "MID\_\!(?<MID>.*?)\s*\!\_\/MID" |
 reverse |
 dedup Message|
 rex field=Message "\:\s+(?<Agent>.*?)\s*\s+is\s+online" |
 rex field=Message "Hello\s+(?<User>.*?)\s*\s+my\s+name"|
eval Initial_Response=if(MID=1,Time,"")|
eval Agent_Online_Time=if(Agent!="",Time,"") |
eval Closing_Time=if(like(Message,"%To start a new chat click on%"),Time,"")|
reverse|
streamstats current=f window=1 last(MID) as prev_MID |
eval id=1 |
eval id=if((prev_MID=1),(id+1),id) |
transaction id|
 table Agent User Initial_Response Agent_Online_Time Closing_Time
0 Karma
Reply

nareshinsvu
Builder
‎08-14-2019 10:09 PM

If this solution helped you, kindly accept and/or upvote to help others with similar request.

0 Karma
Reply

rajaguru2790
Explorer
‎08-15-2019 12:04 AM

Hi Naresh, It is not working and 5 events are coming up. Can you please help me with Regex to match the second occuring agent line and return the timestamp. If I give k now it is taking the last occuring agent line and returning that line's timestamp.

Current Regex matching Agent last response line occurence: 

      index=main | rex field="Session Log" "(?s)(?<agent>\w+) is online for chatting.*!_TIME_!(?<time>[^!]+)!_\/TIME_!.!_NAME_!\k<agent>!_\/NAME_!.!_TEXT_!(?!System Message)" | table agent time
0 Karma
Reply

nareshinsvu
Builder
‎08-15-2019 12:19 AM

Can you tell me what timestamp are you after in your example logs? 1/1/2019 2:55:12 AM ? or 1/1/2019 2:42:55 AM?
Did you run the code which I provided? below is the output from my snippet

Agent   User    Initial_Response            Agent_Online_Time             Closing_Time
Rohi    Saj      1/1/2019 2:42:55 AM      1/1/2019 2:42:59 AM         1/1/2019 2:55:12 AM
0 Karma
Reply

rajaguru2790
Explorer
‎08-14-2019 05:01 PM

Regex 101 link

link text

https://regex101.com/r/IHmHqx/1

0 Karma
Reply

rajaguru2790
Explorer
‎08-14-2019 05:00 PM

Entire Log

 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:55 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!Hi Team!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!1!_/MID_!!_UTCEPOCHTIME_!1546328575000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:56 AM!_/TIME_!
 !_NAME_!System!_/NAME_!
 !_TEXT_!The following associated data has been added:<ul><li>Customer Information</li></ul>!_SM+msg_DataAdded+Customer InformationSM_!!_/TEXT_!!_NAMEID_!system@email.com!_/NAMEID_!!_MID_!3!_/MID_!!_UTCEPOCHTIME_!1546328576000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:42:59 AM!_/TIME_!
 !_NAME_!Rohi!_/NAME_!
 !_TEXT_!System Message: Rohi is online for chatting.!_SM+msg_AgentOnline+RohiSM_!!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!4!_/MID_!!_UTCEPOCHTIME_!1546328579000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:09 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!Wish you a very happy ne year!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!5!_/MID_!!_UTCEPOCHTIME_!1546328589000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:12 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!new*!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!6!_/MID_!!_UTCEPOCHTIME_!1546328592000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:25 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!I need to close this ticket 10936307!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!7!_/MID_!!_UTCEPOCHTIME_!1546328605000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:43:32 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!please help me in closing the same!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!8!_/MID_!!_UTCEPOCHTIME_!1546328612000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:45:07 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!Anyone there ?!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!9!_/MID_!!_UTCEPOCHTIME_!1546328719000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:47:13 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!??!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!10!_/MID_!!_UTCEPOCHTIME_!1546328833000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:49:23 AM!_/TIME_!
 !_NAME_!Saj!_/NAME_!
 !_TEXT_!?? Hi Rohi You there?!_/TEXT_!!_NAMEID_!sajg6@test.com!_/NAMEID_!!_MID_!11!_/MID_!!_UTCEPOCHTIME_!1546328963000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:16 AM!_/TIME_!
 !_NAME_!Rohi!_/NAME_!
 !_TEXT_!Hello Saj my name is Rohi. How can I help you today?!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!12!_/MID_!!_UTCEPOCHTIME_!1546329076000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:51:27 AM!_/TIME_!
 !_NAME_!Rohi!_/NAME_!
 !_TEXT_!Yes!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!13!_/MID_!!_UTCEPOCHTIME_!1546329087000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:53:47 AM!_/TIME_!
 !_NAME_!Rohi!_/NAME_!
 !_TEXT_!Hello Saj my name is Rohi. How can I help you today?!_/TEXT_!!_NAMEID_!rohi@test.com!_/NAMEID_!!_MID_!14!_/MID_!!_UTCEPOCHTIME_!1546329227000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:54:38 AM!_/TIME_!
 !_NAME_!System!_/NAME_!
 !_TEXT_!System Message: Saj G has closed the browser!_SM+msg_hasClosed+Saj GSM_!!_/TEXT_!!_NAMEID_!system@email.com!_/NAMEID_!!_MID_!15!_/MID_!!_UTCEPOCHTIME_!1546329278000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
 !_CI_!!_L_!en!_/LO_!!_TIME_!1/1/2019 2:55:12 AM!_/TIME_!
 !_NAME_!System!_/NAME_!
 !_TEXT_!System Message: rohi has closed and abandoned. To start a new chat click on &quot;Chat now&quot;.!_SM+msg_UserAbandoned+rohiSM_!!_/TEXT_!!_NAMEID_!system@email.com!_/NAMEID_!!_MID_!16!_/MID_!!_UTCEPOCHTIME_!1546329312000!_/UTCEPOCHTIME_!!_/CINST_!
 --------------------------------------------------------------------------------------
0 Karma
Reply
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...