I am trying extract "user20" from rest of "_9a4ab75c_239_process.log".  tried multiple ways but unable to separate the user name from underscore.   /app_5/appname/appanem2/logs/agent-machine4/sre/query/prot/user20_tt65rft67uu87_process.log" ... View more

Currently I am using below query to extract the list of employee_ID column has less then 9 digit employee ID.  However, I have another requirement in same table to extract the employee ID with alphanumeric- (like N0001234, etc) and any special characters.  So overall we need data which is less than 9 digits, more than 9 digits, any alphanumeric characters, special characters. index=QQQQQ sourcetype="XXXXX*" source=TTTTTT Extension="*" MSID="*" Employee_Active="*" Employee_Id=* last_name="*" first_name="*"| rename Extension as DN| dedup Employee_Id | eval emplength=len(Employee_Id)| stats count by DN, MSID, Employee_Active, emplength,Employee_Id, last_name, first_name| where emplength>9 ] | table DN, MSID, Employee_Active, emplength,Employee_Id, last_name, first_name ... View more

EventAgentLogin ==================   2022-12-14 06:39:03.875 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1' received message ''EventAgentLogin' (73) attributes: AttributeReasons [bstr] = KVList:               'referrer' [str] = "https://aaa.test.com"               'clientApp' [str] = "asdfsfdf"               'location' [str] = "server" AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0077MP"               'geo-location-agent' [str] = "CTC" AttributeAgentWorkMode [int] = 0 [Unknown] AttributeEventSequenceNumber [long] = 744434548 TimeStamp:               AttributeTimeinSecs [int] = 1671021543               AttributeTimeinuSecs [int] = 882837'   -------------------------------------------------------------------------------------   agent ready state" =======================   2022-12-14 07:59:12.764 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1' received message ''EventAgentReady' (75) attributes: AttributeReasons [bstr] = KVList:               'site' [str] = "CTC" AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0078C6"               'geo-location-agent' [str] = "CTC" AttributeAgentWorkMode [int] = 0 [Unknown] AttributeEventSequenceNumber [long] = 744780812 TimeStamp:               AttributeTimeinSecs [int] = 1671026352               AttributeTimeinuSecs [int] = 766178'   -------------------------------------------------------- EventAgentNotReady ================== 2022-12-14 08:01:31.602 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1' received message ''EventAgentNotReady' (76) attributes: AttributeReasons [bstr] = KVList: AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0078C6"               'geo-location-agent' [str] = "CTC" AttributeAgentWorkMode [int] = 0 [Unknown] AttributeEventSequenceNumber [long] = 744808316 TimeStamp: ---------------------------------------------------------- Training ==========   2022-12-14 08:02:47.211 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1' received message ''EventAgentNotReady' (76) attributes: AttributeReasons [bstr] = KVList:               'ReasonCode' [str] = "Training" AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0078C6"               'geo-location-agent' [str] = "CTC" AttributeAgentWorkMode [int] = 0 [Unknown] AttributeEventSequenceNumber [long] = 744821504 TimeStamp:               AttributeTimeinSecs [int] = 1671026567               AttributeTimeinuSecs [int] = 209306'   ------------------------------------------------------------------------- "EventAgentNotReady" AND "Break" ====================================   2022-12-14 08:04:34.025 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1'  received message ''EventAgentNotReady' (76) attributes: AttributeReasons [bstr] = KVList:               'ReasonCode' [str] = "Break" AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0078C6"               'geo-location-agent' [str] = "CTC" AttributeAgentWorkMode [int] = 0 [Unknown] AttributeEventSequenceNumber [long] = 744838251 TimeStamp:               AttributeTimeinSecs [int] = 1671026674               AttributeTimeinuSecs [int] = 24284'   ---------------------------------------------------------------------------------- AfterCallWork ===============   2022-12-14 08:07:31.310 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1'  received message ''EventAgentNotReady' (76) attributes: AttributeReasons [bstr] = KVList: AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0078C6"               'ReasonCode' [str] = "ManualSetACWPeriod"               'WrapUpTime' [str] = "untimed"               'geo-location-agent' [str] = "CTC" AttributeAgentWorkMode [int] = 3 [AfterCallWork] AttributeEventSequenceNumber [long] = 744864731 TimeStamp:               AttributeTimeinSecs [int] = 1671026851               AttributeTimeinuSecs [int] = 319075'   -------------------------------------------------------------------------------------------- EventAgentLogout =====================   2022-12-14 08:10:09.778 TRACE 12632 --- [New I/O client worker #1-6] c.i.e.g.workflows.ServerEventFactory     : Endpoint 'SERVER1'  received message ''EventAgentLogout' (74) attributes: AttributeThisDN [str] = "1990613829" AttributeAgentID [str] = "34434343" AttributeExtensions [bstr] = KVList:               'AgentSessionID' [str] = "027DK7N0IC9H5D7NF885C2LAES0078C6"               'geo-location-agent' [str] = "CTC" AttributeEventSequenceNumber [long] = 744889386 TimeStamp:               AttributeTimeinSecs [int] = 1671027009               AttributeTimeinuSecs [int] = 779569'     I have different event log format but I am trying to extract the common fields (highlighted and underlined fields). Since the  log format is different I am not sure how to extract the values using single rex or regex query. The field I am looking for is: Endpoint = SERVER1 received message = EventAgentLogout AttributeThisDN = 1990613829 AttributeAgentID= 34434343 AttributeAgentWorkMode = AfterCallWork ReasonCode = Break   There are few fields like "Reasoncode" does not present in few logs events. But the command field is "AttributeThisDN" which will be unique in all the event           ... View more

Currently I am trying to extract the crossReferenceId value using below rex query.  Its working fine and I can extract the data. But seems below rex query is not extract all the values from the logs. For example, if I search the individual "agentname"  I cannot find that in the search (however I can find the same agentname without rex).  Seems below rex is not extracting the complete values. May be some are missing out.    index=xyz "crossReferenceId" | rex"\{\"crossReferenceId\"\:\"(?<agentname>\w*)\"\,\"providerInstanceId\"\:\"(?<providerInstanceId>\w*............................)\"\,\"userId\"\:\"(?<userid>\w*............................)\"\,\"dateModified\"\:\"(?<modifieddate>\d*................)\"\}" | search agentname="*" providerInstanceId="*" userid="*" modifieddate="*" | stats count by agentname, providerInstanceId, userid, modifieddate | table agentname, providerInstanceId, userid, modifieddate   2022-09-21 21:18:23.046 TRACE 5028 --- [pool-3-thread-2] i.e.p.c.p.OAuthAuthenticationInterceptor : Host-Client Response: GET | 200 from https://xyz.com.com/api/crossReferences?$filter=p: Payload: {"@odata.context":"$metadata#crossReferences","value":[{"crossReferenceId":"asdfdsf","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"336d6a6f-3124-4c7c-b57a-692fa5114c2e","dateModified":"2022-08-09T12:17:06Z"},{"crossReferenceId":"dgsgdf","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"79729cc5-d454-44dc-ad60-0a9caadef580","dateModified":"2022-07-23T11:35:32Z"},{"crossReferenceId":"wqruytuere","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"6fe5f478-fbcb-460f-99b8-af1757c03bc5","dateModified":"2021-06-27T11:07:43Z"},{"crossReferenceId":"yuiyiyui","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"511da6bf-c21f-40bf-a18a-23c9ad472a9d","dateModified":"2022-05-26T11:49:18Z"},{"crossReferenceId":"ttttttt","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"251a6976-1460-49b8-a3cc-5126cb2caa00","dateModified":"2022-08-23T11:11:47Z"},{"crossReferenceId":"ytujty","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"7c17da4f-2181-4392-abe9-0e8ea8290234","dateModified":"2020-10-24T11:25:46Z"},{"crossReferenceId":"iljkljlhl","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"54e850d8-e69e-4749-8244-f2700eec4d0f","dateModified":"2022-03-26T11:33:12Z"},{"crossReferenceId":"xcvxcvvcvx","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"6465cce8-2d40-4661-bc9a-6473e4a09597","dateModified":"2022-04-09T11:27:12Z"},{"crossReferenceId":"ertwetret","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"c679dbe2-e803-4057-92ca-106ed48370b8","dateModified":"2022-09-08T11:23:50Z"},{"crossReferenceId":"tyutyutu","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"8e63a413-f4e4-46cd-aa10-bf86206079de","dateModified":"2021-11-22T12:17:43Z"},{"crossReferenceId":"aaaaaaa","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"71255798-366e-4d1e-8654-c7adcbeb7473","dateModified":"2022-06-23T11:36:02Z"},{"crossReferenceId":"erererere","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"20e39e30-d31b-4ad2-8993-b087104e34fa","dateModified":"2021-09-13T11:10:05Z"},{"crossReferenceId":"yutyuyutyu","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"6735fd0b-1148-4193-8971-f7a3afadb807","dateModified":"2022-07-25T11:20:29Z"},{"crossReferenceId":"ertrtrttr","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"bf3ffa03-83e8-4973-a292-817d0fd9a412","dateModified":"2022-08-23T11:11:47Z"},{"crossReferenceId":"tyuyuyuyu","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"5e622f17-7dce-4f2b-a264-1224fc709469","dateModified":"2022-08-30T21:07:02Z"},{"crossReferenceId":"wewewewewe","providerInstanceId":"c8d1a13b-2ebc-4762-acd0-c788bdd79125","userId":"b46acff6-aedf-45ab-b353-2ce699c0c454","dateModified":"2022-08-23T11:35:20Z"}]} ... View more

  Trying to setup alert for two scenarios as metioned below: Scenario 1: to determine if the connection between Xyz and the abc service is healthy, check for the string “IEX API Call Successfully got agent schedules data” This message occurs in batches roughly every 5 minutes. Good threshold might be to alert if This message is not seen in >= 10 minutes. Scenario 2: Another item to check would be the connection between the service and the xyz host. The String for that is “Schedule successfully posted to the provider API”. The cadence for those messages is the same so an absence of > 10 minutes may be a good place to start. Below are the samnple splunk events. I would like to setup an alert if these keywords event does not appears in last 10 minutes then send e-mail alert. Please help. 3/21/22 4:44:13.000 AM 2022-03-21 04:44:13 [pool-6-thread-2] INFO c.i.e.i.s.c.i.AgentResourceServiceImpl - IEX API Call Successfully got agent schedules data. 3/21/22 4:44:13.000 AM 2022-03-21 04:44:13 [pool-6-thread-2] INFO c.i.e.i.s.c.i.AgentResourceServiceImpl - IEX API Call Successfully got agent schedules data. 3/21/22 4:44:13.000 AM 2022-03-21 04:44:13 [pool-6-thread-2] INFO c.i.e.i.s.c.i.AgentResourceServiceImpl - IEX API Call Successfully got agent schedules data. 3/21/22 4:44:13.000 AM 2022-03-21 04:44:13 [pool-6-thread-2] INFO c.i.e.f.a.w.s.i.SchedulesServiceImpl - Schedule successfully posted to the provider Api.   ... View more

    12/27/21 6:42:50.000 AM PSComputerName Name Memory -------------- ---- ------ Host1 dfdf_Svc.exe 16024 Host1 sssService.exe 13142056 Host1 abcservice.exe 31380 Host1 xyzservice.exe 114340 Host1 rrrrr.exe 29304 12/27/21 6:42:50.000 AM PSComputerName Name Memory -------------- ---- ------ Host2 dfdf_Svc.exe 16064 Host2 sssService.exe 13144028 Host2 abcservice.exe 114708 Host2 xyzservice.exe 32248 Host2 rrrrr.exe 33616 I have these splunk output event in splunk logs. 1 event is for one specific server only. Under one server we have 5 services running and associated memory information. These output is in table format. I like to create regular expression so that I can create table format output as below: Servername Servicename Memory(in MB) (since above memory in bytes)     ... View more

Apr 30 09:13:30 localhost haproxy[22865]: 10.10.10.10:31872 [30/Apr/2021:09:13:30.362] verint rest_service/rest-hostname-8780 0/0/0/10/12 302 1973 - X-CSRF-TOKEN=NtOTKgh2hfTpjwTuRmx269ZR5qQhDRUtAOf0 ---- 32/32/6/0/0 0/0 {} "GET /test/te/ping/login HTTP/1.1" Apr 30 09:13:30 localhost haproxy[22865]: 10.10.10.10:52353 [30/Apr/2021:09:13:30.322] verint rest_service/rest-hostname-8680 0/0/0/1/1 200 11537 - - ---- 32/32/6/1/0 0/0 {} "GET /filterservices/css/filters.css HTTP/1.1" Apr 30 09:13:30 localhost haproxy[22865]: 10.10.10.10:42112 [30/Apr/2021:09:13:30.059] verint rest_service/rest-hostname-8780 0/0/12/143/202 200 122948 - - ---- 32/32/7/0/0 0/0 {} "GET /verintkm/js/tree.jquery.js HTTP/1.1" the below rex expression is working fine until the port number for above events. Now I am trying add expression for "0/0/12/143/202 200". After the port group I need to create another group name (response time) for the value 202 which is the last value after forward slash.[expr/expres/expre/expres/group name]   \[[^\]]+\]\s\w+\s(?<service>[^\/]+)\/\w+\-(?<hostname>\w+)\-(?<port>\d+)\s+   ... View more

Event: [{"hostname":"BBBBBBBBB","contentSourceName":"Authored","contentSourceType":"Authored","incremental":true,"skipCrawl":false,"isBulk":false,"startTime":1616802303335,"endTime":1616802355772,"reportStatus":"Success","documentsFound":20,"documentsFailed":0,"documentsSucceeded":16,"documentsFiltered":0,"documentsUnchanged":0,"contentProcessed":16,"contentAdded":0,"contentUpdated":0,"contentDeleted":0,"pdfSlices":0,"pdfDocCount":0,"exceptionCount":0,"generalExceptionCount":0,"warningCount":0,"processorFailureCount":0,"generalizedFailureCount":0,"heritrixErrorCount":0,"duplicateItemCount":0,"duplicateReportRelativeFilename":null,"jobId":-1}, {"hostname":"AAAAAAAA","contentSourceName":"Authored","contentSourceType":"Authored","incremental":true,"skipCrawl":false,"isBulk":false,"startTime":1616801520297,"endTime":1616801578765,"reportStatus":"Success","documentsFound":40,"documentsFailed":0,"documentsSucceeded":28,"documentsFiltered":0,"documentsUnchanged":0,"contentProcessed":28,"contentAdded":0,"contentUpdated":0,"contentDeleted":0,"pdfSlices":0,"pdfDocCount":0,"exceptionCount":0,"generalExceptionCount":0,"warningCount":0,"processorFailureCount":0,"generalizedFailureCount":0,"heritrixErrorCount":0,"duplicateItemCount":0,"duplicateReportRelativeFilename":null,"jobId":-1}, {"hostname":"ZZZZZZZZZ","contentSourceName":"Authored","contentSourceType":"Authored","incremental":true,"skipCrawl":false,"isBulk":false,"startTime":1616797920257,"endTime":1616797999256,"reportStatus":"Success","documentsFound":104,"documentsFailed":0,"documentsSucceeded":59,"documentsFiltered":0,"documentsUnchanged":0,"contentProcessed":59,"contentAdded":0,"contentUpdated":0,"contentDeleted":0,"pdfSlices":0,"pdfDocCount":0,"exceptionCount":0,"generalExceptionCount":0,"warningCount":0,"processorFailureCount":0,"generalizedFailureCount":0,"heritrixErrorCount":0,"duplicateItemCount":0,"duplicateReportRelativeFilename":null,"jobId":-1}, {"hostname":"YYYYYYYY","contentSourceName":"Authored","contentSourceType":"Authored","incremental":true,"skipCrawl":false,"isBulk":false,"startTime":1616794883261,"endTime":1616795120383,"reportStatus":"Success","documentsFound":236,"documentsFailed":3,"documentsSucceeded":121,"documentsFiltered":0,"documentsUnchanged":0,"contentProcessed":121,"contentAdded":0,"contentUpdated":0,"contentDeleted":0,"pdfSlices":0,"pdfDocCount":0,"exceptionCount":0,"generalExceptionCount":0,"warningCount":0,"processorFailureCount":3,"generalizedFailureCount":3,"heritrixErrorCount":0,"duplicateItemCount":0,"duplicateReportRelativeFilename":null,"jobId":-1}, {"hostname":"XXXXXXXX","contentSourceName":"Authored","contentSourceType":"Authored","incremental":true,"skipCrawl":false,"isBulk":false,"startTime":1616742071025,"endTime":1616794342113,"reportStatus":"Success","documentsFound":83004,"documentsFailed":640,"documentsSucceeded":81533,"documentsFiltered":0,"documentsUnchanged":0,"contentProcessed":81528,"contentAdded":0,"contentUpdated":0,"contentDeleted":0,"pdfSlices":0,"pdfDocCount":0,"exceptionCount":0,"generalExceptionCount":0,"warningCount":0,"processorFailureCount":640,"generalizedFailureCount":640,"heritrixErrorCount":0,"duplicateItemCount":0,"duplicateReportRelativeFilename":null,"jobId":-1}] ================================ We get above data in one event. I like to extract few data from above event in dashboard in table format: Hostname | contentSourceName | incremental| startTime| endTime|Duration| reportStatus | documentsFound | documentsFailed | The extra column needs to be add is i.e. "Duration" that can be extracted from StartTime and EndTime. Start date and end date is in Unix epoch time that needs to be converted into human readable format. Please help ... View more