Getting Data In

json events not breaking properly

fisuser1
Contributor

trying to break events before incidentTicket event, but not breaking properly with BREAK_ONLY_BEFORE.

props:
[prd_spog_inc_data]
BREAK_ONLY_BEFORE={"incidentTicket
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT
MAX_TIMESTAMP_LOOKAHEAD=null
SHOULD_LINEMERGE=false
category=Structured
description=A variant of the JSON source type, with support for nonexistent timestamps
disabled=false
pulldown_type=true
NO_BINARY_CHECK=true

_raw:
{"Last24HourDetail":[{"incidentTicket":"1369862","application":"Adaptiv Suite","stage":"Final","category":"Functional","startTime":"2020-01-03T11:15:00Z","endTime":"2020-01-03T12:00:00Z","incidentManager":"","currentUpdate":"02:20 EST - MDM Phase 2 market data generation process initiated.\n06:00 EST - MDM Phase 2 market data generation completes running ~ 120 mins longer than expacted duration.\n06:01 EST - MR EOD starts.\n07:00 EST MR EOD expected completion","businessImpact":"CIBCs Market risk end of day has experienced a delay following the market data generation processing overrunning. \nAs a result the MR EOD 06:00 EST SLA metric was missed. \nThere is a 60 min delay to the market data generation process and Market Risk EOD is expected to complete at 07:00 EST.","l5":null,"l10":null,"lastUpdate":"2020-01-03T11:41:36Z","differenceMinute":"45","resolutionCoordinator":"","lastUpdatedMinute":"172","causedBy":"","currentStatus":"02:20 EST - MDM Phase 2 market data generation process initiated.\n06:00 EST - MDM Phase 2 market data generation completes running ~ 120 mins longer than expacted duration.\n06:01 EST - MR EOD starts.\n07:00 EST MR EOD expected completion","slaMissed":"","differenceHours":"0:45","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"TC2 Dockland London"},{"incidentTicket":"1369729","application":"Ubix","stage":"Update","category":"Degraded","startTime":"2020-01-03T06:25:00Z","endTime":null,"incidentManager":"","currentUpdate":"Update 2- FIS CX team received confirmation from client that they are not interested to rerun EOD for today.The issue will handover to NAM team to check once the client is back ( 2:30 PM Eastern time) around 12 hours from now.","businessImpact":"Initial Margin is showing Zero in Client statement having open positions in NZFOE exchange. ","l5":null,"l10":null,"lastUpdate":"2020-01-03T09:12:31Z","differenceMinute":"0","resolutionCoordinator":"Pritesh Naidu / Feriel Bayoudh","lastUpdatedMinute":"321","causedBy":"","currentStatus":"Update 2- FIS CX team received confirmation from client that they are not interested to rerun EOD for today.The issue will handover to NAM team to check once the client is back ( 2:30 PM Eastern time) around 12 hours from now.","slaMissed":"","differenceHours":"","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"Hopkins"},{"incidentTicket":"VNG-266","application":"InTrader","stage":"Final","category":"Degraded","startTime":"2020-01-03T06:20:00Z","endTime":"2020-01-03T13:10:00Z","incidentManager":"","currentUpdate":"7:10 - The accounting run completed and the expected report transmission is also complete. Further research will be done on what the issue was causing the slow down. \n\n\n\n\n \n","businessImpact":"There is no current impact at this time as all transmissions are done. The customer can access the system when the report section is complete. The completion of the accounting run is due at 6:00\nThe customer verified it is not critical that they get into the system at 6:00. They are still in the parallel phase of converting to InTrader. The report transfer that runs at 10:00 is a separate job that runs outside of the accounting run and not the file the customer was expecting.","l5":null,"l10":null,"lastUpdate":"2020-01-03T13:24:58Z","differenceMinute":"410","resolutionCoordinator":"Brenda Russ","lastUpdatedMinute":"69","causedBy":"","currentStatus":"7:10 - The accounting run completed and the expected report transmission is also complete. Further research will be done on what the issue was causing the slow down. \n\n\n\n\n \n","slaMissed":"","differenceHours":"6:50","priority":"3","fisVertical":"Capital Markets","status":"Closed","location":"Hopkins"},{"incidentTicket":"1369701","application":"Ubix","stage":"Final","category":"Degraded","startTime":"2020-01-03T04:31:00Z","endTime":"2020-01-03T05:40:00Z","incidentManager":"","currentUpdate":"Resolved","businessImpact":"Initial Margin is showing Zero in Client statements.","l5":null,"l10":null,"lastUpdate":"2020-01-03T06:04:25Z","differenceMinute":"69","resolutionCoordinator":"Piyush Patil","lastUpdatedMinute":"509","causedBy":"","currentStatus":"Resolved","slaMissed":"No","differenceHours":"1:09","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"Hopkins"},{"incidentTicket":"1368719","application":"Protegent PTA","stage":"Initial","category":"Functional","startTime":"2020-01-02T21:29:00Z","endTime":null,"incidentManager":"Emily Wuenstel","currentUpdate":"","businessImpact":"Wellington recently upgraded from 17.1 to 18.3, and during the upgrade process the Adjust Holdings: Expired Security task was enabled by mistake. This has since been turned off, however, it created 8,000+ adjustments in Wellingtons site, and they are requesting an exception for them to be deleted as soon as possible, as its impacting core PTA functions and report authentication.","l5":null,"l10":null,"lastUpdate":"2020-01-02T21:31:52Z","differenceMinute":"0","resolutionCoordinator":"","lastUpdatedMinute":"1022","causedBy":"","currentStatus":"","slaMissed":"","differenceHours":"","priority":"4","fisVertical":"Capital Markets","status":"Open","location":""},{"incidentTicket":"1369339","application":"Control Center,InvestOne","stage":"Final","category":"Degraded","startTime":"2020-01-02T20:40:00Z","endTime":"2020-01-02T21:51:00Z","incidentManager":"","currentUpdate":"Citi has authorized FIS to do rolling restarts of Enterprise servers to help clear the issue with error on Control Center sweep tasks.","businessImpact":"Citi is unable to run sweeps from Control Center. They receive errors which cause them to have to manually sweep in InvestOne directly and override Control Center.","l5":null,"l10":null,"lastUpdate":"2020-01-02T22:03:09Z","differenceMinute":"71","resolutionCoordinator":"Kathy Gillespie","lastUpdatedMinute":"990","causedBy":"","currentStatus":"Citi has authorized FIS to do rolling restarts of Enterprise servers to help clear the issue with error on Control Center sweep tasks.","slaMissed":"","differenceHours":"1:11","priority":"3","fisVertical":"Capital Markets","status":"Closed","location":"Voorhees"},{"incidentTicket":"2000007995","application":"IBS Insight","stage":"Final","category":"Degraded","startTime":"2020-01-02T20:05:00Z","endTime":"2020-01-02T20:29:00Z","incidentManager":"Rick Yost / Katherine Montales","currentUpdate":"No impact or latency observed since 14:29 CT. Teams checked the logs to determine what caused the latency, and saw DB2 deadlock errors which indicated database contention. However, these errors were seen few minutes after latency was observed initially and would have been more of a symptom than root cause. Teams looked at the JVMs on the application side and saw that the downstream impact on the database tier being slow might have caused the application to be unresponsive. No conclusive root cause has been determined yet. Teams will monitor the application and will reconvene tomorrow, 01/03/2020 at 08:00 CT to discuss and research root cause and to have all the necessary teams if the issue reoccurs.","businessImpact":"The Business Impact was that clients who were attempting to access the application were spinning at login. Some clients also reported that if they were logged in they were kicked out of the application. ","l5":null,"l10":null,"lastUpdate":"2020-01-03T14:26:29Z","differenceMinute":"44","resolutionCoordinator":"Duane Wagner / Devan Bremer / Navjot Singh","lastUpdatedMinute":"7","causedBy":"FIS","currentStatus":"No impact or latency observed since 14:29 CT. Teams checked the logs to determine what caused the latency, and saw DB2 deadlock errors which indicated database contention. However, these errors were seen few minutes after latency was observed initially and would have been more of a symptom than root cause. Teams looked at the JVMs on the application side and saw that the downstream impact on the database tier being slow might have caused the application to be unresponsive. No conclusive root cause has been determined yet. Teams will monitor the application and will reconvene tomorrow, 01/03/2020 at 08:00 CT to discuss and research root cause and to have all the necessary teams if the issue reoccurs.","slaMissed":"","differenceHours":"0:24","priority":"2","fisVertical":"Banking Solutions","status":"Pending","location":"Little Rock"},{"incidentTicket":"1369071","application":"VPM - Base System","stage":"Final","category":"Degraded","startTime":"2020-01-02T16:43:00Z","endTime":"2020-01-03T02:36:00Z","incidentManager":"","currentUpdate":"Resolved. After rebooting the app server, MSDTC service came back up properly and BC Partners users will now be able to change any details on existing contracts in VPM.","businessImpact":"All users at BC Partners PROD were unable to change any details on existing contracts in VPM.","l5":null,"l10":null,"lastUpdate":"2020-01-03T08:29:38Z","differenceMinute":"593","resolutionCoordinator":"Eric Appleman","lastUpdatedMinute":"364","causedBy":"","currentStatus":"Resolved. After rebooting the app server, MSDTC service came back up properly and BC Partners users will now be able to change any details on existing contracts in VPM.","slaMissed":"","differenceHours":"9:53","priority":"4","fisVertical":"Capital Markets","status":"Closed","location":"Voorhees"},{"incidentTicket":"IN20001533508","application":"FIS Monitoring Tools","stage":"Final","category":"Outage","startTime":"2020-01-02T15:30:00Z","endTime":"2020-01-02T21:55:00Z","incidentManager":"Derek Potratz","currentUpdate":"Update Final 16:00 CT:\nFIS networking teams identified that at 09:21 CT traffic for LDAP on the primary server had been moved to UDP from TCP and from ephemeral source ports to 1. This caused the traffic to be dropped while attempting to connect. Technical teams recycled the application services but that did not resolve the issue. It was then identified that the primary applic
{"Last24HourDetail":[{"incidentTicket":"1369862","application":"Adaptiv Suite","stage":"Final","category":"Functional","startTime":"2020-01-03T11:15:00Z","endTime":"2020-01-03T12:00:00Z","incidentManager":"","currentUpdate":"02:20 EST - MDM Phase 2 market data generation process initiated.\n06:00 EST - MDM Phase 2 market data generation completes running ~ 120 mins longer than expacted duration.\n06:01 EST - MR EOD starts.\n07:00 EST MR EOD expected completion","businessImpact":"CIBCs Market risk end of day has experienced a delay following the market data generation processing overrunning. \nAs a result the MR EOD 06:00 EST SLA metric was missed. \nThere is a 60 min delay to the market data generation process and Market Risk EOD is expected to complete at 07:00 EST.","l5":null,"l10":null,"lastUpdate":"2020-01-03T11:41:36Z","differenceMinute":"45","resolutionCoordinator":"","lastUpdatedMinute":"171","causedBy":"","currentStatus":"02:20 EST - MDM Phase 2 market data generation process initiated.\n06:00 EST - MDM Phase 2 market data generation completes running ~ 120 mins longer than expacted duration.\n06:01 EST - MR EOD starts.\n07:00 EST MR EOD expected completion","slaMissed":"","differenceHours":"0:45","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"TC2 Dockland London"},{"incidentTicket":"1369729","application":"Ubix","stage":"Update","category":"Degraded","startTime":"2020-01-03T06:25:00Z","endTime":null,"incidentManager":"","currentUpdate":"Update 2- FIS CX team received confirmation from client that they are not interested to rerun EOD for today.The issue will handover to NAM team to check once the client is back ( 2:30 PM Eastern time) around 12 hours from now.","businessImpact":"Initial Margin is showing Zero in Client statement having open positions in NZFOE exchange. ","l5":null,"l10":null,"lastUpdate":"2020-01-03T09:12:31Z","differenceMinute":"0","resolutionCoordinator":"Pritesh Naidu / Feriel Bayoudh","lastUpdatedMinute":"320","causedBy":"","currentStatus":"Update 2- FIS CX team received confirmation from client that they are not interested to rerun EOD for today.The issue will handover to NAM team to check once the client is back ( 2:30 PM Eastern time) around 12 hours from now.","slaMissed":"","differenceHours":"","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"Hopkins"},{"incidentTicket":"VNG-266","application":"InTrader","stage":"Final","category":"Degraded","startTime":"2020-01-03T06:20:00Z","endTime":"2020-01-03T13:10:00Z","incidentManager":"","currentUpdate":"7:10 - The accounting run completed and the expected report transmission is also complete. Further research will be done on what the issue was causing the slow down. \n\n\n\n\n \n","businessImpact":"There is no current impact at this time as all transmissions are done. The customer can access the system when the report section is complete. The completion of the accounting run is due at 6:00\nThe customer verified it is not critical that they get into the system at 6:00. They are still in the parallel phase of converting to InTrader. The report transfer that runs at 10:00 is a separate job that runs outside of the accounting run and not the file the customer was expecting.","l5":null,"l10":null,"lastUpdate":"2020-01-03T13:24:58Z","differenceMinute":"410","resolutionCoordinator":"Brenda Russ","lastUpdatedMinute":"68","causedBy":"","currentStatus":"7:10 - The accounting run completed and the expected report transmission is also complete. Further research will be done on what the issue was causing the slow down. \n\n\n\n\n \n","slaMissed":"","differenceHours":"6:50","priority":"3","fisVertical":"Capital Markets","status":"Closed","location":"Hopkins"},{"incidentTicket":"1369701","application":"Ubix","stage":"Final","category":"Degraded","startTime":"2020-01-03T04:31:00Z","endTime":"2020-01-03T05:40:00Z","incidentManager":"","currentUpdate":"Resolved","businessImpact":"Initial Margin is showing Zero in Client statements.","l5":null,"l10":null,"lastUpdate":"2020-01-03T06:04:25Z","differenceMinute":"69","resolutionCoordinator":"Piyush Patil","lastUpdatedMinute":"508","causedBy":"","currentStatus":"Resolved","slaMissed":"No","differenceHours":"1:09","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"Hopkins"},{"incidentTicket":"1368719","application":"Protegent PTA","stage":"Initial","category":"Functional","startTime":"2020-01-02T21:29:00Z","endTime":null,"incidentManager":"Emily Wuenstel","currentUpdate":"","businessImpact":"Wellington recently upgraded from 17.1 to 18.3, and during the upgrade process the Adjust Holdings: Expired Security task was enabled by mistake. This has since been turned off, however, it created 8,000+ adjustments in Wellingtons site, and they are requesting an exception for them to be deleted as soon as possible, as its impacting core PTA functions and report authentication.","l5":null,"l10":null,"lastUpdate":"2020-01-02T21:31:52Z","differenceMinute":"0","resolutionCoordinator":"","lastUpdatedMinute":"1021","causedBy":"","currentStatus":"","slaMissed":"","differenceHours":"","priority":"4","fisVertical":"Capital Markets","status":"Open","location":""},{"incidentTicket":"1369339","application":"Control Center,InvestOne","stage":"Final","category":"Degraded","startTime":"2020-01-02T20:40:00Z","endTime":"2020-01-02T21:51:00Z","incidentManager":"","currentUpdate":"Citi has authorized FIS to do rolling restarts of Enterprise servers to help clear the issue with error on Control Center sweep tasks.","businessImpact":"Citi is unable to run sweeps from Control Center. They receive errors which cause them to have to manually sweep in InvestOne directly and override Control Center.","l5":null,"l10":null,"lastUpdate":"2020-01-02T22:03:09Z","differenceMinute":"71","resolutionCoordinator":"Kathy Gillespie","lastUpdatedMinute":"989","causedBy":"","currentStatus":"Citi has authorized FIS to do rolling restarts of Enterprise servers to help clear the issue with error on Control Center sweep tasks.","slaMissed":"","differenceHours":"1:11","priority":"3","fisVertical":"Capital Markets","status":"Closed","location":"Voorhees"},{"incidentTicket":"2000007995","application":"IBS Insight","stage":"Final","category":"Degraded","startTime":"2020-01-02T20:05:00Z","endTime":"2020-01-02T20:29:00Z","incidentManager":"Rick Yost / Katherine Montales","currentUpdate":"No impact or latency observed since 14:29 CT. Teams checked the logs to determine what caused the latency, and saw DB2 deadlock errors which indicated database contention. However, these errors were seen few minutes after latency was observed initially and would have been more of a symptom than root cause. Teams looked at the JVMs on the application side and saw that the downstream impact on the database tier being slow might have caused the application to be unresponsive. No conclusive root cause has been determined yet. Teams will monitor the application and will reconvene tomorrow, 01/03/2020 at 08:00 CT to discuss and research root cause and to have all the necessary teams if the issue reoccurs.","businessImpact":"The Business Impact was that clients who were attempting to access the application were spinning at login. Some clients also reported that if they were logged in they were kicked out of the application. ","l5":null,"l10":null,"lastUpdate":"2020-01-03T14:26:29Z","differenceMinute":"44","resolutionCoordinator":"Duane Wagner / Devan Bremer / Navjot Singh","lastUpdatedMinute":"6","causedBy":"FIS","currentStatus":"No impact or latency observed since 14:29 CT. Teams checked the logs to determine what caused the latency, and saw DB2 deadlock errors which indicated database contention. However, these errors were seen few minutes after latency was observed initially and would have been more of a symptom than root cause. Teams looked at the JVMs on the application side and saw that the downstream impact on the database tier being slow might have caused the application to be unresponsive. No conclusive root cause has been determined yet. Teams will monitor the application and will reconvene tomorrow, 01/03/2020 at 08:00 CT to discuss and research root cause and to have all the necessary teams if the issue reoccurs.","slaMissed":"","differenceHours":"0:24","priority":"2","fisVertical":"Banking Solutions","status":"Pending","location":"Little Rock"},{"incidentTicket":"1369071","application":"VPM - Base System","stage":"Final","category":"Degraded","startTime":"2020-01-02T16:43:00Z","endTime":"2020-01-03T02:36:00Z","incidentManager":"","currentUpdate":"Resolved. After rebooting the app server, MSDTC service came back up properly and BC Partners users will now be able to change any details on existing contracts in VPM.","businessImpact":"All users at BC Partners PROD were unable to change any details on existing contracts in VPM.","l5":null,"l10":null,"lastUpdate":"2020-01-03T08:29:38Z","differenceMinute":"593","resolutionCoordinator":"Eric Appleman","lastUpdatedMinute":"363","causedBy":"","currentStatus":"Resolved. After rebooting the app server, MSDTC service came back up properly and BC Partners users will now be able to change any details on existing contracts in VPM.","slaMissed":"","differenceHours":"9:53","priority":"4","fisVertical":"Capital Markets","status":"Closed","location":"Voorhees"},{"incidentTicket":"IN20001533508","application":"FIS Monitoring Tools","stage":"Final","category":"Outage","startTime":"2020-01-02T15:30:00Z","endTime":"2020-01-02T21:55:00Z","incidentManager":"Derek Potratz","currentUpdate":"Update Final 16:00 CT:\nFIS networking teams identified that at 09:21 CT traffic for LDAP on the primary server had been moved to UDP from TCP and from ephemeral source ports to 1. This caused the traffic to be dropped while attempting to connect. Technical teams recycled the application services but that did not resolve the issue. It was then identified that the primary applic
{"Last24HourDetail":[{"incidentTicket":"1369862","application":"Adaptiv Suite","stage":"Final","category":"Functional","startTime":"2020-01-03T11:15:00Z","endTime":"2020-01-03T12:00:00Z","incidentManager":"","currentUpdate":"02:20 EST - MDM Phase 2 market data generation process initiated.\n06:00 EST - MDM Phase 2 market data generation completes running ~ 120 mins longer than expacted duration.\n06:01 EST - MR EOD starts.\n07:00 EST MR EOD expected completion","businessImpact":"CIBCs Market risk end of day has experienced a delay following the market data generation processing overrunning. \nAs a result the MR EOD 06:00 EST SLA metric was missed. \nThere is a 60 min delay to the market data generation process and Market Risk EOD is expected to complete at 07:00 EST.","l5":null,"l10":null,"lastUpdate":"2020-01-03T11:41:36Z","differenceMinute":"45","resolutionCoordinator":"","lastUpdatedMinute":"170","causedBy":"","currentStatus":"02:20 EST - MDM Phase 2 market data generation process initiated.\n06:00 EST - MDM Phase 2 market data generation completes running ~ 120 mins longer than expacted duration.\n06:01 EST - MR EOD starts.\n07:00 EST MR EOD expected completion","slaMissed":"","differenceHours":"0:45","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"TC2 Dockland London"},{"incidentTicket":"1369729","application":"Ubix","stage":"Update","category":"Degraded","startTime":"2020-01-03T06:25:00Z","endTime":null,"incidentManager":"","currentUpdate":"Update 2- FIS CX team received confirmation from client that they are not interested to rerun EOD for today.The issue will handover to NAM team to check once the client is back ( 2:30 PM Eastern time) around 12 hours from now.","businessImpact":"Initial Margin is showing Zero in Client statement having open positions in NZFOE exchange. ","l5":null,"l10":null,"lastUpdate":"2020-01-03T09:12:31Z","differenceMinute":"0","resolutionCoordinator":"Pritesh Naidu / Feriel Bayoudh","lastUpdatedMinute":"319","causedBy":"","currentStatus":"Update 2- FIS CX team received confirmation from client that they are not interested to rerun EOD for today.The issue will handover to NAM team to check once the client is back ( 2:30 PM Eastern time) around 12 hours from now.","slaMissed":"","differenceHours":"","priority":"4","fisVertical":"Capital Markets","status":"Open","location":"Hopkins"},{"incidentTicket":"VNG-266","application":"InTrader","stage":"Final","category":"Degraded","startTime":"2020-01-03T06:20:00Z","endTime":"2020-01-03T13:10:00Z","incidentManager":"","currentUpdate":"7:10 - The accounting run completed and the expected report transmission is also complete. Further research will be done on what the issue was causing the slow down. \n\n\n\n\n \n","businessImpact":"There is no current impact at this time as all transmissions are done. The customer can access the system when the report section is complete. The completion of the accounting run is due at 6:00\nThe customer verified it is not critical that they get into the system at 6:00. They are still in the parallel phase of converting to InTrader. The report transfer that runs at 10:00 is a separate job that runs outside of the accounting run and not the file the customer was expecting.","l5":null,"l10":null,"lastUpdate":"2020-01-03T13:24:58Z","differenceMinute":"410","resolutionCoordinator":"Brenda Russ","lastUpdatedMinute":"67","causedBy":"","currentStatus":"7:10 - The accounting run completed and the expected report transmission is also complete. Further research will be done on what the issue was causing the slow down. \n\n\n\n\n \n","slaMissed":"","differenceHours":"6:50","priority":"3","fisVertical":"Capital Markets","status":"Closed","location":"Hopkins"},

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The BREAK_ONLY_BEFORE attribute is active when SHOULD_LINEMERGE=true.
With SHOULD_LINEMERGE=false try LINE_BREAKER=(){"incidentTicket.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...