Unfortunately, ResponsePayload field would not be normally extracted.  There are multiple challenges with this one.  One is sheer size of the raw events.  This may be overcome by tweaking limits.conf if you have control of it.  Then, many field values are not properly quoted for automatic extraction.  Using custom regex extraction in props.conf may help.  But the irregular use of line breaks (at least as shown in the illustration) can make this very difficult. In addition, you need to have some prior knowledge about the actual dataset in relation to the desired results. Keys "createdDate", "paymentTransactionId", "currencyCode", "operationStepAmount" are as you noted, inside the JSON field ResonsePayload.  But they are part of an array of arrays.  How do you want to handle array elements when there are multiple?  ResponsePayload has another array, paymentInstruments.  How do you want to handle multiple elements in this field?  Keys "itineraryNumber" and "travelProductId" are in a different JSON field RequestPayload, which is simpler to extract.  However, this field is also not automatically extracted due to lack of proper quotes. In the following, I will assume that elements of ResponsePayload.paymentInstruments should always be retained as an array, but each element of ResponsePayload.payments, and each element of ResponsePayload.payments.hits should be viewed individually.  The method used to break array elements, mvexpand, has its limits and can have performance consequences if the arrays are too big. | rex "@RequestPayload=(?<RequestPayload>.+)" | spath input=RequestPayload ``` RequestPayload is very easy to extract ``` | rex mode=sed "s/.+ActivityGUID=.+\n// s/@.+(URL|Headers)=.+\n//g s/@RequestPayload=.+// s/@ResponsePayload=//" ``` the complex sed is needed because of irregular line breaks within RequestPayload ``` | rename _raw as ResponsePayload | spath input=ResponsePayload errorInformation | spath input=ResponsePayload paymentInstruments{} | spath input=ResponsePayload payments{} | fields - ResponsePayload | mvexpand payments{} | spath input=payments{} scrollId | spath input=payments{} totalHits | spath input=payments{} hits{} | fields - payments{} | mvexpand hits{} | spath input=hits{} | fields - hits{} In the above, I have already examined keys in those arrays, and assume that the keys are always the same.  If there can be more than 3 keys in each array, or if keys are indeterministic, you can use a more traditional, conservative approach | rex "@RequestPayload=(?<RequestPayload>.+)" | spath input=RequestPayload ``` RequestPayload is very easy to extract ``` | rex mode=sed "s/.+ActivityGUID=.+\n// s/@.+(URL|Headers)=.+\n//g s/@RequestPayload=.+// s/@ResponsePayload=//" ``` the complex sed is needed because of irregular line breaks within RequestPayload ``` | rename _raw as ResponsePayload | spath input=ResponsePayload | fields - payments{}.* | spath input=ResponsePayload payments{} | fields - ResponsePayload | mvexpand payments{} | spath input=payments{} | fields - hits{}.* | spath input=payments{} hits{} | fields - payments{} | mvexpand hits{} | spath input=hits{} | fields - hits{} Here is the emulation I use.  Play with it and compare with real data | makeresults | fields - _time | eval _raw = "2025-08-27 16:14:15,006 ActivityGUID=931eecb0-8570-4045-8f83-d232065374ab TransactionGUID=9911aa1c-54bc-4912-8b4c-7b8d1b15434f ProductName=yyy-priv-collector HostLocal=10.177.774.774 ActivityName=endpoint OperationName=searchPayments RequestMethod=POST Duration=181 TimestampStart=2025-08-27 09:14:14,824 Timestamp=2025-08-27 09:14:15,005 DurationN=181 StatusCode=0 Client-ID=privCaseManagement HostRemote=127.0.0.6 TimestampEnd=2025-08-27 09:14:15,005 ResponseStatus=200 ActivityStep=rs @RequestURL=http://yyy-priv-collector-prod.rcp.deptandpriv.prod-cts.exp-aws.net/v1/searchPayments/ @ResponseHeaders={Content-Type=[application/json], X-Content-Type-Options=[nosniff], X-XSS-Protection=[0], Cache-Control=[no-cache, no-store, max-age=0, must-revalidate], Pragma=[no-cache], Expires=[0], X-Frame-Options=[DENY]} @RequestHeaders={host=[yyy-priv-collector-prod.rcp.deptandpriv.prod-cts.exp-aws.net], accept=[application/json], transaction-guid=[9911aa1c-54bc-4912-8b4c-7b8d1b15434f], message-guid=[92d969c4-f659-45f4-b4ec-97a9f1e38e69], content-type=[application/json], client-id=[RCM], authorization=#SCRAMBLED#, x-datadog-trace-id=[1798983498446935042], x-datadog-parent-id=[1394352222710180060], x-datadog-sampling-priority=[0], content-length=[222], user-agent=[Apache-HttpClient/4.5.14 (Java/17.0.11)], accept-encoding=[gzip,deflate], x-forwarded-for=[10.172.234.74], x-forwarded-proto=[https], x-request-id=[9eb3657b-da6e-4f0e-84b9-41ca00dae160], x-envoy-attempt-count=[1], x-envoy-internal=[true], x-forwarded-client-cert=[By=spiffe://deptandpriv.prod-cts.exp-aws.net/ns/priv-tech-platform/sa/yyy-priv-collector-prod-us-west-2-prod-template;Hash=c2dc4556e3158a274d6eef6700c9fc5088993a732c77568766b809838dcb3e00;Subject=\"\";URI=spiffe://deptandpriv.prod-cts.exp-aws.net/ns/istio-system/sa/istio-ingressgateway-service-account]} @ResponsePayload={\"errorInformation\":null,\"payments\":[{\"hits\":[{\"currentState\":{\"createdDate\":\"2025-08-26T16:33:51.899Z\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|b6815ff3-6742-4c4b-b69a-fbf99ddb24fb|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"3e3cbd48-30ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":null,\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:33:51.899Z\",\"stateName\":\"ChargeCompleted\",\"stateTransition\":null,\"transactionStateId\":\"b6815ff3-6742-5c6b-b69a-fbf99ddb24fb\",\"updatedDate\":\"2025-08-26T16:33:51.899Z\"},\"operations\":{\"abandonOperations\":null,\"captureDepositActivityOperations\":null,\"captureSettlementOperations\":null,\"chargeOperations\":[{\"attributes\":{\"additionalAttributes\":null,\"alternateCurrencyCode\":null,\"alternatePaymentAmount\":null,\"captureId\":null,\"createdDate\":\"2025-08-26T16:14:24.152Z\",\"currencyCode\":\"USD\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|e43db6f8-78a0-0e20-2636-d5b173280df3|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"operation\",\"operationTypeCode\":\"CHR\",\"paymentAmount\":944.6,\"paymentOperationId\":\"e43db6f8-78a0-0e20-2636-d5b173280df3\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"rateQuoteId\":null,\"rateQuoteServiceId\":null,\"refundId\":null,\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:15:01.647Z\"},\"chargeSteps\":null,\"commitSteps\":[{\"additionalAttributes\":null,\"attemptId\":0,\"authorizationCode\":null,\"authorizationDate\":\"null\",\"createdDate\":\"2025-08-26T16:15:01.582Z\",\"currencyCode\":\"USD\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|e43db6f8-78a0-0e20-2636-d5b173280df3|7d2f68fc-db1f-0238-2a35-8611bc8a90c4|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"step\",\"gatewayOperationId\":\"527ee823-4eba-4580-aa8c-cef59fc4a0db\",\"gatewayResultStatuses\":null,\"gatewayResults\":[],\"gatewayService\":\"PaymentExecutor\",\"internalReferenceId\":null,\"merchantId\":null,\"operationStepAmount\":944.6,\"operationStepPresentedAmount\":944.6,\"operationStepTransactedAmount\":null,\"operationStepTypeCode\":\"CMT\",\"paymentOperationId\":\"e43db6f8-78a0-0e20-2636-d5b173280df3\",\"paymentOperationStepId\":\"7d2f68fc-db1f-0238-2a35-8611bc8a90c4\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"processor\":null,\"processorReferenceId\":null,\"schemaVersion\":\"1.0\",\"statusCode\":\"0\",\"statusCodeCategoryName\":\"Success\",\"statusCodeMessageText\":\"Successfully processed!\",\"statusCodeNamespace\":\"com.expedia.e3.es.payment.common.errorhandling.CommonStatusReportingTemplates\",\"stepStatus\":\"success\",\"successful\":true,\"updatedDate\":\"2025-08-26T16:15:01.637Z\",\"partnerOperationId\":null,\"platformOperationId\":null}],\"holdSteps\":[{\"additionalAttributes\":null,\"attemptId\":0,\"authorizationCode\":null,\"authorizationDate\":\"null\",\"createdDate\":\"2025-08-26T16:14:26.341Z\",\"currencyCode\":\"USD\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|e43db6f8-78a0-0e20-2636-d5b173280df3|793b4768-1ea0-07df-20d9-a280205711c1|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"step\",\"gatewayOperationId\":\"8859e2e7-6a85-4899-87ce-fb4bd7360944\",\"gatewayResultStatuses\":null,\"gatewayResults\":[],\"gatewayService\":\"PaymentExecutor\",\"internalReferenceId\":null,\"merchantId\":null,\"operationStepAmount\":944.6,\"operationStepPresentedAmount\":944.6,\"operationStepTransactedAmount\":null,\"operationStepTypeCode\":\"HLD\",\"paymentOperationId\":\"e43db6f8-78a0-0e20-2636-d5b173280df3\",\"paymentOperationStepId\":\"793b4768-1ea0-07df-20d9-a280205711c1\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"processor\":null,\"processorReferenceId\":null,\"schemaVersion\":\"1.0\",\"statusCode\":\"0\",\"statusCodeCategoryName\":\"Success\",\"statusCodeMessageText\":\"Successfully processed!\",\"statusCodeNamespace\":\"com.expedia.e3.es.payment.common.errorhandling.CommonStatusReportingTemplates\",\"stepStatus\":\"success\",\"successful\":true,\"updatedDate\":\"2025-08-26T16:14:26.369Z\",\"partnerOperationId\":null,\"platformOperationId\":null}],\"redirectSteps\":null,\"rollbackSteps\":null}],\"chargebackOperations\":null,\"chargebackReversalOperations\":null,\"chargebackReversalSettlementOperations\":null,\"chargebackSettlementOperations\":null,\"creditOperations\":null,\"refundDepositActivityOperations\":null,\"refundOperations\":null,\"refundSettlementOperations\":null,\"verifyOperations\":[{\"attributes\":{\"additionalAttributes\":null,\"alternateCurrencyCode\":null,\"alternatePaymentAmount\":null,\"captureId\":null,\"createdDate\":\"2025-08-26T16:14:24.116Z\",\"currencyCode\":\"USD\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|81025324-2e27-0aef-2fb3-ff5ab2a756c2|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"operation\",\"operationTypeCode\":\"VFY\",\"paymentAmount\":944.6,\"paymentOperationId\":\"81025324-2e27-0aef-2fb3-ff5ab2a756c2\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"rateQuoteId\":null,\"rateQuoteServiceId\":null,\"refundId\":null,\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:14:25.331Z\"},\"verifySteps\":[{\"additionalAttributes\":null,\"attemptId\":0,\"authorizationCode\":\"ul425i9o0ur\",\"authorizationDate\":\"2025-08-26T16:14:25.031Z\",\"createdDate\":\"2025-08-26T16:14:24.199Z\",\"currencyCode\":\"USD\",\"entityDataVersion\":3243270965,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|81025324-2e27-0aef-2fb3-ff5ab2a756c2|e2e1bd80-a27e-031e-25b7-87c5369ae5e1|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c6p5jw-ed84cd28-fe3f-47e0-a445-139c6a6b8dca_132_8899\",\"entityType\":\"step\",\"gatewayOperationId\":\"cf737c29-d68d-4c99-9d39-1be40b5e84fe\",\"gatewayResultStatuses\":[{\"acquirerStatusCode\":null,\"description\":null,\"errorResultType\":null,\"internalReferenceId\":\"o98kxkapu3y\",\"merchantId\":\"EXPEDIAAUTH\",\"paymentProcessorStatusCodeType\":null,\"processor\":\"Bibit\",\"processorReferenceId\":\"ul425i9o0ur\",\"ptxGatewayResultStatuses\":[{\"acquirerStatusCode\":null,\"description\":\"The acquirer responds that the address details partially match the details at the issuer.\",\"errorResultType\":\"Technical\",\"ptxGatewayResultStatusTypeCode\":\"avs\",\"rawStatusCode\":\"PARTIAL_APPROVED\",\"statusCategory\":\"EXP\",\"statusCode\":\"3\"},{\"acquirerStatusCode\":null,\"description\":\"The acquirer responds that the CVC code matches the details at the issuer\",\"errorResultType\":\"Technical\",\"ptxGatewayResultStatusTypeCode\":\"cvv\",\"rawStatusCode\":\"APPROVED\",\"statusCategory\":\"EXP\",\"statusCode\":\"7\"},{\"acquirerStatusCode\":null,\"description\":\"Authorized\",\"errorResultType\":\"None\",\"ptxGatewayResultStatusTypeCode\":\"gateway\",\"rawStatusCode\":\"AUTHORISED\",\"statusCategory\":\"EXTERNAL\",\"statusCode\":\"SUCCESSFUL\"}],\"rawStatusCode\":null,\"statusCategory\":null,\"statusCode\":null}],\"gatewayResults\":[],\"gatewayService\":\"PaymentExecutor\",\"internalReferenceId\":\"o98kxkapu3y\",\"merchantId\":\"EXPEDIAAUTH\",\"operationStepAmount\":944.6,\"operationStepPresentedAmount\":944.6,\"operationStepTransactedAmount\":0.0,\"operationStepTypeCode\":\"VFY\",\"paymentOperationId\":\"81025324-2e27-0aef-2fb3-ff5ab2a756c2\",\"paymentOperationStepId\":\"e2e1bd80-a27e-031e-25b7-87c5369ae5e1\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"processor\":\"Bibit\",\"processorReferenceId\":\"ul425i9o0ur\",\"reusedVerificationId\":\"487f7891-06f9-4e01-bb91-a34cbdecc6fb\",\"reusing\":false,\"schemaVersion\":\"1.0\",\"statusCode\":\"0\",\"statusCodeCategoryName\":\"Success\",\"statusCodeMessageText\":\"MSTERR_NO_ERROR\",\"statusCodeNamespace\":\"com.expedia.e3.es.payment.processor.HAPSResponseCode.MSTE\",\"stepStatus\":\"success\",\"successful\":true,\"updatedDate\":\"2025-08-26T16:14:25.277Z\"}]}] },\"stateHistory\":[{\"createdDate\":\"2025-08-26T16:16:01.534Z\",\"entityDataVersion\":3243274925,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|fa9cfff3-0cd5-49e7-a583-ca9965d75f82|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-cplwgn-98f56f82-560c-4b69-b18e-86e1c1ff7aeb_151_9275\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":null,\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:16:01.534Z\",\"stateName\":\"ChargeCompleted\",\"stateTransition\":null,\"transactionStateId\":\"fa9cfff3-0cd5-49e7-a583-ca9965d75f82\",\"updatedDate\":\"2025-08-26T16:16:01.534Z\"},{\"createdDate\":\"2025-08-26T16:14:25.385Z\",\"entityDataVersion\":3243270965,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|12713ce5-6e3a-4d79-8804-64de8383abaa|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c6p5jw-ed84cd28-fe3f-47e0-a445-139c6a6b8dca_132_8899\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":\"45a30bc0-6aae-496d-a45f-f03afd458eaf\",\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:14:25.385Z\",\"stateName\":\"Verified\",\"stateTransition\":\"New:Verified\",\"transactionStateId\":\"12713ce5-6e3a-4d79-8804-64de8383abaa\",\"updatedDate\":\"2025-08-26T16:14:25.385Z\"},{\"createdDate\":\"2025-08-26T16:14:26.479Z\",\"entityDataVersion\":3243271007,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|6aee2039-cf71-440a-9c60-d49c7d34342f|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-cvnv9c-e3efcc74-a5e0-437f-84d3-f5e1691dbf67_157_4357\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":\"d1f42d9c-9e3b-46e4-82a4-c14bef915f8e\",\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:14:26.479Z\",\"stateName\":\"HoldCompleted\",\"stateTransition\":\"Verified:HoldCompleted\",\"transactionStateId\":\"6aee2039-cf71-440a-9c60-d49c7d34342f\",\"updatedDate\":\"2025-08-26T16:14:26.479Z\"},{\"createdDate\":\"2025-08-26T16:15:01.752Z\",\"entityDataVersion\":3243272446,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|3deafd0f-f794-4535-9fa5-ae6c76efe0da|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-cplwgn-98f56f82-560c-4b69-b18e-86e1c1ff7aeb_151_9068\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":\"833dec67-6be9-4e2b-a76c-00b677963ee6\",\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:15:01.752Z\",\"stateName\":\"ChargePending\",\"stateTransition\":\"HoldCompleted:ChargePending\",\"transactionStateId\":\"3deafd0f-f794-4535-9fa5-ae6c76efe0da\",\"updatedDate\":\"2025-08-26T16:15:01.752Z\"},{\"createdDate\":\"2025-08-26T16:14:25.385Z\",\"entityDataVersion\":3243270965,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|45a30bc0-6aae-496d-a45f-f03afd458eaf|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c6p5jw-ed84cd28-fe3f-47e0-a445-139c6a6b8dca_132_8899\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":null,\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:14:25.385Z\",\"stateName\":\"New\",\"stateTransition\":null,\"transactionStateId\":\"45a30bc0-6aae-496d-a45f-f03afd458eaf\",\"updatedDate\":\"2025-08-26T16:14:25.385Z\"},{\"createdDate\":\"2025-08-26T16:15:01.752Z\",\"entityDataVersion\":3243272446,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|75ca49ea-eb88-4473-b029-75a55846048d|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-cplwgn-98f56f82-560c-4b69-b18e-86e1c1ff7aeb_151_9068\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":\"3deafd0f-f794-4535-9fa5-ae6c76efe0da\",\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:15:01.752Z\",\"stateName\":\"ChargeCompleted\",\"stateTransition\":\"ChargePending:ChargeCompleted\",\"transactionStateId\":\"75ca49ea-eb88-4473-b029-75a55846048d\",\"updatedDate\":\"2025-08-26T16:15:01.752Z\"},{\"createdDate\":\"2025-08-26T16:33:51.899Z\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|b6815ff3-6742-4c4b-b69a-fbf99ddb24fb|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"state\",\"lifecycleName\":\"CreditCardWith3DS\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"previousTransactionStateId\":null,\"schemaVersion\":\"1.0\",\"stateDate\":\"2025-08-26T16:33:51.899Z\",\"stateName\":\"ChargeCompleted\",\"stateTransition\":null,\"transactionStateId\":\"b6815ff3-6742-4c4b-b69a-fbf99ddb24fb\",\"updatedDate\":\"2025-08-26T16:33:51.899Z\"}],\"transaction\":{\"bankId\":null,\"bankIdentificationNumber\":\"403476\",\"billingMerchantCode\":\"ThirdParty\",\"bookingDirectoryParentId\":null,\"brandName\":\"Visa\",\"businessLiabilityCategory\":null,\"chargeId\":null,\"cardPresentCode\":\"WebPaymentByCustomer\",\"clientRequestGuid\":\"80224b87-a1ce-4fcf-8e03-8f6fad71d73c:100000\",\"clientTransactionId\":\"e35c6097-5829-4c0c-9c3a-57f7c03b34d7\",\"collectingLegalEntity\":null,\"companyCode\":\"10126\",\"createdDate\":\"2025-08-26T16:14:24.103Z\",\"eapid\":\"0\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"transaction\",\"expediaPurchaseTypeId\":\"21\",\"gpid\":\"0\",\"instrumentVerificationDataId\":\"6d7532f0-5a2b-4105-bf2c-a4d3a8f3ff8c\",\"instrumentVerificationType\":\"cvv\",\"itineraryBookingDirectoryId\":null,\"jurisdiction\":\"USA\",\"languageCode\":\"en\",\"lastFourDigits\":\"7288\",\"lifeCycleModelVersion\":\"V1\",\"locale\":\"en_US\",\"managementUnit\":\"1255\",\"mandateAcceptanceDate\":\"null\",\"mandateId\":null,\"mandateType\":null,\"numberOfInstallments\":null,\"orderId\":\"-9223371999231674674\",\"orderNumber\":\"9076832532564\",\"orderOperationCorrelationId\":null,\"paymentAllocationRecordRefCode\":null,\"paymentDescription\":\"Itinerary: 73221117050482, Start Date: 10/7/2025, End Date: 10/8/2025, Contains: Agency Air\",\"paymentEndDate\":\"null\",\"paymentInstrumentId\":\"90a9fca1-b36e-7710-f8f2-fc03bb9871f3\",\"paymentInstrumentService\":\"PV\",\"paymentIntentId\":null,\"paymentItemCorrelationId\":\"e1849640-1f57-4ba8-8ac3-25f73d63b118\",\"paymentPlanCorrelationId\":\"f6957b2b-8a54-4725-8cea-16a0fd58985a\",\"paymentPlanId\":\"a3e6a9e0-43a7-x67b-af2a-4ec0aeeb06ab\",\"paymentProcessorClientGuid\":\"payorch\",\"paymentProviderId\":\"12\",\"paymentReasonCode\":null,\"paymentReasonId\":\"0\",\"paymentSubType\":\"Visa\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"paymentType\":\"CreditCard\",\"pointOfSaleOrderReferenceNumber\":\"73221117050482\",\"proxyForPayerId\":\"0\",\"scheduledDate\":\"null\",\"schemaVersion\":\"1.0\",\"siteId\":null,\"transactionModelName\":\"TwoStepCommit\",\"transferType\":null,\"travServerDbInstance\":\"TravServerUS\",\"travelProductId\":\"80001\",\"trl\":\"421159069\",\"tuid\":\"744159015\",\"tuidLogon\":\"744159015\",\"updatedDate\":\"2025-08-26T16:15:01.658Z\",\"partnerTransactionId\":null,\"platformTransactionId\":null,\"partnerAccountId\":null,\"paymentPlanRoutingId\":\"ern:pay:ptx:r2::60566b68-85c4-0a43-27df-169fc5cb285f\"},\"transactionAttributes\":{\"descriptorAttributes\":[{\"attributeName\":\"DESCRIPTOR\",\"attributeTypeCode\":\"DESCRIPTOR\",\"createdDate\":\"null\",\"descriptorPhone\":\"tvly.com \",\"descriptorText\":\"TRAVELOCITY*73221117050482\",\"entityDataVersion\":3243270965,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|DESCRIPTOR|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c6p5jw-ed84cd28-fe3f-47e0-a445-139c6a6b8dca_132_8899\",\"entityType\":\"attribute\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:14:25.385Z\"}],\"stringAttributes\":[{\"attributeName\":\"PaymentProviderID\",\"attributeTypeCode\":\"STRING\",\"createdDate\":\"null\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|PaymentProviderID|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"attribute\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:33:51.899Z\",\"value\":\"12\"},{\"attributeName\":\"SupplierMerchantName\",\"attributeTypeCode\":\"STRING\",\"createdDate\":\"null\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|SupplierMerchantName|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"attribute\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:33:51.899Z\",\"value\":\"British Airways\"},{\"attributeName\":\"LegacyPaymentType\",\"attributeTypeCode\":\"STRING\",\"createdDate\":\"null\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|LegacyPaymentType|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"attribute\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:33:51.899Z\",\"value\":\"Full\"}],\"taggingAttributes\":[{\"attributeName\":\"ManagementUnit\",\"attributeTypeCode\":\"TAGGING\",\"createdDate\":\"null\",\"entityDataVersion\":3243315992,\"entityId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf|ManagementUnit|\",\"entitySequenceId\":\"pay-payin-executor-service-prod-us-west-2-prod-primary-2-c9pvrx-a6b87555-8ceb-480a-bea8-1697c5a6f9ca_132_4982\",\"entityType\":\"attribute\",\"paymentTransactionId\":\"5e5cbc48-29ea-0565-2bb4-f7370681ecbf\",\"schemaVersion\":\"1.0\",\"updatedDate\":\"2025-08-26T16:33:51.899Z\",\"value\":\"1255\"}],\"threeDSAttributes\":null}}] ,\"scrollId\":null,\"totalHits\":1}],\"paymentInstruments\":[{\"paymentInstrumentID\":\"90A9FCA1-B36E-7710-F8F2-FC03BB9871F3\",\"paymentMethod\":\"BankIssuedCard\",\"paymentSubMethod\":\"Visa\",\"brandName\":\"Visa\",\"cardNumber\":null,\"cardType\":\"CREDIT\",\"token\":\"XXXX\",\"instrumentVerificationType\":null,\"instrumentDescription\":null,\"expirationDate\":\"XXXX\",\"bin\":\"403476\",\"last4Digits\":\"XXXX\",\"bankRoutingNumber\":null,\"bankAccountNumber\":null,\"bankId\":null,\"bankBranchCode\":null,\"mandateType\":null,\"mandateID\":null,\"mandateAcceptanceDate\":null,\"pin\":null,\"accountId\":null,\"linkedPaymentInstruments\":null,\"customerInfo\":{\"personName\":{\"personalTitle\":null,\"firstName\":\"XXXX\",\"middleName\":null,\"lastName\":\"XXXX\",\"suffixName\":null},\"phoneNumber\":{\"phoneCategoryCode\":null,\"phoneCountryCode\":\"XXXX\",\"phoneAreaCode\":\"XXXX\",\"phoneNumber\":\"XXXX\",\"phoneExtensionNumber\":null},\"address\":{\"addressCategoryCode\":null,\"companyNameAddressLine\":null,\"firstAddressLine\":\"XXXX\",\"secondAddressLine\":null,\"thirdAddressLine\":null,\"fourthAddressLine\":null,\"fifthAddressLine\":null,\"cityName\":\"XXXX\",\"provinceName\":\"XXXX\",\"postalCode\":\"XXXX\",\"countryCode\":\"XXXX\",\"personName\":null,\"phoneNumber\":null,\"addressStatus\":null},\"emailAddress\":\"XXXX\",\"taxId\":null,\"taxIdType\":null,\"payerId\":null,\"payerStatus\":null,\"payerCountry\":null,\"payerBusiness\":null,\"cardHolderName\":null},\"links\":null,\"presentedFormOfPayment\":null}]} @RequestPayload={\"paymentInstrumentId\":null,\"paymentIntentIds\":null,\"itineraryNumber\":\"73221117050482\",\"travelProductId\":\"80001\",\"merchantOrderCode\":null,\"tuid\":null,\"internalReferenceId\":null,\"acquirerReferenceNumber\":null,\"userId\":null}" ... View more

As @gcusello indicates, you will need a common/correlation field between host message and router message.  It is best for you to illustrate your data, even mock data, and illustrate how you want the output to be. (You have just explained the logic between input and desired output.  This is good.) Short of your illustrations, let me assume that the common field in host message (sourcetype=host_message) is named gateway, and  that in router message (sourcetype=router_message) is named side_a.  Assume further that there is a field named status in both. (But statuses are independent despite a common name.) While the above assumptions about data are reasonable, it is even more critical to have your input about your use case, i.e., desired results.  Here, I imagine that you want Splunk to output all host failure messages if their common gateway is up, but if a common gateway is down, you want to suppress host failure messages, only output router failure message. This is how you should describe your use case using data: host: gateway host sourcetype status 10.0.0.1 host1 host_message up 10.0.0.1 host2 host_message down 10.0.1.1 host3 host_message down 10.0.0.1 host4 host_message up 10.0.1.1 host5 host_message down 10.0.0.1 host6 host_message down 10.0.1.1 host7 host_message down router: host side_a sourcetype status router1 10.0.0.1 router_message up router2 10.0.1.1 router_message down Desired results: gateway host side_a sourcetype status 10.0.0.1 host2   host_message down 10.0.0.1 host6   host_message down   router2 10.0.1.1 router_message down Logic between data and desired results: When a gateway router is down, I don't want to see host failure messages using that gateway. This is a search to get such output from illustrated data: sourcetype IN (host_message, router_message) status = down | eval down_router = if(sourcetype == "router_message", host, null()) | eval router = coalesce(gateway, side_a) | eventstats values(down_router) as down_router by router | where isnull(down_router) or sourcetype == "router_message" | fields - down_router router Here is an emulation for you to play with and compare with real data. | makeresults format=csv data="host,gateway,status host1, 10.0.0.1, up host2, 10.0.0.1, down host3, 10.0.1.1, down host4, 10.0.0.1, up host5, 10.0.1.1, down host6, 10.0.0.1, down host7, 10.0.1.1, down" | eval sourcetype = "host_message" | append [ makeresults format=csv data="host, side_a, status router1, 10.0.0.1, up router2, 10.0.1.1, down" | eval sourcetype = "router_message"] | search status = down ``` the above emulates sourcetype IN (host_message, router_message) status = down ```   ... View more

Thank you for clarifying your question with mock code. Is there a way to limit the number of timestamps per record to 1?  Or is there a way to somehow designate which event type the timestamp should be pulled from? First, are teh two values of timestampStr represent the same time?  If yes, the answer is easy.  Just use the value. If the two strings are different, only YOU can answer the question correctly.  What is the logic of using one vs the other?  Is one "better" than the other?  If it doesn't matter which, you can use first(timestampStr), last(timestampStr), earliest(timestampStr), latest(timestampStr), or any number of stats functions that yields a single value. However, if you have a preference, you must describe your preference before any volunteer can help here. ... View more

I agree with @livehybrid and @ITWhisperer: you do NOT go with union.  Especially because you haven't shown anything in your use case that cannot be achieved using what @livehybrid suggests.  It takes some practice to stay away from SQL-like constructs but you will reap the benefit along the way. Collect some data from your various criteria (can be mock data), and illustrate the result you want from these data.  People here will for sure show you a much more efficient search without reaching for union. SPL does have a union command.  But unless there is no alternative, just don't touch it. ... View more

This eventstats needs to be grouped by app | eventstats dc(version) as number_of_versions by AppName | where number_of_versions>1 ... View more

I believe that you are looking in a Splunk views.  Across nearly all interfaces, Splunk always displays the field named _time as calendar time.  You can try this to see if that field is epic or string: | eval question = if(isnum(_time), "is", "is not") . " numeric" You can also display arbitrary numeric value as calendar entry.  For example | makeresults format=csv data="anynumber 1234" | eval _time = anynumber _time anynumber 1969-12-31 16:20:34 1234 ... View more

Building a lookup is perhaps the easiest.  Assuming domain is already extracted, you could do sourcetype = mailstuff domain=* earliest=0 | stats min(_time) as first_seen by domain | output lookup DomainFirstAppeared  Of course, you need to define lookup DomainFirstAppeared. After this, you can add this field first_seen in any search using lookup command, like this sourcetype = mailstuff | lookup DomainFirstAppeared domain   ... View more

Let's go back to the basics: To ask an answerable data analytics question, follow these golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search (SPL that volunteers here do not have to look at). Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. The last point is perhaps moot because you do not have error.  But you still need to explain what is the difference between what you want achieved is different from what chain search is designed for. ... View more

Exactly.  If the raw event contains both non-JSON and JSON components, extract that JSON component first, then use spath to extract keys from JSON. ... View more

I'm not sure if I fully understand the question.  But first of all, do not use join.  Second, if you must use join, just create a chain search to invoke that join.  Place your first search in a main search - in the editor, this is called a Search.  When you create chain search, the editor asks you to select a parent search. As you have already set up the main search, all you do is to put the rest of search into a chain search that chains to that main search.  You do not "reference" a base search.  You chain to it.  Like this After you make that selection, your main search will be displayed in a grayed area Thirdly, do not use join if you can help it. ... View more

 I want to divide Scada_count/dda_count. This is my use case. You haven't answered the real question.  To quote myself: > However, your first search, if it has output, is dominated by | stats count as Scada_count by Area.  This means that if there are more than one values for Area, this search will have more than one row of Scada_count. (If there is ever only one value of Area, why bother group by Area?) On the other hand, the second search can ever only produce one row of dda_count.  What exactly do you expect append to achieve? Here is another quote: To ask an answerable data analytics question, follow these golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search (SPL that volunteers here do not have to look at). Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. ... View more

Instead of just illustrating all SPLs, the best way to obtain concrete help is to first illustrate data - in your case, sample output from svc_summary - even data mockups to illustrate your observed variations, then illustrate the results you want from illustrated data, then explain the logic between data and desired results.  Like @livehybrid said, summary index or _internal is not the problem.  The problem is that volunteers here have difficulty understand your business logic. ... View more

In addition to what @PickleRick already points out, you need to clearly describe your use case - as opposed to throwing complex SPL searches at volunteers if you want to receive concrete help.  What kind of data are in indices si_error and internal_statistics? (Exemplify dataset.) What exactly do you expect as a result of "append" - or any other command? (Expected results.) What is the logic that connects your dataset to expected results?  As @SanjayReddy notes, there is no reason to expect "append" to not "always giving me only one section of the results."  You didn't even show sample results from each search to prove that "append" or any other command should give you more than "only one section of the results." I want to dig deeper on logic.  @SanjayReddy assumes that you want two fields (columns) Scada_count and dda_count in the same rows.  However, your first search, if it has output, is dominated by | stats count as Scada_count by Area.  This means that if there are more than one values for Area, this search will have more than one row of Scada_count. (If there is ever only one value of Area, why bother group by Area?) On the other hand, the second search can ever only produce one row of dda_count.  What exactly do you expect append to achieve? ... View more

You query shows the correct result but the fields are not in a order how I want to display. The easiest way to get the display order you wanted is to use table command, like | table _time API Total AvgDur TXN_1000 1sec-2sec 2sec-5sec 5sec+  @ITWhisperer already pointed out the mistake in your data emulation that is fixed by strptime.  The two method gives exactly the same result.  The only difference is in field names, which can easily be coordinated. ... View more

Your "table" must come with a sequence or the whole problem is unsolvable.  The sequence may come in the form of a _time field, or a special field such as sequence_number, or in the form of sheer order of the table. The whole point is, you can use transaction command to get what you need if your table Has a _time field and Is in reverse time order. | transaction "change #" "user ID" startswith="Mod_type=OLD" endswith="Mod_type=NEW" If, for any reason, your "table" doesn't come with a _time field, you can always make sure it is in reverse time order, and make up a _time field.  You can also use stats to do the same.  The bottom line is: join is seldom the answer. Here is a data emulation for you to play with and compare with real data. | makeresults format=csv data="Mod_type, user ID,Email,change #,Active NEW,123,Me@hotmail.com,152,Yes OLD,123,Me@hotmail.com,152,No" | eval _time = now() ... View more

I know you accepted an answer in Is there a way to group the data by time and another field? There is a simpler, perhaps more intuitive approach. index=my_app sourcetype=my_logs:hec (source=my_Logger) msgsource="*" msgtype="*MyClient*" host=* [ inputlookup My_Application_Mapping.csv | search Client="SomeBank" | table appl ] | eval total_milliseconds = 1000 * (strptime("00:" . elapsed, "%T.%N") - relative_time(now(), "-0d@d")) | eval timebucket = case(total_milliseconds <= 1000, "TXN_1000", total_milliseconds <= 2000, "1sec-2sec", total_milliseconds <= 5000, "2sec-5sec", true(), "5sec+") | rename msgsource as API | bucket _time span=1d | eventstats avg(total_milliseconds) as AvgDur by _time API | stats count by AvgDur _time API timebucket | tojson output_field=api_time _time API AvgDur | chart values(count) over api_time by timebucket | addtotals | spath input=api_time | rename time as _time | fields - api_time Main ideas: Organizing time bucket by case function is easier to maintain. Use chart command to perform transpose over one composite field (api_time). Use tojson to pack all information needed after transpose. In addition, using strptime to calculate total_milliseconds is more maintainable. (It would be even simpler if Splunk doesn't have a bug near epoc zero.) ... View more

| inputlookup append=t myhosts.csv Good suggestion.  I always forget the append option🙂 ... View more

Tried this, but I get a new row in table3.csv That is exactly what appendpipe is supposed to do.  If this is not acceptable, do not use appendpipe.  Have you tried fillnull @gcusello and I have suggested? ... View more

Make sure to apply fillnull before table | inputlookup table3.csv | fillnull field1,field2,comment3 value=- | table field1,field2,comment3 | outputlookup table3.csv   ... View more