Hello,
I'm having problems parsing our client's JSON events. When I add them locally with the Add data menu, it works just fine. Even if I index them, I search on my test index and the parsing works fine.
When I tried doing that in our client's environment, I had the same result. In add data works fine, but the events start to break. (my ST is present at every component, universal forwarder, indexer and search head)
I think it may be some limits configuration related thing. Not sure if the problem is in the universal forwarder, in the indexer, or in the search head.
Here is my sourcetype:
[my_st]
BREAK_ONLY_BEFORE = ^{
NO_BINARY_CHECK = true
TIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%Q"
TIME_PREFIX = \"(?:inicio|timestamp)\"\:\s?\s?\s?
category = Structured
disabled = false
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 25
LINE_BREAKER = ^{
MAX_EVENTS = 999999
TRUNCATE = 0
here is a sample event:
{ "id": "c7a27d54-565d-480d-9647-6ebcccc184f1", "logid": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "server": "SAMPLESERVER001", "module": "Core", "moduleVersion": "", "VersaoApiCore": "v1.0.21", "rawPath": "/v1/SAMPLESX/v1/SAMPLESX/:offer_id", "inicio": "2019-02-27T12:28:39.240", "fim": "2019-02-27T12:28:39.630", "elapsed": 390, "status": 200, "route": { "path": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "method": "GET", "cached": "false" }, "messages": [ { "order": 0, "level": "info", "message": "PROCESSOR processor SAMPLE-samplex-processor requisitado" }, { "order": 1, "level": "debug", "message": "Requisitando o modulo lodash" }, { "order": 2, "level": "debug", "message": "Requisitando o modulo lodash" } ], "request": { "headers": [ { "name": "connection", "value": "keep-alive" }, { "name": "content-length", "value": "0" }, { "name": "content-type", "value": "application/json" }, { "name": "accept", "value": "application/vnd.SAMPLE.v1+json" }, { "name": "accept-encoding", "value": "gzip,deflate" }, { "name": "host", "value": "SAMPLESERVER001.SAMPLE.sample.corp.SAMPLEbank.com" }, { "name": "user-agent", "value": "Jersey/2.26 (Apache HttpClient 4.5.3)" }, { "name": "transaction_id", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "SAMPLE-chave", "value": "aa40f83b-260f-436d-8ded-cf1539735ef2" }, { "name": "SAMPLE-log-id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "session_id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "tirinha", "value": "mobile" }, { "name": "flowid", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "x-original-url", "value": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "query": [], "params": [ { "name": "offer_id", "value": "d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "body": [], "pool": false }, "response": { "headers": [ { "name": "x-powered-by", "value": "Express" }, { "name": "access-control-allow-origin", "value": "*" }, { "name": "access-control-allow-methods", "value": "GET,PUT,POST,DELETE,PATCH" }, { "name": "access-control-allow-headers", "value": "Content-Type" }, { "name": "content-type", "value": "application/json" } ], "status": 200, "content": { "data": { "packages": [ { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en dólares tienen un promedio mínimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrará una comisión de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> domésticas online sin costo." }, { "commission_description": "A partir del 9º retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompañante es nuestro invitado, con pop y refresco de regalo. Tenés caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompañanate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mecánica ligera las 24 hs, los 365 días del año, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papelerías, farmacias, supermercados, combustibles, almuerzos y más.", "category_id": "exclusivo" } ], "package_id": "3", "package_title": "Mix de Cuentas", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cuentas corrientes en dólares tienen un promedio mínimo exigido de <strong>U$S 2.500</strong>, por debajo del cual se te cobrará una comisión de <strong>U$S 15</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> domésticas online sin costo." }, { "commission_description": "A partir del 9º retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompañante es nuestro invitado, con pop y refresco de regalo. Tenés caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompañanate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mecánica ligera las 24 hs, los 365 días del año, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papelerías, farmacias, supermercados, combustibles, almuerzos y más.", "category_id": "exclusivo" } ], "package_id": "4", "package_title": "Cuentas corrientes", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 2.500" }, "account_type": "CC" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en dólares tienen un promedio mínimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrará una comisión de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> domésticas online sin costo." }, { "commission_description": "A partir del 9º retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompañante es nuestro invitado, con pop y refresco de regalo. Tenés caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompañanate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mecánica ligera las 24 hs, los 365 días del año, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papelerías, farmacias, supermercados, combustibles, almuerzos y más.", "category_id": "exclusivo" } ], "package_id": "2", "package_title": "Cajas de ahorro", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en dólares tienen un promedio mínimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrará una comisión de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> domésticas online sin costo." }, { "commission_description": "A partir del 9º retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompañante es nuestro invitado, con pop y refresco de regalo. Tenés caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompañanate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mecánica ligera las 24 hs, los 365 días del año, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papelerías, farmacias, supermercados, combustibles, almuerzos y más.", "category_id": "exclusivo" } ], "package_id": "1", "package_title": "Caja de ahorro pesos", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] } ] } } }, "services": [ { "order": 0, "id": "root-service-SAMPLE-samplex-service", "service": "SAMPLE-samplex-service", "backend": "rest", "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/:offer_id", "metodo": "GET", "contingency": false, "envio": { "timestamp": "2019-02-27T12:28:39.255Z", "trancode": { "query": {}, "body": {}, "header": { "Content-Type": "application/json", "Accept": "application/json", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4", "Cache-Control": "no-cache", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36", "session_id": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "transaction_id": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "metodo": "GET", "envioXml": false } }, "resposta": { "timestamp": "2019-02-27T12:28:39.630Z", "trancode": { "data": "********************" } }, "elapsed": 375, "cached": false, "cache": { "forced": false, "refreshed": false }, "cacheDetails": {} } ], "servico": "posicao-core-response", "PID": 4776, "processIndex": 0 }
Thanks!
... View more