JSON files truncated in index time (I think)

[3DGjos]

Hello,

I'm having problems parsing our client's JSON events. When I add them locally with the Add data menu, it works just fine. Even if I index them, I search on my test index and the parsing works fine.

When I tried doing that in our client's environment, I had the same result. In add data works fine, but the events start to break. (my ST is present at every component, universal forwarder, indexer and search head)

I think it may be some limits configuration related thing. Not sure if the problem is in the universal forwarder, in the indexer, or in the search head.

Here is my sourcetype:

[my_st]

BREAK_ONLY_BEFORE = ^{
NO_BINARY_CHECK = true
TIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%Q"
TIME_PREFIX = \"(?:inicio|timestamp)\"\:\s?\s?\s?
category = Structured
disabled = false
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 25
LINE_BREAKER = ^{
MAX_EVENTS = 999999
TRUNCATE = 0

here is a sample event:

{ "id": "c7a27d54-565d-480d-9647-6ebcccc184f1", "logid": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "server": "SAMPLESERVER001", "module": "Core", "moduleVersion": "", "VersaoApiCore": "v1.0.21", "rawPath": "/v1/SAMPLESX/v1/SAMPLESX/:offer_id", "inicio": "2019-02-27T12:28:39.240", "fim": "2019-02-27T12:28:39.630", "elapsed": 390, "status": 200, "route": { "path": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "method": "GET", "cached": "false" }, "messages": [ { "order": 0, "level": "info", "message": "PROCESSOR processor SAMPLE-samplex-processor requisitado" }, { "order": 1, "level": "debug", "message": "Requisitando o modulo lodash" }, { "order": 2, "level": "debug", "message": "Requisitando o modulo lodash" } ], "request": { "headers": [ { "name": "connection", "value": "keep-alive" }, { "name": "content-length", "value": "0" }, { "name": "content-type", "value": "application/json" }, { "name": "accept", "value": "application/vnd.SAMPLE.v1+json" }, { "name": "accept-encoding", "value": "gzip,deflate" }, { "name": "host", "value": "SAMPLESERVER001.SAMPLE.sample.corp.SAMPLEbank.com" }, { "name": "user-agent", "value": "Jersey/2.26 (Apache HttpClient 4.5.3)" }, { "name": "transaction_id", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "SAMPLE-chave", "value": "aa40f83b-260f-436d-8ded-cf1539735ef2" }, { "name": "SAMPLE-log-id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "session_id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "tirinha", "value": "mobile" }, { "name": "flowid", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "x-original-url", "value": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "query": [], "params": [ { "name": "offer_id", "value": "d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "body": [], "pool": false }, "response": { "headers": [ { "name": "x-powered-by", "value": "Express" }, { "name": "access-control-allow-origin", "value": "*" }, { "name": "access-control-allow-methods", "value": "GET,PUT,POST,DELETE,PATCH" }, { "name": "access-control-allow-headers", "value": "Content-Type" }, { "name": "content-type", "value": "application/json" } ], "status": 200, "content": { "data": { "packages": [ { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "3", "package_title": "Mix de Cuentas", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cuentas corrientes en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 2.500</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 15</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "4", "package_title": "Cuentas corrientes", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 2.500" }, "account_type": "CC" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "2", "package_title": "Cajas de ahorro", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "1", "package_title": "Caja de ahorro pesos", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] } ] } } }, "services": [ { "order": 0, "id": "root-service-SAMPLE-samplex-service", "service": "SAMPLE-samplex-service", "backend": "rest", "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/:offer_id", "metodo": "GET", "contingency": false, "envio": { "timestamp": "2019-02-27T12:28:39.255Z", "trancode": { "query": {}, "body": {}, "header": { "Content-Type": "application/json", "Accept": "application/json", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4", "Cache-Control": "no-cache", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36", "session_id": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "transaction_id": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "metodo": "GET", "envioXml": false } }, "resposta": { "timestamp": "2019-02-27T12:28:39.630Z", "trancode": { "data": "********************" } }, "elapsed": 375, "cached": false, "cache": { "forced": false, "refreshed": false }, "cacheDetails": {} } ], "servico": "posicao-core-response", "PID": 4776, "processIndex": 0 }

Thanks!

[3DGjos]

here is a sample event:

{ "id": "c7a27d54-565d-480d-9647-6ebcccc184f1", "logid": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "server": "SAMPLESERVER001", "module": "Core", "moduleVersion": "", "VersaoApiCore": "v1.0.21", "rawPath": "/v1/SAMPLESX/v1/SAMPLESX/:offer_id", "inicio": "2019-02-27T12:28:39.240", "fim": "2019-02-27T12:28:39.630", "elapsed": 390, "status": 200, "route": { "path": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "method": "GET", "cached": "false" }, "messages": [ { "order": 0, "level": "info", "message": "PROCESSOR processor SAMPLE-samplex-processor requisitado" }, { "order": 1, "level": "debug", "message": "Requisitando o modulo lodash" }, { "order": 2, "level": "debug", "message": "Requisitando o modulo lodash" } ], "request": { "headers": [ { "name": "connection", "value": "keep-alive" }, { "name": "content-length", "value": "0" }, { "name": "content-type", "value": "application/json" }, { "name": "accept", "value": "application/vnd.SAMPLE.v1+json" }, { "name": "accept-encoding", "value": "gzip,deflate" }, { "name": "host", "value": "SAMPLESERVER001.SAMPLE.sample.corp.SAMPLEbank.com" }, { "name": "user-agent", "value": "Jersey/2.26 (Apache HttpClient 4.5.3)" }, { "name": "transaction_id", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "SAMPLE-chave", "value": "aa40f83b-260f-436d-8ded-cf1539735ef2" }, { "name": "SAMPLE-log-id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "session_id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "tirinha", "value": "mobile" }, { "name": "flowid", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "x-original-url", "value": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "query": [], "params": [ { "name": "offer_id", "value": "d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "body": [], "pool": false }, "response": { "headers": [ { "name": "x-powered-by", "value": "Express" }, { "name": "access-control-allow-origin", "value": "*" }, { "name": "access-control-allow-methods", "value": "GET,PUT,POST,DELETE,PATCH" }, { "name": "access-control-allow-headers", "value": "Content-Type" }, { "name": "content-type", "value": "application/json" } ], "status": 200, "content": { "data": { "packages": [ { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "3", "package_title": "Mix de Cuentas", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cuentas corrientes en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 2.500</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 15</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "4", "package_title": "Cuentas corrientes", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 2.500" }, "account_type": "CC" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "2", "package_title": "Cajas de ahorro", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&oacute;lares tienen un promedio m&iacute;nimo exigido de <strong>U$S 750</strong>, por debajo del cual se te cobrar&aacute; una comisi&oacute;n de <strong>U$S 10</strong>." }, { "commission_description": "<strong>Hasta 8 transferencias</strong> dom&eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&ordm; retiro por SAMPLEREDed, se cobraran <strong>U$$ 1,5 c / u </strong>." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&aacute;nica ligera las 24 hs, los 365 d&iacute;as del a&ntilde;o, brindado por <strong>Auxicar</strong> (ver condiciones). Coordinalo llamando al <strong>1784</strong>.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&iacute;as, farmacias, supermercados, combustibles, almuerzos y m&aacute;s.", "category_id": "exclusivo" } ], "package_id": "1", "package_title": "Caja de ahorro pesos", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] } ] } } }, "services": [ { "order": 0, "id": "root-service-SAMPLE-samplex-service", "service": "SAMPLE-samplex-service", "backend": "rest", "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/:offer_id", "metodo": "GET", "contingency": false, "envio": { "timestamp": "2019-02-27T12:28:39.255Z", "trancode": { "query": {}, "body": {}, "header": { "Content-Type": "application/json", "Accept": "application/json", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4", "Cache-Control": "no-cache", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36", "session_id": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "transaction_id": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "metodo": "GET", "envioXml": false } }, "resposta": { "timestamp": "2019-02-27T12:28:39.630Z", "trancode": { "data": "********************" } }, "elapsed": 375, "cached": false, "cache": { "forced": false, "refreshed": false }, "cacheDetails": {} } ], "servico": "posicao-core-response", "PID": 4776, "processIndex": 0 }

[woodcock]

Try this (DO NOT MODIFY/ADD ANY SETTINGS):

[<ORIGINAL_SOURCETYPE_HERE>]
LINE_BREAKER = ^{
SHOULD_LINEMERGE = false
TIME_PREFIX = \"(?:inicio|timestamp)\"\:\s?\s?\s?
TIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%Q"
MAX_TIMESTAMP_LOOKAHEAD = 25
MAX_EVENTS = 999999
TRUNCATE = 0
NO_BINARY_CHECK = true
category = Structured
disabled = false
pulldown_type = true

If you have overriden/overwritten the sourcetype, use the ORIGINAL sourcetype.
Deploy to the first full instance of Splunk that handles the events (HF or Indexer tier).
Restart all Splunk instances there.
Only new events will be correct (old events will stay forever broken).
When validating, use All time for the Time picker and add index_earliest=-5m to your search string.

[3DGjos]

Hello, thanks for your answer!

It didn't work. I have a configuration that works. It works on my lab, it works when I go to "add data" on my client's server, I have the props in the forwarder-indexer-SH, but the result is the same, some events get truncated. I will put an example.

Thanks

[n0str0m08]

Hi,

Please check LOOKAHEAD attribute in transforms.conf file in a place where events are parsed. In your case, with UF only, it should be on the Indexer. A default value for this attribute is 4096 characters and everything above is truncated. Just increase it to e.g. 10240 and check if it helped.

props.conf

[my_st]
...
REPORT-lookhead = change_lookahead

transforms.conf

[change_lookahead]
LOOKAHEAD = 10240

From Splunk Doc: transforfms.conf

LOOKAHEAD = <integer>
* NOTE: This option is valid for all index time transforms, such as
  index-time field creation, or DEST_KEY modifications.
* Optional. Specifies how many characters to search into an event.
* Default: 4096
  * You may want to increase this value if you have event line lengths that
    exceed 4096 characters (before linebreaking).

Regards

[3DGjos]

Thanks for your answer, In the next webex I have with my client i will try.
Can I put the LOOKAHEAD parameter in system/local of the indexer?

btw, I found some other parameters to try in limits.conf, like:

    extraction_cutoff = 5000
    limit
    maxchars
    maxvalues

my version is 6.6.3

[n0str0m08]

I have corrected my post as tranforms.conf has to be called by props.conf

Yes, these files can be placed in 'system/local'. However, bear in mind that in case of other configurations this folder has almost always the highest precedence.

Expand truncated event and from dropdown menu Event Action click on Show Source -> It will show you a raw events. Check if event is truncated or not. If it is then you are sure that it was done during index time. How many characters is in a single, truncated event? Copy/paste truncated event to e.g. notepad++ and check the lenght of event. It will give you an information about a value of limit and attribute you have to look for. It will narrow a bit a field of troubleshooting.

I have never had to change these attributes listed by you. However, please find below my insights about them:

extraction_cutoff - it seems to be an attribute applied during search-time,
maxchars - the same as above,
limit - do you extract fields during index-time besides the default one? If not, this also is applied during search-time
maxvalues - default value is 100000 what is quite a big value. I would look on it as the last point to check.

[lakshman239]

can you share sample data (pls remove any sensitive data)