<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: JSON files truncated in index time (I think) in Getting Data In</title>
    <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417853#M73742</link>
    <description>&lt;P&gt;Try this (DO NOT MODIFY/ADD ANY SETTINGS): &lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;[&amp;lt;ORIGINAL_SOURCETYPE_HERE&amp;gt;]
LINE_BREAKER = ^{
SHOULD_LINEMERGE = false
TIME_PREFIX = \"(?:inicio|timestamp)\"\:\s?\s?\s?
TIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%Q"
MAX_TIMESTAMP_LOOKAHEAD = 25
MAX_EVENTS = 999999
TRUNCATE = 0
NO_BINARY_CHECK = true
category = Structured
disabled = false
pulldown_type = true
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;If you have overriden/overwritten the sourcetype, use the ORIGINAL sourcetype.&lt;BR /&gt;
Deploy to the first full instance of Splunk that handles the events (HF or Indexer tier).&lt;BR /&gt;
Restart all Splunk instances there.&lt;BR /&gt;
Only new events will be correct (old events will stay forever broken).&lt;BR /&gt;
When validating, use &lt;CODE&gt;All time&lt;/CODE&gt; for the &lt;CODE&gt;Time picker&lt;/CODE&gt; and add &lt;CODE&gt;index_earliest=-5m&lt;/CODE&gt; to your search string.&lt;/P&gt;</description>
    <pubDate>Sun, 03 Mar 2019 03:44:54 GMT</pubDate>
    <dc:creator>woodcock</dc:creator>
    <dc:date>2019-03-03T03:44:54Z</dc:date>
    <item>
      <title>JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417848#M73737</link>
      <description>&lt;P&gt;Hello, &lt;/P&gt;

&lt;P&gt;I'm having problems parsing our client's JSON events. When I add them locally with the Add data menu, it works just fine. Even if I index them, I search on my test index and the parsing works fine. &lt;/P&gt;

&lt;P&gt;When I tried doing that in our client's environment, I had the same result. In add data works fine, but the events start to break. (my ST is present at every component, universal forwarder, indexer and search head)&lt;/P&gt;

&lt;P&gt;I think it may be some limits configuration related thing. Not sure if the problem is in the universal forwarder, in the indexer, or in the search head.&lt;/P&gt;

&lt;P&gt;Here is my sourcetype:&lt;/P&gt;

&lt;P&gt;[my_st]&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;BREAK_ONLY_BEFORE = ^{
NO_BINARY_CHECK = true
TIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%Q"
TIME_PREFIX = \"(?:inicio|timestamp)\"\:\s?\s?\s?
category = Structured
disabled = false
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 25
LINE_BREAKER = ^{
MAX_EVENTS = 999999
TRUNCATE = 0
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;here is a sample event:&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;{ "id": "c7a27d54-565d-480d-9647-6ebcccc184f1", "logid": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "server": "SAMPLESERVER001", "module": "Core", "moduleVersion": "", "VersaoApiCore": "v1.0.21", "rawPath": "/v1/SAMPLESX/v1/SAMPLESX/:offer_id", "inicio": "2019-02-27T12:28:39.240", "fim": "2019-02-27T12:28:39.630", "elapsed": 390, "status": 200, "route": { "path": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "method": "GET", "cached": "false" }, "messages": [ { "order": 0, "level": "info", "message": "PROCESSOR processor SAMPLE-samplex-processor requisitado" }, { "order": 1, "level": "debug", "message": "Requisitando o modulo lodash" }, { "order": 2, "level": "debug", "message": "Requisitando o modulo lodash" } ], "request": { "headers": [ { "name": "connection", "value": "keep-alive" }, { "name": "content-length", "value": "0" }, { "name": "content-type", "value": "application/json" }, { "name": "accept", "value": "application/vnd.SAMPLE.v1+json" }, { "name": "accept-encoding", "value": "gzip,deflate" }, { "name": "host", "value": "SAMPLESERVER001.SAMPLE.sample.corp.SAMPLEbank.com" }, { "name": "user-agent", "value": "Jersey/2.26 (Apache HttpClient 4.5.3)" }, { "name": "transaction_id", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "SAMPLE-chave", "value": "aa40f83b-260f-436d-8ded-cf1539735ef2" }, { "name": "SAMPLE-log-id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "session_id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "tirinha", "value": "mobile" }, { "name": "flowid", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "x-original-url", "value": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "query": [], "params": [ { "name": "offer_id", "value": "d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "body": [], "pool": false }, "response": { "headers": [ { "name": "x-powered-by", "value": "Express" }, { "name": "access-control-allow-origin", "value": "*" }, { "name": "access-control-allow-methods", "value": "GET,PUT,POST,DELETE,PATCH" }, { "name": "access-control-allow-headers", "value": "Content-Type" }, { "name": "content-type", "value": "application/json" } ], "status": 200, "content": { "data": { "packages": [ { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 750&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 10&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "3", "package_title": "Mix de Cuentas", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cuentas corrientes en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 2.500&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 15&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "4", "package_title": "Cuentas corrientes", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 2.500" }, "account_type": "CC" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 750&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 10&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "2", "package_title": "Cajas de ahorro", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 750&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 10&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "1", "package_title": "Caja de ahorro pesos", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] } ] } } }, "services": [ { "order": 0, "id": "root-service-SAMPLE-samplex-service", "service": "SAMPLE-samplex-service", "backend": "rest", "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/:offer_id", "metodo": "GET", "contingency": false, "envio": { "timestamp": "2019-02-27T12:28:39.255Z", "trancode": { "query": {}, "body": {}, "header": { "Content-Type": "application/json", "Accept": "application/json", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4", "Cache-Control": "no-cache", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36", "session_id": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "transaction_id": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "metodo": "GET", "envioXml": false } }, "resposta": { "timestamp": "2019-02-27T12:28:39.630Z", "trancode": { "data": "********************" } }, "elapsed": 375, "cached": false, "cache": { "forced": false, "refreshed": false }, "cacheDetails": {} } ], "servico": "posicao-core-response", "PID": 4776, "processIndex": 0 }
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;Thanks!&lt;/P&gt;</description>
      <pubDate>Thu, 28 Feb 2019 14:00:50 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417848#M73737</guid>
      <dc:creator>3DGjos</dc:creator>
      <dc:date>2019-02-28T14:00:50Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417849#M73738</link>
      <description>&lt;P&gt;can you share sample data (pls remove any sensitive data)&lt;/P&gt;</description>
      <pubDate>Thu, 28 Feb 2019 14:08:04 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417849#M73738</guid>
      <dc:creator>lakshman239</dc:creator>
      <dc:date>2019-02-28T14:08:04Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417850#M73739</link>
      <description>&lt;P&gt;Hi,&lt;/P&gt;

&lt;P&gt;Please check &lt;CODE&gt;LOOKAHEAD&lt;/CODE&gt; attribute in &lt;CODE&gt;transforms.conf&lt;/CODE&gt; file in a place where events are parsed. In your case, with UF only, it should be on the Indexer. A default value for this attribute is &lt;CODE&gt;4096&lt;/CODE&gt; characters and everything above is truncated. Just increase it to e.g. &lt;CODE&gt;10240&lt;/CODE&gt; and check if it helped.&lt;/P&gt;

&lt;P&gt;&lt;STRONG&gt;props.conf&lt;/STRONG&gt;&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;[my_st]
...
REPORT-lookhead = change_lookahead
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;&lt;STRONG&gt;transforms.conf&lt;/STRONG&gt;&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;[change_lookahead]
LOOKAHEAD = 10240
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;From Splunk Doc: &lt;A href="https://docs.splunk.com/Documentation/Splunk/7.2.4/Admin/Transformsconf"&gt;transforfms.conf&lt;/A&gt;&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;LOOKAHEAD = &amp;lt;integer&amp;gt;
* NOTE: This option is valid for all index time transforms, such as
  index-time field creation, or DEST_KEY modifications.
* Optional. Specifies how many characters to search into an event.
* Default: 4096
  * You may want to increase this value if you have event line lengths that
    exceed 4096 characters (before linebreaking).
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;Regards&lt;/P&gt;</description>
      <pubDate>Thu, 28 Feb 2019 15:15:32 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417850#M73739</guid>
      <dc:creator>n0str0m08</dc:creator>
      <dc:date>2019-02-28T15:15:32Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417851#M73740</link>
      <description>&lt;P&gt;Thanks for your answer, In the next webex I have with my client i will try.&lt;BR /&gt;
Can I put the LOOKAHEAD parameter in system/local of the indexer?&lt;/P&gt;

&lt;P&gt;btw, I found some other parameters to try in limits.conf, like:&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;    extraction_cutoff = 5000
    limit
    maxchars
    maxvalues
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;my version is 6.6.3&lt;/P&gt;</description>
      <pubDate>Thu, 28 Feb 2019 20:32:32 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417851#M73740</guid>
      <dc:creator>3DGjos</dc:creator>
      <dc:date>2019-02-28T20:32:32Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417852#M73741</link>
      <description>&lt;P&gt;I have corrected my post as tranforms.conf has to be called by props.conf&lt;/P&gt;

&lt;P&gt;Yes, these files can be placed in 'system/local'. However, bear in mind that in case of other configurations this folder has almost always the highest precedence.&lt;/P&gt;

&lt;P&gt;Expand truncated event and from dropdown menu &lt;CODE&gt;Event Action&lt;/CODE&gt; click on &lt;CODE&gt;Show Source&lt;/CODE&gt; -&amp;gt; It will show you a raw events. Check if event is truncated or not. If it is then you are sure that it was done during index time. How many characters is in a single, truncated event? Copy/paste truncated event to e.g. notepad++ and check the lenght of event. It will give you an information about a value of limit and attribute you have to look for. It will narrow a bit a field of troubleshooting.&lt;/P&gt;

&lt;P&gt;I have never had to change these attributes listed by you. However, please find below my insights about them:&lt;/P&gt;

&lt;P&gt;&lt;CODE&gt;extraction_cutoff&lt;/CODE&gt; - it seems to be an attribute applied during search-time,&lt;BR /&gt;
&lt;CODE&gt;maxchars&lt;/CODE&gt; - the same as above,&lt;BR /&gt;
&lt;CODE&gt;limit&lt;/CODE&gt; -  do you extract fields during index-time besides the default one? If not, this also is applied during search-time&lt;BR /&gt;
&lt;CODE&gt;maxvalues&lt;/CODE&gt; - default value is 100000 what is quite a big value. I would look on it as the last point to check.&lt;/P&gt;</description>
      <pubDate>Fri, 01 Mar 2019 14:00:32 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417852#M73741</guid>
      <dc:creator>n0str0m08</dc:creator>
      <dc:date>2019-03-01T14:00:32Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417853#M73742</link>
      <description>&lt;P&gt;Try this (DO NOT MODIFY/ADD ANY SETTINGS): &lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;[&amp;lt;ORIGINAL_SOURCETYPE_HERE&amp;gt;]
LINE_BREAKER = ^{
SHOULD_LINEMERGE = false
TIME_PREFIX = \"(?:inicio|timestamp)\"\:\s?\s?\s?
TIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%Q"
MAX_TIMESTAMP_LOOKAHEAD = 25
MAX_EVENTS = 999999
TRUNCATE = 0
NO_BINARY_CHECK = true
category = Structured
disabled = false
pulldown_type = true
&lt;/CODE&gt;&lt;/PRE&gt;

&lt;P&gt;If you have overriden/overwritten the sourcetype, use the ORIGINAL sourcetype.&lt;BR /&gt;
Deploy to the first full instance of Splunk that handles the events (HF or Indexer tier).&lt;BR /&gt;
Restart all Splunk instances there.&lt;BR /&gt;
Only new events will be correct (old events will stay forever broken).&lt;BR /&gt;
When validating, use &lt;CODE&gt;All time&lt;/CODE&gt; for the &lt;CODE&gt;Time picker&lt;/CODE&gt; and add &lt;CODE&gt;index_earliest=-5m&lt;/CODE&gt; to your search string.&lt;/P&gt;</description>
      <pubDate>Sun, 03 Mar 2019 03:44:54 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417853#M73742</guid>
      <dc:creator>woodcock</dc:creator>
      <dc:date>2019-03-03T03:44:54Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417854#M73743</link>
      <description>&lt;P&gt;Hello, thanks for your answer!&lt;/P&gt;

&lt;P&gt;It didn't work. I have a configuration that works. It works on my lab, it works when I go to "add data" on my client's server, I have the props in the forwarder-indexer-SH, but the result is the same, some events get truncated. I will put an example.&lt;/P&gt;

&lt;P&gt;Thanks&lt;/P&gt;</description>
      <pubDate>Wed, 06 Mar 2019 12:36:39 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417854#M73743</guid>
      <dc:creator>3DGjos</dc:creator>
      <dc:date>2019-03-06T12:36:39Z</dc:date>
    </item>
    <item>
      <title>Re: JSON files truncated in index time (I think)</title>
      <link>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417855#M73744</link>
      <description>&lt;P&gt;here is a sample event:&lt;/P&gt;

&lt;PRE&gt;&lt;CODE&gt;{ "id": "c7a27d54-565d-480d-9647-6ebcccc184f1", "logid": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "server": "SAMPLESERVER001", "module": "Core", "moduleVersion": "", "VersaoApiCore": "v1.0.21", "rawPath": "/v1/SAMPLESX/v1/SAMPLESX/:offer_id", "inicio": "2019-02-27T12:28:39.240", "fim": "2019-02-27T12:28:39.630", "elapsed": 390, "status": 200, "route": { "path": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "method": "GET", "cached": "false" }, "messages": [ { "order": 0, "level": "info", "message": "PROCESSOR processor SAMPLE-samplex-processor requisitado" }, { "order": 1, "level": "debug", "message": "Requisitando o modulo lodash" }, { "order": 2, "level": "debug", "message": "Requisitando o modulo lodash" } ], "request": { "headers": [ { "name": "connection", "value": "keep-alive" }, { "name": "content-length", "value": "0" }, { "name": "content-type", "value": "application/json" }, { "name": "accept", "value": "application/vnd.SAMPLE.v1+json" }, { "name": "accept-encoding", "value": "gzip,deflate" }, { "name": "host", "value": "SAMPLESERVER001.SAMPLE.sample.corp.SAMPLEbank.com" }, { "name": "user-agent", "value": "Jersey/2.26 (Apache HttpClient 4.5.3)" }, { "name": "transaction_id", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "SAMPLE-chave", "value": "aa40f83b-260f-436d-8ded-cf1539735ef2" }, { "name": "SAMPLE-log-id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "session_id", "value": "cdd365b1-0b3e-418c-8d59-e3060eb859a0" }, { "name": "tirinha", "value": "mobile" }, { "name": "flowid", "value": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, { "name": "x-original-url", "value": "/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "query": [], "params": [ { "name": "offer_id", "value": "d7eced74-37be-4b4d-8da0-bf4e50f911b7" } ], "body": [], "pool": false }, "response": { "headers": [ { "name": "x-powered-by", "value": "Express" }, { "name": "access-control-allow-origin", "value": "*" }, { "name": "access-control-allow-methods", "value": "GET,PUT,POST,DELETE,PATCH" }, { "name": "access-control-allow-headers", "value": "Content-Type" }, { "name": "content-type", "value": "application/json" } ], "status": 200, "content": { "data": { "packages": [ { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 750&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 10&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "3", "package_title": "Mix de Cuentas", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cuentas corrientes en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 2.500&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 15&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "4", "package_title": "Cuentas corrientes", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CC", "overdraft_limit": { "currency": .sampleU", "amount": 25000, "decimals": 0 } }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 2.500" }, "account_type": "CC" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 50000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 750&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 10&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "2", "package_title": "Cajas de ahorro", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" }, { "currency": "USD", "general_info": { "product_type": "C", "product_description": "promedio m nimo - U$S 750" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] }, { "prices": [ { "currency": .sampleU", "amount": 600, "decimals": 0 }, { "currency": "UIs", "amount": 152, "decimals": 0 } ], "commissions": [ { "commission_description": "Las cajas de ahorro en d&amp;amp;oacute;lares tienen un promedio m&amp;amp;iacute;nimo exigido de &amp;lt;strong&amp;gt;U$S 750&amp;lt;/strong&amp;gt;, por debajo del cual se te cobrar&amp;amp;aacute; una comisi&amp;amp;oacute;n de &amp;lt;strong&amp;gt;U$S 10&amp;lt;/strong&amp;gt;." }, { "commission_description": "&amp;lt;strong&amp;gt;Hasta 8 transferencias&amp;lt;/strong&amp;gt; dom&amp;amp;eacute;sticas online sin costo." }, { "commission_description": "A partir del 9&amp;amp;ordm; retiro por SAMPLEREDed, se cobraran &amp;lt;strong&amp;gt;U$$ 1,5 c / u &amp;lt;/strong&amp;gt;." }, { "commission_description": "Operaciones por buzoneras sin costo." }, { "commission_description": "Algunas transacciones por cajero humano tienen costo." } ], "benefits": [ { "id": "Movie", "title": "en Movie", "type": "common", "benefit_description": "tu acompa&amp;amp;ntilde;ante es nuestro invitado, con pop y refresco de regalo. Ten&amp;amp;eacute;s caja preferencial para la compra de entradas y los viernes, en Teatro Movie tu acompa&amp;amp;ntilde;anate no paga.", "category_id": "cine" }, { "id": "Helados", "title": "2x1 en helados", "type": "common", "benefit_description": "de litro y cucuruchos grandes, en Freddo y Las Delicias.", "category_id": "helado" }, { "id": "Auxilio", "title": "auxilio mec nico", "type": "common", "benefit_description": "servicio de emergencia mec&amp;amp;aacute;nica ligera las 24 hs, los 365 d&amp;amp;iacute;as del a&amp;amp;ntilde;o, brindado por &amp;lt;strong&amp;gt;Auxicar&amp;lt;/strong&amp;gt; (ver condiciones). Coordinalo llamando al &amp;lt;strong&amp;gt;1784&amp;lt;/strong&amp;gt;.", "category_id": "auto" }, { "id": "exclusivas", "title": "descuentos exclusivos", "type": "common", "benefit_description": "en papeler&amp;amp;iacute;as, farmacias, supermercados, combustibles, almuerzos y m&amp;amp;aacute;s.", "category_id": "exclusivo" } ], "package_id": "1", "package_title": "Caja de ahorro pesos", "package_description": "Tarjeta de d bito", "account_SAMPLESX": [ { "currency": .sampleU", "general_info": { "product_type": "C", "product_description": "sin promedio m nimo" }, "account_type": "CA" } ], "card_SAMPLESX": [ { "limit": { "currency": .sampleU", "amount": 75000, "decimals": 0 }, "general_info": { "product_type": "TC" } } ] } ] } } }, "services": [ { "order": 0, "id": "root-service-SAMPLE-samplex-service", "service": "SAMPLE-samplex-service", "backend": "rest", "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/:offer_id", "metodo": "GET", "contingency": false, "envio": { "timestamp": "2019-02-27T12:28:39.255Z", "trancode": { "query": {}, "body": {}, "header": { "Content-Type": "application/json", "Accept": "application/json", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4", "Cache-Control": "no-cache", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36", "session_id": "cdd365b1-0b3e-418c-8d59-e3060eb859a0", "transaction_id": "14bb5314-1062-4b0b-995f-035e7924ae7f" }, "url": "https://desa.SAMPLE.com.sample/SAMPLECUENTAScuentas/SAMPLESX/v1/SAMPLESX/d7eced74-37be-4b4d-8da0-bf4e50f911b7", "metodo": "GET", "envioXml": false } }, "resposta": { "timestamp": "2019-02-27T12:28:39.630Z", "trancode": { "data": "********************" } }, "elapsed": 375, "cached": false, "cache": { "forced": false, "refreshed": false }, "cacheDetails": {} } ], "servico": "posicao-core-response", "PID": 4776, "processIndex": 0 }
&lt;/CODE&gt;&lt;/PRE&gt;</description>
      <pubDate>Wed, 06 Mar 2019 12:43:52 GMT</pubDate>
      <guid>https://community.splunk.com/t5/Getting-Data-In/JSON-files-truncated-in-index-time-I-think/m-p/417855#M73744</guid>
      <dc:creator>3DGjos</dc:creator>
      <dc:date>2019-03-06T12:43:52Z</dc:date>
    </item>
  </channel>
</rss>

