DBConnect Events Truncate Despite props.conf

[graememeyer]

It seems that DBConnect inputs does no respect the props.conf configuration for event truncation.

Example props:

[epo]
LINE_BREAKER_LOOKBEHIND = 100000
TRUNCATE = 0
MAX_EVENTS = 100000

(Applied in System + Local).

/answers/225697 has a reply suggesting to use [source::...tpl_*.dbmonevt], but I am unsure what this is and a child reply suggests that this isn't effective either.

Does anyone know of a way to get DBConnect to stop truncating the events?

[graememeyer · Accepted solution]

Copying my solution here in case anyone else needs it:

Set TRUNCATE=0 in the following locations:

/opt/splunk_hf/etc/apps/dbx/default

[source::...kv_*.dbmonevt]
SHOULD_LINEMERGE = false
HEADER_MODE = firstline
LINE_BREAKER = ([\r\n]+)
priority = 101
TRUNCATE=0


/opt/splunk_hf/etc/apps/dbx/local

[sourcetype]
LINE_BREAKER_LOOKBEHIND = 100000
TRUNCATE = 0
MAX_EVENTS = 100000


/opt/splunk_hf/etc/system/local

[sourcetype]
TRUNCATE = 0

Replace "sourcetype" with whatever your sourcetype is called (mine was "epo")

[cpetterborg]

How are the event getting truncated? Are you losing columns, or are you losing data from the middle of a column? How many characters are you getting in each event? Etc.

Is epo the sourcetype?

[graememeyer]

EPO is the sourcetype, events are being truncated at 10K characters and it's cutting off in the middle of a column, but not printing the remaining columns afterwards.

[cpetterborg]

Look at this article and see if it has an answer for your:

http://answers.splunk.com/answers/86931/why-does-the-splunk-indexes-only-50-fields-out-of-120-fields...

[graememeyer]

Thanks cpetterborg but I solved the issue. Seems there's another props file in /opt/splunk_hf/etc/apps/dbx/default and you have to set TRUNCATE=0 in the [source::...kv_*.dbmonevt] sourcetype as well.

So for anyone coming in from Google, I have set TRUNCATE=0 in the following locations:

/opt/splunk_hf/etc/apps/dbx/default

[source::...kv_*.dbmonevt]
SHOULD_LINEMERGE = false
HEADER_MODE = firstline
LINE_BREAKER = ([\r\n]+)
priority = 101
TRUNCATE=0


/opt/splunk_hf/etc/apps/dbx/local

[sourcetype]
LINE_BREAKER_LOOKBEHIND = 100000
TRUNCATE = 0
MAX_EVENTS = 100000


/opt/splunk_hf/etc/system/local

[sourcetype]
TRUNCATE = 0

And that's done the trick 🙂

Replace "sourcetype" with whatever your sourcetype is called (mine was "epo")