Why splunk logging truncates rather than wrapping to multiline?

[shandman]

I'm having issues getting this to work. I have played around with the props.conf but can't seem to get this going. pass an argument to props.conf truncate=0 .

Have tried several configuration attempts. Latest one is to change linemerge = false to linemerge = true .

After trying these it still chops off the event. Any ideas?

[eddit0r]

For the multi-line events you need to configure the linebreaking.

For the best performance use SHOULD_LINEMERGE = false & LINE_BREAKER in props.conf

See http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

When left to its own devices Splunk and SHOULD_LINEMERGE = true, Splunk will attempt to break on datestamps.

When using LINE_BREAKER there needs to be a capturing group in the regex - eg ([\r\n]+) the default is any number of new lines or carriage returns. That denotes the end of the event and the start of a new one - the captured data is removed.

For you, something like:
props.conf
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+).*(?:\[\d{4}-\d{1,2}-\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2},\d*\])
TRUNCATE = 0

That should break on new lines that contain the date/timestamp in the square brackets.
*note (?:xxxx) is a non-capturing regex group, that data is not removed.

[shandman]

I tried adding that stanza and it still isn't working. Here is the results from the search.
Splunk search criteria: source=udp:9514 host=broker-p* "Broker Registration Info"

broker-p3.vsp.com app=BrokerApp [2014-11-25 13:58:38,636] INFO {abcdZw7rGW9P_gAWe8ONu} AppointmentCompleteAction.execute: Broker Registration Info: com.vsp.broker.model.AppointmentFormInfo@7ce1034f[licenses={0=com.vsp.broker.model.BrokerLicense@1e42d36e[state=MD,licenseNum=164213,effectiveDate=com.vsp.portal.util.Chrono@3546ea47[day=1,month=6,year=2014,value=,format=MMMM|dd#yyyy]], 1=com.vsp.broker.model.BrokerLicense@f75170b[state=PA,licenseNum=330247,effectiveDate=com.vsp.portal.util.Chrono@503bdb1a[day=3,month=3,year=2004,value=,format=MMMM|dd#yyyy]], 2=com.vsp.broker.model.BrokerLicense@abdcf8b[state=WV,licenseNum=6836793,effectiveDate=com.vsp.portal.util.Chrono@3384b42f[day=2,month=3,year=2002,value=,format=MMMM|dd#yyyy]], 3=com.vsp.broker.model.BrokerLicense@1aa29b40[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@297e9469[day=,month=,year=,value=,format=MM...[Mag: Next line continuation is missing]

[cpetterborg]

Send an example of the data (not too many lines, but representative data) and the contents of the props.conf file for this sourcetype.

[shandman]

broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,143] INFO {abcpnhFLDq4THhWxDqVKu} LogInterceptor.before: Invoking appointment-complete
broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,149] INFO {abcpnhFLDq4THhWxDqVKu}AppointmentCompleteAction.execute: Broker Registration Info:com.vsp.broker.model.AppointmentFormInfo@61bf56bd[licenses={0=com.vsp.broker.model.BrokerLicense@439a95a[state=MS,licenseNum=10265706,effectiveDate=com.vsp.portal.util.Chrono@659d55e0[day=1,month=10,year=2012,value=,format=MMMM|dd#yyyy]],1=com.vsp.broker.model.BrokerLicense@384bc948[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@778bb2d7[day=,month=,year=,value=,format=MMMM|dd#yyyy]],2=com.vsp.broker.model.BrokerLicense@824cf37[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@28a0280e[day=,month=,year=,value=,format=MMMM|dd#yyyy]],3=com.vsp.broker.model.BrokerLicense@3caf42c7[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@164935f1[day=,month=,year=,value=,fo...
broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,317] INFO {abcpnhFLDq4THhWxDqVKu} MailBlock.doAfterBody: Sending 'VSP Resource Center Registration' to jennleebush@aol.com

that's the sample of data.

[cpetterborg]

From this data you have three events, each of which is one line.

Are you seeing the second line itself truncated to be shorter?

Do you want to have a multi-line event, or do you want to have one line split into more than one event?

[shandman]

-sh-4.1$ sudo more props.conf
[datapower]
NO_BINARY_CHECK = 1
pulldown_type = 1

[PMIServlet]
NO_BINARY_CHECK = 1
pulldown_type = 1

[host::SCHQVVCACDEM1*]
TRANSFORMS-anonymizer = password-anonymizer

[host::broker-*]
TRANSFORMS-index = ClientRedirect
TRUNCATE=0

[host::client-*]
TRANSFORMS-index = ClientRedirect
TRUNCATE=0

[host::pt*]
TRANSFORMS-index = TrueFarmRedirect

[host::st*]
TRANSFORMS-index = TrueFarmRedirect

[host::member-*]
TRANSFORMS-index = MemberRedirect

[host::doctor-*]
TRANSFORMS-index = DoctorRedirect

[host::www-*]
TRANSFORMS-index = GlobalRedirect

[host::sa-portals-*]
TRANSFORMS-index = StrategicRedirect

[source::udp:8514]
TRANSFORMS-ClientHostOverride = ClientHostOverride
SHOULD_LINEMERGE = true
TRANSFORMS-ClientRawOverride = ClientRawOverride
TRANSFORMS-ClientShRawOverride = ClientShRawOverride
TRANSFORMS-ClientShortOverride = ClientShortOverride
TRANSFORMS-ClientTempOverride = ClientTempOverride

[source::udp:9514]
SHOULD_LINEMERGE = false
TRANSFORMS-BrokerHostOverride = BrokerHostOverride
TRANSFORMS-BrokerRawOverride = BrokerRawOverride
TRANSFORMS-BrokerShRawOverride = BrokerShRawOverride
TRANSFORMS-BrokerShortOverride = BrokerShortOverride
TRANSFORMS-BrokerTempOverride = BrokerTempOverride

shandman gravatar image

Answer by shandman
53 minutes ago

[shandman]

broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,149] INFO {abcpnhFLDq4THhWxDqVKu} AppointmentCompleteAction.execute: Broker Registration Info: LOG STARTINGcom.vsp.broker.model.AppointmentFormInfo@61bf56bd[licenses={0=com.vsp.broker.model.BrokerLicense@439a95a[state=MS,licenseNum=10265706,effectiveDate=com.vsp.portal.util.Chrono@659d55e0[day=1,month=10,year=2012,value=,format=MMMM|dd#yyyy]], 1=com.vsp.broker.model.BrokerLicense@384bc948[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@778bb2d7[day=,month=,year=,value=,format=MMMM|dd#yyyy]], 2=com.vsp.broker.model.BrokerLicense@824cf37[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@28a0280e[day=,month=,year=,value=,format=MMMM|dd#yyyy]], 3=com.vsp.broker.model.BrokerLicense@3caf42c7[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@164935f1[day=,month=,year=,value=,fo... LINE ENDING and NEXT LINE TRUNCATED
I don't want this truncated. I want a continuation of this event. Does that make sense?