topic 24284 bytes long Event log message causes splunkd to crash - workaround in Monitoring Splunk

24284 bytes long Event log message causes splunkd to crash - workaround

FloydATC — Fri, 04 Apr 2014 13:02:46 GMT

One of my Exchange 2010 hubcas servers is logging monstrous messages into the Event Log about conflicting updates from two ActiveSync devices accessing the same mailbox. (Each event Message is 24284 bytes and contains several debug traces and contains pretty much everything except what that particular Microsoft employee had for breakfast that day)

Anyway, when I try to select that Message into a table, the splunk server craps itself and must be manually restarted:
... | table _time, host, LogName, Message

Re: 24284 bytes long Event log message causes splunkd to crash - workaround

FloydATC — Fri, 04 Apr 2014 13:03:04 GMT

Workaround: Limit the potentially huge string to a more sane length, say 1024 characters

... | eval msg=substr(Message,0,1024) | table _time, host, LogName, msg

Re: 24284 bytes long Event log message causes splunkd to crash - workaround

ahall_splunk — Fri, 04 Apr 2014 15:33:09 GMT

Also, I suspect that the Message field has a certain format. View the event in the event viewer and then extract just the information you need and display that. I suspect the Device ID, Device Type and username are logged in there. The actual processing doesn't care what size the event is - only the display.

Re: 24284 bytes long Event log message causes splunkd to crash - workaround

FloydATC — Fri, 04 Apr 2014 16:14:58 GMT

Yes, the problem only seems to affect 'table' view of the 'Message' field, showing the '_raw' event works as expected.