Monitoring Splunk

24284 bytes long Event log message causes splunkd to crash - workaround

FloydATC
Explorer

One of my Exchange 2010 hubcas servers is logging monstrous messages into the Event Log about conflicting updates from two ActiveSync devices accessing the same mailbox. (Each event Message is 24284 bytes and contains several debug traces and contains pretty much everything except what that particular Microsoft employee had for breakfast that day)

Anyway, when I try to select that Message into a table, the splunk server craps itself and must be manually restarted:
... | table _time, host, LogName, Message

Tags (4)
0 Karma

ahall_splunk
Splunk Employee
Splunk Employee

Also, I suspect that the Message field has a certain format. View the event in the event viewer and then extract just the information you need and display that. I suspect the Device ID, Device Type and username are logged in there. The actual processing doesn't care what size the event is - only the display.

0 Karma

FloydATC
Explorer

Yes, the problem only seems to affect 'table' view of the 'Message' field, showing the '_raw' event works as expected.

0 Karma

FloydATC
Explorer

Workaround: Limit the potentially huge string to a more sane length, say 1024 characters

... | eval msg=substr(Message,0,1024) | table _time, host, LogName, msg

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Agent Mode Engaged! Enchaining Agentic Operations with Splunk AI Assistant 2.0

    Are you ready to transform how your team handles complex data requests? We invite you to our upcoming ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...