Why is Windows event log message data being truncated and only the first 2 lines are getting forwarded?

[andrefriedmann]

Hi

I am having a strange issue where some of the message or 'EventData' is missing from the forwarded Windows event logs.

The majority of the event gets forwarded correctly, however, the message only gets the first 2 lines, then chops the rest off!

If I use renderXml = 1 in the inputs, then it works and I can see the full event as I do in Windows. However, I really would rather not have the events forwarded in the XML format!

Any ideas?

Thanks

[wkupersa · Accepted solution]

Thank you for all the help. I finally figured out what was going on and it was on the windows side. The box in question is an event collector. Apparently I needed to modify the format of the forwarded events with the following statement

wecutil ss "subscription Name" /cf:Events

where "wecutil es" will list all the subscription names

[wkupersa]

More on this. Making the change above allowed me to see what was going on and get the rest of the event. What I saw was that it was dropping at "message=" because the tool that I was logging wasn't installed on my event collector. When I installed that tool, I ended up having to change the event format back to RenderedText (wecutil ss "subscription name" /cf:RenderedText" in order to get everything to parse correctly. Quite the adventure.

[ryandg]

That's an interesting nuance with your Windows Event Logs, thanks for sharing the fix!

[ryandg]

you are sending this over TCP right and not UDP? The fact that it chops it off at 2 lines to me doesn't look like a truncation issue on the props and transforms but more like the data is getting split into small segments like a UDP connection would do for Windows events.

[wkupersa]

Thanks for the response. I should have been more clear. I my case, the log message is being truncated after 14 lines. All my log entries end with "message ="

I am using tcp.

I've turned on DEBUG for TailingProcessor, WatchedFile, TailReader, and the WinEventLog options on the forwarder and I am reviewing the splunkd.log but so far, I haven't seen anything useful.

[ryandg]

Is the forwarder sending straight to the indexer tier or is it going to a heavy forwarder?

[javiergn]

If you search in your internal logs for truncated lines, can you see anything there at all?

index=_internal truncated

Are all your events getting the message field truncated? If not, do those events have anything in common?
Can you run btool to identify which settings are applying and if other apps are conflicting?

[wkupersa]

In my case, I see all sorts of notifications regarding things being truncated in the _internal index and splunkd.log, but nothing talking about the ForwardedEvents file that I am concerned about.

Btools only shows me the config from inputs.conf
[WinEventLog://ForwardedEvents]
current_only = 0
disabled = 0
evt_dc_name =
evt_dns_name =
index = FEvents

The default section has nothing defined.

[wkupersa]

Having same problem