Ossec 2.6, Splunk splunk-4.2.3-105575 and Splunk app ossec-1.1.88
Recently upgraded Ossec from 2.0 to 2.6 and added Splunk. Both reside on the same server which has over 900 active agents. When using the Splunk web interface only a few (36) agents show up and as part of troubleshooting running the ossec_agent_status.py I see that it gets an error.
Also, even when in the web interface under "Agent Status" the status column drops letters (doesn't finish the word like - "disco" instead of "disconnected" or "Never con" instead of "Never connected" for some of the agents. I don't know if that is part of this same problem or something different.
I hope someone can help with this as I would really like to show off Splunk using an existing Ossec installation base.
The most likely reason for the timeout is just the relatively large number of agents - the default timeout is 5 seconds. I've made a note to increase that and/or make it configurable.
In the mean time, try editing bin/pyOSSEC.py. At line 331, change:
p = pexpect.spawn(cmd, timeout=5)
p = pexpect.spawn(cmd, timeout=30)
and see if that solves the timeout concern.
There's a good chance that the truncation is caused by the timeouts. The partial output from pexpect will print out 100 characters of context for diagnostic purposes.
In the example you posted above, it's the section that looks like this:
before (last 100 chars): pvap020, IP: 10.180.5.151, Active
ID: 1036, Name: w1pvap003, IP: 10.180.5.152, Active
Here, it truncated what would have been the ID value. If the truncation had occurred toward the end of a line instead of toward the beginning, you would get something ending with a partial word that would be extracted into the Status column.
It might still be something else, but the best approach will be to fix the truncation issue and look more closely at this if the problem persists.
If you search on sourcetype=oseec_agent_control, I would expect to see see lines where the raw data shows the same truncation. (Assuming that's the case, it supports the conjecture that it's a data collection problem and not a field extraction problem).