I'm monitoring a number of processes on a dozen or so Windows hosts.
I've not written a script, the input is simply defined something like:
Get-Process | Where-Object {$_.Name -like "vendor*.exe"}
Assume the vendor executables all start with Vendor. This worked fine initially, I picked up 4 or 5 processes and got all the details for each process. This is admittedly a lot of information but I figured I'd rather gather too much at this time.
As we started to use the application, some values like VirtualMemorySize started going up.
Now, one of the processes with a large (almost 2 GB) value for VirtualMemorySize simply stopped appearing in Splunk. Others still work fine.
When trying the exact same Get-Process | Where-Object {$_.Name -like "vendor*.exe"} on the Windows host in a Powershell command window, I get results for all the processes, including the one I'm missing in Splunk, I notice on the table view, some values are a bit mangled, presumeably because the values are large.
I suspect somehow output where some values are large, they get dropped or are in some other way lost. I've not been able to pinpoint exactly what, but for example this Powershell Input works and picks up all processes:
Get-Process | Where-Object {$_.Name -like "vendor*.exe"} | Select-Object Name,CPU
But this does not
Get-Process | Where-Object {$_.Name -like "vendor*.exe"} | Select-Object Name,CPU,VirtualMemorySize
Anyone got any insight into this?