All Apps and Add-ons
Highlighted

Splunk Add-on for IPFIX: Why is ipfix_collector.py using 99.5% CPU and 70% Memory on our heavy forwarder? Getting "ParseError="Template not known (yet).""

Communicator

Hi guys,

We have the Splunk Add-on for IPFIX installed on one of our Heavy Forwarders.

I got noticed that one of the Python scripts is causing a daily crash of that HWF host.

-Path of the .py script: /opt/splunk/splunk/etc/apps/Splunk_TA_IPFIX_UDP_NIX/bin/ipfix_collector.py

-Checked splunkd.log in the heavy-weight forwarder, could not find any information related to ipfix_collector.py.

-Checked appflow.log, log had stopped for more than 20h. Also find some error like this:

TimeStamp="2016-03-02T20:35:33"; Template="265"; Observer="0"; Address=""; Port="<>"; ParseError="Template not known (yet).";

-Checked debug.log, it is full of

Have not implemented parsing for 'None' of length 8 (5951:319) which is needed for template 284.

-Ping the Netscaler, PING OK

-Restarted host and Splunk heavy-weight forwarder, still with no luck.

Has anyone seen this before?

Any advice will be much appreciated!

Thank you very much in advance.

Cheers,
Vincent

alt text

alt text

0 Karma
Highlighted

Re: Splunk Add-on for IPFIX: Why is ipfix_collector.py using 99.5% CPU and 70% Memory on our heavy forwarder? Getting "ParseError="Template not known (yet).""

Communicator

Each time restarting Splunk. That python script stop for about 10 mins. then it will starting running again and use 100% of CPU. The memory also keep increasing from 0% to 70%. Not long after, the host will crash.

I had confirmed there is enough HDD space for running the script.

But why the python script use all CPU and the memory ever increasing and eventually crash the host?

Could it be ipfix_collector.py script doesn't have permission to write into the disk?

0 Karma
Highlighted

Re: Splunk Add-on for IPFIX: Why is ipfix_collector.py using 99.5% CPU and 70% Memory on our heavy forwarder? Getting "ParseError="Template not known (yet).""

Splunk Employee
Splunk Employee

hi, you're getting this error because you need to add enterprise information elements for the device you're trying to parse data from.

http://docs.splunk.com/Documentation/AddOns/latest/IPFIX/ConfigureEnterpriseInformationElements

View solution in original post

0 Karma
Highlighted

Re: Splunk Add-on for IPFIX: Why is ipfix_collector.py using 99.5% CPU and 70% Memory on our heavy forwarder? Getting "ParseError="Template not known (yet).""

Communicator

Hi Jcoates,

Thanks for your response. Likely we have not configure template for this Citrix device yet. That partially answers my question.

However,

We have another HWF which is receiving template correctly, but the ipfix_collector.py script still progressively growing in memory usage(Same memory issue as my earlier comment. It used 100% CPU, and memory grow gradually...). Is this expected for this python script?

0 Karma
Highlighted

Re: Splunk Add-on for IPFIX: Why is ipfix_collector.py using 99.5% CPU and 70% Memory on our heavy forwarder? Getting "ParseError="Template not known (yet).""

Path Finder

Hi anyone there.
So how to configure template for Citrix device?

0 Karma