topic Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting in All Apps and Add-ons

Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

danicarmelo — Thu, 31 Oct 2019 07:37:52 GMT

I have a host were TA-eStreamer is deployed, it was working fine last 2018 but it is now not running. This is the estreamer.log when it was working then stopped until the time I tried to start splencore.sh.

2018-11-22 11:20:50,027 Monitor INFO Running. 23229500 handled; average rate 45.3 ev/sec;
2018-11-22 11:23:06,795 Monitor INFO Running. 23230900 handled; average rate 45.29 ev/sec;
2018-11-22 11:23:11,190 Service INFO Splunk is not running.
2018-11-22 11:23:11,191 Service INFO Stopping
2018-11-22 11:23:11,691 Controller INFO Stopping...
2018-11-22 11:23:17,300 SubscriberParser INFO Stop message received
2018-11-22 11:23:27,808 SubscriberParser INFO Exiting
2018-11-22 11:23:27,829 Controller INFO Process 22262 (Process-1) exit code: 0
2018-11-22 11:23:27,835 Decorator INFO Stop message received
2018-11-22 11:23:27,840 Decorator INFO Error state. Clearing queue
2018-11-22 11:23:27,840 Cache INFO Saving cache to $SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/
2018-11-22 11:23:34,042 Decorator INFO Exiting
2018-11-22 11:23:34,154 Controller INFO Process 22263 (Process-2) exit code: 0
2018-11-22 11:23:34,155 Transformer INFO Stop message received
2018-11-22 11:23:34,160 Transformer INFO Error state. Clearing queue
2018-11-22 11:23:34,160 Transformer INFO Exiting
2018-11-22 11:23:34,160 Controller INFO Process 22264 (Process-3) exit code: 0
2018-11-22 11:23:34,161 Writer INFO Stop message received
2018-11-22 11:23:34,166 Writer INFO Error state. Clearing queue
2018-11-22 11:23:34,166 Writer INFO Exiting
2018-11-22 11:23:34,166 Controller INFO Process 22266 (Process-4) exit code: 0
2018-11-22 11:23:34,166 Monitor INFO Stopping Monitor.
2018-11-22 11:23:34,331 Controller INFO Goodbye
2019-10-30 20:07:59,466 Controller INFO eNcore version: 3.5.3

As you can see from the logs that splunk is not running when estreamer logs stopped that time.

But I've verified before and after I've started splencore.sh that splunk is running, but I still see the same message that splunk is not running.

2019-10-31 15:44:39,776 Decorator INFO Starting process.
2019-10-31 15:44:39,777 Transformer INFO Starting process.
2019-10-31 15:44:39,777 Monitor INFO Starting Monitor.
2019-10-31 15:44:39,777 Writer INFO Starting process.
2019-10-31 15:44:39,793 Service INFO Splunk is not running.
2019-10-31 15:44:39,794 Service INFO Stopping

estreamer.logs doesnt really show me why its failing to start.

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

douglashurd — Wed, 06 Nov 2019 17:19:54 GMT

Please update to the latest version of the TA.
https://splunkbase.splunk.com/app/3662/

If you still have the problem just copy / paste new log data in this forum and we'll make a few suggestions.

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

danicarmelo — Wed, 30 Sep 2020 02:58:08 GMT

Hi @douglashurd
I have upgraded to the latest version but I am encountering this error message when i am starting encore:

2019-11-15 21:59:36,939 Diagnostics ERROR The FMC eStreamer server has closed the connection. There are a number of possible causes which may show above in the error log.\n\nIf you see no errors then this could be that:\n * the server is shutting down\n * there has been a client authentication failure (please check that your outbound IP address matches that associated with your certificate - note that if your device is subject to NAT then the certificate IP must match the upstream NAT IP)\n * there is a problem with the server. If you are running FMC v6.0, you may need to install "Sourcefire 3D Defense Center S3 Hotfix AZ 6.1.0.3-1"\n
2019-11-15 21:59:36,940 Controller ERROR ConnectionClosedException: Connection closed\nTraceback (most recent call last):\n File "$SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/estreamer/controller.py", line 244, in start\n diagnostics.execute()\n File "$SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/estreamer/diagnostics.py", line 96, in execute\n response = connection.response()\n File "$SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/estreamer/connection.py", line 181, in response\n dataBuffer = self.__read( 8 )\n File "$SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/estreamer/connection.py", line 158, in __read\n raise estreamer.ConnectionClosedException('Connection closed')\nConnectionClosedException: Connection closed\n
2019-11-15 21:59:36,940 Controller INFO Stopping...
2019-11-15 21:59:36,940 Monitor INFO Stopping Monitor.
2019-11-15 21:59:36,941 Controller INFO Goodbye

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

vinz2020 — Tue, 10 Mar 2020 10:01:30 GMT

I am having the same issue with the new app 3.6.8
https://splunkbase.splunk.com/app/3662/

and FMC v6.4.0.7

I can collect the logs a few minutes (cisco:estreamer:data) and then i received
"Process subscriberParser is dead"

any idea ?
thanks a lot

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

vik_splunk — Wed, 23 Jun 2021 15:13:22 GMT

Hi @vinz2020 ,

Did you ever manage to resolve this? We are running into the same issue now

We use the app ver. 4.6.0 on Splunk 8.1.3 with an FMC version of 6.6.0 and are encountering the same issue

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

vinz2020 — Thu, 24 Jun 2021 09:04:37 GMT

Yes I fixed it ... but unfortunately I can't remember how 😕

Now I am running app 4.6, Splunk 8.1.3 and FMC 6.5

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

vik_splunk — Thu, 24 Jun 2021 19:15:57 GMT

Thanks. @vinz2020 Any thing that you can recollect and provide inputs will be highly appreciated.

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

_joe — Fri, 02 Jul 2021 12:44:56 GMT

It would seem 6.4.0 was released with a couple of bugs. My instance just failed 3-4 days after install. Upgrade to 6.4.2.

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

computermathguy — Tue, 09 Jan 2024 17:00:10 GMT

Recently we upgraded FMC from 6.x to 7.x and noticed no data was being streamed into the /opt/splunk/etc/apps/TA-eStreamer/bin/encore/data/splunk directory. We then started getting a firewall error when testing the connection..

Does anyone know if FMC 7.x is compatible with the TA-eStreamer add-on?

./splencore.sh test
Diagnostics ERROR [no message or attrs]: Could not connect to eStreamer Server at all. Are you sure the host and port are correct? If so then perhaps it is a firewall issue.

Re: Cisco eStreamer eNcore Add-on for Splunk: eNcore process not starting

vikesh05 — Wed, 27 Mar 2024 13:56:43 GMT

For me, it turned out to be an incorrect FMC IP. Post proper IP configuration it worked
Splunk - 9.2.0.1
eStreamer - 5.2.9