All Apps and Add-ons

Splunk Machine Learning Toolkit: how can I upload a custom algorithm?

ravishbhupesh
Engager

I followed the link (http://docs.splunk.com/Documentation/MLApp/2.4.0/API/Registeranalgorithm) to load an algorithm MLPRegressor from scikit into Splunk.

I did the entry in algos.conf as "[MLPRegressor]"
I created a new file in "SPLUNK_HOME\etc\apps\Splunk_ML_Toolkit\bin\algos" as MLPRegressor.py and copied the algorithm code in it.

Restarted Splunk. Now when I am applying the algorithm to Predict Numeric Fields it gives an error as below
I am running as
SEARCH


| inputlookup server_power.csv |fit MLPRegressor hidden_layer_sizes=1 activation=logistic


ERROR


09-13-2017 12:42:16.516 INFO ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\python.exe" "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py"
09-13-2017 12:42:17.028 INFO ChunkedExternProcessor - stderr: Running C:\Program Files\Splunk\etc\apps\Splunk_SA_Scientific_Python_windows_x86_64\bin\windows_x86_64\python.exe C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py
09-13-2017 12:42:18.650 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
09-13-2017 12:42:18.650 INFO UserManager - Unwound user context: NULL -> NULL


0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Hi @RavishBhubesh - I see from your comments that you are trying to use an algorithm that does not exist in the version of sklearn in the PSC app. The sklearn version of the Python for Scientific Computing app is set to 0.17.

aljohnson_splun
Splunk Employee
Splunk Employee

HI @Ravishbhubesh

Glad to hear you are using the custom algorithms! As you probably saw, the key line is this one:

09-13-2017 12:42:18.650 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"

Most likely its a bad import or some bad syntax or something of the sort.

Could you paste the source code of MLPRegressor?

ravishbhupesh
Engager

Please excuse me for late reply.

Yes we are writing our own Neural Network algorithm. Till its ready we were trying to import an existing algorithm to see how it works(importing).

Because less some karma points I cannot attach the file and the code is quite big to paste here, but since I took this algorithm from sci kit. Here is the link of the source.
http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html
https://github.com/scikit-learn/scikit-learn/blob/ef5cb84a/sklearn/neural_network/multilayer_percept...

0 Karma

ravishbhupesh
Engager

Only error I see in mlspl.log is this only

1505806913.766000 2017-09-19 09:41:53,766 DEBUG [mlspl.chunked_controller] [initialize_processor] Error while initializing processor "FitBatchProcessor": Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
1505806913.766000 2017-09-19 09:41:53,766 ERROR [mlspl] [die] Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
1505806913.767000 2017-09-19 09:41:53,766 ERROR [mlspl] [die] Traceback (most recent call last):
File "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\cexc_init.py", line 151, in run
while self._handle_chunk():
File "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\cexc__init
.py", line 277, in _handle_chunk
ret = self.handler(metadata, body)
File "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py", line 121, in handler
return self.setup()
File "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py", line 91, in setup
self.controller = ChunkedController(self.getinfo, self.controller_options)
File "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\chunked_controller.py", line 56, in __init
_
self.processor = self.initialize_processor(self.controller_options['processor'], process_options, searchinfo)
File "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\chunked_controller.py", line 119, in initialize_processor
raise RuntimeError(str(e))
RuntimeError: Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
Blockquote

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

@aljohnson - Is it possible the algo is dependent on a missing module, or would that be a different error?

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

There is additional logging verbosity in $SPLUNK_HOME/var/log/mlspl.log that may help us determine the actual source of the error.

http://docs.splunk.com/Documentation/MLApp/2.3.0/API/CustomLogging

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...