All Apps and Add-ons

Splunk Machine Learning Toolkit: how can I upload a custom algorithm?

I followed the link (http://docs.splunk.com/Documentation/MLApp/2.4.0/API/Registeranalgorithm) to load an algorithm MLPRegressor from scikit into Splunk.

I did the entry in algos.conf as "[MLPRegressor]"
I created a new file in "SPLUNKHOME\etc\apps\SplunkML_Toolkit\bin\algos" as MLPRegressor.py and copied the algorithm code in it.

Restarted Splunk. Now when I am applying the algorithm to Predict Numeric Fields it gives an error as below
I am running as
SEARCH


| inputlookup serverpower.csv |fit MLPRegressor hiddenlayer_sizes=1 activation=logistic


ERROR


09-13-2017 12:42:16.516 INFO ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\python.exe" "C:\Program Files\Splunk\etc\apps\SplunkMLToolkit\bin\fit.py"
09-13-2017 12:42:17.028 INFO ChunkedExternProcessor - stderr: Running C:\Program Files\Splunk\etc\apps\SplunkSAScientificPythonwindowsx8664\bin\windowsx8664\python.exe C:\Program Files\Splunk\etc\apps\SplunkMLToolkit\bin\fit.py
09-13-2017 12:42:18.650 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
09-13-2017 12:42:18.650 INFO UserManager - Unwound user context: NULL -> NULL


0 Karma

Splunk Employee
Splunk Employee

Hi @RavishBhubesh - I see from your comments that you are trying to use an algorithm that does not exist in the version of sklearn in the PSC app. The sklearn version of the Python for Scientific Computing app is set to 0.17.

Splunk Employee
Splunk Employee

HI @Ravishbhubesh

Glad to hear you are using the custom algorithms! As you probably saw, the key line is this one:

09-13-2017 12:42:18.650 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"

Most likely its a bad import or some bad syntax or something of the sort.

Could you paste the source code of MLPRegressor?

Please excuse me for late reply.

Yes we are writing our own Neural Network algorithm. Till its ready we were trying to import an existing algorithm to see how it works(importing).

Because less some karma points I cannot attach the file and the code is quite big to paste here, but since I took this algorithm from sci kit. Here is the link of the source.
http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html
https://github.com/scikit-learn/scikit-learn/blob/ef5cb84a/sklearn/neural_network/multilayer_percept...

0 Karma

Only error I see in mlspl.log is this only

1505806913.766000 2017-09-19 09:41:53,766 DEBUG [mlspl.chunkedcontroller] [initializeprocessor] Error while initializing processor "FitBatchProcessor": Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
1505806913.766000 2017-09-19 09:41:53,766 ERROR [mlspl] [die] Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
1505806913.767000 2017-09-19 09:41:53,766 ERROR [mlspl] [die] Traceback (most recent call last):
File "C:\Program Files\Splunk\etc\apps\SplunkMLToolkit\bin\cexc_init.py", line 151, in run
while self.
handlechunk():
File "C:\Program Files\Splunk\etc\apps\Splunk
MLToolkit\bin\cexc_init.py", line 277, in handlechunk
ret = self.handler(metadata, body)
File "C:\Program Files\Splunk\etc\apps\SplunkMLToolkit\bin\fit.py", line 121, in handler
return self.setup()
File "C:\Program Files\Splunk\etc\apps\SplunkMLToolkit\bin\fit.py", line 91, in setup
self.controller = ChunkedController(self.getinfo, self.controlleroptions)
File "C:\Program Files\Splunk\etc\apps\Splunk
MLToolkit\bin\chunkedcontroller.py", line 56, in __init

self.processor = self.initializeprocessor(self.controlleroptions['processor'], processoptions, searchinfo)
File "C:\Program Files\Splunk\etc\apps\Splunk
MLToolkit\bin\chunkedcontroller.py", line 119, in initialize_processor
raise RuntimeError(str(e))
RuntimeError: Error while initializing algorithm "MLPRegressor": Failed to load algorithm "algos.MLPRegressor"
Blockquote

0 Karma

SplunkTrust
SplunkTrust

@aljohnson - Is it possible the algo is dependent on a missing module, or would that be a different error?

0 Karma

Splunk Employee
Splunk Employee

There is additional logging verbosity in $SPLUNK_HOME/var/log/mlspl.log that may help us determine the actual source of the error.

http://docs.splunk.com/Documentation/MLApp/2.3.0/API/CustomLogging

0 Karma