About pdrieger_splunk

pdrieger_splunk · ‎07-14-2023

Hi @Slawamba - can you look into the search.log (available over the job inspector from the search bar) and see what exact error is close to the call of the fit endpoint? If there is something with SSL error, please manually pull the 5.1.0 golden image again and see if this resolved the issue. If it's something different, please let me know.

pdrieger_splunk · ‎12-02-2022

Thanks @Gabriel I resolved your issue on https://github.com/splunk/splunk-mltk-container-docker/issues/30 by adding the correct feature selection with X = df[param['feature_variables']] - please consider marking this answer here as solved if this resolves your issue. Thanks and have a nice weekend 🙂

pdrieger_splunk · ‎12-02-2022

Hi @Gabriel , thanks for working with DSDL and sharing your findings here with the community. This behaviour is indeed not what is expected. Let me ask a few question and share some thoughts to find a way to resolve this issue. 1. Did you implement the def load() and def save() in your notebook? If so, how did you serialise the KMeans model object (which would contain the trained state after running your fit). 2. MLTK's fit and apply commands have some specific behaviour: fit is an eventing command, apply is a stateful streaming command. But if your model is properly loaded you should not see this deviation. 3. While I appreciate to see you implementing in DSDL, you probably know that MLTK has KMeans out of the box and might be easier to use to achieve the same goal. Would this be an alternative? 4. For DSDL you can also consider opening a support case for this issue. If your notebook and sample data is anonymised and contains no sensitive information, you could share it so we can reproduce your issue. Hope this is helpful for you. Please let us know!

pdrieger_splunk · ‎03-02-2022

Hi @landon , thanks again for sharing the details. It looks like this situation can occur when some synchronisation function was not yet completed. The list of running containers is actively checked and synced with a .conf file. This config is then used for communicating with the container. When the error is not persistent and does not show up anymore then I assume things are working fine? Can you confirm or do you see this error happening more frequently?

pdrieger_splunk · ‎01-26-2022

Hi @landon can you please share more details on the version you use and what specific issue you face?

pdrieger_splunk · ‎03-24-2021

Hi @ColinRobinson - it looks like this is related to an issue with config files on windows. See https://github.com/splunk/splunk-mltk-container-docker/issues/14 and https://github.com/splunk/deep-learning-toolkit/issues/66 - actively looking into it.

pdrieger_splunk · ‎02-01-2021

Hi @robinSplunk - did you check the other answers around windows and docker access issues, e.g. https://community.splunk.com/t5/All-Apps-and-Add-ons/Splunk-Deep-Learning-Toolkit-Setup-Python-Traceback-WinError/m-p/523148 and linked answers? There has been reported issues with reading conf files with UTF-8-BOM under windows - this might be related here as well.

pdrieger_splunk · ‎02-01-2021

Hi @muyouming , for your 1. question can you check either the user have the role mltk_container_user OR their role capability includes list_mltk_container and control_mltk_container as this should give the the rights to access and control containers. Your 2. question: assuming that both environments are isolated, e.g. without shared storage, then any notebook from dev would need to get copied over to prod. If you have any versioning / CI/CD workflows or tools in place than you could connect your dev environment e.g. to a GIT repo which is also accessible from the prod environment and only pulls the relevant production parts. Currently there is no build in functions in DLTK so you would need to manage those workflows on your own based on your requirements. Hope that's helpful? Thanks, Philipp

pdrieger_splunk · ‎02-01-2021

Hi @brandy81 , thanks for asking - it seems this is related to a known issue as listed here: https://github.com/splunk/splunk-mltk-container-docker/issues/8 - as a workaround you could leave out the ... into app:server_power_regression when using mode=stage.

pdrieger_splunk · ‎01-29-2021

Hi @force1024 - thanks for sharing - this is likely related to a bug that happens on windows when reading conf files with different encoding. See https://github.com/splunk/deep-learning-toolkit/issues/66 with a proposed fix which is to be implemented.

pdrieger_splunk · ‎11-11-2020

Hi @SplunkHead10 - it looks like either a connection issue to your docker host. Depending on your docker host system you might need to expose the management port - e.g. when running Docker on Windows. There has been a few similar question being answered on these topics here, can you have a look if those help: https://community.splunk.com/t5/All-Apps-and-Add-ons/How-can-I-get-rid-of-quot-Connection-refused-quot-for-Deep/m-p/461398 Otherwise: can you share more information about your setup: which OS are your running on? Is your SH on the same instance as docker or do you have a separate instances?

pdrieger_splunk · ‎10-26-2020

Hi @andrewtrobec - did you see any popup appear after you clicked or any errors in your browser console? Could be a browser related issue as @SplunkHead10 mentioned above. If there is no specific need for running version 2.3 I would recommend to use the latest version 3.3 available from splunkbase.

pdrieger_splunk · ‎10-26-2020

Hi @bplaxco - currently the recommendation would be to add all specific security requirements into the container itself and build according to your needs. There is an open source GitHub repo for building own containers: https://github.com/splunk/splunk-mltk-container-docker As there are various deployment scenarios for different container environments this would give you most control. However, we're currently also working to standardise and better secure DLTK within the latest (work in progress) version 4.0 available on https://github.com/splunk/deep-learning-toolkit

pdrieger_splunk · ‎07-06-2020

This makes sense. Yes, you need to get the image downloaded. Either you manually docker pull the image OR when you start the dev container for the first time it should try to automatically pull the image if it is not available locally.

pdrieger_splunk · ‎07-06-2020

Hi @state_larson_ti , typically when you start a development container from the DLTK container management page, the stanzas in the containers.conf are created automatically. When you have a running dev container enabled like in this screen, your local/containers.conf should look similar like this: [default] [__dev__] api_url = http://localhost:32768 cluster = docker id = fa99644474e76340b97d1a3c3ce3c2ec4298debfc591b0b2d47f625586aeecd5 image = mltk-container-golden-image-gpu jupyter_url = http://localhost:8888 runtime = None tensorboard_url = http://localhost:6006 spark_url = http://localhost:4040 Hope this is helpful?

pdrieger_splunk · ‎05-15-2020

Hi @davietch , sure that's easy 🙂 Q: What is the password for Jupyter Lab? A: Please have a look at the Model Development Guide page in the Deep Learning Toolkit app. https://splunkbase.splunk.com/app/4607/#/details Hope this helps. Best, Philipp

pdrieger_splunk · ‎05-13-2020

Hi @frozenpy , that should be easy: you can find the password noted in the DLTK app page Model Development Guide accessible over the home content dashboard or from the DLTK app menu Overview > Model Development Guide. Best, Philipp

pdrieger_splunk · ‎04-27-2020

Hi @davietch , thanks for your question. Let me answer all 3 questions that I identified in your post: That's indeed an unintended behaviour and will be fixed in the upcoming version 3.1. As a workaround you can modify the relevant | rest ... searches on your dashboard, e.g. with | rest splunk_server=local services/mltk-container/status . By adding splunk_server=local the search does not propagate to the indexers and will resolve this issue. As a pointer: please also update other saved searches like the MLTK Container Sync , too. DLTK currently does not support execution on the indexers. So the short answer here is that there is no benefit in installing DLTK on your indexers. It seems that there was no module code generated from the Jupyter Notebook in the container. Simply resolve by opening the binary_nn_classifier.ipynb in Jupyter Lab and save it. This automatically generates a python module in the container that you can verify as file in the /app/model/binary_nn_classifier.py - you should see the example dashboard working as soon as this file exists. Hope this answers all your questions 🙂

pdrieger_splunk · ‎03-31-2020

Hi @davietch , to me it looks like your docker host can not be reached from the DLTK which tries to establish a connection but fails. for the last case: Can you cross check with this post here if you haven't: https://github.com/docker/docker-py/issues/290 for your initial connection, assuming your docker connection was defined this way: Docker Host: unix://var/run/docker.sock Endpoint URL: localhost External URL: localhost Can you double check that your docker deamon socket is enabled: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-socket-option Let us know if it can be resolved with that.

pdrieger_splunk · ‎03-31-2020

Hi @davietch, can you please cross check this answer and see if connecting to tcp instead of the unix://var/run/docker.sock helps in your case, too: https://answers.splunk.com/answers/811981/deep-learning-toolkit-setup.html

pdrieger_splunk · ‎03-24-2020

Thanks for sharing: I see you mixed in a | rest call in your search above: | inputlookup diabetes.csv \n | rest services/mltk-container/sync \n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response Can you have a look what's in your .../etc/apps/mltk-container/local/containers.conf ? Typically you should see a [dev] stanza with filled attributes like api_url=... - In case you have a valid running container you should see the correct urls there - if those are missing then this would lead to the error you describe above. Can you verify?

pdrieger_splunk · ‎03-23-2020

Can you check how your local containers.conf looks like? It seems the url can not be retrieved properly. If you can share or attach your search.log that contains the python errors as occurred in the MLTKContainer.py this will also be helpful to troubleshoot. I remember I saw this error one time, but it didn't happen any more. Thanks for your help to share more infos!

pdrieger_splunk · ‎03-23-2020

Hi @farideh1371 , seems your question is exactly the same as here: https://answers.splunk.com/answers/812585/deep-learning-toolkit-setup-1.html - see my answer there.

pdrieger_splunk · ‎03-23-2020

Hi @hari7696, thanks for your question! In order to be able to apply a model trained as you described, please ensure that you have implemented the load and save method (Stage 5 and 6 in your Jupyter lstm_autoencoder notebook) and saved the notebook. Reason: when you call | apply lstm_autoencoder, implicitly the model is loaded with the given name by calling your load() method so that it can be applied to your test_50 data. Let me know if this resolves the issue. If you don't implement your save() and load() methods the consequence is that your model is initialised with a potentially unknown state or data inputs shape definition etc. so it is likely to fail in the way you described above. Did you find the Model Development Guide in the DLTK app useful? I agree, there can always be more documentation 🙂 Thanks for your feedback

pdrieger_splunk · ‎03-23-2020

Hi @farideh1371, you likely need wait for DLTK to resync your new container and the containers.conf file which has not yet a valid url for your container listed. You can force resync with | rest services/mltk-container/sync in the DLTK. Can you check if that resolved it? Thanks.

Posts	40
Solutions	6
Karma Given	2
Karma Received	12
Member Since	‎01-26-2015

Online Status	Offline
Date Last Visited	‎07-16-2023 09:54 AM

No content is shown in Machine Learning Toolkit Sh...

Re: Why is DLTK search showing error for sending t...

Re: DSDL - Kmeans yields different results for fit...

Re: DSDL - Kmeans yields different results for fit...

Re: DLTK search error for sending to container

Re: DLTK search error for sending to container

Re: Splunk Deep Learning Toolkit Setup url variabl...

Re: Deep learning Toolkit windows error

Re: How to use DLTK for multiple non-Admin users?

Re: DLTK search error for sending to container

Re: DLTK Setup (Windows w/ Docker Desktop)

Re: Splunk Deep Learning Toolkit Setup - Python Tr...

Re: Deep Learning Toolkit Setup Issue with Docker

Re: Deep Learning Toolkit for Splunk Container Sec...

Re: Deep learning toolkit setup

Re: Deep learning toolkit setup

Re: How can I add model.binary_nn_classifier modul...

Re: How to Access Jupyter notebook in Deep learnin...

Re: How can I add model.binary_nn_classifier modul...

Re: How can I get rid of "Connection refused" for ...

Re: How can I get rid of "Connection refused" for ...

Re: Deep learning toolkit setup

Re: Deep learning toolkit setup

Re: Deep Learning toolkit use cases

Re: Deep Learning Toolkit-Issue with apply command

Re: Deep learning toolkit setup