All Apps and Add-ons

Deep learning toolkit setup

farideh1371
Explorer

I have running docker with image"mlkt-container-tf-cpu" in deep learning toolkit also I have access to the jupyter notebook in the toolkit but when I want to run a use case for instance "neural network classifier" I get an error which is "Error in fit command. Error while initializing algorithm 'MLTKContainer': local variable 'url' referenced before assigned"

0 Karma
1 Solution

farideh1371
Explorer

thank you for following up on my issue.

this is the search.log (I couldn't attach it 😞

03-23-2020 20:55:50.100 INFO dispatchRunner - Search process mode: freestanding (build a7f645ddaf91).
03-23-2020 20:55:50.102 INFO dispatchRunner - initing LicenseMgr in search process: nonPro=0
03-23-2020 20:55:50.102 INFO LicenseMgr - Initing LicenseMgr
03-23-2020 20:55:50.105 INFO ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
03-23-2020 20:55:50.105 INFO ServerConfig - Host name option is "".
03-23-2020 20:55:50.116 INFO ServerConfig - SSL session cache path enabled 0 session timeout on SSL server 300.000
03-23-2020 20:55:50.117 INFO LMConfig - serverName=MUCL95035 guid=6D139D41-58E6-407B-A1C9-8E1D9F452F44
03-23-2020 20:55:50.117 INFO LMConfig - connection_timeout=30
03-23-2020 20:55:50.119 INFO LMConfig - send_timeout=30
03-23-2020 20:55:50.119 INFO LMConfig - receive_timeout=30
03-23-2020 20:55:50.119 INFO LMConfig - key=license_warnings_update_interval not found in licenser stanza of server.conf, defaulting=0
03-23-2020 20:55:50.119 INFO LMConfig - squash_threshold=2000
03-23-2020 20:55:50.119 INFO LMConfig - strict_pool_quota=1
03-23-2020 20:55:50.119 INFO LMConfig - key=pool_suggestion not found in licenser stanza of server.conf, defaulting=''
03-23-2020 20:55:50.119 INFO LMConfig - key=test_aws_metering not found in licenser stanza of server.conf, defaulting=0
03-23-2020 20:55:50.119 INFO LMConfig - key=test_aws_product_code not found in licenser stanza of server.conf, defaulting=0
03-23-2020 20:55:50.119 INFO LicenseMgr - Initing LiacenseMgr runContext_splunkd=false
03-23-2020 20:55:50.119 INFO LMStackMgr - closing stack mgr
03-23-2020 20:55:50.119 INFO LMSlaveInfo - all slaves cleared
03-23-2020 20:55:50.119 INFO LMStackMgr - Initalized license_warnings_update_interval=auto
03-23-2020 20:55:50.120 INFO LMStack - Added type=download-trial license, from file=enttrial.lic, to stack=download-trial of group=Trial
03-23-2020 20:55:50.120 INFO LMStackMgr - created stack='download-trial'
03-23-2020 20:55:50.120 INFO LMStackMgr - have to auto-set active stack group='Trial' reason='invalid/missing group id' gidStr='' oldGid=Invalid
03-23-2020 20:55:50.120 INFO LMStackMgr - added pool auto_generated_pool_download-trial to stack download-trial
03-23-2020 20:55:50.120 INFO LMStackMgr - added pool auto_generated_pool_forwarder to stack forwarder
03-23-2020 20:55:50.120 INFO LMStackMgr - added pool auto_generated_pool_free to stack free
03-23-2020 20:55:50.120 INFO LMStackMgr - Initialized hideQuotaWarning = "0"
03-23-2020 20:55:50.120 INFO LMStackMgr - init completed [6D139D41-58E6-407B-A1C9-8E1D9F452F44,Trial,runContext_splunkd=false]
03-23-2020 20:55:50.120 INFO LicenseMgr - StackMgr init complete...
03-23-2020 20:55:50.120 INFO LMTracker - Setting default product type='enterprise'
03-23-2020 20:55:50.120 INFO LMTracker - this is not splunkd, will perform partial init
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=Acceleration state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=AdvancedSearchCommands state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=AdvancedXML state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=Alerting state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ArchiveToHdfs state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=Auth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=CustomRoles state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=DeployClient state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=DeployServer state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=DistSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=FwdData state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=GuestPass state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=KVStore state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=LDAPAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=LocalSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=MultifactorAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=MultisiteClustering state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=NontableLookups state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=RcvData state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=RcvSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=RollingWindowAlerts state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SAMLAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScheduledAlerts state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScheduledReports state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScheduledSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScriptedAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SearchheadPooling state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SigningProcessor state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SplunkWeb state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SubgroupId state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SyslogOutputProcessor state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=UnisiteClustering state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LicenseMgr - Tracker init complete...
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'licenses'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'pools'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'stacks'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'groups'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'slaves'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'localslave'
03-23-2020 20:55:50.125 INFO AdminManagerDispatch - added factory for admin handler: 'licensermessages'
03-23-2020 20:55:50.125 INFO AdminManagerDispatch - added factory for admin handler: 'scriptedwarning'
03-23-2020 20:55:50.125 INFO AdminManagerDispatch - added factory for admin handler: 'licenseusage'
03-23-2020 20:55:50.125 INFO dispatchRunner - Per-process handle limit is 512
03-23-2020 20:55:50.125 INFO dispatchRunner - Increasing per-process handle limit from '512' to '2048'
03-23-2020 20:55:50.125 INFO dispatchRunner - Successfully increased per-process handle limit from '512' to '2048'
03-23-2020 20:55:50.125 INFO dispatchRunner - registering build time modules, count=1
03-23-2020 20:55:50.125 INFO dispatchRunner - registering search time components of build time module name=vix
03-23-2020 20:55:50.125 INFO dispatchRunner - Getting search configuration data from: C:\Program Files\Splunk\etc\modules\parsing\config.xml
03-23-2020 20:55:50.128 INFO BundlesSetup - Setup stats for C:\Program Files\Splunk\etc: wallclock_elapsed_msec=32, cpu_time_used=0.03125, shared_services_generation=1, shared_services_population=1
03-23-2020 20:55:50.132 INFO UserManagerPro - Load authentication: forcing roles="admin, power, user"
03-23-2020 20:55:50.132 INFO PipelineComponent - Ingestion pipeline sets have been configured to use pipeline set policy=round_robin for multiple pipeline sets.
03-23-2020 20:55:50.135 INFO UserManager - Setting user context: splunk-system-user
03-23-2020 20:55:50.135 INFO UserManager - Done setting user context: NULL -> splunk-system-user
03-23-2020 20:55:50.135 INFO UserManager - Unwound user context: splunk-system-user -> NULL
03-23-2020 20:55:50.135 INFO UserManager - Setting user context: fbazangani
03-23-2020 20:55:50.135 INFO UserManager - Done setting user context: NULL -> fbazangani
03-23-2020 20:55:50.136 INFO dispatchRunner - search context: user="fbazangani", app="mltk-container", bs-pathname="C:\Program Files\Splunk\etc"
03-23-2020 20:55:50.138 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.140 INFO dispatchRunner - SearchHeadInitSearchMs=0
03-23-2020 20:55:50.140 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.140 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.140 INFO dispatchRunner - Executing the Search orchestrator and iterator model (dfs=0).
03-23-2020 20:55:50.140 INFO SearchOrchestrator - SearchOrchestrator getting constructed
03-23-2020 20:55:50.140 INFO SearchOrchestrator - Initialized the SRI
03-23-2020 20:55:50.141 INFO ISplunkDispatch - Not running in splunkd. Bundle replication not triggered.
03-23-2020 20:55:50.141 INFO SearchOrchestrator - Initialzing the run time settings for the orchestrator.
03-23-2020 20:55:50.142 INFO UserManager - Setting user context: fbazangani
03-23-2020 20:55:50.142 INFO UserManager - Done setting user context: NULL -> fbazangani
03-23-2020 20:55:50.143 INFO SearchOrchestrator - Creating the search DAG.
03-23-2020 20:55:50.143 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.296 WARN SearchOperator:inputcsv - sid:328a15c033a3708b_tmp The lookup table 'diabetes.csv' is invalid.
03-23-2020 20:55:50.296 INFO SearchOperator:rest - built uri=services/mltk-container/sync?count=0, server=https://127.0.0.1:8089
03-23-2020 20:55:50.296 ERROR DispatchThread - Error in 'rest' command: This command must be the first command of a search.
03-23-2020 20:55:50.298 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.298 WARN SearchOperator:inputcsv - sid:1584993349.17069 The lookup table 'diabetes.csv' is invalid.
03-23-2020 20:55:50.298 INFO SearchOperator:rest - built uri=services/mltk-container/sync?count=0, server=https://127.0.0.1:8089
03-23-2020 20:55:50.299 INFO ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\Python3.exe" "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py"
03-23-2020 20:55:51.946 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:52.341 INFO ScopedTimer - search.optimize 2.047
03-23-2020 20:55:52.341 INFO SearchPhaseGenerator - Failed to create phases using AST:Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment. Falling back to 2 phase mode.
03-23-2020 20:55:52.341 INFO SearchPhaseGenerator - Executing two phase fallback for the search=| inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:52.341 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:52.341 WARN SearchOperator:inputcsv - sid:1584993349.17069 The lookup table 'diabetes.csv' is invalid.
03-23-2020 20:55:52.341 INFO SearchOperator:rest - built uri=services/mltk-container/sync?count=0, server=https://127.0.0.1:8089
03-23-2020 20:55:52.342 INFO ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\Python3.exe" "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py"
03-23-2020 20:55:53.980 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.384 ERROR SearchPhaseGenerator - Fallback to two phase search failed:Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.389 ERROR SearchOrchestrator - Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.389 ERROR SearchStatusEnforcer - sid:1584993349.17069 Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.389 INFO SearchStatusEnforcer - State changed to FAILED due to: Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.393 INFO SearchStatusEnforcer - Enforcing disk quota = 10485760000
03-23-2020 20:55:54.404 INFO DispatchStorageManager - Remote storage disabled for search artifacts.
03-23-2020 20:55:54.404 INFO DispatchManager - DispatchManager::dispatchHasFinished(id='1584993349.17069', username='fbazangani')
03-23-2020 20:55:54.404 INFO UserManager - Unwound user context: fbazangani -> NULL
03-23-2020 20:55:54.404 INFO UserManager - Unwound user context: fbazangani -> NULL
03-23-2020 20:55:54.404 INFO LookupProviderFactory - Clearing out lookup shared provider map
03-23-2020 20:55:54.409 INFO ShutdownHandler - Shutting down splunkd
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Begin"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FileIntegrityChecker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_JustBeforeKVStore"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_KVStore"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_DFM"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Thruput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpInput1"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ExecSendInitialSigterm"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpOutput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_UdpInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FifoInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueueInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_WinEventLogInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SelfStorageArchiver"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_BucketMergeWorker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Scheduler"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SyslogOutput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ArchiveAndOneshot"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_MainThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Exec"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Tailing"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_PeerManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailQueueServiceThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeMonitor"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeManagerProcessor"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_DeploymentClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpClientPollingThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AsyncQueuedMessageDispatcherThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueueOutput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_OfflineFlusher"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_UploadReplicatedBucketThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Slave"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_CacheManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SlaveSearch"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Deployer"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Captain"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Select"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_IdataDO_Collector"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpOutput2"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_IndexerService"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Database1"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_LastIndexerLevel"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AWSMetering"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpInput2"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueueInputWorker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueue"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteFileSystem"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_LocalFileSystem"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SearchDispatch"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SSCCacheClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ExternalRendezvous"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_CatalogBackedConfSingleton"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_LoadLDAPUsers"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_MetricsManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Pipeline"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Queue"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_CallbackRunner"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_DmcProxyHttpClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Duo2FAHttpClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ApplicationLicenseChecker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_S3ConnectionPoolManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - Shutdown complete in 0 microseconds
03-23-2020 20:55:54.412 ERROR dispatchRunner - RunDispatch::runDispatchThread threw error: Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment

View solution in original post

0 Karma

farideh1371
Explorer

thank you for following up on my issue.

this is the search.log (I couldn't attach it 😞

03-23-2020 20:55:50.100 INFO dispatchRunner - Search process mode: freestanding (build a7f645ddaf91).
03-23-2020 20:55:50.102 INFO dispatchRunner - initing LicenseMgr in search process: nonPro=0
03-23-2020 20:55:50.102 INFO LicenseMgr - Initing LicenseMgr
03-23-2020 20:55:50.105 INFO ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
03-23-2020 20:55:50.105 INFO ServerConfig - Host name option is "".
03-23-2020 20:55:50.116 INFO ServerConfig - SSL session cache path enabled 0 session timeout on SSL server 300.000
03-23-2020 20:55:50.117 INFO LMConfig - serverName=MUCL95035 guid=6D139D41-58E6-407B-A1C9-8E1D9F452F44
03-23-2020 20:55:50.117 INFO LMConfig - connection_timeout=30
03-23-2020 20:55:50.119 INFO LMConfig - send_timeout=30
03-23-2020 20:55:50.119 INFO LMConfig - receive_timeout=30
03-23-2020 20:55:50.119 INFO LMConfig - key=license_warnings_update_interval not found in licenser stanza of server.conf, defaulting=0
03-23-2020 20:55:50.119 INFO LMConfig - squash_threshold=2000
03-23-2020 20:55:50.119 INFO LMConfig - strict_pool_quota=1
03-23-2020 20:55:50.119 INFO LMConfig - key=pool_suggestion not found in licenser stanza of server.conf, defaulting=''
03-23-2020 20:55:50.119 INFO LMConfig - key=test_aws_metering not found in licenser stanza of server.conf, defaulting=0
03-23-2020 20:55:50.119 INFO LMConfig - key=test_aws_product_code not found in licenser stanza of server.conf, defaulting=0
03-23-2020 20:55:50.119 INFO LicenseMgr - Initing LiacenseMgr runContext_splunkd=false
03-23-2020 20:55:50.119 INFO LMStackMgr - closing stack mgr
03-23-2020 20:55:50.119 INFO LMSlaveInfo - all slaves cleared
03-23-2020 20:55:50.119 INFO LMStackMgr - Initalized license_warnings_update_interval=auto
03-23-2020 20:55:50.120 INFO LMStack - Added type=download-trial license, from file=enttrial.lic, to stack=download-trial of group=Trial
03-23-2020 20:55:50.120 INFO LMStackMgr - created stack='download-trial'
03-23-2020 20:55:50.120 INFO LMStackMgr - have to auto-set active stack group='Trial' reason='invalid/missing group id' gidStr='' oldGid=Invalid
03-23-2020 20:55:50.120 INFO LMStackMgr - added pool auto_generated_pool_download-trial to stack download-trial
03-23-2020 20:55:50.120 INFO LMStackMgr - added pool auto_generated_pool_forwarder to stack forwarder
03-23-2020 20:55:50.120 INFO LMStackMgr - added pool auto_generated_pool_free to stack free
03-23-2020 20:55:50.120 INFO LMStackMgr - Initialized hideQuotaWarning = "0"
03-23-2020 20:55:50.120 INFO LMStackMgr - init completed [6D139D41-58E6-407B-A1C9-8E1D9F452F44,Trial,runContext_splunkd=false]
03-23-2020 20:55:50.120 INFO LicenseMgr - StackMgr init complete...
03-23-2020 20:55:50.120 INFO LMTracker - Setting default product type='enterprise'
03-23-2020 20:55:50.120 INFO LMTracker - this is not splunkd, will perform partial init
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=Acceleration state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=AdvancedSearchCommands state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=AdvancedXML state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=Alerting state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ArchiveToHdfs state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=Auth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=CustomRoles state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=DeployClient state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=DeployServer state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=DistSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=FwdData state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=GuestPass state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=KVStore state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=LDAPAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=LocalSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=MultifactorAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=MultisiteClustering state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=NontableLookups state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=RcvData state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=RcvSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=RollingWindowAlerts state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SAMLAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScheduledAlerts state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScheduledReports state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScheduledSearch state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=ScriptedAuth state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SearchheadPooling state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SigningProcessor state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SplunkWeb state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SubgroupId state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=SyslogOutputProcessor state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LMTracker - Setting feature=UnisiteClustering state=ENABLED (featureStatus=1)
03-23-2020 20:55:50.120 INFO LicenseMgr - Tracker init complete...
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'licenses'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'pools'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'stacks'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'groups'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'slaves'
03-23-2020 20:55:50.124 INFO AdminManagerDispatch - added factory for admin handler: 'localslave'
03-23-2020 20:55:50.125 INFO AdminManagerDispatch - added factory for admin handler: 'licensermessages'
03-23-2020 20:55:50.125 INFO AdminManagerDispatch - added factory for admin handler: 'scriptedwarning'
03-23-2020 20:55:50.125 INFO AdminManagerDispatch - added factory for admin handler: 'licenseusage'
03-23-2020 20:55:50.125 INFO dispatchRunner - Per-process handle limit is 512
03-23-2020 20:55:50.125 INFO dispatchRunner - Increasing per-process handle limit from '512' to '2048'
03-23-2020 20:55:50.125 INFO dispatchRunner - Successfully increased per-process handle limit from '512' to '2048'
03-23-2020 20:55:50.125 INFO dispatchRunner - registering build time modules, count=1
03-23-2020 20:55:50.125 INFO dispatchRunner - registering search time components of build time module name=vix
03-23-2020 20:55:50.125 INFO dispatchRunner - Getting search configuration data from: C:\Program Files\Splunk\etc\modules\parsing\config.xml
03-23-2020 20:55:50.128 INFO BundlesSetup - Setup stats for C:\Program Files\Splunk\etc: wallclock_elapsed_msec=32, cpu_time_used=0.03125, shared_services_generation=1, shared_services_population=1
03-23-2020 20:55:50.132 INFO UserManagerPro - Load authentication: forcing roles="admin, power, user"
03-23-2020 20:55:50.132 INFO PipelineComponent - Ingestion pipeline sets have been configured to use pipeline set policy=round_robin for multiple pipeline sets.
03-23-2020 20:55:50.135 INFO UserManager - Setting user context: splunk-system-user
03-23-2020 20:55:50.135 INFO UserManager - Done setting user context: NULL -> splunk-system-user
03-23-2020 20:55:50.135 INFO UserManager - Unwound user context: splunk-system-user -> NULL
03-23-2020 20:55:50.135 INFO UserManager - Setting user context: fbazangani
03-23-2020 20:55:50.135 INFO UserManager - Done setting user context: NULL -> fbazangani
03-23-2020 20:55:50.136 INFO dispatchRunner - search context: user="fbazangani", app="mltk-container", bs-pathname="C:\Program Files\Splunk\etc"
03-23-2020 20:55:50.138 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.140 INFO dispatchRunner - SearchHeadInitSearchMs=0
03-23-2020 20:55:50.140 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.140 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.140 INFO dispatchRunner - Executing the Search orchestrator and iterator model (dfs=0).
03-23-2020 20:55:50.140 INFO SearchOrchestrator - SearchOrchestrator getting constructed
03-23-2020 20:55:50.140 INFO SearchOrchestrator - Initialized the SRI
03-23-2020 20:55:50.141 INFO ISplunkDispatch - Not running in splunkd. Bundle replication not triggered.
03-23-2020 20:55:50.141 INFO SearchOrchestrator - Initialzing the run time settings for the orchestrator.
03-23-2020 20:55:50.142 INFO UserManager - Setting user context: fbazangani
03-23-2020 20:55:50.142 INFO UserManager - Done setting user context: NULL -> fbazangani
03-23-2020 20:55:50.143 INFO SearchOrchestrator - Creating the search DAG.
03-23-2020 20:55:50.143 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.296 WARN SearchOperator:inputcsv - sid:328a15c033a3708b_tmp The lookup table 'diabetes.csv' is invalid.
03-23-2020 20:55:50.296 INFO SearchOperator:rest - built uri=services/mltk-container/sync?count=0, server=https://127.0.0.1:8089
03-23-2020 20:55:50.296 ERROR DispatchThread - Error in 'rest' command: This command must be the first command of a search.
03-23-2020 20:55:50.298 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:50.298 WARN SearchOperator:inputcsv - sid:1584993349.17069 The lookup table 'diabetes.csv' is invalid.
03-23-2020 20:55:50.298 INFO SearchOperator:rest - built uri=services/mltk-container/sync?count=0, server=https://127.0.0.1:8089
03-23-2020 20:55:50.299 INFO ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\Python3.exe" "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py"
03-23-2020 20:55:51.946 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:52.341 INFO ScopedTimer - search.optimize 2.047
03-23-2020 20:55:52.341 INFO SearchPhaseGenerator - Failed to create phases using AST:Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment. Falling back to 2 phase mode.
03-23-2020 20:55:52.341 INFO SearchPhaseGenerator - Executing two phase fallback for the search=| inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:52.341 INFO SearchParser - PARSING: | inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response* *
03-23-2020 20:55:52.341 WARN SearchOperator:inputcsv - sid:1584993349.17069 The lookup table 'diabetes.csv' is invalid.
03-23-2020 20:55:52.341 INFO SearchOperator:rest - built uri=services/mltk-container/sync?count=0, server=https://127.0.0.1:8089
03-23-2020 20:55:52.342 INFO ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\Python3.exe" "C:\Program Files\Splunk\etc\apps\Splunk_ML_Toolkit\bin\fit.py"
03-23-2020 20:55:53.980 ERROR ChunkedExternProcessor - Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.384 ERROR SearchPhaseGenerator - Fallback to two phase search failed:Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.389 ERROR SearchOrchestrator - Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.389 ERROR SearchStatusEnforcer - sid:1584993349.17069 Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.389 INFO SearchStatusEnforcer - State changed to FAILED due to: Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment
03-23-2020 20:55:54.393 INFO SearchStatusEnforcer - Enforcing disk quota = 10485760000
03-23-2020 20:55:54.404 INFO DispatchStorageManager - Remote storage disabled for search artifacts.
03-23-2020 20:55:54.404 INFO DispatchManager - DispatchManager::dispatchHasFinished(id='1584993349.17069', username='fbazangani')
03-23-2020 20:55:54.404 INFO UserManager - Unwound user context: fbazangani -> NULL
03-23-2020 20:55:54.404 INFO UserManager - Unwound user context: fbazangani -> NULL
03-23-2020 20:55:54.404 INFO LookupProviderFactory - Clearing out lookup shared provider map
03-23-2020 20:55:54.409 INFO ShutdownHandler - Shutting down splunkd
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Begin"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FileIntegrityChecker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_JustBeforeKVStore"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_KVStore"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_DFM"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Thruput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpInput1"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ExecSendInitialSigterm"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpOutput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_UdpInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FifoInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueueInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_WinEventLogInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpInput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SelfStorageArchiver"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_BucketMergeWorker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Scheduler"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SyslogOutput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ArchiveAndOneshot"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_MainThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Exec"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Tailing"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_PeerManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailQueueServiceThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeMonitor"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeManagerProcessor"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_DeploymentClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpClientPollingThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AsyncQueuedMessageDispatcherThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueueOutput"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_OfflineFlusher"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_UploadReplicatedBucketThread"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Slave"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_CacheManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SlaveSearch"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Deployer"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Captain"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Select"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_IdataDO_Collector"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpOutput2"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_IndexerService"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Database1"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_LastIndexerLevel"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_AWSMetering"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpInput2"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueueInputWorker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteQueue"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_RemoteFileSystem"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_LocalFileSystem"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SearchDispatch"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_SSCCacheClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ExternalRendezvous"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_CatalogBackedConfSingleton"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_LoadLDAPUsers"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_MetricsManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Pipeline"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Queue"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_CallbackRunner"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_DmcProxyHttpClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_Duo2FAHttpClient"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_ApplicationLicenseChecker"
03-23-2020 20:55:54.409 INFO ShutdownHandler - shutting down level "ShutdownLevel_S3ConnectionPoolManager"
03-23-2020 20:55:54.409 INFO ShutdownHandler - Shutdown complete in 0 microseconds
03-23-2020 20:55:54.412 ERROR dispatchRunner - RunDispatch::runDispatchThread threw error: Error in 'fit' command: Error while initializing algorithm "MLTKContainer": local variable 'url' referenced before assignment

View solution in original post

0 Karma

pdrieger_splunk
Splunk Employee
Splunk Employee

Thanks for sharing: I see you mixed in a | rest call in your search above:
| inputlookup diabetes.csv \n| rest services/mltk-container/sync\n| fit MLTKContainer algo=binary_nn_classifier epochs=10 batch_size=8 response from BMI age blood_pressure diabetes_pedigree glucose_concentration number_pregnant serum_insulin skin_thickness into app:diabetes_classifier_model \n| eval response_prediction = if('predicted_response'>=0.5,1,0) \n| table response

Can you have a look what's in your .../etc/apps/mltk-container/local/containers.conf ?
Typically you should see a [dev] stanza with filled attributes like api_url=... - In case you have a valid running container you should see the correct urls there - if those are missing then this would lead to the error you describe above. Can you verify?

state_larson_ti
Explorer

@pdrieger_splunk 

I am having this issue, and your instructions were very helpful.  my .../etc/apps/mltk-container/local/containers.conf only has a [default] stanza.

I am going to try to specify api_url and more, but could you point me to a "standard" [dev] example?

Thanks,

T

0 Karma

pdrieger_splunk
Splunk Employee
Splunk Employee

Hi @state_larson_ti , typically when you start a development container from the DLTK container management page, the stanzas in the containers.conf are created automatically.

Screenshot 2020-07-06 at 13.31.35.png

When you have a running dev container enabled like in this screen, your local/containers.conf should look similar like this:

[default]

[__dev__]
api_url = http://localhost:32768
cluster = docker
id = fa99644474e76340b97d1a3c3ce3c2ec4298debfc591b0b2d47f625586aeecd5
image = mltk-container-golden-image-gpu
jupyter_url = http://localhost:8888
runtime = None
tensorboard_url = http://localhost:6006
spark_url = http://localhost:4040

 

Hope this is helpful?

state_larson_ti
Explorer

Thank you @pdrieger_splunk . 

When attempting to start the Golden Image GPU (3.2), I get the error "404 Client Error: Not Found ("No such image: phdrieger/mltk-container-golden-image-gpu:latest")".

mltk.jpg

 

You were correct, I did not know I needed to start the containers before testing a Model.

-T

0 Karma

pdrieger_splunk
Splunk Employee
Splunk Employee

This makes sense. Yes, you need to get the image downloaded. Either you manually docker pull the image OR when you start the dev container for the first time it should try to automatically pull the image if it is not available locally.

state_larson_ti
Explorer

Okay - Docker is new to me, so when I went to make the manual pull:

docker pull phdrieger/mltk-container-golden-image-gpu

 

and it stopped with 

write /var/lib/docker/tmp/GetImageBlob582606830: no space left on device

 

That I can fix :-).  Thanks so much!

0 Karma

pdrieger_splunk
Splunk Employee
Splunk Employee

Hi @farideh1371, you likely need wait for DLTK to resync your new container and the containers.conf file which has not yet a valid url for your container listed. You can force resync with | rest services/mltk-container/sync in the DLTK. Can you check if that resolved it? Thanks.

0 Karma

farideh1371
Explorer

Hello,
Thank you for your answer,
I did what you suggested but still have the same problem.

0 Karma

pdrieger_splunk
Splunk Employee
Splunk Employee

Can you check how your local containers.conf looks like? It seems the url can not be retrieved properly. If you can share or attach your search.log that contains the python errors as occurred in the MLTKContainer.py this will also be helpful to troubleshoot. I remember I saw this error one time, but it didn't happen any more. Thanks for your help to share more infos!

0 Karma