How to know/measure if a model is learning?

[komnina]

Hi there!

I'm a newbie in machine learning world and I have a doubt.

After training and deploying a model, how to know if it is learning? Or, how to teach it?

Is this an automatic process for each use?

[aoliner_splunk]

Hi komnina,

The 'fit' command trains or updates a model; this is the 'learning' step in machine learning. The 'apply' command uses a trained model to make predictions. The model doesn't learn anything new when you apply it. You can fit the model on a schedule, using a saved search, if you'd like it to continue learning from new data, but you should validate that it's learning what you want it to.

The Splunk Machine Learning Toolkit has guided modeling dashboards called 'assistants' that walk you through the process of fitting, validating, and deploying a model (including re-training on a schedule).

Hope that helps!

[komnina]

Hi friend!

Thanks for your response.

So the 'machine' learn from several re-trainings (schedule), is this?

Can I use 'assistents' (like 'Predict Numeric Fields' and your statistics) to evaluate whether the model improved from a training to another? i.e. comparing statistics of re-trainings. Is it a good way?

kind regards.

[aoliner_splunk]

Correct: the machine "learns" by training or updating a model. You can do each of those either manually or on a schedule.

The assistants have a "Load Existing Settings" tab, which will show you the previous models you built, what their various parameters were, and what validation statistics they achieved. Comparing the statistics of different models is exactly what that tab is for. 🙂

[komnina]

Hi, good morning.

Thank you very much. It is very cool.

;>)

kind regards.