Understanding Sweep Parameters module in Azure Machine Learning

Tweet about this on TwitterShare on LinkedInShare on Google+Share on Facebook

When you train a machine learning algorithm it is very important that you choose right set of parameters. When you don’t understand the in-side out of that algorithm it might be very difficult to choose and fine tune the parameters. Even if you understand the algorithm well, it might be daunting to run different iteration of training and evaluate a model with different combination of parameters – consider Neural Networks.

Microsoft Azure Machine Learning comes with a handy option to address the same with a module called Sweep Parameters. This module takes an untrained model along with training and validation data set and generates optimum parameter settings with just clicks.

I will give you a walk through on how to train and optimize a two-class neural network model with Sweep Parameters.

Below is the model to train the neural network with the default parameter settings.

two class neural network

Below is the performance of the model we found by visualizing the output of the Evaluate Model. To understand the Evaluate Model and its output, I have a separate blog post here.

visualising the output of the Evaluate Model

As you can notice the accuracy of the model is 0.971. Let’s try to improve the accuracy through finding optimized the parameter setting with Sweep Parameters. You can access the official documentation page for the same here.

sweep parameters

Here we have chosen “Entire grid” for Sweeping mode. This option is useful for cases where you don’t know what the best parameter settings might be and want to try many parameters. It loops over a grid predefined by the system to try different combinations and identify the best learner.

Alternatively you can choose “Random Sweep” and specify the maximum number of runs that you want the module to execute. It will randomly select parameter values over a system-defined range. This option is suggested when you want to increase model performance using the metrics of your choice and conserve computing resources.

Let’s now investigate what optimized parameters the module is suggesting:

visualize output of the sweep parameters

Either you can pick up best value from above or try visualizing the other output of the Sweep Parameters, to just find suggested best possible parameters. And then allow to the parameters to your model to get the best performance.

Well, this might not be the ultimate way to improve the performance of your model but certainly a handy tool during experiment phase.

About Sumit Mund

Sumit Mund is a big data analytics consultant with about a decade of industry experience. At Mund Consulting Ltd, he is a director and acts as the lead consultant. He is an expert in machine learning, predictive analytics, Apache Spark, Python, C#, R, and Scala; Sumit has an active interest in Artificial Intelligence. He has extensive experience working with most of Microsoft Data Analytics tools and Big Data platforms like HDInsight. He is a Certified Developer on Apache Spark and also Microsoft Certified Solution Expert (MCSE in Business Intelligence). Sumit regularly engages on social media platforms through his tweets, blogs, and LinkedIn profile, and often gives talks at industry conferences and local user group meetings.

One thought on “Understanding Sweep Parameters module in Azure Machine Learning

  1. Hi Sumit

    Do you know how Azure calculate “Sweep results” (output of Tune Model Hyperparameters) or kindly direct me to the places I can find more information?

    I follow Azure tutorial on “How to perform cross-validation with a parameter sweep” here (https://msdn.microsoft.com/library/azure/038d91b6-c2f2-42a1-9215-1f2c20ed1b40) using a neural network regression. The evaluation results from the cross validated model is much worse than those reported in the Sweep results (output of Tune Model Hyperparameters). I assumed the mean evaluation results (across 10 folds) from cross validate model should match the best results from ” Sweep results” because they share the same data input and trained by the same best model. The results are very different though, In my experiment, mean evaluation results are much worse than the best Sweep results. Could you provide any insights? Thank you.

    Best regards.


Leave a Reply

Your email address will not be published. Required fields are marked *

− 3 = one