Hyperparameter Tuning In Machine Learning

In machine learning, we have two types of parameters:

1. Model Parameter:

   Model parameters are configuration variables that are internal to the model, and a model learns them on its own.

2. Model Hyperparameter:

   Hyperparameters are those parameters that are explicitly defined by the user to control the learning process.

Hyperparameter Tunning Implementation

1. Importing dependencies

    import numpy as np
    import pandas as pd
    import sklearn.datasets
    from sklearn.svm import SVC
    from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
   

2. Loading breast cancer dataset

    breast_cancer_data = sklearn.datasets.load_breast_cancer()
   

3. Creating data frame

    df = pd.DataFrame(data=breast_cancer_data.data,columns=breast_cancer_data.feature_names)
    # Adding target column in datafram
    df['Target'] = breast_cancer_data.target
   

4. Checking for rows and columns

    df.shape
   

5. Checking for missing values in the data

    df.isnull().sum()
   

6. Checking for distribution of target feature

    df['Target'].value_counts()
   

7. Segregating the features and target

    X = df.drop(columns='Target',axis=1)
    y = df['Target']
   

8. GridSearchCV

    # Loading model 
    model = SVC()
    # Hyperparameter tunning
    parameters = {
        'kernel':['linear','poly','rbf','sigmoid'],
        'C':[1,5,10,20]
    }
    # Grid Search
    classifier = GridSearchCV(model,param_grid=parameters,cv=5)
    # Fitting the data to our model
    classifier.fit(X,y)
   

   

1. Checking for best parameters selected by GridSearchCV

    classifier.best_params_
   

   Output:

   {'C': 10, 'kernel': 'linear'}

2. Printing accuracy of SVC model with GridSearchCV

    print(f'The highest accuracy for svc is {round(classifier.best_score_*100,2)}%')
   

   Output:

   The highest accuracy for svc is 95.26%

3. RandomizedSearchCV

    # Loading model 
    model = SVC()
    # Hyperparameter tunning
    parameters = {
        'kernel':['linear','poly','rbf','sigmoid'],
        'C':[1,5,10,20]
    }
    # Randomizes Search CV
    classifier = RandomizedSearchCV(model,parameters,cv=5)
    classifier.fit(X,y)
   

   

4. Checking for best parameters selected by RandomizedSearchCV

    classifier.best_params_
   

   Output:

   {'kernel': 'linear', 'C': 10}

5. Printing accuracy of SVC model with RandomizedSearchCV

    print(f'The highest accuracy for svc is {round(classifier.best_score_*100,2)}%')
   

   Output:

   The highest accuracy for svc is 95.26%