Gaussian Naive Bayes

Implement Gaussian Naive Bayes classifier using scikit-learn on iris dataset.

Step 01: Import the library

Imported various modules like datasets, mertics, and GaussianNB from differnt libraries.

Step 02: Setup the Data

Here used datasets to load the inbuilt iris dataset and created objects X and y to store the data and the target value respectively.

Step 03: Model and its Score

• Take GaussianNB as a Machine Learning model to fit the data.

• Then predicted the output by passing X_test and also stored real target in expected_y.

• then printed classification report and confusion matrix for the classifier.

To Identify Overfitting Machine Learning Models In Scikit-Learn

 

Identify Overfitting Machine Learning Models In Scikit-Learn

STEP 01

To evaluate decision tree performance on train and test sets with different tree depths

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn.tree import DecisionTreeClassifier
from matplotlib import pyplot

STEP 02

synthetic classification dataset

from sklearn.datasets import make_classification

STEP 03 

define dataset

X, y = make_classification(n_samples=10000, n_features=20, n_informative=5, n_redundant=15, random_state=1)

STEP 04 

summarize the dataset

print(X.shape, y.shape)

STEP 05 

To Split a dataset into train and test sets

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

STEP 06 

create dataset

X, y = make_classification(n_samples=10000, n_features=20, n_informative=5, n_redundant=15, random_state=1)

STEP 07 

To split into train test sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

STEP 08 

summarize the shape of the train and test sets

print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)

STEP 09 

define the tree depths to evaluate

values = [i for i in range(1, 51)]

STEP 10 

To evaluate a decision tree for each depth for i in values:

1. To configure the model

model = DecisionTreeClassifier(max_depth=i)

2. To fit model on the training dataset

model.fit(X_train, y_train)

3. To evaluate on the train dataset

train_yhat = model.predict(X_train)
train_acc = accuracy_score(y_train, train_yhat)
train_scores.append(train_acc)

4. To evaluate on the test dataset

test_yhat = model.predict(X_test)
test_acc = accuracy_score(y_test, test_yhat)
test_scores.append(test_acc)

5. To summarize progress

print(‘>%d, train: %.3f, test: %.3f’ % (i, train_acc, test_acc))

To display Pearson r and p- value in seaborn for visualization plots

To display Pearson r and p- value in seaborn:

Decision tree example

Decision tree example on Loan dataset

Step#01

Import necessary libraries 

import pandas as pd

import seaborn as sns

import matplotlib.pyplot as plt

%matplotlib inline

import numpy as np

Step#02

Read the dataset frm directory Path

df=pd.read_csv('/content/drive/MyDrive/dataset/loan_data.csv')

df.head() # shows first 5 rows from dataset

Step#03

I have choose Google Colab, so importing dataset from my drive directly

from google.colab import drive

drive.mount('/content/drive')

Step#04

first need to convert string values in variable'purpose' to some dummy variables. As, the purpose column is categorical.
That means we need to transform them using dummy variables so sklearn will be able to understand them. Let's do this in one clean step
using pd.get_dummies.
Let's show you a way of dealing with these columns that can be expanded to multiple categorical features if necessary.
1. Create a list of 1 element containing the string 'purpose'. Call this list cat_feats

cat_feats = ['purpose']

Step#05

2. Now use pd.get_dummies(loans,columns=cat_feats,drop_first=True) to create a fixed larger dataframe that has new feature columns with dummy variables. Set this dataframe as final_data.

final_data = pd.get_dummies(df, columns = cat_feats, drop_first = True)

Step#06

Compare both dataset now

df.head()

final_data.head()

Step#07

from sklearn.model_selection import train_test_split

X = final_data.drop('not.fully.paid', axis = 1) #drop not.fully.paid variable from x-axis

y = final_data['not.fully.paid']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20 , random_state= 101)

Step#08

Let's start by training a single decision tree.
first Import DecisionTreeClassifier
and then Create an instance of DecisionTreeClassifier() called dtree and fit it to the training data.

from sklearn.tree import DecisionTreeClassifier

dtree = DecisionTreeClassifier()

dtree.fit(X_train, y_train,sample_weight=None, check_input=True, X_idx_sorted=None)

Step#09

Create predictions from the test set and create a classification report.

y_predict = dtree.predict(X_test)

from sklearn.metrics import classification_report

print(classification_report(y_test, y_predict))