Day 3

 Lets start with Random Forest 

1. It combines the output of multiple decision tree to reach the single result.

2. It handles both regression and classification problems so we wont be having problems we encountered on Ordinary Square Method.

3. it is made of many decision tree but I am yet to learn decision tree.

Lets move back and learn decision tree first.

1. Similar to Random forest as it can handle both regression and classification. 

Lets drive into some math before we start:

1. Entropy (Information Gain):

Measure's the impurity or disorder of set of data. High entropy means the data is more mixed up (e.g., equal numbers of different classes), while low entropy means it's more pure (mostly one class).

2. Information Gain

it is a decrease in entropy achieved by splitting the data on particular attribute. One of the main attribute of decision tree is that it gives highest information gain, as this leads to most information splits.

Formula for Entropy:

Entropy(S) = - Σ (p_i * log2(p_i))
where:
  • S is the set of examples
  • p_i is the proportion of examples in S that belong to class i
Formula for Information Gain:
InformationGain(S, A) = Entropy(S) - Σ ((|S_v| / |S|) * Entropy(S_v)) 
where:
  • S is the set of examples
  • A is the attribute we're considering splitting on
  • S_v is the subset of examples in S that have value v for attribute A
2. Gini Impurity
lower the Gini means more pure node

Formula for Gini Impurity:
Gini(S) = 1 - Σ (p_i)^2
3. Attribute Selection
For every node, decision tree information gain or gini impurity to choose the best attribute
to split on at each node. The goal is the maximize the purity of the resulting child node
4. Recursive Partitioning
Decision trees build their structure by recursively splitting the data based on the chosen
attributes. This process continues until a stopping criterion is met (e.g., maximum depth, 
minimum number of samples per leaf).


Comments

Post a Comment

Popular posts from this blog

I am starting a new course Course objective In this course, we are going to focus on the following learning objectives: 1. Understand the theory and intuition behind Multiple Linear Regression. 2. Import Key python libraries, dataset and perform data visualization 3. Perform exploratory data analysis and standardize the training and testing data. 4. Train and Evaluate linear regression model using Sci-kit Learn library. 5. Understand the difference between various regression models KPIs such as MSE, RMSE, MAE, R2, and adjusted R2. 6. Assess the performance of regression models and visualize the performance of the best model using various KPIs. 7. Understand the distribution and relationship of data Project Structure The hands on project on Life Expectancy Prediction Using Machine Learning is divided into following tasks: Task #1: Understand the Problem Statement and Business Case Task #2: Import Datasets and Libraries Task #3: Perform Data Visualization and Exploratory Data Analysis T...
Read more

Linear Models Battle

Image
Linear Models Battle: Who Wins Disaster Tweet Prediction? On day three of my ML journey, I'm exploring different linear models to see how they work and which ones are most accurate. Before I start coding, I want to understand the two main learning approaches in machine learning: Supervised Learning 1. 1 Linear model Let's start by understanding what a linear model is. It's like a special tool that helps us make guesses (or predictions) when one thing seems to be related to another. Imagine this: Your favorite basketball player, Steph Curry, is practicing his 3-pointers. You notice a pattern: Scenario 1: He takes 100 shots and makes 10 of them. Scenario 2: He takes 200 shots and makes 40 of them. Now, you want to guess how many 3-pointers he might make if he takes 250 shots. A linear model can help us with that! Think of it like this: The Dots: Each scenario (100 shots, 10 makes; 200 shots, 40 makes) is like a dot on a piece of graph paper. The Line: A linear model trie...
Read more