## Data Science Day 9:

### Feature Selection:

It is used to minimize the dimensionality of the dataset by selecting the particular “important” attributes (features), and dropping the less important ones.

**Sequential Forward Selection (SFS)**

Forward Selection is a selection method starting from the null set, and building an attribute set by ** adding the feature that maximizes the value of the Objective Function** in each step.

**Sequential Forward Selection Algorithm**

- Y= ∅.
- where F(x) is maximized.
- Y∪{x}, and repeat step 2.

Apply the complete SFS Algorithm, we will have Y=X. We can also set a stop criterion if we are satisfied with an output value.

**Example:**

**Apply feature selection on the objective function without a stopping criterion.**

**Solution:**

- Check the Objective function value for x1, x2, x3 and x4.

If x1=1, we have F(1,0,0,0)=0

If x2=1, we have F(0,1,0,0)=0

If x3=1, we have F(0,0,1,0)=-1

If x4=1, we have F(0,0,0,1)=4

Since x4 produce the highest value for the objective function, we have a winner x4.

2. Check the Objective function value for {x4}∪{xi}

If x1=1, we have F(1,0,0,1)=4

If x2=1, we have F(0,1,0,1)=4

If x3=1, we have F(0,0,1,1)=3

Since x1 and x2 produce the same value, we can pick either x1 or x2.

I will pick x1 for simplicity.

3. Check the Objective function value for {x4,x1}∪{xi}

If x2=1, we have F(1,1,0,1)=7

If x3=1, we have F(1,0,1,1)=3

Since x2 produce the highest value for the objective function, 7, we have a winner x2 in step 3.

3. Check the Objective function value for {x4,x1,x2}∪{x3}

If x3=1, we have F(1,1,1,1)= 6

By finishing this step, we produced the whole set.

I see Feature selection, Forward Selection everywhere, thanks to *Douglas Rumbaugh*‘s Data Mining Class notes, I finally understood the reason behind it!

**Happy studying!** 🦀