Feature Selection 1

Data Science Day 9:

Feature Selection:
It is used to minimize the dimensionality of the dataset by selecting the particular “important” attributes (features), and dropping the less important ones.

Sequential Forward Selection (SFS)

Forward Selection is a selection method starting from the null set, and building an attribute set by adding the feature that maximizes the value of the Objective Function in each step.

Sequential Forward Selection Algorithm

  1. Y= ∅.
  2. x in X where F(x) is maximized.
  3. Y∪{x}, and repeat step 2.

Apply the complete SFS Algorithm, we will have Y=X. We can also set a stop criterion if we are satisfied with an output value.

Example:

Apply feature selection on the objective function F(x_{i}) without a stopping criterion.

F(x1,x2,x3,x4)= 3x1*x2 -x3+ 4x4

qimono / Pixabay

Solution:

F(x1,x2,x3,x4)= 3x1*x2 -x3+ 4x4

  1. Check the Objective function value for x1, x2, x3 and x4.

If x1=1, we have F(1,0,0,0)=0
If x2=1, we have F(0,1,0,0)=0
If x3=1, we have F(0,0,1,0)=-1
  If x4=1, we have F(0,0,0,1)=4 
Since x4 produce the highest value for the objective function, we have a winner x4.

2. Check the Objective function value for {x4}∪{xi}
If x1=1, we have F(1,0,0,1)=4
If x2=1, we have F(0,1,0,1)=4
If x3=1, we have F(0,0,1,1)=3
Since x1 and x2 produce the same value, we can pick either x1 or x2.
I will pick x1 for simplicity.

3. Check the Objective function value for {x4,x1}∪{xi}
If x2=1, we have F(1,1,0,1)=7
If x3=1, we have F(1,0,1,1)=3
Since x2 produce the highest value for the objective function, 7, we have a winner x2 in step 3.

3. Check the Objective function value for {x4,x1,x2}∪{x3}
If x3=1, we have F(1,1,1,1)= 6
By finishing this step, we produced the whole set.

 

I see Feature selection, Forward Selection everywhere, thanks to Douglas Rumbaugh‘s Data Mining Class notes, I finally understood the reason behind it!

Happy studying! 🦀

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Social media & sharing icons powered by UltimatelySocial
%d bloggers like this: