create account

advanced mode

Machine Learning with Scikit-Learn - [Part 44] by cristi

View this thread on steemit.com
View on official site
· @cristi ·
$12.80
Machine Learning with Scikit-Learn - [Part 44]
In  this tutorial we're going to discuss and code another method of automatic feature selection in scikit-learn, which is model based selection. 

According to the textbook we are following, model based selection uses a supervised model to compute the importance that each feature carries. After making the selection, it will only keep the most importance features. 

Since it needs something to determine the importance of each feature, this means that the algorithm used has to be able to do that. The algorithm has to have one or more methods to determine feature importance. And in scikit-learn, we know that two of these models are Decision Trees and ensembles of trees, like Random Forests. 

In this tutorial we're going to use a RandomForestClassifier for our model based selection example. The algorithm in scikit-learn for model based selection is SelectFromModel and the parameters it requires include:

- the algorithm to determine the importance (in this case RandomForestClassifier)
- parameters for the classifier (n_estimators, etc)
- and a threshold - to make the selection - in this case 'median'

Once we have it, we fit it onto the data and then we apply it onto our training set. We then look at both the original training set and the training set after we applied the select method. We will ultimately do some visualization and then train an algorithm on both sets to be able to compare their performances. 

The trained algorithm on the data with the select method applied has a better performance than the one trained on the original dataset. Please see the full video to have a comprehensive understanding of this:

<center><iframe width="560" height="315" src="https://www.youtube.com/embed/VvJcmxnAmxA" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></center>
___
### <center>To stay in touch with me, follow @cristi</center>   
___

[Cristi Vlad](http://cristivlad.com) Self-Experimenter and Author
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
json_metadata{"tags":["machine-learning","python","programming","science"],"users":["cristi"],"image":["https://img.youtube.com/vi/VvJcmxnAmxA/0.jpg"],"links":["https://www.youtube.com/embed/VvJcmxnAmxA","http://cristivlad.com"],"app":"steemit/0.1","format":"markdown"}
created2018-01-27 16:19:09
active2018-01-27 19:45:03
last_payout2018-02-03 16:19:09
depth0
children5
net_rshares0
vote_rshares0
children_abs_rshares0
cashout_time1969-12-31 23:59:59
max_cashout_time1969-12-31 23:59:59
total_vote_weight0
reward_weight10,000
total_payout_value11.418 SBD
curator_payout_value1.378 SBD
author_rewards2,410
net_votes30
root_authorcristi
root_permlinkmachine-learning-with-scikit-learn-part-44
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
allow_repliestrue
allow_votestrue
allow_curation_rewardstrue
beneficiaries[]
root_title"Machine Learning with Scikit-Learn - [Part 44]"
pending_payout_value0.000 SBD
total_pending_payout_value0.000 STEEM
author_reputation128,152,010,851,740
promoted0.000 SBD
body_length0
reblogged_by[]
author_curate_reward""
vote details (30)

vote your-acct "cristi" "machine-learning-with-scikit-learn-part-44" 100 true
post_comment your-acct "re-cristi-machine-learning-with-scikit-learn-part-44-20181115t023539936z" "cristi" "machine-learning-with-scikit-learn-part-44" "" "your reply.." "{}" true

View on official site
@dani74 ·
Nice post
👍  
json_metadata{"tags":["machine-learning"],"app":"steemit/0.1"}
created2018-01-27 16:20:24
active2018-01-27 16:20:24
last_payout2018-02-03 16:20:24
depth1
children0
net_rshares0
vote_rshares0
children_abs_rshares0
cashout_time1969-12-31 23:59:59
max_cashout_time1969-12-31 23:59:59
total_vote_weight0
reward_weight10,000
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
author_rewards0
net_votes1
root_authorcristi
root_permlinkmachine-learning-with-scikit-learn-part-44
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
allow_repliestrue
allow_votestrue
allow_curation_rewardstrue
beneficiaries[]
root_title"Machine Learning with Scikit-Learn - [Part 44]"
pending_payout_value0.000 SBD
total_pending_payout_value0.000 STEEM
author_reputation80,059,701,542
promoted0.000 SBD
body_length0
reblogged_by[]
author_curate_reward""
vote details (1)

vote your-acct "dani74" "re-cristi-machine-learning-with-scikit-learn-part-44-20180127t162004350z" 100 true
post_comment your-acct "re-dani74-re-cristi-machine-learning-with-scikit-learn-part-44-20181115t023539943z" "dani74" "re-cristi-machine-learning-with-scikit-learn-part-44-20180127t162004350z" "" "your reply.." "{}" true

View on official site
@khanaasim ·
$0.17
noise is selected by the selection function and is preferred over the real ones. why is it so?
👍  
json_metadata{"tags":["machine-learning"],"app":"steemit/0.1"}
created2018-01-27 16:23:42
active2018-01-27 19:45:03
last_payout2018-02-03 16:23:42
depth1
children2
net_rshares0
vote_rshares0
children_abs_rshares0
cashout_time1969-12-31 23:59:59
max_cashout_time1969-12-31 23:59:59
total_vote_weight0
reward_weight10,000
total_payout_value0.150 SBD
curator_payout_value0.018 SBD
author_rewards32
net_votes1
root_authorcristi
root_permlinkmachine-learning-with-scikit-learn-part-44
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
allow_repliestrue
allow_votestrue
allow_curation_rewardstrue
beneficiaries[]
root_title"Machine Learning with Scikit-Learn - [Part 44]"
pending_payout_value0.000 SBD
total_pending_payout_value0.000 STEEM
author_reputation198,310,394,747
promoted0.000 SBD
body_length0
reblogged_by[]
author_curate_reward""
vote details (1)

vote your-acct "khanaasim" "re-cristi-machine-learning-with-scikit-learn-part-44-20180127t162339171z" 100 true
post_comment your-acct "re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20181115t023539949z" "khanaasim" "re-cristi-machine-learning-with-scikit-learn-part-44-20180127t162339171z" "" "your reply.." "{}" true

View on official site
@cristi ·
I just answered this question on the video. It seems that some noise features have more importance over some of the original ones. Basically, some of the original features may be completely irrelevant to the training of the algorithm...
👍  
json_metadata{"tags":["machine-learning"],"app":"steemit/0.1"}
created2018-01-27 16:38:36
active2018-01-27 19:45:03
last_payout2018-02-03 16:38:36
depth2
children1
net_rshares0
vote_rshares0
children_abs_rshares0
cashout_time1969-12-31 23:59:59
max_cashout_time1969-12-31 23:59:59
total_vote_weight0
reward_weight10,000
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
author_rewards0
net_votes1
root_authorcristi
root_permlinkmachine-learning-with-scikit-learn-part-44
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
allow_repliestrue
allow_votestrue
allow_curation_rewardstrue
beneficiaries[]
root_title"Machine Learning with Scikit-Learn - [Part 44]"
pending_payout_value0.000 SBD
total_pending_payout_value0.000 STEEM
author_reputation128,152,010,851,740
promoted0.000 SBD
body_length0
reblogged_by[]
author_curate_reward""
vote details (1)

vote your-acct "cristi" "re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20180127t164105243z" 100 true
post_comment your-acct "re-cristi-re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20181115t023539953z" "cristi" "re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20180127t164105243z" "" "your reply.." "{}" true

View on official site
@khanaasim ·
may be it is so but confusion still persists
json_metadata{"tags":["machine-learning"],"app":"steemit/0.1"}
created2018-01-27 19:45:03
active2018-01-27 19:45:03
last_payout2018-02-03 19:45:03
depth3
children0
net_rshares0
vote_rshares0
children_abs_rshares0
cashout_time1969-12-31 23:59:59
max_cashout_time1969-12-31 23:59:59
total_vote_weight0
reward_weight10,000
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
author_rewards0
net_votes0
root_authorcristi
root_permlinkmachine-learning-with-scikit-learn-part-44
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
allow_repliestrue
allow_votestrue
allow_curation_rewardstrue
beneficiaries[]
root_title"Machine Learning with Scikit-Learn - [Part 44]"
pending_payout_value0.000 SBD
total_pending_payout_value0.000 STEEM
author_reputation198,310,394,747
promoted0.000 SBD
body_length0
reblogged_by[]

vote your-acct "khanaasim" "re-cristi-re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20180127t194500682z" 100 true
post_comment your-acct "re-khanaasim-re-cristi-re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20181115t023539957z" "khanaasim" "re-cristi-re-khanaasim-re-cristi-machine-learning-with-scikit-learn-part-44-20180127t194500682z" "" "your reply.." "{}" true

View on official site
@hernanjosegb ·
Good tutorial my friend, very simple to explain, thank you very much and greetings my brother, good content in Steemit!
json_metadata{"tags":["machine-learning"],"app":"steemit/0.1"}
created2018-01-27 16:29:03
active2018-01-27 16:29:03
last_payout2018-02-03 16:29:03
depth1
children0
net_rshares0
vote_rshares0
children_abs_rshares0
cashout_time1969-12-31 23:59:59
max_cashout_time1969-12-31 23:59:59
total_vote_weight0
reward_weight10,000
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
author_rewards0
net_votes0
root_authorcristi
root_permlinkmachine-learning-with-scikit-learn-part-44
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
allow_repliestrue
allow_votestrue
allow_curation_rewardstrue
beneficiaries[]
root_title"Machine Learning with Scikit-Learn - [Part 44]"
pending_payout_value0.000 SBD
total_pending_payout_value0.000 STEEM
author_reputation199,530,640,596
promoted0.000 SBD
body_length0
reblogged_by[]

vote your-acct "hernanjosegb" "re-cristi-machine-learning-with-scikit-learn-part-44-20180127t163140024z" 100 true
post_comment your-acct "re-hernanjosegb-re-cristi-machine-learning-with-scikit-learn-part-44-20181115t023539960z" "hernanjosegb" "re-cristi-machine-learning-with-scikit-learn-part-44-20180127t163140024z" "" "your reply.." "{}" true