<center>![Resources #91.png](https://steemitimages.com/DQmTPxdiK19inHzTKrsVstyBwC4Y2ib6prX2DCQxM2CiBXZ/Resources%20%2391.png)</center> ___ While studying backpropagation, I came across this great mathematical explanation or breakdown of how it goes. Matt Mazur, a data scientist working for _Help Scout_, wrote about it on his personal blog. I am so excited about the mathematical details that power neural networks that I enjoy writing down on a piece of paper how the actual computations work, both for forward propagation (the logits and the activations) and for backpropagation with gradient descent (the partial derivatives). This passion and excitement rose inside of me as doing the courses in the Deep Learning specializations of Professor Andrew Ng at Coursera. This part of neural networks is greatly explain in the first course. Anyway, backpropagation simply means back propagating the error you get as you reach the output of a network, with the purpose of updating the weights to minimize this error. Matt details both the forward pass and the backward pass through a neural network, with the equations and derivatives that they imply. So, if you're a geek that's excited about this stuff, I'd recommend taking a piece of paper and replicating the work done by Matt on his blog. Also, do check the visualizations he provides as they enhance the understanding of backprop. <center>[Backpropagation Step by Step - [The Mathematical Approach]](https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/)</center> ___ ### <center>To stay in touch with me, follow @cristi</center> ___ [Cristi Vlad](http://cristivlad.com) Self-Experimenter and Author
post_id | 30,572,386 |
---|---|
author | cristi |
permlink | backpropagation-step-by-step-the-mathematical-approach |
category | deep-learning |
json_metadata | "{"format": "markdown", "links": ["https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/", "http://cristivlad.com"], "app": "steemit/0.1", "tags": ["deep-learning", "science", "programming", "python"], "users": ["cristi"], "image": ["https://steemitimages.com/DQmTPxdiK19inHzTKrsVstyBwC4Y2ib6prX2DCQxM2CiBXZ/Resources%20%2391.png"]}" |
created | 2018-02-05 13:40:39 |
last_update | 2018-02-05 13:40:39 |
depth | 0 |
children | 11 |
net_rshares | 1,582,985,490,529 |
last_payout | 2018-02-12 13:40:39 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 9.776 SBD |
curator_payout_value | 1.212 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 1,687 |
author_reputation | 128,167,460,043,555 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
author_curate_reward | "" |
voter | weight | wgt% | rshares | pct | time |
---|---|---|---|---|---|
pharesim | 0 | 76,449,164,698 | 0.02% | ||
gavvet | 0 | 288,552,104,770 | 2% | ||
dragonslayer109 | 0 | 44,954,727,690 | 2% | ||
thecryptofiend | 0 | 19,655,938,781 | 25% | ||
coinbitgold | 0 | 45,252,845,457 | 100% | ||
hitmeasap | 0 | 14,622,678,566 | 7% | ||
jacor | 0 | 1,232,038,952 | 2% | ||
jens84 | 0 | 39,718,707,151 | 25% | ||
team-leibniz | 0 | 31,098,411,654 | 40% | ||
ace108 | 0 | 197,669,030,312 | 18% | ||
jasonstaggers | 0 | 69,280,483,794 | 25% | ||
timsaid | 0 | 10,112,729,670 | 2% | ||
cristi | 0 | 399,935,013,020 | 100% | ||
scaredycatguide | 0 | 71,207,144,217 | 20% | ||
mada | 0 | 24,579,797,273 | 100% | ||
ionescur | 0 | 20,294,008,411 | 100% | ||
geke | 0 | 5,430,833,022 | 5% | ||
anomaly | 0 | 200,978,557 | 1% | ||
mrtv2 | 0 | 56,779,185,822 | 100% | ||
mitchelljaworski | 0 | 5,335,933,066 | 25% | ||
brobear1995 | 0 | 220,925,789 | 100% | ||
mandela | 0 | 10,778,054,602 | 2% | ||
adrianobalan | 0 | 81,276,862,853 | 11% | ||
dkmathstats | 0 | 21,742,793,795 | 100% | ||
bape682 | 0 | 3,981,198,665 | 100% | ||
veleje | 0 | 11,467,516,270 | 100% | ||
sportspodium | 0 | 7,929,300,772 | 2% | ||
josuhimovic | 0 | 584,197,580 | 100% | ||
kedi | 0 | 18,265,897,016 | 100% | ||
hxr | 0 | 613,696,269 | 100% | ||
luisaeren16 | 0 | 596,024,155 | 100% | ||
ivanoff | 0 | 614,466,952 | 100% | ||
jessienumb3rs | 0 | 289,526,819 | 51.06% | ||
laxam | 0 | 478,273,645 | 100% | ||
amokrane | 0 | 605,245,710 | 100% | ||
jobsua2018 | 0 | 605,238,812 | 100% | ||
coinchuu | 0 | 574,515,942 | 100% |
Super. Very nice
post_id | 30,572,636 |
---|---|
author | jobsua2018 |
permlink | re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t134150301z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 13:41:57 |
last_update | 2018-02-05 13:44:18 |
depth | 1 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 13:41:57 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 16 |
author_reputation | 4,641,588,833 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
@cristi can you help me out in derivatives and integration i will be happy ?
post_id | 30,572,995 |
---|---|
author | zshah |
permlink | re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t134350253z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "users": ["cristi"], "tags": ["deep-learning"]}" |
created | 2018-02-05 13:43:51 |
last_update | 2018-02-05 13:43:51 |
depth | 1 |
children | 1 |
net_rshares | 19,996,750,651 |
last_payout | 2018-02-12 13:43:51 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.128 SBD |
curator_payout_value | 0.007 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 76 |
author_reputation | 202,094,993,819 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
author_curate_reward | "" |
voter | weight | wgt% | rshares | pct | time |
---|---|---|---|---|---|
cristi | 0 | 19,996,750,651 | 5% |
Sure, can you be more specific?
post_id | 30,574,218 |
---|---|
author | cristi |
permlink | re-zshah-re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t135345324z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 13:51:12 |
last_update | 2018-02-05 13:51:12 |
depth | 2 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 13:51:12 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 31 |
author_reputation | 128,167,460,043,555 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
Very interesting @cristi, as a mathematician Neural Networks and Genetical Algorithms are one my top list of Topics.
post_id | 30,573,699 | ||||||
---|---|---|---|---|---|---|---|
author | ivanoff | ||||||
permlink | re-cristi-201825t9484348z | ||||||
category | deep-learning | ||||||
json_metadata | "{"app": "esteem/1.5.1", "format": "markdown+html", "community": "esteem", "tags": ["deep-learning", "science", "programming", "python"]}" | ||||||
created | 2018-02-05 13:48:06 | ||||||
last_update | 2018-02-05 13:48:06 | ||||||
depth | 1 | ||||||
children | 3 | ||||||
net_rshares | 19,996,750,651 | ||||||
last_payout | 2018-02-12 13:48:06 | ||||||
cashout_time | 1969-12-31 23:59:59 | ||||||
total_payout_value | 0.121 SBD | ||||||
curator_payout_value | 0.003 SBD | ||||||
pending_payout_value | 0.000 SBD | ||||||
promoted | 0.000 SBD | ||||||
body_length | 116 | ||||||
author_reputation | 19,749,475,012 | ||||||
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" | ||||||
beneficiaries |
| ||||||
max_accepted_payout | 1,000,000.000 SBD | ||||||
percent_steem_dollars | 10,000 | ||||||
author_curate_reward | "" |
voter | weight | wgt% | rshares | pct | time |
---|---|---|---|---|---|
cristi | 0 | 19,996,750,651 | 5% |
You got yourself a new follower :) Saludos!
post_id | 30,574,416 |
---|---|
author | cristi |
permlink | re-ivanoff-re-cristi-201825t9484348z-20180205t135501445z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 13:52:27 |
last_update | 2018-02-05 13:52:27 |
depth | 2 |
children | 2 |
net_rshares | 586,815,939 |
last_payout | 2018-02-12 13:52:27 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 43 |
author_reputation | 128,167,460,043,555 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
author_curate_reward | "" |
voter | weight | wgt% | rshares | pct | time |
---|---|---|---|---|---|
ivanoff | 0 | 586,815,939 | 100% |
I new follower. Thanks
post_id | 30,578,917 |
---|---|
author | jobsua2018 |
permlink | re-cristi-re-ivanoff-re-cristi-201825t9484348z-20180205t141725748z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 14:17:30 |
last_update | 2018-02-05 14:17:30 |
depth | 3 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 14:17:30 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 22 |
author_reputation | 4,641,588,833 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
You got yourself one too!!!
post_id | 30,591,284 |
---|---|
author | ivanoff |
permlink | re-cristi-re-ivanoff-re-cristi-201825t9484348z-20180205t152624824z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 15:26:24 |
last_update | 2018-02-05 15:26:24 |
depth | 3 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 15:26:24 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 27 |
author_reputation | 19,749,475,012 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
I always use chain rule of differentiation to understand backpropagation. Since neural network can be represented as f(g(h(...(input)...))) where each function is a layer of the neural network, while trying to minimize the error function which is another function of (predicted-actual), we differentiate this function of function representation, resulting in usage of chain rule.
post_id | 30,594,881 |
---|---|
author | brobear1995 |
permlink | re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t154558326z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 15:45:57 |
last_update | 2018-02-05 15:45:57 |
depth | 1 |
children | 0 |
net_rshares | 34,939,981,148 |
last_payout | 2018-02-12 15:45:57 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.180 SBD |
curator_payout_value | 0.059 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 379 |
author_reputation | 375,356,935,218 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
author_curate_reward | "" |
voter | weight | wgt% | rshares | pct | time |
---|---|---|---|---|---|
cristi | 0 | 34,939,981,148 | 9% |
Thanks for the link, looks very interesting.
post_id | 30,598,039 |
---|---|
author | laxam |
permlink | re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t160432534z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 16:04:33 |
last_update | 2018-02-05 16:04:33 |
depth | 1 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 16:04:33 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 44 |
author_reputation | 3,880,510,732,210 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
great information the one of your publication, generally I practice at every moment the mathematics since my university career is based on pure calculations. Greetings cristi
post_id | 30,602,469 |
---|---|
author | josuhimovic |
permlink | re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t165903945z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 16:30:39 |
last_update | 2018-02-05 16:30:39 |
depth | 1 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 16:30:39 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 174 |
author_reputation | 146,404,880,873 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |
I always take very good with math since I was in the school of Christ. regards
post_id | 30,603,161 |
---|---|
author | luisaeren16 |
permlink | re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t170259183z |
category | deep-learning |
json_metadata | "{"app": "steemit/0.1", "tags": ["deep-learning"]}" |
created | 2018-02-05 16:34:33 |
last_update | 2018-02-05 16:34:33 |
depth | 1 |
children | 0 |
net_rshares | 0 |
last_payout | 2018-02-12 16:34:33 |
cashout_time | 1969-12-31 23:59:59 |
total_payout_value | 0.000 SBD |
curator_payout_value | 0.000 SBD |
pending_payout_value | 0.000 SBD |
promoted | 0.000 SBD |
body_length | 78 |
author_reputation | 52,079,483,285 |
root_title | "Backpropagation Step by Step - [The Mathematical Approach]" |
beneficiaries | [] |
max_accepted_payout | 1,000,000.000 SBD |
percent_steem_dollars | 10,000 |