Backpropagation Step by Step - [The Mathematical Approach] by cristi

View this thread on steempeak.com
· @cristi ·
$10.99
Backpropagation Step by Step - [The Mathematical Approach]
<center>![Resources #91.png](https://steemitimages.com/DQmTPxdiK19inHzTKrsVstyBwC4Y2ib6prX2DCQxM2CiBXZ/Resources%20%2391.png)</center>
___
While studying backpropagation, I came across this great mathematical explanation or breakdown of how it goes. Matt Mazur, a data scientist working for _Help Scout_, wrote about it on his personal blog. 

I am so excited about the mathematical details that power neural networks that I enjoy writing down on a piece of paper how the actual computations work, both for forward propagation (the logits and the activations) and for backpropagation with gradient descent (the partial derivatives). 

This passion and excitement rose inside of me as doing the courses in the Deep Learning specializations of Professor Andrew Ng at Coursera. This part of neural networks is greatly explain in the first course. 

Anyway, backpropagation simply means back propagating the error you get as you reach the output of a network, with the purpose of updating the weights to minimize this error. Matt details both the forward pass and the backward pass through a neural network, with the equations and derivatives that they imply. 

So, if you're a geek that's excited about this stuff, I'd recommend taking a piece of paper and replicating the work done by Matt on his blog. Also, do check the visualizations he provides as they enhance the understanding of backprop. 

<center>[Backpropagation Step by Step - [The Mathematical Approach]](https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/)</center>
___
### <center>To stay in touch with me, follow @cristi</center>   
___

[Cristi Vlad](http://cristivlad.com) Self-Experimenter and Author
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
properties (23)
post_id30,572,386
authorcristi
permlinkbackpropagation-step-by-step-the-mathematical-approach
categorydeep-learning
json_metadata"{"format": "markdown", "links": ["https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/", "http://cristivlad.com"], "app": "steemit/0.1", "tags": ["deep-learning", "science", "programming", "python"], "users": ["cristi"], "image": ["https://steemitimages.com/DQmTPxdiK19inHzTKrsVstyBwC4Y2ib6prX2DCQxM2CiBXZ/Resources%20%2391.png"]}"
created2018-02-05 13:40:39
last_update2018-02-05 13:40:39
depth0
children11
net_rshares1,582,985,490,529
last_payout2018-02-12 13:40:39
cashout_time1969-12-31 23:59:59
total_payout_value9.776 SBD
curator_payout_value1.212 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length1,687
author_reputation128,167,460,043,555
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
author_curate_reward""
vote details (37)
@jobsua2018 · (edited)
Super. Very nice
properties (22)
post_id30,572,636
authorjobsua2018
permlinkre-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t134150301z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 13:41:57
last_update2018-02-05 13:44:18
depth1
children0
net_rshares0
last_payout2018-02-12 13:41:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length16
author_reputation4,641,588,833
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
@zshah ·
$0.14
@cristi can you help me out in derivatives and integration i will be happy ?
👍  
properties (23)
post_id30,572,995
authorzshah
permlinkre-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t134350253z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "users": ["cristi"], "tags": ["deep-learning"]}"
created2018-02-05 13:43:51
last_update2018-02-05 13:43:51
depth1
children1
net_rshares19,996,750,651
last_payout2018-02-12 13:43:51
cashout_time1969-12-31 23:59:59
total_payout_value0.128 SBD
curator_payout_value0.007 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length76
author_reputation202,094,993,819
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
author_curate_reward""
vote details (1)
@cristi ·
Sure, can you be more specific?
properties (22)
post_id30,574,218
authorcristi
permlinkre-zshah-re-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t135345324z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 13:51:12
last_update2018-02-05 13:51:12
depth2
children0
net_rshares0
last_payout2018-02-12 13:51:12
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length31
author_reputation128,167,460,043,555
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
@ivanoff ·
$0.12
Very interesting @cristi, as a mathematician Neural Networks and Genetical Algorithms are one my top list of Topics.
👍  
properties (23)
post_id30,573,699
authorivanoff
permlinkre-cristi-201825t9484348z
categorydeep-learning
json_metadata"{"app": "esteem/1.5.1", "format": "markdown+html", "community": "esteem", "tags": ["deep-learning", "science", "programming", "python"]}"
created2018-02-05 13:48:06
last_update2018-02-05 13:48:06
depth1
children3
net_rshares19,996,750,651
last_payout2018-02-12 13:48:06
cashout_time1969-12-31 23:59:59
total_payout_value0.121 SBD
curator_payout_value0.003 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length116
author_reputation19,749,475,012
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries
0.
accountesteemapp
weight1,000
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
author_curate_reward""
vote details (1)
@cristi ·
You got yourself a new follower :) Saludos!
👍  
properties (23)
post_id30,574,416
authorcristi
permlinkre-ivanoff-re-cristi-201825t9484348z-20180205t135501445z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 13:52:27
last_update2018-02-05 13:52:27
depth2
children2
net_rshares586,815,939
last_payout2018-02-12 13:52:27
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length43
author_reputation128,167,460,043,555
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
author_curate_reward""
vote details (1)
@jobsua2018 ·
I new follower. Thanks
properties (22)
post_id30,578,917
authorjobsua2018
permlinkre-cristi-re-ivanoff-re-cristi-201825t9484348z-20180205t141725748z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 14:17:30
last_update2018-02-05 14:17:30
depth3
children0
net_rshares0
last_payout2018-02-12 14:17:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length22
author_reputation4,641,588,833
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
@ivanoff ·
You got yourself one too!!!
properties (22)
post_id30,591,284
authorivanoff
permlinkre-cristi-re-ivanoff-re-cristi-201825t9484348z-20180205t152624824z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 15:26:24
last_update2018-02-05 15:26:24
depth3
children0
net_rshares0
last_payout2018-02-12 15:26:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length27
author_reputation19,749,475,012
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
@brobear1995 ·
$0.24
I always use chain rule of differentiation to understand backpropagation. Since neural network can be represented as f(g(h(...(input)...))) where each function is a layer of the neural network, while trying to minimize the error function which is another function of (predicted-actual), we differentiate this function of function representation, resulting in usage of chain rule.
👍  
properties (23)
post_id30,594,881
authorbrobear1995
permlinkre-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t154558326z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 15:45:57
last_update2018-02-05 15:45:57
depth1
children0
net_rshares34,939,981,148
last_payout2018-02-12 15:45:57
cashout_time1969-12-31 23:59:59
total_payout_value0.180 SBD
curator_payout_value0.059 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length379
author_reputation375,356,935,218
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
author_curate_reward""
vote details (1)
@laxam ·
Thanks for the link, looks very interesting.
properties (22)
post_id30,598,039
authorlaxam
permlinkre-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t160432534z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 16:04:33
last_update2018-02-05 16:04:33
depth1
children0
net_rshares0
last_payout2018-02-12 16:04:33
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length44
author_reputation3,880,510,732,210
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
@josuhimovic ·
great information the one of your publication, generally I practice at every moment the mathematics since my university career is based on pure calculations. Greetings cristi
properties (22)
post_id30,602,469
authorjosuhimovic
permlinkre-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t165903945z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 16:30:39
last_update2018-02-05 16:30:39
depth1
children0
net_rshares0
last_payout2018-02-12 16:30:39
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length174
author_reputation146,404,880,873
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000
@luisaeren16 ·
I always take very good with math since I was in the school of Christ. regards
properties (22)
post_id30,603,161
authorluisaeren16
permlinkre-cristi-backpropagation-step-by-step-the-mathematical-approach-20180205t170259183z
categorydeep-learning
json_metadata"{"app": "steemit/0.1", "tags": ["deep-learning"]}"
created2018-02-05 16:34:33
last_update2018-02-05 16:34:33
depth1
children0
net_rshares0
last_payout2018-02-12 16:34:33
cashout_time1969-12-31 23:59:59
total_payout_value0.000 SBD
curator_payout_value0.000 SBD
pending_payout_value0.000 SBD
promoted0.000 SBD
body_length78
author_reputation52,079,483,285
root_title"Backpropagation Step by Step - [The Mathematical Approach]"
beneficiaries[]
max_accepted_payout1,000,000.000 SBD
percent_steem_dollars10,000