Multiclass logistic regression (Softmax) and its derivative
While watching a lecture, “Lecture 4.3 - The softmax output function”, taught by Geoffrey Hinton, I have noticed that he did not show steps to find the derivative of cross entropy cost function with respect to a variable, zi, which resulted in si - yi in his lecture. For beginners in calculus, this was quite a mystery to find the final formula as shown below:
where the cross entropy cost function is
So, I have decided to take this challenge, and as a result of my effort, I was able to describe detailed steps to find the derivative of cost function, and I hope this post is helpful for those who are not familiar with calculus (like me).
Unfortunately, I cannot add MathJax library to Blogger to show detailed steps using LaTeX. So I have decided to create a separate post using github.io site. To see the derivative of cross entropy cost function in action, click this link: https://jpark7ca.github.io/softmax.html I hope this helps.
where the cross entropy cost function is
So, I have decided to take this challenge, and as a result of my effort, I was able to describe detailed steps to find the derivative of cost function, and I hope this post is helpful for those who are not familiar with calculus (like me).
Unfortunately, I cannot add MathJax library to Blogger to show detailed steps using LaTeX. So I have decided to create a separate post using github.io site. To see the derivative of cross entropy cost function in action, click this link: https://jpark7ca.github.io/softmax.html I hope this helps.
Comments
Post a Comment