Rob Hicks (Posts about mle)http://rlhick.people.wm.edu/enTue, 25 Feb 2020 10:01:57 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rssTensorflow with Custom Maximum Likelihood Functionshttp://rlhick.people.wm.edu/posts/custom-likes-tensorflow.html<div><div class="HTML">
<p>
<script type="text/javascript" async
src="<a href="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML</a>">
MathJax.Hub.Config({ jax: ["input/TeX","output/HTML-CSS"], displayAlign: "center",TeX: { equationNumbers: { autoNumber: "AMS" }} });
</script>
</p>
</div>
<p>
This post builds on earlier ones dealing with ?? and estimation with ??
The goal of the post is threefold:
</p>
<ol class="org-ol">
<li>Define a custom log-likelihood function in tensorflow and perform differentiation within tensorflow over model parameters to illustrate how under the hood, tensorflow's model graph is designed to uncover derivatives "free of charge".</li>
<li>Use this log-likelihood to estimate the maximum likelihood using tensorflow's optimization capabilities.</li>
<li>Illustrate how the <code>tensorflow.probability.mcmc</code> libraries can be used with custom log-likelihoods.</li>
</ol>
<p>
So why bother? The earlier posts outlined above show python's <code>autograd</code> or <code>theano</code> has auto differentiation capabilities which we used successfully to speedup some models. Unfortunately, those libraries aren't optimal for the current revolution in machine learning enabled by gpu computing<sup><a id="fnr.1" class="footref" href="http://rlhick.people.wm.edu/posts/custom-likes-tensorflow.html#fn.1">1</a></sup>. Tensorflow is a library that opens doors for econometricians enabling much faster processing (and massive parallelization of models) that simply isn't possible on cpu's. In a followup post, I hope to illustrate the types of speedup one gets from gpu processing, but before we get there it is necessary to delve into tensorflow for model building since gpu speedup can only be achieved if tensorflow primitives are used for all math and linear algebra operations.
</p>
<p><a href="http://rlhick.people.wm.edu/posts/custom-likes-tensorflow.html">Read moreā¦</a> (6 min remaining to read)</p></div>likelihoodmcmcmletensorflowhttp://rlhick.people.wm.edu/posts/custom-likes-tensorflow.htmlMon, 24 Feb 2020 08:15:50 GMT