Rob Hicks (Posts about matlab)http://rlhick.people.wm.edu/enFri, 13 Apr 2018 08:49:25 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rssEstimating Custom Maximum Likelihood Models in Python (and Matlab)http://rlhick.people.wm.edu/posts/estimating-custom-mle.html<div><p>
In this post I show various ways of estimating "generic" maximum likelihood models in python. For each, we'll recover standard errors.
</p>
<p>
We will implement a simple ordinary least squares model like this
</p>
\begin{equation}
\mathbf{y = x\beta +\epsilon}
\end{equation}
<p>
where \(\epsilon\) is assumed distributed i.i.d. normal with mean 0 and variance \(\sigma^2\). In our simple model, there is only a constant and one slope coefficient (\(\beta = \begin{bmatrix} \beta_0 & \beta_1 \end{bmatrix}\)).
</p>
<p>
For this model, we would probably never bother going to the trouble of manually implementing maximum likelihood estimators as we show in this post. However, for more complicated models for which there is no established package or command, there are benefits to knowing how to build your own likelihood function and use it for estimation. It is also worthwhile noting that most of the methods shown here don't use analytical gradiants or hessians, so are likely (1) to have longer execution times and (2) to be less precise than methods where known analytical gradiants and hessians are built into the estimation method. I might explore those issues in a later post.
</p>
<p>
<b>tl;dr</b>: There are numerous ways to estimate custom maximum likelihood models in Python, and what I find is:
</p>
<ol class="org-ol">
<li>For the most features, I recommend using the <a href="http://rlhick.people.wm.edu/posts/estimating-custom-mle.html#statsmodels"><code>Genericlikelihoodmodel</code> class from Statsmodels</a> even if it is the least intuitive way for programmers familiar with Matlab. If you are comfortable with object oriented programming you should definitely go this route.
</li>
<li>For fastest run times and computationally expensive problems Matlab will most likely be significantly even with lots of code optimizations.
</li>
</ol>
<p><a href="http://rlhick.people.wm.edu/posts/estimating-custom-mle.html">Read more…</a> (6 min remaining to read)</p></div>mathjaxmatlabmaximum likelihoodpythonhttp://rlhick.people.wm.edu/posts/estimating-custom-mle.htmlSat, 06 May 2017 08:15:50 GMTReproducible Research and Literate Programming for Econometricshttp://rlhick.people.wm.edu/posts/reproducible-research.html<div><p>
<a href="http://rlhick.people.wm.edu/posts/reproducible-research.html#Stata-and-Org-Mode">Jump straight to the discussion on Stata and Emacs Org Mode</a>
</p>
<p>
This post describes my experience implementing reproducible research and literate programming methods for commonly used econometric software packages. Since literate programming aims to store the accumulated scientific knowledge of the research project in one document, the software package must allow for the reproduction of data cleaning and data analysis steps, store the record of methods used, generate results dynamically and use these for the writeup, and be executable by including the computational environment.
</p>
<p>
Perhaps most importantly, this dynamic document can be executed to produce the <b>academic paper</b>. The researcher shares this file with other researchers rather than the only a pdf of the paper, making the research fully reproducible by executing the dynamic document. It is my view that this will be expected in most scientific journals over the next few decades.
</p>
<p><a href="http://rlhick.people.wm.edu/posts/reproducible-research.html">Read more…</a> (12 min remaining to read)</p></div>emacsjupytermathjaxmatlaborg modepythonstatahttp://rlhick.people.wm.edu/posts/reproducible-research.htmlThu, 02 Mar 2017 10:30:50 GMTPart II: Comparing the Speed of Matlab versus Python/Numpyhttp://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy-partii.html<div tabindex="-1" id="notebook" class="border-box-sizing">
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>In this note, I extend a <a href="http://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy.html">previous post</a> on comparing run-time speeds of various econometrics packages by</p>
<ol>
<li>Adding Stata to the original comparison of Matlab and Python</li>
<li>Calculating runtime speeds by<ul>
<li>Comparing full OLS estimation functions for each package<ul>
<li>Stata: <code>reg</code></li>
<li>Matlab: <code>fitlm</code></li>
<li>Python: <code>regression.linear_model.OLS</code> from the <code>statsmodels</code> module.</li>
</ul>
</li>
<li>Comparing the runtimes for calculations using linear algebra code for the OLS model: $ (x'x)^{-1}x'y $ </li>
</ul>
</li>
<li>Since Stata and Matlab automatically parralelize some calculations, we parallelize the python code using the <code>Parallel</code> module.<p><a href="http://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy-partii.html">Read more…</a> (11 min remaining to read)</p></li></ol></div></div></div></div></div>bootstrapipythonmatlabnotebookhttp://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy-partii.htmlThu, 09 Apr 2015 12:06:21 GMTComparing the Speed of Matlab versus Python/Numpyhttp://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy.html<div tabindex="-1" id="notebook" class="border-box-sizing">
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><strong>Update 1</strong>: A more complete and updated speed comparison can be found <a href="http://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy-partii.html">here</a>.</p>
<p><strong>Update 2</strong>: Python and Matlab code edited on 4/5/2015.</p>
<p>In this short note, we compare the speed of matlab and the scientific computing platform of python for a simple bootstrap of an ordinary least squares model. Bottom line (with caveats): matlab is faster than python with this code. One might be able to further optimize the python code below, but it isn't an obvious or easy process (see for example <a href="http://scipy-lectures.github.io/advanced/optimizing/">advanced optimization techniques</a>).</p>
<p>As an aside, this note demonstrates that even if one can't optimize python code significantly enough, it is possible to do computationally expensive calculations in matlab and return results to the ipython notebook.</p>
<h3 id="Data-Setup">Data Setup<a class="anchor-link" href="http://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy.html#Data-Setup">¶</a></h3><p>We will bootstrap the ordinary least squares model (ols) using 1000 replicates. For generating the toy dataset, the true parameter values are
$$
\beta=\begin{bmatrix}
10\\-.5\\.5
\end{bmatrix}
$$</p>
<p>We perform the experiment for 3 different sample sizes ($n = \begin{bmatrix}1,000 & 10,000 & 100,000 \end{bmatrix}$). For each of the observations in the toy dataset, the independent variables are drawn from</p>
<p>$$
\mu_x = \begin{bmatrix} 10\\10 \end{bmatrix}, \sigma_x = \begin{bmatrix} 4 & 0 \\ 0 & 4 \end{bmatrix}
$$</p>
<p>The dependent variable is constructed by drawing a vector of random normal variates from Normal(0,1). Denoting this vector as $\epsilon$ calculate the dependent variable as
$$
\mathbf{Y=Xb+\epsilon}
$$
</p><p><a href="http://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy.html">Read more…</a> (2 min remaining to read)</p></div></div></div></div></div>bootstrapipythonmatlabnotebookhttp://rlhick.people.wm.edu/posts/comparing-the-speed-of-matlab-versus-pythonnumpy.htmlThu, 19 Mar 2015 12:07:34 GMTRunning R and Matlab Commands in an Ipython Notebookhttp://rlhick.people.wm.edu/posts/running-r-and-matlab-commands-in-an-ipython-notebook.html<div tabindex="-1" id="notebook" class="border-box-sizing">
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The ipython notebook environment is a superb environment for empirical research. Sometimes, though, you would like to access the capabilities of other software. This post shows how to incorporate R and Matlab into ipython notebooks.</p>
<p><a href="http://rlhick.people.wm.edu/posts/running-r-and-matlab-commands-in-an-ipython-notebook.html">Read more…</a> (2 min remaining to read)</p></div></div></div></div></div>ipythonmatlabnotebookRhttp://rlhick.people.wm.edu/posts/running-r-and-matlab-commands-in-an-ipython-notebook.htmlThu, 26 Feb 2015 16:06:21 GMT