How Does AI Work?

I had a nice discussion with Robert Wright on his podcast Nonzero on the technical workings of AI aimed at a nontechnical audience. We go into the details about transformers (the architecture underlying models like ChatGPT), aimed both at a high level understanding as well as going into the weeds of the internal mechanisms. Along the way, I talk about precursor algorithms like Word2Vec and what is and is not surprising about machine learning in general, providing intuitions and making analogies for a nontechnical audience.

In the extended version of our interview (which subscribers to Nonzero have access to), we briefly discuss the story behind Eric Weinstein’s economics workshop talk at The University of Chicago, which was the basis for my rebuttal paper. This side topic has nothing to do with AI, but is a follow up to my first podcast episode with Bob from two years ago which was provocatively titled “Is Eric Weinstein a Crackpot?”.