Had a great time chatting with fellow podcaster Tim Scarfe over at Machine Learning Street Talk on my recent paper Understanding Transformers via N-gram Statistics: https://www.youtube.com/watch?v=W485bz0_TdI
Year: 2024
Michael Freedman | A Fields Medalist Panorama
Michael Freedman is a mathematician who was awarded the Fields Medal in 1986 for his solution of the 4-dimensional Poincare conjecture. Mike has also received numerous other awards for his scientific contributions including a MacArthur Fellowship and the National Medal of Science. In 1997, Mike joined Microsoft Research and in 2005 became the director of … Continue reading Michael Freedman | A Fields Medalist Panorama
Understanding Transformers via N-Gram Statistics
I released a preprint of my paper Understanding Transformers via N-Gram Statistics last Friday, which provides insights into the ways in which LLM behavior can be described in terms of simple statistical rules. I wrote a detailed X thread summarizing the paper, so I don't have anything else to add to that for now. I'm … Continue reading Understanding Transformers via N-Gram Statistics
Timothy Nguyen 