It was a pretty lively summer for me, as I was fortunate enough to be invited to a few festivals and talks, all focused on AI. I wanted to share some thoughts and photos from each event to document the fun I had and the wonderful people I met. HowTheLightGetsIn Hay Festival The Institute for … Continue reading A Summer of AI Debates and Talks
Tag: machine-learning
Open sourced my work on LLMs and n-gram statistics
Just open sourced the datasets used in my NeurIPS 2024 paper Understanding Transformers via N-Gram Statistics. It includes training data and associated n-gram data to enable the research community to replicate and build upon my work measuring to what extent LLM predictions can be described in terms of n-gram statistics.
Understanding Transformers via N-Gram Statistics
I released a preprint of my paper Understanding Transformers via N-Gram Statistics last Friday, which provides insights into the ways in which LLM behavior can be described in terms of simple statistical rules. I wrote a detailed X thread summarizing the paper, so I don't have anything else to add to that for now. I'm … Continue reading Understanding Transformers via N-Gram Statistics
Timothy Nguyen 