Open Source Software

Google DeepMind: transformer_ngrams
Releases datasets used in Understanding Transformers via N-Gram Statistics to enable studying how well LLM predictions can be described in terms of n-gram statistics.

Google Research: KIP
Implements Kernel Inducing Points and Label Solve algorithm introduced in arxiv:2011.00050 and releases distilled datasets obtained using large scale distributed meta-learning as described in arxiv:2107.13034. Featured on the GoogleAI blog: Training Machine Learning Models More Efficiently with Dataset Distillation

Top: Natural (i.e., unmodified) CIFAR-10 images. Bottom: Distilled dataset (1 image per class) on CIFAR-10 classification task. Using only these 10 synthetic images as training data, a model can achieve test set accuracy of ~51%.