Ask-E: An Environment for Calibrated Question Generation

Sarah Pratt, Jae Sung Park, Scott Geng, Ali Farhadi

In submission, NeurIPS 2026 (Datasets Track)

We present Ask-E, an environment that benchmarks and trains models on their ability to write questions at a given skill level, rather than answer them. A question is successfully calibrated if exactly one of two boundary models can solve it. Training on this environment improves downstream math benchmarks with no new math data, no interaction with stronger models, and no correctness-based reward.

The Emergence of Complex Behavior in Large-Scale Ecological Environments

J. Bejjani, C. Van Amburg, C. Wang, C. H. Su, Sarah Pratt, Y. Mazloumi, N. Khoshnevis, S. Kakade, K. Brantley, A. Walsman

Explores how physical scale and population size affect behaviors in simulated ecological environments.

Superposed decoding: Multiple generations from a single autoregressive inference pass

Ethan Shen, Alan Fan, Sarah Pratt, Jae Sung Park, Matthew Wallingford, Sham Kakade, Ari Holtzman, Ranjay Krishna, Ali Farhadi, Aditya Kusupati

This work proposes Superposed Decoding, a new decoding algorithm that generates k drafts at the computation cost of one autoregressive inference pass.

Can Language Models Use Forecasting Strategies?

Sarah Pratt, Seth Blumberg, Pietro Kreitlon Carolino, Meredith Ringel Morris

This work describes experiments using a novel dataset of real world events and associated human predictions, an evaluation metric to measure forecasting ability, and the accuracy of a number of different LLM based forecasting designs on the provided dataset.

DataComp-LM: In Search of the Next Generation of Training Sets for Language Models

J. Li, A. Fang, G. Smyrnis, M. Ivgi, M. Jordan, ... Sarah Pratt, ... V. Shankar (59 authors)

NeurIPS 2024 (Datasets Track)

A testbed and benchmark for designing high-quality training sets for language models.

DataComp: In Search of the Next Generation of Multimodal Datasets

S. Y. Gadre, G. Ilharco, A. Fang, ... Sarah Pratt, ... L. Schmidt (34 authors)

NeurIPS 2023 (Datasets Track)

A testbed for dataset experiments centered around a candidate pool of 12.8B image-text pairs for training CLIP models.

What does a platypus look like? Generating customized prompts for zero-shot image classification

Sarah Pratt, Ian Covert, Rosanne Liu, Ali Farhadi

This work combines open vocabulary models with large language models (LLMs) to create Customized Prompts via Language models (CuPL). We leverage the knowledge contained in LLMs to generate descriptive sentences for zero-shot image classification.

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

Sarah Pratt, Luca Weihs, Ali Farhadi

We argue for an introspective agent, which considers its own abilities in the context of its environment. Different environments yield vastly different optimal designs.

Learning Generalizable Visual Representations via Interactive Gameplay

Luca Weihs, Ani Kembhavi, Kiana Ehsani, Sarah Pratt, Winson Han, Alvaro Herrasti, Eric Kolve, Dustin Schwenk, Roozbeh Mottaghi, Ali Farhadi

Oral Presentation at ICLR 2021

Grounded Situation Recognition

Sarah Pratt, Mark Yatskar, Luca Weihs, Ali Farhadi, Ani Kembhavi

Spotlight at ECCV 2020

Grounded Situation Recognition builds upon situation recognition and requires one to not just identify the situation observed in the image but also visually ground the identified roles within the corresponding image.

University of Washington

CSE 493: Deep Learning - Instructor (Winter 2026)

CSE 493: Deep Learning - Instructor (Autumn 2024)

CSE 493: Deep Learning - Instructor (Winter 2024)

CSE 493: Deep Learning - Teaching Assistant (Spring 2023)

Selected Lecture Slides:

Brown University

CS022: Discrete Math and Probability - Head Teaching Assistant (Spring 2018)

CS022: Discrete Math and Probability - Teaching Assistant (Spring 2017)

CS015: Introduction to Object Oriented Programming - Teaching Assistant (Fall 2016)