A program starts as a high-level description. At each layer, the compiler fixes one more
decision — the representation, then the schedule, then the target — until it becomes code
for a specific machine. Because these choices are made separately, changing the data
format or the hardware does not mean rewriting the program.
This separation is what makes a single library or language polymorphic over data structures
and portable across machines.
01
Languages
What you write
High-level languages and libraries over tensors, relations, graphs, and objects in space.
02
Compiler abstractions
What the compiler reasons about
Iteration models and intermediate representations that capture the computation on its own, apart from any data structure or machine.
03
Representations & formats
How the data is stored
Dense and sparse data structures, layouts, and formats — chosen without changing what is computed.
04
Schedules & execution
How the work is organized
Loop orders, tiling, fusion, and distribution — the strategy for running the computation well.
05
Machine targets
Where it runs
CPUs, GPUs, accelerators, and distributed clusters, reached from the same source.
Publications
Recent work
The group's latest papers. Every paper, with search and filters, is on the publications page.
@inproceedings{soi2026optimal,
author = {Soi, Rupanshu and Yadav, Rohan and Kjolstad, Fredrik and Aiken, Alex and Dehnavi, Maryam Mehri and Garland, Michael and Bauer, Michael},
title = {Optimal Software Pipelining and Warp Specialization for Tensor Core {GPUs}},
booktitle = {20th USENIX Symposium on Operating Systems Design and Implementation (OSDI 26)},
year = {2026},
month = jul,
pages = {1875--1890},
publisher = {USENIX Association},
address = {Seattle, WA},
isbn = {978-1-939133-55-7},
url = {https://www.usenix.org/conference/osdi26/presentation/soi}
}
Cite
Rupanshu Soi, Rohan Yadav, Fredrik Kjolstad, Alex Aiken, Maryam Mehri Dehnavi, Michael Garland, and Michael Bauer. Optimal Software Pipelining and Warp Specialization for Tensor Core GPUs. OSDI, 2026.
@article{root2026bonsai,
author = {Root, Alexander J and Gyurgyik, Christophe and Goel, Purvi and Fatahalian, Kayvon and Ragan-Kelley, Jonathan and Adams, Andrew and Kjolstad, Fredrik},
title = {{Bonsai}: Compiling Queries to Pruned Tree Traversals},
journal = {Proceedings of the ACM on Programming Languages},
year = {2026},
month = jun,
volume = {10},
number = {PLDI},
articleno = {178},
numpages = {29},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
doi = {10.1145/3808256},
url = {https://doi.org/10.1145/3808256},
keywords = {compilation, data independence, acceleration structures, tree data structures}
}
Cite
Alexander J Root, Christophe Gyurgyik, Purvi Goel, Kayvon Fatahalian, Jonathan Ragan-Kelley, Andrew Adams, and Fredrik Kjolstad. Bonsai: Compiling Queries to Pruned Tree Traversals. PLDI, 2026.
@article{koul2026onyx,
author = {Koul, Kalhan and Hsu, Olivia and Mei, Yuchen and Ravipati, Sai Gautham and Strange, Maxwell and Melchert, Jackson and Carsello, Alex and Kong, Taeyoung and Chen, Po-Han and Ke, Huifeng and Zhang, Keyi and Liu, Qiaoyi and Nyengele, Gedeon and Xie, Zhouhua and Balasingam, Akhilesh and Adivarahan, Jayashree and Sharma, Ritvik and Torng, Christopher and Emer, Joel S. and Kjolstad, Fredrik and Horowitz, Mark and Raina, Priyanka},
title = {{Onyx}: A 12-nm Programmable Accelerator for Dense and Sparse Applications},
journal = {IEEE Journal of Solid-State Circuits},
year = {2026},
month = may,
volume = {61},
number = {5},
pages = {2336--2348},
publisher = {IEEE},
doi = {10.1109/JSSC.2025.3604724},
url = {https://doi.org/10.1109/JSSC.2025.3604724},
keywords = {Coarse-grained reconfigurable array (CGRA), compilers, computer vision, image processing, machine learning (ML), reconfigurable accelerators, sparse matrices}
}
Cite
Kalhan Koul, Olivia Hsu, Yuchen Mei, Sai Gautham Ravipati, Maxwell Strange, Jackson Melchert, Alex Carsello, Taeyoung Kong, Po-Han Chen, Huifeng Ke, Keyi Zhang, Qiaoyi Liu, Gedeon Nyengele, Zhouhua Xie, Akhilesh Balasingam, Jayashree Adivarahan, Ritvik Sharma, Christopher Torng, Joel S. Emer, Fredrik Kjolstad, Mark Horowitz, and Priyanka Raina. Onyx: A 12-nm Programmable Accelerator for Dense and Sparse Applications. IEEE JSSC, 2026.
Software
Systems we build
Our research ships as compilers, languages, and libraries — a connected body of work rather than isolated tools.
A library that adds sparse tensors and JIT-compiled kernels to PyTorch with a compatible API, automating loop ordering, tiling, and format inference for sparse machine-learning workloads.
A technique that generates machine code by copying pre-built binary stencils and patching in constants and addresses, producing baseline code far faster than a traditional compiler backend.
A meta-compiler that takes a dynamic language's bytecode semantics, written as C++ functions, and generates a high-performance interpreter and a Copy-and-Patch baseline JIT.
A compiler for distributed dense tensor algebra that lets users specify how data is distributed and how computation is distributed independently, then compiles to a task-based runtime.
A distributed, accelerated drop-in replacement for the scipy.sparse library that runs unmodified sparse SciPy programs across clusters of CPUs and GPUs.
A streaming-dataflow abstract machine and intermediate representation for compiling sparse tensor algebra to reconfigurable and fixed-function dataflow accelerators.
We welcome prospective PhD students, current Stanford students, and collaborators who
are curious about languages, compilers, and the systems that run underneath.