Programming Languages × Systems
We build programming languages and compilers that make sophisticated software fast and portable across changing data representations and hardware.
01 · Abstraction
A(i,j) = Σk B(i,k) · C(k,j)
A high-level expression, independent of layout or machine.
02 · Representation
A nested iteration space the compiler can reshape.
03 · Structure
A sparse layout: only nonzeros are stored and visited.
04 · Execution
A partitioned mapping across CPUs, GPUs, and accelerators.
Research
How can a single high-level program stay fast as its data and target keep changing? Our work approaches that question from several directions.
Programs should move between dense, sparse, graph, and custom layouts without being rewritten around each representation.
ExploreMapping the same high-level intent onto CPUs, GPUs, accelerators, distributed clusters, and heterogeneous systems.
ExploreMeta-compilers, VM generators. Instead of building one compiler at a time, we build methods and generators for creating whole compiler families.
ExploreCompiling tensor, array, and relational computation over sparse and irregular data without sacrificing performance.
ExploreFrom abstractions to machines
A program starts as a high-level description. At each layer, the compiler fixes one more decision — the representation, then the schedule, then the target — until it becomes code for a specific machine. Because these choices are made separately, changing the data format or the hardware does not mean rewriting the program.
This separation is what makes a single library or language polymorphic over data structures and portable across machines.
High-level languages and libraries over tensors, relations, graphs, and objects in space.
Iteration models and intermediate representations that capture the computation on its own, apart from any data structure or machine.
Dense and sparse data structures, layouts, and formats — chosen without changing what is computed.
Loop orders, tiling, fusion, and distribution — the strategy for running the computation well.
CPUs, GPUs, accelerators, and distributed clusters, reached from the same source.
Software
Our research ships as compilers, languages, and libraries — a connected body of work rather than isolated tools.
Sparse tensor & array compilation
A compiler for a sparse array language that supports shape operators — such as reshaping and concatenating sparse arrays — alongside compute operators, generating fused code over reshaped views.
Sparse tensor & array compilation
A library that adds sparse tensors and JIT-compiled kernels to PyTorch with a compatible API, automating loop ordering, tiling, and format inference for sparse machine-learning workloads.
Meta-compilers, JITs & virtual machines
A technique that generates machine code by copying pre-built binary stencils and patching in constants and addresses, producing baseline code far faster than a traditional compiler backend.
Meta-compilers, JITs & virtual machines
A meta-compiler that takes a dynamic language's bytecode semantics, written as C++ functions, and generates a high-performance interpreter and a Copy-and-Patch baseline JIT.
Distributed sparse computation
A compiler for distributed dense tensor algebra that lets users specify how data is distributed and how computation is distributed independently, then compiles to a task-based runtime.
Distributed sparse computation
A distributed, accelerated drop-in replacement for the scipy.sparse library that runs unmodified sparse SciPy programs across clusters of CPUs and GPUs.
Publications
A few papers that show the range of the group. Every paper, with search and filters, is on the publications page.
@inproceedings{xu2026deegen,
title = {Deegen: A JIT-Capable VM Generator for Dynamic Languages},
author = {Haoran Xu and Fredrik Kjolstad},
booktitle = {ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications},
year = {2026},
month = {oct},
doi = {10.1145/3798246},
url = {https://fredrikbk.com/publications/deegen.pdf}
} Haoran Xu and Fredrik Kjolstad. Deegen: A JIT-Capable VM Generator for Dynamic Languages. OOPSLA, 2026 (to appear).
@inproceedings{root2026bonsai,
title = {Bonsai: Compiling Queries to Pruned Tree Traversals},
author = {Alexander J Root and Christophe Gyurgyik and Purvi Goel and Kayvon Fatahalian and Jonathan Ragan-Kelley and Andrew Adams and Fredrik Kjolstad},
booktitle = {ACM SIGPLAN Conference on Programming Language Design and Implementation},
year = {2026},
month = {jun},
url = {https://fredrikbk.com/publications/bonsai.pdf}
} Alexander J Root, Christophe Gyurgyik, Purvi Goel, Kayvon Fatahalian, Jonathan Ragan-Kelley, Andrew Adams, and Fredrik Kjolstad. Bonsai: Compiling Queries to Pruned Tree Traversals. PLDI, 2026.
@inproceedings{zhang2024dataflow,
title = {The Dataflow Abstract Machine Simulator Framework},
author = {Nathan Zhang and Rubens Lacouture and Gina Sohn and Paul Mure and Qizheng Zhang and Fredrik Kjolstad and Kunle Olukotun},
booktitle = {International Symposium on Computer Architecture},
year = {2024},
month = {jun},
doi = {10.1109/ISCA59077.2024.00046},
url = {https://fredrikbk.com/publications/dam.pdf}
} Nathan Zhang, Rubens Lacouture, Gina Sohn, Paul Mure, Qizheng Zhang, Fredrik Kjolstad, and Kunle Olukotun. The Dataflow Abstract Machine Simulator Framework. ISCA, 2024.
Distributed Sparse tensor algebra
@inproceedings{yadav2023legate,
title = {Legate Sparse: Distributed Sparse Computing in Python},
author = {Rohan Yadav and Wonchan Lee and Melih Elibol and Manolis Papadakis and Taylor Lee-Patti and Michael Garland and Alex Aiken and Fredrik Kjolstad and Michael Bauer},
booktitle = {International Conference for High Performance Computing, Networking, Storage and Analysis},
year = {2023},
month = {nov},
doi = {10.1145/3581784.3607033},
url = {https://fredrikbk.com/publications/legate-sparse.pdf}
} Rohan Yadav, Wonchan Lee, Melih Elibol, Manolis Papadakis, Taylor Lee-Patti, Michael Garland, Alex Aiken, Fredrik Kjolstad, and Michael Bauer. Legate Sparse: Distributed Sparse Computing in Python. SC, 2023.
Sparse tensor algebra IR & iteration
@inproceedings{bansal2023mosaic,
title = {Mosaic: An Interoperable Compiler for Tensor Algebra},
author = {Manya Bansal and Olivia Hsu and Kunle Olukotun and Fredrik Kjolstad},
booktitle = {ACM SIGPLAN Conference on Programming Language Design and Implementation},
year = {2023},
month = {jun},
doi = {10.1145/3591236},
url = {https://fredrikbk.com/publications/mosaic.pdf}
} Manya Bansal, Olivia Hsu, Kunle Olukotun, and Fredrik Kjolstad. Mosaic: An Interoperable Compiler for Tensor Algebra. PLDI, 2023.
We welcome prospective PhD students, current Stanford students, and collaborators who are curious about languages, compilers, and the systems that run underneath.