Research
Research
We study a single question: how can one high-level program stay correct and fast while the data it works on and the machine it runs on keep changing?
The problem
Computation is written once. It has to run everywhere.
A matrix multiply, a graph traversal, a physical simulation — each is easy to state at a high level. But making it fast forces a cascade of decisions: how is the data stored, dense or sparse and in which format? How is the work tiled and ordered? Does it run on a CPU, a GPU, an accelerator, or a cluster?
Today those decisions are usually baked into the program. Change the data format or the hardware, and much of the code has to be rewritten. That is expensive, and it is why so much high-performance software is narrow and brittle.
Our work is about breaking that coupling — separating what a program computes from how its data is represented and where it runs, and building compilers that fill in the rest.
The approach
One computation, lowered through a stack of decisions
Each layer fixes one more decision. Because the layers are separate, a change at the bottom — a new format, a new machine — does not require rewriting the top.
- 01
Languages
What you writeHigh-level languages and libraries over tensors, relations, graphs, and objects in space.
- 02
Compiler abstractions
What the compiler reasons aboutIteration models and intermediate representations that capture the computation on its own, apart from any data structure or machine.
- 03
Representations & formats
How the data is storedDense and sparse data structures, layouts, and formats — chosen without changing what is computed.
- 04
Schedules & execution
How the work is organizedLoop orders, tiling, fusion, and distribution — the strategy for running the computation well.
- 05
Machine targets
Where it runsCPUs, GPUs, accelerators, and distributed clusters, reached from the same source.
Research areas
Where the work happens
These threads all serve the same goal. Follow any one into the publications that develop it.
-
Sparse tensor algebra
Compiling tensor algebra that is polymorphic over dense and sparse data structures.
Papers -
Sparse & structured arrays
Array programming models and shape operators over irregular data.
Papers -
Data representations
Decoupling the layouts and formats of data structures from the algorithms that use them.
Papers -
Compiler representations
Intermediate representations and iteration models for fused, structured computation.
Papers -
Meta-compilation, JITs & VMs
Compilers that generate compilers, fast baseline JITs, and virtual-machine generators.
Papers -
Distributed computation
Mapping computation onto clusters and task-based runtimes at scale.
Papers -
Accelerators & dataflow
Hardware/software co-design for reconfigurable and dataflow architectures.
Papers -
Domain-specific languages
Languages for simulation, graphics, and other structured domains.
Papers -
Scheduling & autotuning
Choosing schedules and formats automatically, including cost models and search.
Papers -
Recurrences
Compiling systems of recurrence equations over dense and sparse arrays.
Papers -
Relational & query compilation
Compiling relational and query operations alongside tensor computation.
Papers
From one program to many machines
The same computation should reach CPUs, GPUs, accelerators, and distributed clusters. We express the mapping to hardware as explicit, separate schedules — data distribution apart from computation distribution — so targeting new hardware means writing a schedule, not a new program. This runs from distributed tensor algebra through to co-designed dataflow accelerators.
Distributed & accelerator workCompilers that generate compilers
If writing a compiler by hand is expensive, generate it. We build meta-compilers, fast baseline JITs, and virtual-machine generators — technology like Copy-and-Patch and Deegen that derives high-performance compilers and interpreters from a description instead of hand-written code.
Meta-compilation workIn practice
Systems that carry the ideas
The research is concrete: it ships as compilers, languages, and libraries you can read and run.
- Burrito: Compilation of Shape Operators on Sparse Arrays A compiler for a sparse array language that supports shape operators — such as reshaping and concatenating sparse arrays — alongside compute operators, generating fused code over reshaped views.
- Scorch: An Optimized Sparse PyTorch A library that adds sparse tensors and JIT-compiled kernels to PyTorch with a compatible API, automating loop ordering, tiling, and format inference for sparse machine-learning workloads.
- Copy-and-Patch Compilation A technique that generates machine code by copying pre-built binary stencils and patching in constants and addresses, producing baseline code far faster than a traditional compiler backend.
- Deegen: A JIT-Capable VM Generator A meta-compiler that takes a dynamic language's bytecode semantics, written as C++ functions, and generates a high-performance interpreter and a Copy-and-Patch baseline JIT.
How we work
We build the systems, then find the abstraction
The group's results tend to come as working systems — compilers, languages, and simulators that run real workloads — alongside the abstractions that make them general. Papers are accompanied by artifacts and open-source releases, and the work spans the full stack, from formal intermediate representations down to co-designed hardware.
Much of it is done with collaborators across programming languages, systems, computer architecture, and graphics, both at Stanford and elsewhere.