Assistant Professor
Department of Computer Science
Stanford University
Office: Gates 486 kjolstad@stanford.edu
CV

I aim to make it easy to program sparse computing applications that can run everywhere with end-to-end performance that matches or exceeds the best hand-optimized codes. A sparse application computes on a system where most parts do not interact—they are sparsely interconnected. Examples include sparse matrices, meshes, and graphical arrangements used to simulate the dynamics of robots, optimize the flow of traffic, compute the rank of web pages, estimate the similarity of genome loci, or train sparse deep neural networks. Sparse computing is ubiquitous and will become increasingly important with the growth of robotic and autonomous vehicle applications, data analytics, and machine learning.

I have developed the taco compiler for computing sparse tensor expressions and the Simit programming language for computing on sparse systems. My broad research interests include compilers, programming models and languages, parallel computing, software engineering, and performance engineering.

My thesis on Sparse Tensor Algebra Compilation is the best place to start reading about the ideas behind the tensor algebra compiler (taco) and sparse iteration model.

PhD Students

Publications

Stardust: Compiling Sparse Tensor Algebra to a Reconfigurable Dataflow Architecture
Olivia Hsu, Alexander Rucker, Tian Zhao, Varun Desai, Kunle Olukotun, and Fredrik Kjolstad
International Symposium on Code Generation and Optimization (CGO), March 2025
(to appear)
Compiling Recurrences over Dense and Sparse Arrays
Shiv Sundram, Muhammad Usman Tariq, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 8, Issue OOPSLA, October 2024
pdf
Compilation of Shape Operators on Sparse Arrays
Alexander J Root, Bobby Yan, Peiming Liu, Christophe Gyurgyik, Aart J.C Bik, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 8, Issue OOPSLA, October 2024
pdf
Compiler Support for Sparse Tensor Convolutions
Peiming Liu, Alexander J Root, Anlun Xu, Yinying Li, Fredrik Kjolstad, and Aart J.C Bik
Proceedings of the ACM on Programming Languages, Volume 8, Issue OOPSLA, October 2024
pdf
The Dataflow Abstract Machine Simulator Framework
Nathan Zhang, Rubens Lacouture, Gina Sohn, Paul Mure, Qizheng Zhang, Fredrik Kjolstad, and Kunle Olukotun
Proceedings of the International Symposium on Computer Architecture (ISCA), June 2024
ISCA Distinguished Artifact Award
pdf
Compilation of Modular and General Sparse Workspaces
Genghan Zhang, Olivia Hsu, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 8, Issue PLDI, June 2024
pdf
Onyx: A 12nm 756 GOPS/W Coarse-Grained Reconfigurable Array for Accelerating Dense and Sparse Applications
Kalhan Koul, Maxwell Strange, Jackson Melchert, Alex Carsello, Yuchen Mei, Olivia Hsu, Taeyoung Kong, Po-Han Chen, Huifeng Ke, Keyi Zhang, Qiaoyi Liu, Gedeon Nyengele, Akhilesh Balasingam, Jayashree Adivarahan, Ritvik Sharma, Zhouhua Xie, Christopher Torng, Joel Emer, Fredrik Kjolstad, Mark Horowitz, and Priyanka Raina
IEEE Symposium on VLSI Technology and Circuits (VLSI), June 2024
Revet: A Language and Compiler for Dataflow Threads
Alexander Rucker, Shiv Sundram, Coleman Smith, Matt Vilim, Raghu Prabhakar, Fredrik Kjolstad, and Kunle Olukotun
IEEE International Symposium on High-Performance Computer Architecture (HPCA), March 2024
pdf
Legate Sparse: Distributed Sparse Computing in Python
Rohan Yadav, Wonchan Lee, Melih Elibol, Manolis Papadakis, Taylor Lee-Patti, Michael Garland, Alex Aiken, Fredrik Kjolstad, and Michael Bauer
ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC), November 2023
pdf
Indexed Streams: A Formal Intermediate Representation for the Fused Execution of Contraction Operations
Scott Kovach, Praneeth Kolichala, Timothy Gu, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 7, Issue PLDI, June 2023
pdf youtube
Mosaic: An Interoperable Compiler for Tensor Algebra
Manya Bansal, Olivia Hsu, Kunle Olukotun, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 7, Issue PLDI, June 2023
Distinguished Paper Award
pdf
BaCO: A Fast and Portable Bayesian Compiler Optimization Framework
Erik Hellsten, Artur Souza, Johannes Lenfers, Rubens Lacouture, Olivia Hsu, Adel Ejjeh, Fredrik Kjolstad, Michel Steuwer, Kunle Olukotun, and Luigi Nardi
International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), April 2023
pdf
The Sparse Abstract Machine
Olivia Hsu, Maxwell Strange, Ritvik Sharma, Jaeyeon Won, Kunle Olukotun, Joel Emer, Mark Horowitz, and Fredrik Kjolstad
Architectural Support for Programming Languages and Operating Systems (ASPLOS), March 2023
pdf youtube
Unified Buffer: Compiling Image Processing and Machine Learning Applications to Push-Memory Accelerators
Qiaoyi Liu, Jeff Setter, Dillon Huff, Maxwell Strange, Kathleen Feng, Mark Horowitz, Priyanka Raina, and Fredrik Kjolstad
ACM Transactions on Architecture and Code Optimization (TACO), March 2023
pdf
Looplets: A Language For Structured Coiteration
Willow Ahrens, Daniel Donenfeld, Fredrik Kjolstad, and Saman Amarasinghe
International Symposium on Code Generation and Optimization (CGO), February 2023
pdf
AHA: An agile approach to the design of coarse-grained reconfigurable accelerators and compilers
Kalhan Koul, Jackson Melchert, Kavya Sreedhar, Leonard Truong, Gedeon Nyengele, Keyi Zhang, Qiaoyi Liu, Jeff Setter, Po-Han Chen, Yuchen Mei, Maxwell Strange, Ross Daly, Caleb Donovick, Alex Carsello, Taeyoung Kong, Kathleen Feng, Dillon Huff, Ankita Nayak, Raj Setaluri, James Thomas, Nikhil Bhagdikar, David Durst, Zachary Myers, Nestan Tsiskaridze, Stephen Richardson, Rick Bahr, Kayvon Fatahalian, Pat Hanrahan, Clark Barrett, Mark Horowitz, Christopher Torng, Fredrik Kjolstad, and Priyanka Raina
ACM Transactions on Embedded Computing Systems (TECS), January 2023
pdf
SpDISTAL: Compiling Distributed Sparse Tensor Computations
Rohan Yadav, Alex Aiken, and Fredrik Kjolstad
ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC), November 2022
pdf
Compiler Support for Sparse Tensor Computations in MLIR
Aart J.C Bik, Penporn Koanantakool, Tatiana Shpeisman, Nicolas Vasilache, Bixia Zheng, and Fredrik Kjolstad
ACM Transactions on Architecture and Code Optimization (TACO), September 2022
pdf youtube
DISTAL: The Distributed Tensor Algebra Compiler
Rohan Yadav, Alex Aiken, and Fredrik Kjolstad
ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2022
pdf youtube
Autoscheduling for Sparse Tensor Algebra with an Asymptotic Cost Model
Peter Ahrens, Fredrik Kjolstad, and Saman Amarasinghe
ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2022
pdf
Copy-and-Patch Compilation
Haoran Xu and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 5, Issue OOPSLA, November 2021
Distinguished Paper Award
pdf youtube
Compilation of Sparse Array Programming Models
Rawn Henry, Olivia Hsu, Rohan Yadav, Stephen Chou, Kunle Olukotun, Saman Amarasinghe, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 5, Issue OOPSLA, November 2021
pdf youtube
A Sparse Iteration Space Transformation Framework for Sparse Tensor Algebra
Ryan Senanayake, Changwan Hong, Ziheng Wang, Amalee Wilson, Stephen Chou, Shoaib Kamil, Saman Amarasinghe, and Fredrik Kjolstad
Proceedings of the ACM on Programming Languages, Volume 4, Issue OOPSLA, November 2020
pdf youtube
Creating an Agile Hardware Design Flow
Rick Bahr, Clark Barrett, Nikhil Bhagdikar, Alex Carsello, Ross Daly, Caleb Donovick, David Durst, Kayvon Fatahalian, Kathleen Feng, Pat Hanrahan, Teguh Hofstee, Mark Horowitz, Dillon Huff, Fredrik Kjolstad, Taeyoung Kong, Qiaoyi Liu, Makai Mann, Jackson Melchert, Ankita Nayak, Aina Niemetz, Gedeon Nyengele, Priyanka Raina, Stephen Richardson, Raj Setaluri, Jeff Setter, Kavya Sreedhar, Maxwell Strange, James Thomas, Christopher Torng, Leonard Truong, Nestan Tsiskaridze, and Keyi Zhang
Design Automation Conference (DAC), July 2020
pdf
Sparse Tensor Transpositions
Suzanne Mueller, Peter Ahrens, Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe
ACM Symposium on Parallelism in Algorithms and Architectures (SPAA brief announcement), July 2020
pdf
Automatic Generation of Efficient Sparse Tensor Format Conversion Routines
Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe
ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2020
pdf youtube
Tensor Algebra Compilation with Workspaces
Fredrik Kjolstad, Peter Ahrens, Shoaib Kamil, and Saman Amarasinghe
International Symposium on Code Generation and Optimization (CGO), February 2019
pdf
Format Abstraction for Sparse Tensor Algebra Compilers
Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe
Proceedings of the ACM on Programming Languages, Volume 2, Issue OOPSLA, October 2018
pdf youtube
Taco: compilation et génération de code d’expressions tensorielles
David Lugato, Fredrik Kjolstad, Stephen Chou, Saman Amarasinghe, and Shoaib Kamil
AVANCÉES, Volume 12, 2018
pdf
taco: A Tool to Generate Tensor Algebra Kernels
Fredrik Kjolstad, Stephen Chou, David Lugato, Shoaib Kamil, and Saman Amarasinghe
32th IEEE/ACM International Conference on Automated Software Engineering (ASE tools paper), November 2017
pdf youtube
The Tensor Algebra Compiler
Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, and Saman Amarasinghe
Proceedings of the ACM on Programming Languages, Volume 1, Issue OOPSLA, October 2017
Distinguished Paper Award
pdf youtube
Simit: A Language for Physical Simulation
Fredrik Kjolstad, Shoaib Kamil, Jonathan Ragan-Kelley, David I. W. Levin, Shinjiro Sueda, Desai Chen, Etienne Vouga, Danny M. Kaufman, Gurtej Kanwar, Wojciech Matusik, and Saman Amarasinghe
ACM Transactions on Graphics (TOG presented at SIGGRAPH 2016), May 2016
pdf youtube
Why New Programming Languages for Simulation?
Gilbert Louis Bernstein and Fredrik Kjolstad
ACM Transactions on Graphics (TOG perspective), May 2016
pdf
MPI Datatype Processing using Runtime Compilation
Timo Schneider, Fredrik Kjolstad, and Torsten Hoefler
20th European MPI Users' Group Meeting, September 2013
Best Paper Award
pdf
Automatic Datatype Generation and Optimization
Fredrik Kjolstad, Torsten Hoefler, and Marc Snir
17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP short paper), February 2012
pdf
Transformation for Class Immutability
Fredrik Kjolstad, Danny Dig, Gabriel Acevedo, and Marc Snir
33rd International Conference on Software Engineering (ICSE), May 2011
pdf
Bringing the HPC Programmer's IDE into the 21st Century through Refactoring
Fredrik Kjolstad, Danny Dig, and Marc Snir
SPLASH 2010 Workshop on Concurrency for the Application Programmer, October 2010
pdf
Ghost Cell Pattern
Fredrik Kjolstad and Marc Snir
2nd Annual Workshop on Parallel Programming Patterns, March 2010
pdf