Fredrik Kjolstad

Assistant Professor
Department of Computer Science
Stanford University
Office: CoDa 456 kjolstad@stanford.edu
CV

My group aims to make it easier to develop applications that are portable across different data representations and different machines. This work includes developing languages on collections of data (tensors, relations, graphs, and objects embedded in space) and developing meta-compilers that generates compilers for different languages and machines.

In particular, we have developed compilers for sparse tensor algebra and other array operations on sparse data, which lets us build tensor algebra and array libraries and languages that are polymorphic over different data structures. This work includes the TACO compiler, the Simit Programming Language, the Burrito compiler, the Distal and Legate Sparse systems, the Sparse Abstract Machine, the Etch Indexed Stream compiler, the RECUMA recurrence compiler, and the Scorch sparse extension to the PyTorch library. We have also developed the Copy-and-Patch compiler technology that makes it possible to derive from LLVM lightning fast online compilers for different ISAs, which have no runtime dependencies on LLVM.

My thesis on Sparse Tensor Algebra Compilation is a good place to start reading about the ideas behind the tensor algebra compiler (taco) and sparse iteration model.

PhD Students

Alumni

Trevor Gale (with Matei Zaharia)

Publications

Designing Programmable Accelerators for Sparse Tensor Algebra Kalhan Koul, Zhouhua Xie, Maxwell Strange, Sai Gautham Ravipati, Bo Wun Cheng, Olivia Hsu, Po-Han Chen, Mark Horowitz, Fredrik Kjolstad, and Priyanka Raina IEEE Micro, April 2025
Automatic Tracing in Task-Based Runtime Systems Rohan Yadav, Michael Bauer, David Broman, Michael Garland, Alex Aiken, and Fredrik Kjolstad International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), April 2025
Composing Distributed Computations Through Task and Kernel Fusion. Rohan Yadav, Shiv Sundram, Wonchan Lee, Michael Garland, Michael Bauer, Alex Aiken, and Fredrik Kjolstad International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), April 2025
Stardust: Compiling Sparse Tensor Algebra to a Reconfigurable Dataflow Architecture Olivia Hsu, Alexander Rucker, Tian Zhao, Varun Desai, Kunle Olukotun, and Fredrik Kjolstad International Symposium on Code Generation and Optimization (CGO), March 2025
Compiling Recurrences over Dense and Sparse Arrays Shiv Sundram, Muhammad Usman Tariq, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 8, Issue OOPSLA, October 2024
Compilation of Shape Operators on Sparse Arrays Alexander J Root, Bobby Yan, Peiming Liu, Christophe Gyurgyik, Aart J.C Bik, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 8, Issue OOPSLA, October 2024
Compiler Support for Sparse Tensor Convolutions Peiming Liu, Alexander J Root, Anlun Xu, Yinying Li, Fredrik Kjolstad, and Aart J.C Bik Proceedings of the ACM on Programming Languages, Volume 8, Issue OOPSLA, October 2024
The Dataflow Abstract Machine Simulator Framework Nathan Zhang, Rubens Lacouture, Gina Sohn, Paul Mure, Qizheng Zhang, Fredrik Kjolstad, and Kunle Olukotun Proceedings of the International Symposium on Computer Architecture (ISCA), June 2024 ISCA Distinguished Artifact Award
Compilation of Modular and General Sparse Workspaces Genghan Zhang, Olivia Hsu, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 8, Issue PLDI, June 2024
Onyx: A 12nm 756 GOPS/W Coarse-Grained Reconfigurable Array for Accelerating Dense and Sparse Applications Kalhan Koul, Maxwell Strange, Jackson Melchert, Alex Carsello, Yuchen Mei, Olivia Hsu, Taeyoung Kong, Po-Han Chen, Huifeng Ke, Keyi Zhang, Qiaoyi Liu, Gedeon Nyengele, Akhilesh Balasingam, Jayashree Adivarahan, Ritvik Sharma, Zhouhua Xie, Christopher Torng, Joel Emer, Fredrik Kjolstad, Mark Horowitz, and Priyanka Raina IEEE Symposium on VLSI Technology and Circuits (VLSI), June 2024
Revet: A Language and Compiler for Dataflow Threads Alexander Rucker, Shiv Sundram, Coleman Smith, Matt Vilim, Raghu Prabhakar, Fredrik Kjolstad, and Kunle Olukotun IEEE International Symposium on High-Performance Computer Architecture (HPCA), March 2024
Legate Sparse: Distributed Sparse Computing in Python Rohan Yadav, Wonchan Lee, Melih Elibol, Manolis Papadakis, Taylor Lee-Patti, Michael Garland, Alex Aiken, Fredrik Kjolstad, and Michael Bauer ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC), November 2023
Indexed Streams: A Formal Intermediate Representation for the Fused Execution of Contraction Operations Scott Kovach, Praneeth Kolichala, Timothy Gu, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 7, Issue PLDI, June 2023
Mosaic: An Interoperable Compiler for Tensor Algebra Manya Bansal, Olivia Hsu, Kunle Olukotun, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 7, Issue PLDI, June 2023 Distinguished Paper Award
BaCO: A Fast and Portable Bayesian Compiler Optimization Framework Erik Hellsten, Artur Souza, Johannes Lenfers, Rubens Lacouture, Olivia Hsu, Adel Ejjeh, Fredrik Kjolstad, Michel Steuwer, Kunle Olukotun, and Luigi Nardi International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), April 2023
The Sparse Abstract Machine Olivia Hsu, Maxwell Strange, Ritvik Sharma, Jaeyeon Won, Kunle Olukotun, Joel Emer, Mark Horowitz, and Fredrik Kjolstad Architectural Support for Programming Languages and Operating Systems (ASPLOS), March 2023
Unified Buffer: Compiling Image Processing and Machine Learning Applications to Push-Memory Accelerators Qiaoyi Liu, Jeff Setter, Dillon Huff, Maxwell Strange, Kathleen Feng, Mark Horowitz, Priyanka Raina, and Fredrik Kjolstad ACM Transactions on Architecture and Code Optimization (TACO), March 2023
Looplets: A Language For Structured Coiteration Willow Ahrens, Daniel Donenfeld, Fredrik Kjolstad, and Saman Amarasinghe International Symposium on Code Generation and Optimization (CGO), February 2023
AHA: An agile approach to the design of coarse-grained reconfigurable accelerators and compilers Kalhan Koul, Jackson Melchert, Kavya Sreedhar, Leonard Truong, Gedeon Nyengele, Keyi Zhang, Qiaoyi Liu, Jeff Setter, Po-Han Chen, Yuchen Mei, Maxwell Strange, Ross Daly, Caleb Donovick, Alex Carsello, Taeyoung Kong, Kathleen Feng, Dillon Huff, Ankita Nayak, Raj Setaluri, James Thomas, Nikhil Bhagdikar, David Durst, Zachary Myers, Nestan Tsiskaridze, Stephen Richardson, Rick Bahr, Kayvon Fatahalian, Pat Hanrahan, Clark Barrett, Mark Horowitz, Christopher Torng, Fredrik Kjolstad, and Priyanka Raina ACM Transactions on Embedded Computing Systems (TECS), January 2023
SpDISTAL: Compiling Distributed Sparse Tensor Computations Rohan Yadav, Alex Aiken, and Fredrik Kjolstad ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC), November 2022
Compiler Support for Sparse Tensor Computations in MLIR Aart J.C Bik, Penporn Koanantakool, Tatiana Shpeisman, Nicolas Vasilache, Bixia Zheng, and Fredrik Kjolstad ACM Transactions on Architecture and Code Optimization (TACO), September 2022
DISTAL: The Distributed Tensor Algebra Compiler Rohan Yadav, Alex Aiken, and Fredrik Kjolstad ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2022
Autoscheduling for Sparse Tensor Algebra with an Asymptotic Cost Model Peter Ahrens, Fredrik Kjolstad, and Saman Amarasinghe ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2022
Copy-and-Patch Compilation Haoran Xu and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 5, Issue OOPSLA, November 2021 Distinguished Paper Award
Compilation of Sparse Array Programming Models Rawn Henry, Olivia Hsu, Rohan Yadav, Stephen Chou, Kunle Olukotun, Saman Amarasinghe, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 5, Issue OOPSLA, November 2021
A Sparse Iteration Space Transformation Framework for Sparse Tensor Algebra Ryan Senanayake, Changwan Hong, Ziheng Wang, Amalee Wilson, Stephen Chou, Shoaib Kamil, Saman Amarasinghe, and Fredrik Kjolstad Proceedings of the ACM on Programming Languages, Volume 4, Issue OOPSLA, November 2020
Creating an Agile Hardware Design Flow Rick Bahr, Clark Barrett, Nikhil Bhagdikar, Alex Carsello, Ross Daly, Caleb Donovick, David Durst, Kayvon Fatahalian, Kathleen Feng, Pat Hanrahan, Teguh Hofstee, Mark Horowitz, Dillon Huff, Fredrik Kjolstad, Taeyoung Kong, Qiaoyi Liu, Makai Mann, Jackson Melchert, Ankita Nayak, Aina Niemetz, Gedeon Nyengele, Priyanka Raina, Stephen Richardson, Raj Setaluri, Jeff Setter, Kavya Sreedhar, Maxwell Strange, James Thomas, Christopher Torng, Leonard Truong, Nestan Tsiskaridze, and Keyi Zhang Design Automation Conference (DAC), July 2020
Sparse Tensor Transpositions Suzanne Mueller, Peter Ahrens, Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe ACM Symposium on Parallelism in Algorithms and Architectures (SPAA brief announcement), July 2020
Automatic Generation of Efficient Sparse Tensor Format Conversion Routines Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), June 2020
Tensor Algebra Compilation with Workspaces Fredrik Kjolstad, Peter Ahrens, Shoaib Kamil, and Saman Amarasinghe International Symposium on Code Generation and Optimization (CGO), February 2019
Format Abstraction for Sparse Tensor Algebra Compilers Stephen Chou, Fredrik Kjolstad, and Saman Amarasinghe Proceedings of the ACM on Programming Languages, Volume 2, Issue OOPSLA, October 2018
Taco: compilation et génération de code d’expressions tensorielles David Lugato, Fredrik Kjolstad, Stephen Chou, Saman Amarasinghe, and Shoaib Kamil AVANCÉES, Volume 12, 2018
taco: A Tool to Generate Tensor Algebra Kernels Fredrik Kjolstad, Stephen Chou, David Lugato, Shoaib Kamil, and Saman Amarasinghe 32th IEEE/ACM International Conference on Automated Software Engineering (ASE tools paper), November 2017
The Tensor Algebra Compiler Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, and Saman Amarasinghe Proceedings of the ACM on Programming Languages, Volume 1, Issue OOPSLA, October 2017 Distinguished Paper Award
Simit: A Language for Physical Simulation Fredrik Kjolstad, Shoaib Kamil, Jonathan Ragan-Kelley, David I. W. Levin, Shinjiro Sueda, Desai Chen, Etienne Vouga, Danny M. Kaufman, Gurtej Kanwar, Wojciech Matusik, and Saman Amarasinghe ACM Transactions on Graphics (TOG presented at SIGGRAPH 2016), May 2016
Why New Programming Languages for Simulation? Gilbert Louis Bernstein and Fredrik Kjolstad ACM Transactions on Graphics (TOG perspective), May 2016
MPI Datatype Processing using Runtime Compilation Timo Schneider, Fredrik Kjolstad, and Torsten Hoefler 20th European MPI Users' Group Meeting, September 2013 Best Paper Award
Automatic Datatype Generation and Optimization Fredrik Kjolstad, Torsten Hoefler, and Marc Snir 17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP short paper), February 2012
Transformation for Class Immutability Fredrik Kjolstad, Danny Dig, Gabriel Acevedo, and Marc Snir 33rd International Conference on Software Engineering (ICSE), May 2011
Bringing the HPC Programmer's IDE into the 21st Century through Refactoring Fredrik Kjolstad, Danny Dig, and Marc Snir SPLASH 2010 Workshop on Concurrency for the Application Programmer, October 2010
Ghost Cell Pattern Fredrik Kjolstad and Marc Snir 2nd Annual Workshop on Parallel Programming Patterns, March 2010