CS343D: Domain-Specific Programming Models and Compilers

Course Description

This class will cover the principles and practices of domain-specific programming models and compilers for dense and sparse applications in scientific computing, data science, and machine learning. We will study programming models from the recent literature, categorize them, and discuss their properties. We will also discuss promising directions for their compilation, including the separation of algorithm, schedule, and data representation, polyhedral compilation versus rewrite rules, and sparse iteration theory. Prerequisites: CS143 or equivalent

Basic Logistics

Professor: Fred Kjolstad

CA: AJ Root

Quarter: Winter 2024

Lecture Times: Tue, Thu 10:30 AM - 11:50 AM

Lecture Location: Littlefield 107

Fred's office hours: Friday 11-12noon in Gates 486 (10-11am on Jan 12th)

AJ's office hours: Wednesday 3-4pm in Gates 4B common area

For more detail see the course info page.

Schedule

	Date	Topic	Readings
Lecture 1	January 9	Introduction	Programming Pearls: little languages A New Golden Age for Computer Architecture
Lecture 2	January 11	Domain-Specific Compilers	There’s plenty of room at the Top: What will drive computer performance after Moore’s law? Can Programming Be Liberated from the von Neumann Style? (except Sections 12, 13, and 14)
Lecture 3	January 16	Building DSLs	Building Domain-Specific Embedded Languages Terra: a multi-stage language for high-performance computing
Lecture 4	January 18	Collection-Oriented Languages	Collection-Oriented Languages An Introduction to the Set Theoretical Language SETL
Lecture 5	January 23	Dense Programming System	Halide: Decoupling Algorithms from Schedules for High-Performance Image Processing
Lecture 6	January 25	Sparse Programming Systems	A Relational Model of Data for Large Shared Data Banks Sparse Tensor Algebra Compilation (Chapters 1 and 9)
Lecture 7	January 30	Sparse Iteration Model I	Sparse Tensor Algebra Compilation (Chapters 2 and 3)
Lecture 8	February 1	Sparse Iteration Model II	Sparse Tensor Algebra Compilation (Chapters 4, 5, and 6)
Lecture 9	February 6	Project Pitch
Lecture 10	February 8	Fast Compilers	Firefox's Baseline WebAssembly Compiler Copy-and-Patch Compilation
Lecture 11	February 13	Project Discussion I
Lecture 12	February 15	Project Discussion II
Lecture 13	February 20	Performance vs. Correctness When Writing Low-Level HPC Code (Gilbert Bernstein)	Verified Tensor-Program Optimization Via High-Level Scheduling Rewrites Exocompilation for Productive Programming of Hardware Accelerators
Lecture 14	February 22	Codesigning Computing Systems for Artificial Intelligence (Suvinay Subramanian)	TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings Why Systolic Architectures?
Lecture 15	February 27	Domain-Specific Accelerators	Computing’s Energy Problem (and what we can do about it) The Sparse Abstract Machine
Lecture 16	February 29	Sparse Shape Operators (AJ)	Compilation of Shape Operators on Sparse Arrays
Lecture 17	March 5	Automatic Scheduling (AJ)	Learning to Optimize Halide with Tree Search and Random Programs Autoscheduling for Sparse Tensor Algebra with an Asymptotic Cost Model
Lecture 18	March 7	Notation (Pat Hanrahan)	Notation as a Tool of Thought Mathematical Notation: Past and Future
Lecture 19	March 12	Project demo
Lecture 20	March 14	Project demo