Juncheng Yang

I am an Assistant Professor in Harvard John A. Paulson School of Engineering and Applied Sciences. I received my Ph.D. in Computer Science from Carnegie Mellon University working with Rashmi Vinayak. I am broadly interested in computer systems with particular focus on workload analysis, design of efficient, reliable and sustainable data storage and machine learning systems.

My works have received Best-Paper Award or Honorable Mention at NSDI'24, NSDI'21, SOSP'21, VALUETOOLS'24, and SYSTOR'16 and have been deployed in production at Google, VMware, Twitter, Redpanda with many open-source libraries contributed by the community. I am a 2020 Meta Fellow, a 2023 Google Cloud Research Innovator, and a 2023 Rising Star in Machine Learning and Systems. And I received the CMU SCS Doctoral Dissertation Award, the ACM SIGOPS Dennis M. Ritchie Doctoral Dissertation Award.

I lead the Harvard Measurements and Design of Computer Systems Group (MadSys Group). And I am fortunate to work with several amazing students.

Storage systems and machine learning systems with a focus on efficiency, scalability and robustness:

• Efficient, sustainable and scalable storage systems
  • Measurement studies [OSDI'20][HotOS'23][VLDB'26]
  • New cache system designs [NSDI'21 ][SOSP'21 ] [NSDI'22] [EuroSys'23]
  • New replacement algorithm designs [FAST'23] [SOSP'23][NSDI'24 ][VLDB'26] [OSDI'26]
  • Storage for Large Models [NSDI'26][SOSP'26]
• Efficient LLM serving and democratizing access to AI
  • Serving bursty workloads [MLSys'26]
  • Serving Prefill workloads [NSDI'27]
  • Hybrid Inference [Eurosys'27]
• Robust and reliable learned systems [VLDB'23] [OSDI'26]

• FreeInference: democratizing access to large language models, currently serving more than 10B tokens daily to users from more than 50 countries speaking 25 different languages using 20+ agents.
• SIEVE (NSDI'24): the first cache eviction algorithm simpler than LRU but yet more effective than state-of-the-art algorithms for web caches. Adopted by software and systems such as Android API, BIND 9, ImmuDB, TiDB, PostgREST Implemented in many open-source libraries, e.g., Golang, Python, JavaScript, Rust, Java, Swift, Ruby, Nim, and Zig. Find more details on SIEVE website.
• S3-FIFO (SOSP'23): a simple and scalable cache eviction algorithm composed of only FIFO queues. Implemented or deployed at companies including Google, VMware and Redpanda, and many open-source libraries. Find more details on S3-FIFO website.
• Segcache (NSDI'21): received a community best-paper award, and deployed at Twitter and Momento.

We release open artifacts so that others can reproduce, extend, and build on our work. Most of them are led and maintained by students in the lab.

• Open-source libraries
  • libCacheSim — a high-performance library for building cache simulators, used by almost 100 research institutes and companies
  • libCacheSim-python — Python bindings for rapid cache simulation experiments (pip install libcachesim)
  • RouteWise — a library for LLM routers to optimize cost and latency across multiple providers
• Open Datasets
  • Cache trace dataset — a large open collection of key-value, object/CDN, and block cache traces (2007–2025) from Meta, Twitter, Alibaba, Tencent, and others
  • Year-long LLM serving dataset (coming soon)
• Free Services
  • FreeInference — a free, OpenAI- and Anthropic-compatible inference API for frontier open-source models, aimed at research, teaching, and prototyping
  • traceViz — a web tool for visualizing and exploring storage and cache workloads
• Open infrastructure
  • A public dashboard for the GPUs and serving engine behind FreeInference (coming soon)

Juncheng Yang is an Assistant Professor in Harvard John A. Paulson School of Engineering and Applied Sciences. He received his Ph.D. in Computer Science from Carnegie Mellon University in 2024. His research interests broadly cover the efficiency, performance, reliability, and sustainability of large-scale data systems.

Juncheng's works have received best paper awards at VALUETOOLS'24, NSDI'24, NSDI'21, SOSP'21, and SYSTOR'16. His OSDI'20 paper was recognized as one of the best storage papers at the conference and invited to ACM TOS'21. Juncheng received a Facebook Ph.D. Fellowship in 2020, was recognized as a Rising Star in machine learning and systems in 2023, and a Google Cloud Research Innovator in 2023. His dissertation on designing efficient and scalable cache management systems received the ACM SIGOPS Dennis M. Ritchie Doctoral Dissertation Award.

His works have been widely adopted. For example, S3-FIFO and SIEVE are adopted for production at Google, VMware, Redpanda, and several others, with over 60 open-source libraries and packages in 18 programming languages available on GitHub. Moreover, the open-source cache simulation library he created, libCacheSim, has been used by almost 100 research institutes and companies.