Chung-Ming Chien (簡仲明)

Santorini, Greece

May 30, 2023

I am a 5th-year Ph.D. student at Toyota Technological Institute at Chicago (TTIC), advised by Karen Livescu. My overarching goal is to create Trustworthy, User-Friendly, and Human-Like Voice AI. My current research focuses on two key directions:

Balancing Intelligence with Interactivity in Voice AI
How can we make full-duplex voice models more intelligent without compromising real-time interactivity? How can we better preserve the knowledge acquired during large-scale text pre-training? How should a voice model decide what to delegate, to whom, and when—whether to a larger language model, a text-based reasoning system, or external tools?
Aligning Speech Language Models (SLMs) with Human Preference
How can we align SLMs with human preferences to improve content fidelity, natural turn-taking, and emotional responsiveness? How to enhance overall user experience?

Aside from these, my past research in Controllable Speech Generation, Efficient Fine-Tuning of SLMs, Flow-Matching & Diffusion Models, and Self-Supervised Speech Representations has provided a strong technical foundation for pursuing these goals.

Prior to joining TTIC, I completed my Master’s degree in Computer Science at National Taiwan University (NTU), where I had the fortune to work with Lin-shan Lee and Hung-yi Lee at the Speech Processing Lab. I also gained valuable practical experience through summer internships at Amazon Alexa TTS Research, FAIR (AI at Meta), NVIDIA NeMo Team, and Kyutai.

NTU

TTIC

Amazon Alexa

news

Sep 27, 2026	I completed my first half marathon in 82:16 🏃🏻‍➡️
Sep 27, 2026	I give a talk at the Interspeech 2026 tutorial On the Landscape of Spoken Language Models in Sydney
Sep 24, 2026	I give a talk at xAI on MoshiRAG
Jul 9, 2026	I will present MoshiRAG at ICML 2026 in Seoul. Orchestrating intelligence with interactivity is the future of voice AI
Apr 29, 2026	MoshiRAG is released as the world’s first full-duplex speech language model with retrieval-augmented generation (RAG)! Check out the paper, code, and blog post. Excited to present it at ICML 2026!
Sep 5, 2025	It was a true pleasure reconnecting with old friends and meeting new colleagues at the Workshop on Foundations of Speech and Audio Foundation Models in TTIC. It is my greatest hope that you found the event valuable and enjoyable.
Jul 17, 2025	I will be co-hosting the Workshop on Foundations of Speech and Audio Foundation Models this September. Join us at TTIC to explore the newest advancements in Speech Language Models.
Apr 18, 2025	This summer, I am heading to France to join Kyutai and work on moshi, the world’s first full-duplex speech assistant. This is a fresh trial, and I can’t wait to see what we achieve! Bonjour Paris
Apr 11, 2025	The long wait is over. Our review paper on SLMs, “On the landscape of spoken language models: A comprehensive survey”, is officially on arXiv. Check out this comprehensive survey -— it was a significant, collective effort and is packed with insights!
Jun 4, 2024	“Learning Fine‑Grained Controllability on Speech Generation via Efficient Fine‑Tuning” is accepted to InterSpeech 2024!

selected publications

ICML 2026

MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models

Chung-Ming Chien, Manu Orsini, Eugene Kharitonov, and 3 more authors

In Forty-third International Conference on Machine Learning

arXiv Bib Blog Code Poster Slides Video

@inproceedings{chien2026moshirag,
  author = {Chien, Chung-Ming and Orsini, Manu and Kharitonov, Eugene and Zeghidour, Neil and Livescu, Karen and D{\'e}fossez, Alexandre},
  booktitle = {Forty-third International Conference on Machine Learning},
  title = {Moshi{RAG}: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models},
  year = {2026},
  month = jul,
  eprint = {2604.12928},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
}

TMLR 2025

On The Landscape of Spoken Language Models: A Comprehensive Survey

Chung-Ming Chien*, Siddhant Arora*, Kai-Wei Chang*, and 7 more authors

*equal contribution

Transactions on Machine Learning Research

arXiv Bib

@article{arora2025on,
  title = {On The Landscape of Spoken Language Models: A Comprehensive Survey},
  author = {Chien*, Chung-Ming and Arora*, Siddhant and Chang*, Kai-Wei and Peng*, Yifan and Wu*, Haibin and Adi, Yossi and Dupoux, Emmanuel and Lee, Hung-Yi and Livescu, Karen and Watanabe, Shinji},
  journal = {Transactions on Machine Learning Research},
  volume = {},
  pages = {},
  year = {2026},
  month = jan,
  issn = {},
  doi = {},
  eprint = {2504.08528},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
}

InterSpeech 2024

Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning

Chung-Ming Chien, Andros Tjandra, Apoorv Vyas, and 3 more authors

In Interspeech 2024

arXiv Bib Poster

@inproceedings{chien2024learning,
  title = {Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning},
  author = {Chien, Chung-Ming and Tjandra, Andros and Vyas, Apoorv and Le, Matt and Shi, Bowen and Hsu, Wei-Ning},
  year = {2024},
  booktitle = {Interspeech 2024},
  month = sep,
  eprint = {2406.06251},
  archiveprefix = {arXiv},
  primaryclass = {eess.AS},
}

ASRU 2023

Few-Shot Spoken Language Understanding via Joint Speech-Text Models

Chung-Ming Chien, Mingjiamei Zhang, Ju-Chieh Chou, and 1 more author

Best Student Paper Award

In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

arXiv Bib Poster Slides Video

@inproceedings{chien2023few,
  title = {Few-Shot Spoken Language Understanding via Joint Speech-Text Models},
  author = {Chien, Chung-Ming and Zhang, Mingjiamei and Chou, Ju-Chieh and Livescu, Karen},
  booktitle = {2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year = {2023},
  month = dec,
  eprint = {2310.05919},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
}

ICASSP 2021

FragmentVC: Any-To-Any Voice Conversion by End-To-End Extracting and Fusing Fine-Grained Voice Fragments with Attention

Chung-Ming Chien*, Yist Y. Lin*, Jheng-Hao Lin, and 2 more authors

*equal contribution

In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

arXiv Bib Code Poster Slides

@inproceedings{chien2020fragmentvc,
  author = {Chien*, Chung-Ming and Lin*, Yist Y. and Lin, Jheng-Hao and Lee, Hung-yi and Lee, Lin-shan},
  booktitle = {ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  title = {FragmentVC: Any-To-Any Voice Conversion by End-To-End Extracting and Fusing Fine-Grained Voice Fragments with Attention},
  year = {2021},
  volume = {},
  number = {},
  pages = {5939-5943},
  doi = {10.1109/ICASSP39728.2021.9413699},
  month = jun,
  eprint = {2010.14150},
  archiveprefix = {arXiv},
  primaryclass = {eess.AS},
}