Chung-Ming Chien (簡仲明)

Santorini, Greece

May 30, 2023

I am a 4th-year Ph.D. student at Toyota Technological Institute at Chicago (TTIC), advised by Karen Livescu. My overarching ambition is to create Trustworthy and User-Friendly Human-Like Speech AI. To achieve this goal, my current research is focused on:

Factual Conversational Speech AI
How to make full-duplex models more intelligent while preserving their interactiveness? How to best preserve the knowledge learned from large-scale text pre-training?
Aligning Speech Language Models (SLMs) with Human Preference
How to improve various aspects of SLMs – such as content fidelity, natural turn-taking, and emotional expressiveness – with human preference? How to enhance overall user experience?

Aside from these, my past research in Controllable Speech Generation, Efficient Fine-Tuning of SLMs, Flow-Matching & Diffusion Models, and Self-Supervised Speech Representations have also been crucial experience and provided me valuable foundation on my way towards achieving the goal.

Prior to joining TTIC, I completed my Master’s degree in Computer Science at National Taiwan University (NTU), where I had the fortune to work with Lin-shan Lee and Hung-yi Lee at the Speech Processing Lab. I also gained valuable practical experience through summer internships at Amazon Alexa TTS Research, FAIR (AI at Meta), NVIDIA NeMo Team, and Kyutai.

NTU

TTIC

Amazon Alexa

news

Sep 5, 2025	It was a true pleasure reconnecting with old friends and meeting new colleagues at the Workshop on Foundations of Speech and Audio Foundation Models in TTIC. It is my greatest hope that you found the event valuable and enjoyable.
Jul 17, 2025	I will be co-hosting the Workshop on Foundations of Speech and Audio Foundation Models this September. Join us at TTIC to explore the newest advancements in Speech Language Models.
Apr 18, 2025	This summer, I am heading to France to join Kyutai and work on moshi, the world’s first full-duplex speech assistant. This is a fresh trial, and I can’t wait to see what we achieve! Bonjour Paris
Apr 11, 2025	The long wait is over. Our review paper on SLMs, “On the landscape of spoken language models: A comprehensive survey”, is officially on arXiv. Check out this comprehensive survey -— it was a significant, collective effort and is packed with insights!
Jun 4, 2024	“Learning Fine‑Grained Controllability on Speech Generation via Efficient Fine‑Tuning” is accepted to InterSpeech 2024!
May 16, 2024	“On the Evaluation of Speech Foundation Models for Spoken Language Understanding” is accepted to Findings of ACL 2024!
Apr 16, 2024	I gave a talk at Midwest Speech and Language Days in Ann Arbor, Michigan
Apr 9, 2024	I successfully passed the qualifying exam of TTIC and will soon become a Ph.D. candidate
Jan 23, 2024	“What Do Self‑Supervised Speech Models Know about Words” is accepted to TACL 2024!
Jan 18, 2024	I will join NVIDIA NeMo Team for my 2024 summer internship and will work on speech language models!

selected publications

TMLR 2026

On The Landscape of Spoken Language Models: A Comprehensive Survey

Chung-Ming Chien*, Siddhant Arora*, Kai-Wei Chang*, and 7 more authors

*equal contribution

Transactions on Machine Learning Research

arXiv Bib

@article{arora2025on,
  title = {On The Landscape of Spoken Language Models: A Comprehensive Survey},
  author = {Chien*, Chung-Ming and Arora*, Siddhant and Chang*, Kai-Wei and Peng*, Yifan and Wu*, Haibin and Adi, Yossi and Dupoux, Emmanuel and Lee, Hung-Yi and Livescu, Karen and Watanabe, Shinji},
  journal = {Transactions on Machine Learning Research},
  volume = {},
  pages = {},
  year = {2026},
  month = jan,
  issn = {},
  doi = {},
  eprint = {2504.08528},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
}

InterSpeech 2024

Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning

Chung-Ming Chien, Andros Tjandra, Apoorv Vyas, and 3 more authors

In Interspeech 2024

arXiv Bib Poster

@inproceedings{chien2024learning,
  title = {Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning},
  author = {Chien, Chung-Ming and Tjandra, Andros and Vyas, Apoorv and Le, Matt and Shi, Bowen and Hsu, Wei-Ning},
  year = {2024},
  booktitle = {Interspeech 2024},
  month = sep,
  eprint = {2406.06251},
  archiveprefix = {arXiv},
  primaryclass = {eess.AS},
}

ASRU 2023

Few-Shot Spoken Language Understanding via Joint Speech-Text Models

Chung-Ming Chien, Mingjiamei Zhang, Ju-Chieh Chou, and 1 more author

Best Student Paper Award

In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

arXiv Bib Poster Slides Video

@inproceedings{chien2023few,
  title = {Few-Shot Spoken Language Understanding via Joint Speech-Text Models},
  author = {Chien, Chung-Ming and Zhang, Mingjiamei and Chou, Ju-Chieh and Livescu, Karen},
  booktitle = {2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year = {2023},
  month = dec,
  eprint = {2310.05919},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
}

ICASSP 2021

FragmentVC: Any-To-Any Voice Conversion by End-To-End Extracting and Fusing Fine-Grained Voice Fragments with Attention

Chung-Ming Chien*, Yist Y. Lin*, Jheng-Hao Lin, and 2 more authors

*equal contribution

In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

arXiv Bib Code Poster Slides

@inproceedings{chien2020fragmentvc,
  author = {Chien*, Chung-Ming and Lin*, Yist Y. and Lin, Jheng-Hao and Lee, Hung-yi and Lee, Lin-shan},
  booktitle = {ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  title = {FragmentVC: Any-To-Any Voice Conversion by End-To-End Extracting and Fusing Fine-Grained Voice Fragments with Attention},
  year = {2021},
  volume = {},
  number = {},
  pages = {5939-5943},
  doi = {10.1109/ICASSP39728.2021.9413699},
  month = jun,
  eprint = {2010.14150},
  archiveprefix = {arXiv},
  primaryclass = {eess.AS},
}