THESIS DESK // 06
research index ↙SENIOR HONORS THESIS · CARNEGIE MELLON · 2024–25
Resource-efficient representation learning, synthetic supervision, and the lab where everyone seemed to speak in loss functions.
I joined Bhiksha Raj’s ML & Signal Processing Lab as an undergraduate and worked on my senior honors thesis around a simple question: can speech and audio pretraining be made more efficient by changing the representation and the supervision rather than simply scaling the model?
research brief // 06
the question
Speech models are expensive partly because their input representations are large and their supervision can be costly to obtain.
My thesis explored two complementary routes: compressing the representation itself, and generating supervision synthetically.
two directions
Can the representation become smaller without becoming useless?
I explored deep compression autoencoders that mapped speech spectrograms from large inputs into much smaller latent representations while attempting to preserve useful acoustic structure.
Can speech errors be generated rather than annotated one by one?
I created controlled phoneme perturbations over TIMIT to generate synthetic speech-error examples and trained a classifier to distinguish error severity.
contribution ledger
Prototyped autoencoder architectures and evaluated how aggressively speech representations could be compressed while retaining acoustic structure.
Built the pipeline producing controlled phoneme-level perturbations over TIMIT.
Measured reconstruction quality and downstream classification performance across experimental settings.
Brought the work together into an honors thesis spanning literature review, system design, experimentation, analysis, and defense.
apprenticeship, really
Pittsburgh
Later, Doha
I joined Bhiksha Raj’s lab as an undergraduate, in an environment where everyone around me seemed to casually speak in loss functions.
Some of the most formative parts of this project were not experiments at all. They were hours in his Pittsburgh office going through papers paragraph by paragraph — not just what a method did, but why the authors had made each decision and how an argument was constructed.
When I returned to Doha, those conversations moved onto Zoom. The medium changed; the patience didn’t.
maybe this is where the professor dream really started: being taught with patience, in all the places no one else sees.
trajectory
This thesis changed how I think about efficiency.
Rather than treating larger models and more data as the default path forward, I became interested in asking what structure actually matters, what can be compressed, and what kinds of supervision can be constructed more carefully.
It also changed how I think about academic life. The technical work mattered, but so did experiencing what sustained mentorship looks like from the student side.
methods
speech representation learning · autoencoders · compression · synthetic supervision · phoneme perturbation · LibriSpeech · TIMIT · PyTorch
thesis