PAIR-Net
Learn the comparison
- neural representations
- prompt-conditioned cross-attention
- pairwise score differences
- cached embeddings
LANGUAGE DESK // 04
research index ↙NATIONAL RESEARCH FUND–SUPPORTED RESEARCH · 2025
Arabic Writing Assessment
Can a model learn writing quality rather than memorizing the question?
As part of a National Research Fund–supported collaboration between Carnegie Mellon and Qatar University, I worked on automated Arabic writing assessment in low-resource settings.
My focus was cross-prompt generalization: training on writing from several prompts, then evaluating whether the model could score essays responding to a prompt it had never seen before.
research brief // 04
At a Glance
- Question
- Can automated writing assessment generalize to completely unseen prompts?
- Setting
- Arabic student writing across QAES and CAST datasets.
- Evaluation
- Leave-one-prompt-out evaluation across eight writing prompts.
- Approaches
- PAIR-Net · neural pairwise scoring
ALiF-Net · Arabic linguistic feature modeling - My role
- Cross-prompt evaluation · model implementation · linguistic feature engineering · augmentation experiments · reproducibility pipeline
- Project context
- National Research Fund–supported CMU × Qatar University research initiative
the problem
Scoring the Writing, Not the Prompt
Most automated essay-scoring systems perform best when training and testing happen on the same writing prompt. That is convenient experimentally, but awkward in real educational settings where new questions appear all the time.
We asked whether Arabic writing quality could be modeled across prompts rather than rebuilding a scoring system every time the question changed.
prompts seen during training
01020304050607
↓one scoring modellearn across prompts↓
unseen prompt
08?two assumptions
Two Ways of Asking the Question
ALiF-Net
Make the language visible
- morphology + syntax
- readability + lexical structure
- prompt adherence
- Arabic-specific linguistic features
Same question, different assumptions about what evidence of writing quality should look like.
contribution ledger
What I Built
Cross-prompt evaluation pipeline
Built the training and evaluation setup used to hold each prompt out completely in turn.
PAIR-Net implementation
Worked on the prompt-aware pairwise scoring architecture and efficient embedding reuse.
Arabic linguistic feature pipeline
Developed features spanning morphology, syntax, readability, lexical structure, and semantic prompt adherence.
Reproducible experimentation
Structured multi-seed runs, logging, cached representations, and shared evaluation metrics.
Cross-lingual augmentation experiments
Tested whether translated English learner essays could expand training coverage without overwhelming human-scored Arabic data.
a side experiment
Can Synthetic Data Help Without Becoming the Dataset?
We also experimented with cross-lingual augmentation: English learner essays were translated into Arabic with prompts designed to preserve learner-like errors rather than silently “fix” the writing.
The useful question was not simply whether more data helped, but when synthetic data complemented — or distorted — the original scoring signal.
translation models have an unfortunate habit of correcting the mistakes you specifically asked them to preserve
trajectory
Why I Still Care About This
Automated assessment is ultimately a model standing in for a human judgment.
This project was one of the first places I became interested in what happens when a model leaves the clean setting it was trained in: new prompts, sparse data, different rubrics, and linguistic structures that were never designed around English.
That question — whether a system remains meaningful when context changes — followed me into much of my later work.
methods
Methods
Arabic NLP · cross-prompt evaluation · pairwise regression · cross-attention · linguistic feature engineering · cross-lingual augmentation · AraBERT · CAMeL Tools · QWK
project context