Experiments and Model Evaluation
This page collects instructions for project preparation, experiment execution, and cross-model comparison.
Research Preparation
Scheduled for Week 7.
Goals:
- Refine the research question.
- Check whether the proposed task fits the four-layer evaluation framework.
- Confirm datasets, models, metrics, and expected failure modes.
- Reduce scope before the midterm presentation.
Experiment Execution
Scheduled for Week 12.
Goals:
- Run experiments.
- Collect and clean data.
- Inspect preliminary results.
- Identify prompt, data, or model issues.
Extended Experiments and Cross-Model Comparison
Scheduled for Week 13.
Goals:
- Test additional models where feasible.
- Compare open-source and closed-source systems.
- Check robustness across prompts, datasets, and task variants.
- Prepare analysis results for the writing workshops.
Notes
Model access and GPU instructions will be posted in Handouts after the infrastructure plan is finalized.