Presentation Round 3: From a Research Plan to Preliminary Evidence

Overview

Round 3 is a progress report on the study you have begun to run. In Round 1, you organized a research area around its central questions. In Round 2, you developed a testable plan. In Round 3, you explain what happened when that plan met actual stimuli, participants, models, and data.

The central question is: what have we learned so far, how confident should we be, and what should we do next?

A useful report includes actual preliminary evidence, from a pilot or an initial experiment. The study does not need to be complete, and the results do not need to support your prediction. What matters is whether the audience can assess the evidence and understand the decisions it motivates. A working interface is an important milestone; interpreting the data it produces is the next step.

These guidelines apply to the Round 3 group presentation listed in the course syllabus; each presentation round runs over two class sessions.

Goals of Round 3

By the end of your presentation, the audience should understand:

• What specific question is the study testing?
• What has actually been implemented and run?
• What do the initial results show about the main comparison?
• What checks make those results interpretable?
• What explanations remain possible?
• Which next experiment or analysis would be most informative?

Start with the Question and the Current State

Give a short reminder of the question and prediction, then explain any important revision since Round 2 or the midterm. Connect each revision to a reason: feedback, a failed assumption, a practical constraint, or an observation from the pilot.

Distinguish three statuses throughout the talk: completed, in progress, and planned. A planned control cannot yet support a conclusion. If a major component is blocked, show the evidence available, diagnose the obstacle, and specify the smallest next step that would resolve it.

Example to Explore: A Research Checkpoint

The Stanford CS341 project checkpoints combine methods, experimental setup, ongoing results, and current challenges. Read the second checkpoint description and ask: could a listener distinguish what our team has already demonstrated from what we still hope to demonstrate?

Show the Study That Was Actually Run

Use one concrete trial to explain what a participant or model received, what it did, and how that response entered the analysis. Show the actual stimulus, instruction or prompt, response options, and scoring rule where relevant. Then locate that example within the full experiment.

Keep the main talk focused on details that affect interpretation. Place lengthy prompts, configuration files, and additional screenshots in backup slides.

Describe the Data Behind Each Result

• Coverage: which conditions, participants or models, stimuli, and sessions have been completed?
• Counts: how many distinct participants or items, trials, and repeated model responses contributed?
• Selection: how were items chosen, and which observations were excluded or missing?
• Protocol: which task, prompt, model version, and scoring procedure generated this batch?

For model evaluations, record the model identifier, run date, sampling settings, prompt version, repetition scheme, and treatment of invalid responses. If a protocol changed during collection, identify the affected batches and explain whether they can be compared or combined.

Check the Measurement Before Interpreting the Score

Walk a few responses through the entire analysis: original input → raw response → extracted answer → assigned score → plotted value. Choose examples that cover both ordinary and problematic cases.

A shuffled-label check is useful only when its expected behavior is clear for your task. A random or majority-class baseline must reflect the available choices and class balance. State what each check tests and what it leaves unresolved.

Example to Explore: Reproducibility

Pineau et al. (2021) discuss reproducibility practices in machine learning. Use the paper to check whether another team could recover your reported result from your recorded setup, data, and analysis. Re-running a script successfully does not by itself establish that the task measures the intended ability.

Present the Critical Result Clearly

Begin each result with the question it addresses. Identify the comparison, show the relevant figure, and state the observed pattern in ordinary language. Explain the axes and reference level before discussing an effect.

Choose a display that preserves the structure of the evidence:

• For paired conditions, show paired observations or their differences.
• For heterogeneous items or participants, show their distribution as well as a summary.
• For several conditions, make the predicted contrast easy to locate.
• For errors, report how often each pattern occurs before showing illustrative cases.

State What the Uncertainty Represents

Always identify what the error bars or intervals represent and the units used to calculate them. Standard deviation describes variation; standard error and confidence intervals address estimation uncertainty under their respective assumptions. They are not interchangeable. See Cumming, Fidler, and Vaux (2007) for a short introduction.

In an AI study, variation across items, repeated responses, and model identities answers different questions. In a human study, repeated trials belong to participants. Preserve these dependencies in the analysis rather than treating every row as an independent sample.

Write a Useful Figure Caption

Points represent [units]. The plotted outcome is [measure] for [conditions]. The summary is [statistic]; intervals represent [type and calculation]. The analysis includes [counts], with [exclusions or missingness].

Complete this caption before presenting the figure. It often reveals an ambiguity in the analysis.

Learn by Looking: What Does the Summary Hide?

Weissgerber et al. (2015) show that similar summaries can conceal different distributions and paired patterns. Use their Figures 1 and 2 as short exercises: make an initial judgment before reading the interpretation, then explain whether the additional information changes your judgment.

• First look: cover panels B–E. What can you infer from panel A? What remains unknown?
• Reveal: compare the distributions in B–E. Their means and standard errors are similar, while the data differ in outliers, clustering, and sample size.
• Discuss: which features would you inspect before interpreting a group difference? An unusual point deserves investigation; its appearance alone does not justify deleting it.

Try it on your data. Keep the data fixed. Make a summary plot and a second plot showing the observations or their distribution, and label what each observation represents. Write one sentence explaining what the second display reveals and whether it changes your conclusion.

A bar chart can be appropriate for counts or totals. For a sample of measurements, check whether an aggregate display hides variation relevant to the claim. Changing chart type does not repair a weak design or create additional evidence.

Who Actually Changed?

Before interpreting a difference between two averages, ask whether the same units were observed in both conditions and whether their changes are consistent.

Cover B–D and describe panel A. Then reveal the paired observations and the distributions of differences below them. Is the change widespread, mixed, or concentrated in a subset? The averages alone cannot answer this.

Try it on your data. If your design uses the same participant or item in both conditions, show that pairing and plot the within-unit difference. If it does not, do not invent pairings to imitate the example. For repeated model outputs, first state how responses belong to items and how item-level summaries are constructed.

A visible cluster suggests a question to investigate, not an established subgroup. Write a cautious description of the pattern and identify what additional evidence would support a subgroup explanation.

Revise Your Own Figure

Examples to Explore: Journal Figure Guides

• The Nature Research Figure Guide illustrates axis labels, colour accessibility, and crowded annotations. Find one relevant issue in your own figure and correct it. Its size and font specifications apply to journal artwork; for slides, test readability at the projected size and viewing distance.
• Streit and Gehlenborg (2014), Bar Charts and Box Plots, compares what each representation makes easier to judge. Use it to justify your chart choice. Full text may require institutional access.
• Wong (2011), The Overview Figure, shows how to organize procedures and conditions in one diagram. Apply the principle of consistent visual meaning to your task diagram: what stays the same, what changes, and what is measured?

Short Learning Activities

These are suggested practice activities, not additional graded submissions.

Bring one revised result figure and caption. Explain what changed, what became easier to assess, and what remains unresolved.

Separate the Finding from Its Interpretation

A result establishes a pattern under a particular procedure. A cognitive interpretation requires an argument connecting that pattern to the question. Make the steps visible:

1. Observation: what changed, by how much, and under which conditions?
2. Interpretation: why is this pattern relevant to the hypothesis?
3. Alternative: what else could have produced it?
4. Next test: what comparison would distinguish those explanations?

Learn from Unexpected and Inconclusive Results

If a prediction is not supported, first check whether the comparison was implemented correctly and whether the task leaves room to observe the predicted change. Inspect possible floor or ceiling effects and the precision of the estimate.

A small observed difference with substantial uncertainty does not establish equivalence. Report the estimate and its limitations. Decide whether the next step should improve measurement, collect more independent evidence, or revise the hypothesis.

Unexpected results can motivate useful analyses. Label these as exploratory, distinguish them from the original predictions, and explain how they will be checked on new data or in a subsequent experiment. If prompts or stimuli were selected using the current data, describe those data as development evidence; reserve fresh evaluation material for the revised claim.

Use Failure Cases to Diagnose a Pattern

Select examples according to a stated rule, such as all errors within a stimulus category or a random sample of failures. Report the category’s frequency and denominator. One memorable response may illustrate a possible explanation, but cannot establish how typical it is.

For language models, treat generated explanations as outputs to analyze. A convincing explanation is not direct proof of the process that produced the answer.

End this part of the report with a bounded statement: “These results suggest X under Y conditions; Z remains unresolved.” Make the next test follow from Z.

A Short Worked Example

A team asks whether an AI system classifies a visual relation when familiar object appearances change. It prepares 40 matched item pairs: one familiar rendering and one unfamiliar rendering of each relation. It collects five model responses per rendering, using the same instructions and settings throughout. The analysis first averages responses within each item and condition, then compares the matched items. Each condition contains 200 responses drawn from 40 distinct items.

Suppose familiar renderings yield 80% accuracy and unfamiliar renderings yield 60%. A paired plot shows whether the 20-point decrease is widespread or concentrated in a few items. Any interval should be calculated from the actual observations with a method appropriate to this paired design.

• Overextended: “The model cannot represent relations.”
• Defensible: “Accuracy decreased for unfamiliar renderings in this stimulus set.”

What still needs explaining?

Prioritize the uncertainty that most threatens the interpretation. If objects are difficult to identify, adding several more models will not resolve that measurement problem. A useful progress-report ending would be:

We observed a decrease with unfamiliar renderings. We have checked the scoring and response completeness. Object identifiability remains a possible explanation, so our next step is to test it with the same stimuli before expanding the evaluation.

Apply it to your project. Write four sentences: the question, the observed result, the strongest unresolved alternative, and the next comparison. Then identify the one figure that would let an audience assess your result. If a sentence depends on an experiment you have not run, mark it as a prediction or plan.

Organize the Presentation Around What You Learned

There is no mandatory slide template, but a strong presentation will often follow question → current experiment → evidence → interpretation → next decision:

1. Question and prediction: the issue the audience should keep in mind.
2. Current experiment: the procedure and the changes needed to interpret it.
3. Preliminary evidence: the main result, its uncertainty, and relevant checks.
4. Interpretation and limits: what the evidence supports and what remains open.
5. Next decision: the experiment or analysis that follows from the open issue.

Spend most of the talk on the actual study, results, and interpretation. For each result slide, ask: what should the audience learn here, and which visible evidence supports that message? Keep additional conditions and technical detail available for questions. A brief demo can clarify the task, but prepare screenshots or a recording so the explanation does not depend on a live system working.

Ask for Feedback on a Decision

End with 2–3 questions that the audience can answer from the evidence you showed. For example:

• Does the current control distinguish cue use from a change in task difficulty?
• Which of these two follow-up comparisons best separates the remaining explanations?
• Is our proposed analysis unit appropriate for the generalization we want to make?

Practical Reminders

• Separate completed, in-progress, and planned work.
• Show one concrete trial from start to score.
• Check the measurement before interpreting the score.
• Say what one point and one error bar represent.
• Show distributions or pairings, not only averages.
• Keep observations distinct from interpretations.
• Let the strongest open alternative choose the next test.
• Rehearse within the assigned time.

Before You Present: Checklist

Study and Data

• Are the research question, current procedure, and completed conditions clear?
• Do the counts distinguish participants or items from trials and repeated responses?
• Have we disclosed protocol changes, exclusions, missingness, and invalid outputs?
• Does at least one example connect a raw response to the reported measurement?

Results and Inference

• Is the main comparison visible, with an appropriate baseline where relevant?
• Does each figure identify its units, counts, summary, and uncertainty?
• Are planned analyses and exploratory observations distinguished?
• Do the conclusions match the scope and precision of the evidence?

Next Steps and Presentation

• Does the strongest remaining alternative have a concrete follow-up test?
• Is planned work visibly separate from completed evidence?
• Have we rehearsed the talk, and are the figures readable?

What You Should Have After Round 3

By the end of Round 3, your project should have:

1. A documented version of the experiment that produced the reported data.
2. A traceable path from those data to the main figures.
3. A concise statement of the preliminary finding and its limits.
4. A prioritized next experiment or analysis, with a reason for doing it.
5. A list of remaining work that fits the time and resources available.

These should begin to form the Methods and Results sections of your final paper. Start drafting them while the details are fresh, keep observations distinct from their interpretation, and keep a record of your original predictions and later revisions.

For the next checkpoint, name the uncertainty to resolve, the output that will answer it, and the person responsible.

References

• Stanford CS341 — Project Information and Checkpoints: how a progress report brings together setup, ongoing results, and unresolved challenges.
• Weissgerber, Milic, Winham, and Garovic (2015). Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm. PLOS Biology, 13(4), e1002128: whether your plot conceals a distribution or paired pattern that matters.
• Cumming, Fidler, and Vaux (2007). Error Bars in Experimental Biology. Journal of Cell Biology, 177(1), 7–11: distinguishing variation from estimation uncertainty.
• Pineau et al. (2021). Improving Reproducibility in Machine Learning Research. Journal of Machine Learning Research, 22(164), 1–20: the documentation someone else would need to reproduce your main figure.
• Nature Research Figure Guide — Preparing Figures: axis labels, colour accessibility, and annotation.
• Streit and Gehlenborg (2014). Bar Charts and Box Plots. Nature Methods, 11, 117: choosing a chart for the comparison you need to show.
• Wong (2011). The Overview Figure. Nature Methods, 8, 365: organizing a task diagram with consistent visual meaning.
• Previous handouts: Round 1: How to Give a Literature Review Presentation and Round 2: From Literature to a Research Plan.