How Research Works
Types of research contribution, the research cycle, how to think like a scientist, choosing problems with good taste, and why most experiments fail.
What counts as a contribution
Every paper must answer one question: what does the reader know or have after reading this that they did not before? In AI, contributions come in a few types, and knowing which one you are making shapes everything else:
| Type | What it offers | What reviewers will demand |
|---|---|---|
| Method | a new model, training procedure, algorithm | strong, fairly tuned baselines; ablations; several datasets |
| Empirical analysis | a careful study of why or when something works or fails | breadth, controlled comparisons, statistics, clear takeaways |
| Dataset / benchmark | a new way to measure something important | quality, documentation (datasheet), baselines, licence and ethics |
| Theory | proofs, bounds, formal understanding | correctness, relevance, ideally an experiment linking to practice |
| Application | a well-done solution to a real problem | a lesson that generalises beyond this one problem, honest evaluation |
| Reproduction / negative result | checks a claimed result, or shows an idea doesn't work | thoroughness; a fair attempt; why it matters |
For a first paper, empirical analyses, reproductions and well-scoped methods on one problem are the most realistic: they need less compute than new large models and reward care, which you control.
Watch it: The research cycle
A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.
Interactive: this part runs in your browser.
The research cycle
Research is a loop, not a line: read → question → hypothesis → experiment → analyse → write → share → read again. Two habits make the loop fast:
- Small, fast experiments first. Test the riskiest assumption on a tiny dataset in an afternoon before spending weeks on the full pipeline.
- Write early. Draft the introduction and the key figure you hope to show before running the big experiments. If you cannot say what result would be interesting, you are not ready to run the experiment.
Expect most ideas to fail. Experienced researchers estimate that only a minority of initial ideas survive contact with experiments. A failed experiment that you understand is progress: it rules something out and often suggests a better question.
Try it: what kind of contribution is it?
Read six one-sentence paper summaries and classify each contribution. Then see what reviewers would ask for.
Interactive: this part runs in your browser.
Thinking like a scientist
- Falsifiable claims. "Our method helps" is not testable; "fine-tuning with method X improves macro-F1 on low-resource languages by at least 2 points over full fine-tuning at the same compute" is.
- Seek disconfirmation. Design the experiment that would show you are wrong. Look hardest for bugs when results are surprisingly good.
- Alternative explanations. Before claiming "A causes the gain", ask what else changed: more compute, more tuning, a different random seed, data leakage.
- Strength of evidence. One run on one dataset is an anecdote. Multiple seeds, datasets and ablations make a finding.
- Honesty about uncertainty. Report what you did not test and where the method fails. Reviewers trust papers that state their limits.
Choosing problems: taste
Good research problems sit where three things overlap: important (someone cares about the answer), tractable (you can make progress with your skills, data and compute in months, not years), and you have an edge (a dataset, domain knowledge, a skill, or a perspective others lack). Richard Hamming's question is worth asking regularly: what are the important problems in your field, and why aren't you working on them?
For a student, an "edge" is often domain knowledge: if you know healthcare work, a language other than English, education in a particular country, or disability support, you can see problems and evaluate solutions in ways most ML researchers cannot. That is a genuine advantage, and supervisors notice it.
Quiz
1. Which question should every paper answer?
2. Which claim is falsifiable?
3. Your new method beats the baseline by 15 points on the first try. First reaction?
4. Which is a realistic first-paper contribution for a student with limited compute?
5. Why draft the introduction before running the big experiments?
📚 Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- A. Karpathy, "A Survival Guide to a PhD" ↗ choosing problems, writing, and PhD life
- J. Eisner, Advice for Research Students ↗ a curated collection of advice on reading, writing and research