What a test is
An automated test is a small program that runs part of your code and checks the result against what should happen. Run after every change, a set of tests answers one question quickly: does everything that used to work still work?
Tests do not prove that code is perfect. They prove specific things, every time, without relying on memory or patience. That is what makes changing code safe.
Kinds of tests
- Unit tests
- Check one function or rule on its own, such as a single grade boundary. Fast and precise.
- Property tests
- Generate many random cases and check a rule that must always hold, such as “the planner agrees with the grading engine”.
- Browser tests
- Open the real page in a browser controlled by a program, click and type like a visitor, and check what appears.
- Content checks
- Check data before it is used, such as every project having the fields the page needs.
- Regression tests
- Written after a bug: they fail on the old code and pass on the fixed code, so the bug cannot quietly return.
How to write a test that can fail
- Decide, in one sentence, what must be true before writing the test.
- Make the check specific: compare with an exact expected value, not just “something appeared”.
- Prove the test can fail: break the code or the data on purpose and confirm the test catches it. A test that never fails proves nothing.
- Write a failure message that says what was expected and what happened, so a failure points to its cause.
- Keep tests away from real outside services by using stand-ins, so a test never sends a real email.
- When behaviour changes on purpose, update the test and say why. When it changes by accident, fix the code.
In GPA Tracker
GPA Tracker is checked with 564 automated checks: 193 unit tests and 371 window-level checks written for the original interface and run against a stand-in for its toolkit, so that interface could be tested without opening windows. Mutation testing went one step further: 89 bugs were planted deliberately, and the tests caught every one. That is the strongest evidence that the tests fail when they should.
In this site
This site has fourteen test files with more than two hundred and seventy checks. Browser tests cover the keyboard, text contrast, layouts from 320 to 1920 pixels wide, the contact form, the dialogs, both themes, the security policy and the project and note pages. A content check validates the data file in about a second; it was itself tested with eight deliberate mistakes and caught all eight, each with a message naming the problem.
A GitHub workflow runs everything on every push, so a mistake shows up as a failed check on the change that caused it.
What tests cannot see
A test checks only what it was written to check. On this site the tests passed while a hidden spam-trap checkbox was visible, and while decorative chips covered the portrait on phones (see the debugging note, cases 5 and 8). Tests and looking at the result belong together.