03 and up have 4 wheel disc brakes.
03 and lower are the only ones with a manual transmission. Also, they had the manual odometers, as opposed to the 04 & 05's digital ones.
Color matched cladding would be nice.
04 & 05 also can come with stock MP3 cd players that support Auxiliary Input, whereas most 01-03 ST radios lack that capability (MP3 & aux capabilities, that is).
Also, no 04 & 05 have the rear-seat radio controls, but I wouldn't consider that a huge loss.
For 1st generation STs, you should shoot for a Job 2 motor. You could also check the fuel filter to see if it has 2 or 3 pipes in, and the intake tube to see if it has an extra hose coming off of it. My 02 has these things, and they're a bit of a pain, but nothing deal breaking, and only a concern if you want to do your own maintenance & install an aftermarket intake system.
If you get a 2nd generation ST, if you get an 07 make sure it has the power rear window and the Aux-in jack, as you can't easily add those later.