By hand is the best way to preserve time stamps and speeds.
Usually it's best to just delete the spurious points altogether, so you may be able to grab several of them at a time with the select (box) point tool.
It may be possible to come up with an algorithm to "clean" tracks automatically. Looking for bad jumps of speed or elevation, and also just the points itself. It would basically be a smoothing method.
I have experimented somewhat with this, and it's not easy to get it to do what you want it to do. A human is much more effective.
'>
