Yep, it would be nice to sync up broken video segments as you describe. I'm not sure there is a standard way to store time stamps for all the various video formats out there.
I'm no mpg or quicktime expert, though. If there is such a format that is fairly widespread I could take a look at it. It would be cool to pull images every, say 30 seconds, and display them on the map much like PhotoFusion.
Among other cool things you could do...