I was talking about just plugging the headphones into the computer, nothing else. When I tried a different setup, that seemed to correct the desync problem.
But if your setup is just plugging straight into your soundcard then I don't know.
Looks like it's back to experimenting...