Here's my take on it:
In the camera, you have the the optical center and the film gate. These are locked together, and won't move relative to each other.
In the scanner, there may be a scanner gate, and then there is also the image center. These two will also be locked to each other.
I don't know scanners all that well, but I'm going to assume for the sake of this answer that the scanner gate, if it exists, is larger than the camera gate.
So, if we can see the camera gate moving on the final scan, then we know that the optical center was also moving. Whether this movement came from the film moving in the camera or the scanner doesn't make a difference.
In an ideal situation, the lens -> digital image transition would be completely locked off. There would be no movement here at all. This happens in digital cameras, but in film there is an intermediate step.
Think of it as having an object with two transformations on. If either or both of those transformations are moving around, the final result will be moving around (unless, but some insane coincidence, they are both moving in opposite directions - but the chance of this happening with something as random as film jitter is negligible)
As Julian mentioned, you can certainly possibly figure out whether the jitter that you're seeing is camera or scanner-based by looking at whether there is motion blur or not - if it was in the camera, there will be motion blur, whereas if it was in the scanner, there won't be (and therefore you can possibly ask them to re-scan it)