While this is true, the time difference should actually be not very large or it is not the same recording anymore. So in general this should still work in most cases. Only if there is e.g. a significant silence added at the beginning of one track the times would be off, of course.
Or this gets implemented per track somehow, which would of course complicate this thing even more
Recordings of different durations can be merged, as long as there is no evidence to suggest that differences in mixing or editing have caused the change in lengths.
Variations in the length of silence at either end of tracks is not a reason to keep recordings separate, since no changes have been made to the audio itself. Similarly, different volume fades at either end of multiple tracks are not reasons to maintain separate recordings - they are considered mastering differences unless they cause the structure of the song to change. The same is true for variations in playback speed between recordings.
This shouldn’t go for “extreme” variations, right? See the example:
The original recording Don’t Push appears on tracks which vary in length from 3:45 to 3:55 because they have been mastered at different speeds.
The fastest version is ~1.05 times faster than the slowest, similar to difference in mastering. A track with the same recording sped up 2 times with nothing else changed wouldn’t be perceived as the same recording, more like a remix (even if it isn’t).