Otherwise it can happen that we regularly switch back and forth between clocks under certain circumstances for no good reason. Also remove redundant comparison when comparing the steps removed between two clocks. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5520>