When we are in buffer mode, adjust the buffering low/high thresholds based on
the total configured latency. If we don't and there is a huge queue or element
with a big latency downstream we might drain the complete queue immediately and
start buffering again.