There is no reason x264enc should enforce a maximum allocation size. The maximum is normally set by buffer pool which cannot grow, but we don't offer a buffer pool. This would lead to stall when used with element that don't implement allocation query. Related to: https://bugzilla.gnome.org/show_bug.cgi?id=738302