When closing the connection, unref the currently used sockets. This should close them when not in use. We need to do this because else we cannot reconnect anymore after a close, the connect function requires that the sockets are NULL.