In terms of server connection, it is slower since the server will try to access your port for a while before realñizing it's not open, though once the connection is established, it'll al be ok.
Also, some big servers tend to allow a limited amount of low-id users to connect to them, so you might find you are unable to connect sometimes to some server. Though this is quite unusual.