Is your ssh taking ages to show the login prompt. It could be a DNS issue.
One quick way to fix this is to disable DNS for ssh. To do this follow
$ sudo vi /etc/ssh/ssh_config
$ sudo service ssh restart
Note that its probably better to fix the DNS issue but in some cases you may not have access to do this.