Is your ssh taking ages to show the login prompt. It could be a DNS issue.
One quick way to fix this is to disable DNS for ssh. To do this follow
these steps

$ sudo vi /etc/ssh/ssh_config

Add:

UseDNS no

$ sudo service ssh restart

Note that its probably better to fix the DNS issue but in some cases you may not have access to do this.