Log into SSH on your dedicated or cloud server.
You can edit the sshd_config file located in /etc/ssh/.
# vi /etc/ssh/sshd_config
By default sshd listens on all IP addresses. If you want to limit it to one address do a search in vim for the following line.
#ListenAddress 0.0.0.0
Uncomment the line (Remove the # at the begining) and put in the IP Address that you wish to have sshd listen to.
Save the file and then restart SSH:
# service sshd restart