Using SSH keys efficiently - intro to the easy to use SSH config

Using SSH keys efficiently - intro to the easy to use SSH config
Photo by Roman Synkevych / Unsplash

SSH (Secure Shell) is a widely used protocol for secure remote access to systems and secure data communication. It allows users to connect securely to remote servers and services, like GitHub, using cryptographic keys.

When managing multiple remote servers or repositories, each requiring a unique SSH key, configuring SSH to handle these keys efficiently becomes crucial. Let's explore how to set up and manage multiple SSH keys on a standard Linux system.

1. Generating SSH Keys

This step can be skipped if you already have a public key on your remote server and a private key at hand.

Start by generating SSH keys for each remote server you need to access:

Remote Server 1

Generate an SSH key for the first remote server:

ssh-keygen -t rsa -b 4096 -C "[email protected]" -f ~/.ssh/server1.key

Remote Server 2 (Optional)

If you have another remote server, create a separate key:

ssh-keygen -t rsa -b 4096 -C "[email protected]" -f ~/.ssh/server2.key

Adjust the commands with the appropriate email address and key filenames for each server.

If you're inexperienced, stuck or don't know how to generate and install SSH keys, consider this guide to connect to your remote server using SSH keys.

2. SSH Config File

The SSH config file (~/.ssh/config) is a crucial component for configuring and managing SSH connections on a Linux system. It allows users to define custom configurations for different hosts, specifying various parameters for connection, authentication, and more. Here's a detailed breakdown of the SSH config file:


The SSH config file resides in the .ssh directory within a user's home directory:


If you do not have one, create one using touch, vim, nano, or any other tool:

touch ~/.ssh/config

File Structure

The SSH config file follows a straightforward structure composed of configuration blocks for different hosts or groups of hosts. Each block contains directives that define how SSH should handle connections to specific hosts.


  1. Host: Defines the host or group of hosts for which the configuration applies. You can use wildcards (*) to match multiple hosts or patterns.
Host example_host
  User username
  IdentityFile ~/.ssh/private_key
  • Host: Specifies the name of the host or a pattern to match multiple hosts. This is the name you'll use to SSH into another machine.
  • HostName: Specifies the domain name or IP address of the host.
  • User: Specifies the username to use when connecting to the host.
  • IdentityFile: Specifies the path to the private key used for authentication. It's best practice to keep your private keys in your ~/.ssh/ directory.

Various other options can be specified within the SSH config file, such as Port, ForwardAgent, ForwardX11, Compression, ServerAliveInterval, and more, to customize SSH behavior according to specific needs.

Multiple Configurations

The SSH config file can contain configurations for multiple hosts or groups of hosts. Each host's configuration is separated by its unique Host directive.


For security reasons, ensure that the SSH config file and any referenced private keys have restricted permissions:

chmod 600 ~/.ssh/config
chmod 600 ~/.ssh/private_key

Trying to SSH with a private key that has overly exposed permissions will cause an error.


  • Simplified Connections: By defining configurations in the SSH config file, users can establish connections to hosts using custom settings without having to remember specific connection details.
  • Enhanced Security: Centralizing SSH configurations allows for consistent and secure authentication methods across multiple hosts.
We'll also show an alternative way of easily remoting, but also explain why SSH configs are still the best way down below!

Testing Configurations

After modifying the SSH config file, it's advisable to test connections to ensure that the configurations are set up correctly:

ssh example_host

Example of a complete SSH config

I use Oracle Cloud's free tier quite rigirously. For this I have two remote servers. I also have a home server called mini.. This is how my SSH config looks like:

Host gameserver
  HostName 176.your.ip.206
  User ubuntu
  Port 22
  IdentityFile ~/.ssh/gameserver.key

Host web
  HostName 122.other.ip.54
  User ubuntu
  Port 22
  IdentityFile ~/.ssh/web.key

  IdentityFile ~/.ssh/github.key

I simply run ssh gameserver,ssh web to remote to my web servers.

Easily connecting without the SSH config file

You can also avoid using the SSH config and still conveniently remote into your server. However, there are some caviats, which will be explained at the end of this section.

My home server's IP address, which I call mini, is defined in my /etc/hosts file at the very bottom:

# Loopback entries; do not change.
# For historical reasons, localhost precedes localhost.localdomain:   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
# See hosts(5) for proper format and other examples:
# foo
# bar mini
  1. I have previously generated a key-pair and added the public key to my home server's ~/.ssh/authorized_keys file.
  2. On my desktop computer, I've placed the private key, mini.key in my ~/.ssh/ directory.

When connecting, ssh tries as much as keys as it can in your ~/.ssh directory until one matches. This is why it seemingly works. We can prove this by seeing what SSH does behind the scenes verbosely, such as ssh -v mini.

However, most SSH servers will only allow a limited amount of attempts, and for this reason it's best to use the SSH config.

The SSH config is a neat place to keep your configurations central. Embrace it!


Effectively managing SSH keys for remote servers on Linux streamlines access and enhances security. By leveraging the SSH config file, associating keys with specific servers becomes a seamless process, allowing easy and secure connections to multiple servers without the hassle of handling various credentials.

With these configurations in place, I hope this article helped you show how you can efficiently manage and access your remote servers using their respective SSH keys.