Setting Up SSH for Remote Access to Your Linux Machine

SSH is one of the first Linux features that makes a machine feel like more than a machine sitting in one place.

The moment it works, the computer changes shape in your mind.

It becomes something you can reach.
Something you can maintain.
Something you can fix without standing in front of it.

But the first SSH setup also comes with a very specific beginner anxiety:

"What if I change one setting and lock myself out?"

That fear is reasonable.
And honestly, it is one of the best reasons to set SSH up in a careful order instead of a rushed one.

The steps and terminology below are checked against the official OpenSSH manual; menu names and behavior can vary by operating-system release and hardware.

SSH is not just “remote terminal”, it is the beginning of trustable remote administration

OpenSSH’s official documentation describes ssh as the client program and sshd as the daemon that accepts remote logins.

That sounds simple.
And it is.

But I think the more useful beginner understanding is this:

SSH is the point where your Linux machine becomes remotely manageable in a serious way.

Once it is working well, you can:

• open a remote shell;
• run commands;
• copy files;
• work through slow infrastructure tasks more comfortably;
• recover or inspect systems without needing a monitor and keyboard attached every time.

That is why SSH often becomes one of the first real “I get it now” moments in Linux.

The right first goal is not hardening everything at once

This matters.

A lot of first-time guides jump too quickly from:

"install SSH"

to:

"disable all passwords immediately, change the port, turn every knob, and never look back."

I understand the instinct.
I do not think it is the best beginner path.

The right sequence is calmer:

1. install the server;
2. start and enable it;
3. confirm you can reach it;
4. test key-based login;
5. validate configuration changes;
6. then tighten the risky options.

That order is what keeps security advice from becoming self-sabotage.

On Ubuntu or Debian, the first package is usually openssh-server

Ubuntu’s official server documentation says to install the OpenSSH server with:

sudo apt install openssh-server

That is one of the cleanest first steps in the whole process.

After that, you are not “done,” but you are at least dealing with a real SSH service instead of a missing package.

Ubuntu’s documentation also notes that the main server configuration lives in:

/etc/ssh/sshd_config

and that configuration snippets can be placed in:

/etc/ssh/sshd_config.d/

That second point matters more than many beginners realize.
Using snippets often makes changes easier to track and safer to undo later.

On Fedora, the package and service are very similar, but the service name usually matters

Fedora’s official server documentation says to install the server with:

sudo dnf install openssh-server

and then start it with:

sudo systemctl start sshd.service

Fedora also documents enabling the service so it starts automatically at boot.

This is a small but important beginner difference:

• Ubuntu and Debian commonly use ssh.service
• Fedora commonly uses sshd.service

That difference is not conceptually deep, but it is exactly the kind of small naming mismatch that makes a beginner wonder whether they broke something.

Usually they did not.
They just used the wrong service name for the distro.

Start the service, then check it before touching config

This is one of the habits I wish more people learned early.

Before editing banners, authentication methods, or root-login rules, first confirm the daemon is actually running.

That usually means commands such as:

sudo systemctl status ssh

or on Fedora:

sudo systemctl status sshd

The exact unit name differs, but the idea does not:

make sure the service exists, starts, and looks healthy before layering complexity on top of it.

Network reachability matters just as much as package installation

This is the part beginners sometimes underestimate.

SSH can be installed correctly and still feel broken if:

• the machine’s IP address is not what you think it is;
• you are connecting to the wrong hostname;
• a firewall is blocking access;
• the machine is asleep or on a different network than expected.

So before changing authentication rules, I would make sure you know:

• the Linux machine’s IP address;
• the username you intend to log in with;
• that both devices can actually see each other on the network.

If the machine is on your local LAN, this is usually where the setup starts feeling concrete.
You are no longer configuring an abstract service.
You are trying to reach a real box.

Your first successful login should happen before you harden the server

I think this is one of the healthiest sequencing rules in SSH.

Get one normal login working first.
Then improve it.

That means if your current path is password-based authentication, I would let that prove the path itself works before I begin locking anything down.

A simple first test might look like:

ssh youruser@your-server-ip

If you reach the host key prompt and then the login prompt, that is progress.
You have proven basic network path, daemon availability, and account reachability.

That is an excellent checkpoint.

SSH keys are where the setup starts becoming properly durable

OpenSSH’s official ssh-keygen manual says that ssh-keygen generates, manages, and converts authentication keys, and that when no key type is specified it generates an Ed25519 key by default.

For a beginner, this is the key idea:

SSH keys let you move from “something I type every time” to “something cryptographically stronger and easier to trust.”

This is usually the command I would start with:

ssh-keygen -t ed25519 -C "yourname@yourdevice"

The OpenSSH documentation also makes clear that the private key and the public key are stored separately, and that a passphrase can protect the private key.

That passphrase matters.

If the key is the thing that proves your identity, the passphrase is the thing that protects the key when your own laptop is not in your hands.

The server cares about authorized_keys

The official sshd documentation says that the default public-key authorization files are:

• ~/.ssh/authorized_keys
• and historically also ~/.ssh/authorized_keys2

In practical modern use, ~/.ssh/authorized_keys is the file beginners should know.

This file lives in the remote user’s home directory and contains the public keys allowed to log in as that user.

This is the mental model I keep:

• your private key stays on the client machine you trust;
• your public key goes into the server user’s authorized_keys.

That is the whole relationship.

Once that clicks, public-key SSH stops feeling magical and starts feeling clean.

Do not disable password authentication until the key login has already worked

Fedora’s official SSH server documentation is very clear here: test the key-based login process before disabling password authentication, especially if you are connected remotely and do not have out-of-band access.

That is exactly the right advice.

Because the dangerous beginner move is this:

1. generate key;
2. assume everything is fine;
3. disable passwords;
4. discover the key setup never worked.

That sequence is how lockouts happen.

The better sequence is:

1. create the key pair;
2. place the public key in authorized_keys;
3. test a fresh login using the key;
4. only then consider disabling password authentication.

That is slower by a few minutes and safer by a lot.

PasswordAuthentication and PermitRootLogin are the two settings beginners hear about first

OpenSSH’s sshd_config documentation includes both directives, and Fedora’s server documentation specifically points administrators toward disabling password authentication once key-based login is confirmed.

These are the settings people usually harden first:

• PasswordAuthentication
• PermitRootLogin

I think the right beginner instinct is not to memorize every sshd_config option.
It is to respect the ones that can cut off your path back in.

In plain language:

• PasswordAuthentication no means the server should stop accepting normal password logins;
• restrictive PermitRootLogin behavior reduces the risk of direct root logins over SSH.

These are good goals.
They are just not step one.

Use a config snippet if your distro supports it

Ubuntu’s official server docs explicitly recommend modular configuration snippets in /etc/ssh/sshd_config.d/, and Fedora’s docs also describe using drop-in files such as /etc/ssh/sshd_config.d/01-local.conf.

This is one of those modern admin habits that makes life calmer later.

Instead of editing one giant main config file until it becomes mysterious, you can keep small local decisions in a clearly named snippet.

That makes it easier to answer questions like:

• what did I change;
• why did I change it;
• what would I remove if troubleshooting gets weird.

For a beginner, that clarity matters.

Validate the SSH configuration before restarting the service

Ubuntu’s documentation explicitly recommends checking the configuration after changing it and before restarting the server.

This is one of the best habits in the whole article.

If you edited SSH configuration, test it first.

A common pattern is:

sudo sshd -t

The point is not the specific command syntax alone.
The point is the habit:

do not restart a remote access service blindly after editing the thing that controls remote access.

That is the kind of mistake you only enjoy making once.

Firewall rules can silently decide whether SSH feels “broken”

Fedora’s firewalld quick docs note that the firewall blocks access other than SSH by default, and their examples include working with firewall-cmd services such as ssh.

That means firewall behavior may help you or hurt you depending on the distro and current rules.

On Fedora, SSH is often already allowed by default.
On other systems, you may need to allow it explicitly.

The beginner lesson is not to memorize every firewall command.
It is to remember that a healthy sshd process does not guarantee a reachable SSH session if the firewall disagrees.

So if SSH looks correct locally but dead remotely, firewall checks should rise quickly on your list.

The best first SSH hardening is boring and disciplined

This is my favorite kind of security advice because it usually survives contact with real life.

For a first Linux machine, I would treat these as the healthy defaults:

• use a normal user account, not direct root login;
• generate an Ed25519 key pair;
• protect the private key with a passphrase;
• get key-based login working fully;
• validate config changes before restarting;
• disable password authentication only after testing;
• keep one existing session open while testing a second session.

That last one is underrated.

If you already have one working SSH session, do not close it while testing changes in a second login window.
It is one of the simplest ways to avoid turning a recoverable mistake into a trip to the console.

Conclusion

Setting up SSH for remote access to a Linux machine is one of the most useful early infrastructure skills because it turns a local box into something you can manage calmly from elsewhere. But the smartest way to do it is sequentially, not aggressively: install OpenSSH, start and enable the correct service for your distro, verify reachability, test a normal login, set up keys, validate configuration changes, and only then disable weaker authentication paths such as passwords or direct root access.

If this is your first remote Linux workflow, trying Linux from a live USB provides a safe foundation, while running lightweight Alpine LXC containers shows where repeatable SSH administration becomes useful in a homelab.

SSH is not hard because the commands are mystical.
It is hard only when the order is careless.