How to Recover a Broken Linux System

While I can’t provide a comprehensive guide how to save your broken Linux system (really, this is a guide to myself about how I should save myself), I’ll list a few things which I’ve learned in the last 15 years.

I’ve probably broken my machine 20-30 times. Usually from copying commands that I didn’t really understand the effects of.

This last time, I messed around with my Nvidia drivers, as one does, while trying to do something with Cuda. And in my ignorance, I rebooted my machine twice to make sure that I hadn’t broken anything. And it was fine.

And then I rebooted one more time after running a few more commands, and all of a sudden the screen is blank.

So here’s what I do when I can’t boot up a Linux machine anymore.

Tips and Tricks

1. Hold escape to enter into the grub menu

Generally, you’ll be able to boot into recovery mode here.

1. If you hold escape too long, you’ll enter the Grub prompt. If so, type “normal”, then hit “esc” (possibly more than once), to get back to the menu

I used to think I had to time my escapes really fast (because it will take you into the grub prompt if you hold it down), but you can just do this one trick!

1. Log into a Recovery Mode kernel

Generally these use the nouveau drivers, so you’ll be able to login.

1. Start the X server

I don’t think you can start this as root, so:

$ su ${USER}

$ startx

1. Document the things that you do and have done

That way you can try not to get more stuck, help your future self, and help others :)

1. Your package history is in /var/log/apt/history.log

If you removed an Nvidia driver via apt uninstall nvidia-, for instance, you’ll be able to see what version was uninstalled.

1. Install nvidia drivers via sudo apt install nvidia-driver-XXX

Where x is whatever version you previously had.