I’m having trouble staying on top of updates for my self hosted applications and infrastructure. Not everything has auto updates baked in and some things you may not want to auto update. How do y’all handle this? How do you keep track of vulnerabilities? Are there e.g. feeds for specific applications I can subscribe to via RSS or email?
You must log in or register to comment.
There are a couple of things to cover here:
-
Keep your software/containers up to date. You can subscribe to the GitHub repo and configure it to get notified for new releases and security alerts. Complementary, you can use RSS feeds, newteleases.io and/or WUD (What’s Up Docker) and add labels to your docker compose files. Personally, I check the notification once a week and change the version for all minor tools I’m using. If there is a major release (or new Immich version) I read the changelog and update instructions (if it’s the case).
-
For container security scans, you can use Trivy, but the problem is that you don’t have a centralized overview of your scan results. For this you can use DefectDojo. Depending on the case/threat model, vulnerability management for self-hosted things might be overkill, but highly recommended of you want to learn more about this. It worth mentioning Trufflehog as secrets scanner and sops as a solution to encrypt sensitive data so you can push it to git/SCM.
-
i subscribe to the release page of the repo in my rss reader. simple and effective.
That is a fantastic idea. Wtf how is this not commonplace? Or am I just way behind 😅
upgrade all things by default
removed by mod
what’s the alternative? Write a PR yourself?
removed by mod
That may work for a handful of projects. It’d be my full time job if I did it for everything I run. Also, I might simply suggest maintainers to adopt dependabot or an alternative before I spend time with manual changes. These things should be automated.
removed by mod
dependabot is a tool for repos, not to apply local changes
removed by mod
This is also a great way to just break everything you’ve set up.
that’s a lot of FUD, topgrade just upgrades using all package managers you have, it doesn’t do the upgrades itself bypassing the manager that installed it, or package authors.
The issue is more that trying to upgrade everything at the same time is a recipe for disaster and a troubleshooting nightmare. Once you have a few interdependent services/VMs/containers/environments/hosts running, what you want to do is upgrade them separately, one at a time, then restart that service and anything that connects to it and make sure everything still works, then move on to updating the next thing.
If you do this shotgun approach for the sake of expediency, what happens is something halfway through the stack of upgrades breaks connectivity with something else, and then you have to go digging through the logs trying to figure out which piece needs a rollback.
Even more fun if two things in the same environment have conflicting dependencies, and one of them upgrades and installs its new dependency version and breaks whatever manual fix you did to get them to play nice together before, and good luck remembering what you did to fix it in that one environment six months ago.
It’s not FUD, it’s experience.
I’ve been doing that for years. Rollbacks are very rare, to the point that it doesn’t make much of a difference whether I do them all at once or not, other than spending more time to do it.
If I wasn’t using containers for everything, sure. Otherwise it’s a bit of an excessive concern.
Does badly count as a way?
I kinda keep an eye on that https://selfh.st/ post that does a weekly roundup of stuff to know when I need to do patching.
No doubt there is a container I could run that would do it for me. I just can’t remember the name of it.
That’s the neat part. I don’t!
I have automatic updates on everything, but if I actually spent time managing updates and vulnerabilities I’d have no time to do anything else in my life.
How do I do it? Everything’s installed and updated via pacman/the AUR, including python packages and nextcloud apps. The only thing I don’t install via that way is Firefox addons.
The only thing I don’t install via that way is Firefox addons.
Any specific reason why? Yesterday I installed LibreWolf and saw at the same time a few addons in the AUR.
Do you know what’s the difference from an AUR addon or the official Firefox addon repo?
I guess It would be for security reasons because you never know if someone has tempered with the addon.
Simply because I haven’t bothered searching for the extensions I have in the AUR. And some extensions aren’t in there (namely 7tv, augmented steam, blacklist autoclose, defund wikipedia, kagi, peertube companion, tampermonkey and unload tabs).
XD okay ! Maybe I put to much fought into it 😅
I just update every month or two, or whenever I remember. I use Docker/podman, and I set the version to whatever minor release I’m using, and manually bump after checking the release notes to look for manual upgrade steps.
It usually takes 5 min and that’s with doing one at a time.
For my docker containers I use what’s up docker which not only alerts me when there is an update but also give a link to the changes, so I can have a look what’s happening !
For my system itself… Just doing
sudo pacman -Syu. Though that’s not great, cause some updates can potentially break my EndeavourOS system… I keep sometimes an eye on the forum when I see some critical changes like the kernel itself or nvidia updates though.95% of things I just don’t expose to the net; so I don’t worry about them.
Most of what I do expose doesn’t really have access to any sensitive info; at most an attacker could delete some replaceable media. Big whoop.
The only thing I expose that has the potential for massive damage is OpenVPN, and there’s enough of a community and money invested in that protocol/project that I trust issues will be found and fixed promptly.
Overall I have very little available to attack, and a pretty low public presence. I don’t really host any services for public use, so there’s very little reason to even find my domain/ip, let alone attack it.
You should try wireguard if you haven’t before, like a breath of fresh air
removed by mod
Unless you have actual tooling (i.e. RedHat erratas + some service on top of that), just don’t even try.
Stop downloading random shit from dockerhub and github. Pick a distro that has whatever you need packaged, install from the repositories and turn on automatic updates. If you need stuff outside of repos, use first party packages and turn on auto updates. If there aren’t any decent packages, just don’t do it. There is a reason people pay RedHat a shitton of money, and that’s because they deal with much of this bullshit for you.
At home, I simply won’t install anything unless I can enable automatic updates. Nixos solves much of it. Two times a year I need to bump the distro version, bump the nextcloud release, and deal with depreciations, and that’s it.
I also highly recommend turning on automatic periodic reboots, so you actually get new kernels running…
Most critical infrastructure like my mail i subscribe to the release and blog rss feed. My OSs send me Update notifications via Mail (apticron), those i handle manual. Everything else auto updates daily.
You still need to check if the software you use is still maintained and receives security updates. This is mostly done by choosing popular and community drive options, since those are less likely to get abandoned.
upgrades:
- distribution packages: unattended-upgrades
- third party software: subscribe to the releases RSS feed (in tt-rss or rss2email), read release notes, bump version number in my ansible playbook, run playbook, done.
vulnerabilities:
- VPN only, nothing exposed
- Host runs openSUSE MicroOS which updates itself daily
- Watchtower updates the containers daily and if something blows up so be it, except for Nextcloud as everyone says it’s brittle as hell.
I have stuff in new releases.io and also GitHub release RSS feeds in nextcloud, I then sit down once a week and see what needs an update. Reboot when required.
