You’re not worrying for nothing. Losing wall power will shut down the drives and as usb-cradle is generally slower than “proper” drive bus it’s more likely that some write operation is going on when power is lost and that’ll potentially cause data corruption. Obviously not every power outage will cause issues, but I’d say it’s a higher risk with USB-drives than with drives on a SATA/m.2 bus.

But no matter what your setup is, raid is not a backup. All kinds of things can happen which cause loss of data and you should plan accordingly. If all you have is two drives on usb-cradles I might choose to use one of them as a offline backup disk and one for ‘live’ data so that it’s more likely that at least one of the drives is functional even after power issues or whatever, but that approach has it’s own problems too.

I haven’t tested lvm on arm-based systems, but if it doesn’t automatically locate them you could try pvscan/vgscan/lvscan.

If you’re talking about just moving the physical volumes (as in the actual hard drives) as is to another computer they’re automatically scanned and ready to go in majority of modern distributions. No need to export/import anything. This is obviously assuming your boot drive isn’t a part of volume group and you have healthy drives at your hands. You can test this with any live-distribution, just boot from USB into a new operating system and verify your physical volumes/volume groups from that.

If you want to move the volume group to a new set of disks simplest way would be to add physical drive(s) to volume group and then removing the old drive(s) from it after data has been copied. Search for pvmove and vgreduce. This obviously requires a working system, if your data drive has already failed it’s a whole another circus.

Even if your router acts as an DNS proxy it shouldn’t overload any pihole installation unless you have a crapload of devices doing millions of queries per hour. My pihole manages all my devices (20-30 individual things) without any problems and even if I hit some rate limit it’s going to be a change to default configuration, not a immovable object on your way. Based on quick glance over that reddit thread a new router might be a good option, but that’s another easy-ish task to accomplish. I use mikrotik device and I’m pretty happy with it but there’s a ton of good options.

For hiring someone to coach you I can see quite a few of potential issues. People who claim to know what they’re talking about but don’t really have the knowledge, straight up scammers obviously, mismatch in personal chemistry which will make learning unnecessarily difficult or even impossible, some people just aren’t good at teaching even if they do know their stuff and so on. By all means, use your money however you like, but I personally strongly advice against it unless you can get some courses on (preferably local) reputable vendor. You can look for online courses too, cisco has a ton of courses on networking, redhat has plenty of linux courses and other big players have their own training and even certificates if you want to go that far.

For pihole you don’t need support from router. It’s convenient if you can adjust dhcp-server settings so that pihole will automatically cover your whole network, but it’s not a requirement, you can just manually set each device to use pihole as DNS server. All you need is a static IP address outside your DHCP -pool. For spesific router configurations, you can ask those too, just include spesific model and possibly screenshots from your router interface.

That iMac of yours is more than enough to get you going. If you plan to run multiple things on it it might be good idea to look for hypervisors like proxmox or ovirt, but basic qemu+libvirt -setup on pretty much any linux-installation will work just fine too.

For the 3rd part, your concerns are mostly about networking and setting up pihole/other servers on your local network will gain you knowledge on how to manage that as well. Also, you can set up nextcloud/immich/whatever locally at first, get familiar with them and then allow access from the internet either via bitwarden or other tunneling or directly over public network. Latter has obviously way bigger threat models than using VPN and accessing stuff that way, but gladly the networking side of things is somewhat it’s own beast from the servers so you can build everything local only at first and then figure out what’s the best approach for you with remote access.

However right now I’m simply feeling overwhelmed and blocked.

I could explain to you in pretty decent detail how to build a setup which could cover pretty much every imaginable scenario for a home gamer, but that would also be suitable to serve a mid-sized company who’ll have multiple people on duty to manage the servers, storages, security, networking and other stuff. Also it’d cost roughly as much as a decent house. That’s close to the ‘big picture’ you’re looking for and equally overwhelming than your current situation. I’ve been earning my living with this stuff for quite a while now and there’s still a ton of things I’m at a very much beginner level. Maybe the difference now vs starting this is that I actually have some idea on things which I don’t know and thus I know when to learn more/ask from more experienced team members.

Just like eating an elephant, this field requires that you take it piece by piece. You’ll learn new things to build both your setup and your knowledge further, but if you try to eat it all at once it just doesn’t happen. First you need to decide a simple goal on what you want to get out of self hosting. DNS-based ad-blocking on your network is pretty neat and setting up pihole will get you started. Also with that you don’t need to allow any external connections to your network. Plus if something goes wrong you can easily just return to where you started from and try again. Setting your own router with DHCP, caching DNS and other stuff is pretty neat too and it’s also pretty simple to isolate from the rest of the network so you’ll have your ‘normal’ stuff still working while you learn for new things. Whatever it is, set up a relatively simple goal to work for. Then you can start to ask questions like ‘is raspberry pi 4 suitable for this’ or ‘what subnet I should use for my homelab’ or even ‘how to install debian on a old laptop to run pihole’.

Or if you really insist on going to the deep end, go to library and pick up TCP/IP Network Administration from O’reilly (altough that might be a bit outdated by now) or something similar and dig in. The o’reilly one has a bit over 700 pages to go trough. There’s equally in-depth books for linux administration, firewalls, network security and so on. Annas archive will most likely have some decent books too if you don’t care about legal issues and want to go trough brick-sized books as pdfs.

Some random thoughts about your points:

1. It’s a pretty damn big picture you’re looking at. Networking, backups, hypervisors, storage solutions, security and a lot of other topics are each big enough that you can make a career out of any of those alone. Obviously you don’t need to know everything about everything but as you learn more you’ll find more and more stuff to learn so I’d say there’s no practical way to learn ‘big picture’ just over a few hours of ‘lessons’. Also there’s a ton of variations on what one might consider as ‘self hosting’. Some will have setup comparable to decent sized company, others will have a single raspberry pi on top of their router.

2. Same goes here, it’s a pretty big field to go trough. The best setup for me is most likely very different from the best setup for you. Also with real world constraints (money, bandwidth, space available, electricity price…) the best setup is practically quaranteed to be some kind of compromise. Also, at least in my opinion, it makes sense to start with what you already have or can cheaply get, so that you’ll get something out of the system with as little investment as possible even if the first iteration might be a bit janky. Also your needs will likely change over time so the ‘optimal’ configuration for today might be wildly different from the configuration tomorrow.

3. This goes hand-in-hand with first point. You need to understand some basic networking, backup scenarios and proper threat mitigation against security threats, hardware failures, power outages and so on. Also there’s no ‘initial setup’ after which the system is complete as, again, your needs will change over time.

4. That’s why we’re here. Just describe your problems in a reasonably sized chunks. Don’t ask how to build a homelab but instead ask for something more spesific which doesn’t have a crapload of variables to figure out before getting to the actual problem.

For the money part, I’ve done stuff like this for companies (getting suitable hardware for their needs, setting it up, offering support…) as a freelancer and at least in here that’ll cost you 80-150€/h commercially. Even as a hobbyist I personally wouldn’t take that kind of contract as I heavily doubt that you’re willing to throw thousands of euros on the table (as properly going trough your list will take quite some time). However, if you can narrow things down and ask for something spesific I’ll happily reply to you around here for free if I happen to have time and/or knowledge about the matter.

So, figure out what you want from the system right now, what’s the first thing you want to build. It might be a hypervisor so you can keep experimenting with virtual machines, it might be a pihole for your network or something else, but you’ll need a pretty spesific goal. Then you can come back and ask more spesific questions and get deeper into the rabbit hole. Also, specially if you’re starting from scratch, there’s no such thing as a perfect setup. I’m working on a decent sized company with offices around the globe and even with those resources there’s still compromises with pretty much everything as cooling capacity, bandwidth, financial, man hours and other things aren’t infinite.

If it tries to start but doesn’t do anything it’s pretty much a lost cause then as the drive gets power but fails to initialize. In theory a simple broken solder joint somewhere might cause that and that might be fixable, but that requires at least somewhat decent soldering station and some experience. Or maybe you could get a donor board and swap out memory chips from the old one, but that’s even more tricky. Hopefully it’s not too expensive lesson.

I’ve had some luck with portable drives by removing the drive from enclosure and attaching it directly to sata-bus instead of USB. Also, as a general rule for anyone who might stumble on this, whenever attempting recovery at first create an image (I use ddrescue) and work with that. That way you’ll minimize risk of causing even more damage.

A while ago we “fixed” couple of hard drives with my brother. All of them had a single faulty diode, apparently it was a known failure point on those drives and brother found instructions online how to bypass that diode. Obviously that doesn’t really fix the drives, but a small piece of wire and some soldering was enough to get drives spinning again long enough that he could copy data over to new drives.

Outlook (new) classic new new final (7). Teams (personal). Multiple products with wildly different appreance and somewhat different functionality. And then the whole 365-environment naming, starting from the platform itself.

Not spesifically helpful with your cgnat-situation, but my jellyfin runs on a isolated network and it’s just directly exposed to the internet via named reverse proxy in order to share the library with family and friends. Should someone get access to that they can obviously use the VM for nefarious purposes, but it’s a known risk for me and the attacker would need to breach trough either my VLAN isolation or out of the virtual environment to my proxmox host if they wanted to access my actually valuable data.

Sure, there’s bots trying every imaginable password combination and such, but in my scenario even if they could breach either the jellyfin server or reverse proxy it’s not that big of a deal. Obviously I keep the setup updated and do my best to keep bad actors out. but as I mentioned, breach for that one server would not be the end of the world.

With cgnat there’s not much else to do than to run a VPN where server is somewhere publicly accessible and route traffic via that tunnel (obviously running a VPN-client on jellyfin-server or otherwise routing traffic to it via VPN). Any common VPN-server should do the trick.

Our opsec is pretty well managed, but I try to squeeze anything I’d need at home to work tasks. I get paid to learn the stuff at work and then I can just implement it on my own environment.

It’s pretty simple to set up. Generate CA, keep key and other private stuff stored securely, distribute public part of CA to whoever you want and sign all the things you wish with your very own CA. There’s loads of howtos and tools around to accomplish that. The tricky part is that manual work is needed to add that CA to every device you want to trust your certificates.

I didn’t know raspberry supports that. Searching for ‘atv remote’ just brings up androind apps, so maybe I misunderstood. Neat thing, but the hardware I have doesn’t support it and seems like usb-cec adapters are more expensive than usb-hid remotes.

I’d rather have a physical remote which acts as a keyboard so it’ll support waking the system up from suspend. Plus I prefer a dedicated device for that instead of a phone as I’m not a only user for the thing. There’s plenty of those around, only problem is to find one that works reliably and local stores don’t seem to have a lot of options so I might need to dig one up on ebay even if it’s a bit of a PITA to order from China to EU today with customs.

I installed Jellyfin on my server and threw kodi on a minipc I dug out of dumpster pile at work. Works pretty well, but my server needs more RAM and the minipc needs either a wireless keyboard or a USB-HID remote controller to finalize the setup. Also ran some wiring in the house and added two network sockets to a room where the whole kodi-tv-gamingpc-whatever-pile is going to live.

On the server RAM I found some on ebay, but if anyone is interested on 64G DDR4 ECC DIMMs I have a few. I thought they were supported on my server motherboard when I took them out from a old server at work but it supports only up to 32G ECC dimms.

Self hosting is not just one thing. You are system adminstrator, network engineer, security specialist, service architect and many other things, specially if you expose anything to anyone outside your very private network. And to get anything even running on that complex mess requires some knowledge on a lot of things. Making them run securely with proper backups requires even more knowledge on things.

Sure, you can just throw some docker images on your old desktop and be happy, even forward ports from the public internet to your things if you like. But that exposes your stuff to quite a lot of dangers and if you just click buttons without any understanding you’ll soon be a part of a botnet or lose your data or lose money if someone decides to mess around with your home automation or something else.

I get what you’re saying, not all of us are very polite and answers can be pretty harsh, but more often than not the generic idea behind those answers is not trying to be an asshole or gatekeep anything. It’s just that there’s a skillset you need to build things safely and if it’s clear from the start that someone looking for answers is way over their head it’s better for everyone to get them take a step back and learn instead of trying to create a meaningful answer since there’s too many variables or it’d just take immense effort to write down comprehensive guide on what to do, why and how for everything from the ground up.

I know for a fact that in my area there’s a bunch of surveillance cameras, home automation stuff and even some farm equipment directly open to the public network just because someone just plugged things in without any idea on the whole picture. Sometimes the correct answer is ‘stop shooting yourself on the foot and learn the basics first, then come back’.

Just for the sake of conversation, I recently did some crude math on this. I have few friends around who are well capable of running a backup server for me (hardware maintenance and stuff is always needed anyways) and at first it seemed like a good plan. Just get a 4TB SSD/NVME and throw that on a Raspberry Pi (or something small to keep electricity consumption low and setup silent), set up encryption, connect that to my network with wireguard or some other VPN and let it do it’s thing.

But I’d need to purchase everything as setting up a remote location with old hardware is just asking for trouble. The drive alone is 300€ (give or take) and the rest is easily another 100€. Currently my storagebox costs ~10€/month for 5TB. Even if I scored a fantastic black week offer and got everything for -50% discount that hardware with multiple single point of failures would cost nearly 2 years worth of cloud backups. And I’d still owe at least few beers to the friend for the trouble.

Your mileage may obviously vary, there’s a million different scenarios, but for me with my current setup it just makes sense to pick couple cloud providers and let them store my bits instead of getting more hardware to maintain and upgrade.

With backups two is one and one is none, so you are very much in a right track. Personally I have my stuff running on proxmox VMs with a proxmox backup server (VM as well) storing backups to Hetzner Storagebox. I’m planning to set up a another host in garage to have “local” backups too, as mine is detached as well the risk of both going up in flames in event of fire is pretty low. However, a voltage spike due to lightning on the grid or something else might blow up both hosts so that’s a threat model to be aware of. Also if your connection to garage is over copper it can cause other problems, fibre or wireless is highly recommended.

With backups it’s largely about the bandwidth available. I personally have enough so uploading to cloud is not an issue, but backing up a terabyte of data over 10Mbps connection might not work out at all.

For more info search for 3-2-1 strategy, that should give you plenty of ideas what you need to think about and what are industry best practises about making sure backups are in order.

Last time they tried our grandparents took 105 days to beat Russia with very little formal military and even less hardware. Sure, we had some help back then too, but today it’s on a whole different level. And we’re just a single small country up north, NATO as a whole is quite a bit bigger.

Russia currently has lost million soldiers and stockpiles of soviet relics are pretty much empty with a strong breeze away from total economy collapse. They don’t have power to conquer a potato field from a modern western country right now, much less against the whole global west.