One of PyFed’s selling points was that it was easier to work with than Lemmy. It’s going to be amusing when that takes a 180 turn and people start complaining.
Python is great for prototyping and iterating on small projects or as glue for modules written in C and C++. What it isn’t great at is linearly scaling on a single node. When the day that throwing more powerful hardware at the problem stops being an option, Kubernetes is going to walk through that door and fuck any semblance of simplicity up.
I think Lemmy has some in-memory data structures that limit the backend to a single node, too. Also postgres is great, but Lemmy really fucked up their database performance somehow.
But yeah large python codebases turn into spaghetti really quickly.
but Lemmy really fucked up their database performance somehow.
Can confirm. I spent like 4 hours one day configuring auto vacuum and other shit on my Lemmy DB because it was bloating to hell and eating up resources. Runs much smoother now but it was a massive PITA to get there.
How to see if autovacuum is already enabled (these are commands you’d run while in sql):
SHOW autovacuum;
View current settings:
SELECT name, setting FROM pg_settings WHERE name LIKE'%autovacuum%';
Monitor which tables need attention:
SELECT schemaname, relname, n_dead_tup, n_live_tup
FROM pg_stat_user_tables
WHERE n_dead_tup >1000ORDERBY n_dead_tup DESC;
I wish I had documented it at the time because info about this for Lemmy specifically is pretty lacking. I was in kind of a dire situation though because my disk space had filled up quickly and my system was struggling. A lot of what I did was basic PostgreSQL maintenance stuff, so you should be able to find some general guides for that.
EDIT: I had Claude help me corral some optimization stuff into a PDF. This is along the lines of what I did to get my db back under control. Make sure to always backup first! https://u.orca.casa/1771250081
large python codebases turn into spaghetti really quickly
I don’t think the language is the problem here. Seeing as Python isn’t somehow severely limited in its expressiveness or organization. Static typing isn’t a cure against spaghetti.
However, code in that particular file doesn’t inspire any faith in the authors’ organizing skill.
People are already complaining (see: this very thread:-), but also note that people are complaining about Lemmy too - e.g. slrpnk.net about to switch to PieFed, due to frustrations with long-standing bugs.
People always complain, the important thing is to move forward with something that is going to work. Personally I think Lemmy will not, though I would be happy to be proven wrong, and I am pinning all hopes on PieFed - worst case there is that they both succeed, which would be fantastic 😍.
Yes I hear what you are saying about resources and complexity (learning Rust is somewhat complicated as well though…), but right now the subscriber counts across the entire Threadiverse are dropping not rising, so that will be a wonderful problem to have to solve one day, if we ever get to that point.
One of PyFed’s selling points was that it was easier to work with than Lemmy. It’s going to be amusing when that takes a 180 turn and people start complaining.
Python is great for prototyping and iterating on small projects or as glue for modules written in C and C++. What it isn’t great at is linearly scaling on a single node. When the day that throwing more powerful hardware at the problem stops being an option, Kubernetes is going to walk through that door and fuck any semblance of simplicity up.
I would agree with that sentiment, but seems like peoples’ actual experiences are a bit different: https://jeena.net/lemmy-switch-to-piefed
Possibly a testament of how software architecture can be more important than any lower level technical decisions.
I think Lemmy has some in-memory data structures that limit the backend to a single node, too. Also postgres is great, but Lemmy really fucked up their database performance somehow.
But yeah large python codebases turn into spaghetti really quickly.
I can confirm this statement.
Can confirm. I spent like 4 hours one day configuring auto vacuum and other shit on my Lemmy DB because it was bloating to hell and eating up resources. Runs much smoother now but it was a massive PITA to get there.
Old post, but can you share what you did? I’m fighting the same problem.
I can’t remember exactly what I did but enabling autovacuum was one of the big ones. I’ll try to provide some info below:
Walks through VACUUM and autovacuum - https://oneuptime.com/blog/post/2026-01-25-use-vacuum-analyze-postgresql/view
How to see if autovacuum is already enabled (these are commands you’d run while in sql):
SHOW autovacuum;View current settings:
SELECT name, setting FROM pg_settings WHERE name LIKE '%autovacuum%';Monitor which tables need attention:
SELECT schemaname, relname, n_dead_tup, n_live_tup FROM pg_stat_user_tables WHERE n_dead_tup > 1000 ORDER BY n_dead_tup DESC;I wish I had documented it at the time because info about this for Lemmy specifically is pretty lacking. I was in kind of a dire situation though because my disk space had filled up quickly and my system was struggling. A lot of what I did was basic PostgreSQL maintenance stuff, so you should be able to find some general guides for that.
EDIT: I had Claude help me corral some optimization stuff into a PDF. This is along the lines of what I did to get my db back under control. Make sure to always backup first! https://u.orca.casa/1771250081
I don’t think the language is the problem here. Seeing as Python isn’t somehow severely limited in its expressiveness or organization. Static typing isn’t a cure against spaghetti.
However, code in that particular file doesn’t inspire any faith in the authors’ organizing skill.
People are already complaining (see: this very thread:-), but also note that people are complaining about Lemmy too - e.g. slrpnk.net about to switch to PieFed, due to frustrations with long-standing bugs.
People always complain, the important thing is to move forward with something that is going to work. Personally I think Lemmy will not, though I would be happy to be proven wrong, and I am pinning all hopes on PieFed - worst case there is that they both succeed, which would be fantastic 😍.
Yes I hear what you are saying about resources and complexity (learning Rust is somewhat complicated as well though…), but right now the subscriber counts across the entire Threadiverse are dropping not rising, so that will be a wonderful problem to have to solve one day, if we ever get to that point.