#lemmy/#kbin has a problem that #mastodon hasn’t even attempted to solve; groups and what happens when they get popular.
#Communities, #groups, #magazines, whatever they are called are implemented as #Actors in #ActivityPub. They are basically just *very* popular users who boost a *lot*.
You can’t just distribute them across instances the way normal actors do. Whichever server hosts @technology@lemmy.ml or @technology@beehaw.org is going to get HOSED on the regular.
Lots of traffic, lots of posts, lots of comments, … a torrent of incoming data. That’s going to need more storage, more bandwidth, more CPU power, higher running costs. The original instance hosting the community bears a higher load than the instances that duplicate it.
Ideally, there would be a way to more evenly distribute this load across instances according to their resources, but from my (currently limited) knowledge, I don’t think Lemmy/ActivityPub is really geared for that kind of distributed computing, and currently I don’t believe that there’s a way to move subs between instances to offload them (although I believe some people may be working on that).
Perhaps the Lemmy back-end could use a distributed architecture for serving requests and storage, such that anyone could run a backend server to donate resources without necessarily hosting an instance.
For example, I currently have access to a fairly powerful spare server. I’m reluctant to host a Lemmy instance on it as I can’t guarantee its availability in the long term (so any communities/user accounts would be lost when it goes down), but while it’s available I’d happily donate CPU/storage/bandwidth to a Lemmy cloud, if such a thing existed.
There are pros and cons to this approach, but it might be worth considering as Lemmy grows in popularity.
I don’t think it’s a problem. If you weren’t using activity pub and just something like reddit then if you were reddit (the sysadmin) you’d also deal with having to scale if your community gets really popular
Stuff that gets linked to also has the same problem
https://www.jwz.org/blog/2022/11/mastodon-stampede/
(Btw I don’t like jwz but he mentions it here)
Funny how you say it’s not a problem, then go on to describe the problem that needs to be dealt with. Dealing with scaling is a problem, and it’s a problem that costs money.
Posts like this: https://lemm.ee/post/58472 suggest it is a problem. The rise in traffic seen by Lemmy in the last few days is absolutely tiny compared to a site like reddit, and already instances are struggling to cope. The recent growth in user registrations represents only about 0.007% of reddit’s active user base. (~60K new Lemmy users vs 861,000,000 active monthly reddit users). A site like reddit costs millions to run.
There are 190+ Lemmy instances last time I checked, yet almost all the brunt of this load has been borne by a handful of servers, which see an inordinate amount of traffic while 100+ other servers sit around idle. Why should a handful of “lucky” servers have to pay all the hosting costs? What if a volunteer-run instance explodes to reddit-like levels of popularity? It will simply fold, unless the volunteer has serious money to throw at the problem.
While this is true, 5 days ago lemmy.ml, the biggest instance, was on a 67 EUR server which is very small. https://news.ycombinator.com/item?id=36270094
This is a scaling problem (having more users means you need more mods) but I disagree with how they handled it and it isn’t a money related thing. My thoughts on this are in an older post when this was first announced https://partizle.com/comment/64178
My initial idea is to use the something awful model of paying a one time fee to register an acount. The problem is that people would just sign up on another instance that doesn’t charge a fee but still add load to the lucky instance. Another approach could be to participate in communities on one of those lucky servers then you need to pay a one time fee to that server (comments would need to be removed by a bot if they’re not made by an approved user). I’m not saying that’s perfect, but it’s an idea. Adsense is another idea.
Again, you say it’s not a money problem… then go on to describe a money problem! 🤦♂️
Also, did you read the link included in the post I linked to? ( https://beehaw.org/post/520044?scrollToComments=true )
That’s a money problem and a time problem. (And time problems are money problems.)
But more generally, high traffic sites need lots of money and resources to run. That’s just a fact.
We can solve this in many ways as Lemmy grows (and I think we will), but to just pretend there aren’t any problems to be solved is naive, IMO.
If Lemmy grows to any significant percentage of reddit traffic, the Lemmy of tomorrow will (necessarily) look quite different to the Lemmy of today.