Hello, fellow lemmings!
I have three updates regarding lemm.ee:
Image uploads
As some of you have already noticed, image uploads are currently not working on lemm.ee at all. The short explanation is that I have been trying to figure out the best way to enable image uploads without incurring big additional costs in terms of storage, bandwidth, and moderation.
I have come to the conclusion that initially, the most sustainable path will be to enable image uploads only for small images (<100kb) - this should be enough to allow folks to upload avatars for themselves and their communities, but it for sure won’t allow uploading many bigger images or gifs for your posts.
For bigger images, I would ask everybody to use an external image hosting service for now, as was usually done on Reddit before they added image uploads.
I am in the process of creating a patch to the image upload code that will allow all Lemmy instance admins to configure a smaller limit on the uploaded image sizes. Once that patch is finished and accepted, I will be deploying it on lemm.ee - at that point, you will be able to finally upload avatars. Sorry for the wait, hopefully it won’t be much longer!
“No pornography” rule
There is a new rule landing in your sidebar - “No pornography”. I was initially against adding any too-restrictive rules, but after many discussions with other instance admins, I have realized that there is a big amount of additional risk to a community when allowing pornography.
As of writing this post, there are no pornography-related communities on lemm.ee, so I’m hopeful that this rule won’t be too controversial. If you were planning to create such a community on lemm.ee, I’m very sorry to ruin your plans and I hope you can find another instance that is more suitable for your community.
Growth of lemm.ee
The rate of new users joining lemm.ee has certainly been much faster than I expected. For now, the growth has been sustainable, and we can still keep growing at this same rate for a while longer without incurring additional costs.
However, if this growth continues for an extended period of time, then additional costs will become unavoidable. At that point, we will have the option of either closing sign-ups to new users, or upgrading our infrastructure to allow continued growth. The latter can only be achieved through some kind of community funding, so I will be asking the community for input if that time ever comes.
I want to be clear about one thing: if we end up closing new sign-ups, then this will not close your communities to new members. As always, Lemmy users from any other instance will be able to subscribe to your communities on lemm.ee, they will be able to participate as normal and even become moderators. The only thing that will change if we close sign-ups is that new users will not be able to choose lemm.ee as their home instance.
That’s all for now, let me know in the comments if you have any feedback or thoughts about any of this!
Indeed. I’m just sampling some from r/pics, even the largest/highest-res photos I saw clock in at ~1MB or just over (jpg, webp). I’m going to dive into Lemmy’s source and see what they’re doing… I imagine the 40mb is an upload limit on the pre-compressed filesize (still enormous). It would be ideal if they shipped with sliders for “compression aggression” or let you pass flags into ImageMagick or whatever it’s using. Even 1MB/image would get expensive fast unless you could prune old images or move into a cheaper cold storage product.
I’ve definitely seen some >10MB files in places like /r/EarthPorn and various wallpaper threads. That’d definitely get impactful fast. Cloudflare will help, especially if you can set nice long TTLs for caching. I’m glad S3 type storage is something that works with Lemmy, I was actually wondering this myself. You could set that to Infrequent or “Cold” access polices after a day or so to throw it into a cheaper bucket for long term storage being that Cloudflare will take the brunt of the hits.
Side note: It warms my DevOps heart to hear this setup. Definitely sounds scalable and easily duplicated or redeployed if need be. Major props and I do thank you for this setup!
Question: I haven’t looked into it, but why does the upload server have to be stateful? Perhaps it could be a lambda or similar?
Lemmy uses pict-rs to process uploaded images, which requires a database to index all the uploaded media. Unfortunately, pict-rs currently only supports a local on-disk database for this purpose. I’ve seen some talk that in the future it could also support postgresql for it’s database (which Lemmy uses anyway) - that could simplify things a bit for sure.