Disclaimer: I wrote this article and made this website.
There was some talk of this issue in the recent fediverse inefficiencies thread. I’m hopeful that in the future we’ll have a decentralized solution for file hosting but for now I deeply believe that users should pay for their own file hosting.
Have you considered providing something like this: https://jortage.com/ and maybe contribute to their efforts to develop a specific API for that? Source code is here: https://github.com/jortage
Jortage is a really interesting approach. It definitely helps reduce the impact of the file hosting problem but it doesn’t fully address the underlying cost issue. The cost of storing files grows every month indefinitely while donations typically don’t.
I would like to see a file hosting pool come to lemmy though. So I will look into it. :)
Pict-rs that is used by Lemmy to store images already supports S3 type storage, so in theory it should work with Jortage, but I don’t think anybody has tested that yet. The people behind Feddit.org might have experimented with it as they expressed interest a while back.
I think the major advantage is the deduplication - when an image goes viral across Mastodon (or Lemmy) it’s currently stored hundreds or thousands of times, each with its own cost. Do you dedupe (for either your customers’ benefit or your own)?
Are the images duplicated when shared? My understanding is that only a link to the file is replicated across servers and duplication comes from users manually uploading the same file to another server.
My website does not do any deduplication at this time.
Yes, for example go to https://infosec.exchange/explore
I see the top post as https://infosec.exchange/@nocontexttrek@mastodon.social/113433063621462027 and the image is https://media.infosec.exchange/infosec.exchange/cache/media_attachments/files/113/433/063/582/671/258/original/71da3801e4e4f08c.png
The link is to the original on https://files.mastodon.social/media_attachments/files/113/433/062/676/773/993/original/f828afef5cc7ed1c.png but when you click image the javascript loads a modal with the local cached version (same image as the thumbnail that infosec.exchange loads.
There’s lots of different codebases across the fediverse so perhaps some hotlink, but local copies is the default.
The Lemmy server config indicates that is an optional setting to improve user privacy so requests don’t ever hit the original server from the client. Those cached files are only temporary and will be deleted after some time. So it’s not really full blown duplication.
The default setting is to only generate the thumbnails and store those locally (indefinitely?) but even that can be turned off. I checked and it appears that lemmy.world has the thumbnail generation disabled so all images from other instances just link to the original on that instance.
Ok, so Lemmy doesn’t cause the same amount of duplication, but I’d still argue that dedupe is valuable: it saves on hosting costs (your costs, in this case) and users will get a small advantage in having slightly higher cache hits.
For sure, I’ll add it to the list. :)