You're going to have outages in production. They're inevitable. The question is how to best minimize outages, both their frequency and duration. Common wi...

cross-posted from: https://lemmy.world/post/21023181

Sharing some lessons I learned from 10 years/millions of users in production. I’ll be in the comments if anyone has any questions!

I hope this series will be useful to the self-hosted and small web crowds—tips for tools to pick and the basics of server management.

  • betweenthesixesEnglish
    1·
    5 hours ago

    I understand this take and the frustration it is born out of, however, extremes are rarely the right answer. If this were a series of posts, I would gladly read along and expect that the author would learn some valuable lessons from this approach.

    Are certain things often needlessly complex? Sure. Does that come at a cost? Sure.

    However as a business (or deployment) grows, it becomes subject to more requirements, regulations, risks, etc. that often demand the complexities that the industry and author have witnessed.

    All business decisions are made through a lens. What is important today? Based on what I know today, what does the future hold? What constraints do I have? The key is a logical, but balanced decision.

  • just_another_person@lemmy.worldEnglish
    154·
    11 days ago

    Anytime you see anyone post something like “THIS Is How You Do The Thing”, it should automatically be ignored. This article is no exception.

    The author is making a big deal about a team of 40 people and “millions of customers”…k.

    Not sure if anyone is supposed to be impressed by that, but the titular argument here isn’t a position for a small team and product, it’s making the case that “ALL THINGS EVERYWHERE ARE TOO COMPLICATED IN PRODUCTION ENVIRONMENTS”, which is just an ignorant position to take.

    If this had come with some sort of metrics, tools, or workflow to back up the claim, then it would be worth reading. Nothing like that here. This is just an inexperienced person’s boasting rant with zero elaboration about what actually works for them, why, or suggestions for others facing some sort of similar situation.

    There is no content here, just time wasted.

    • something_random_tho@lemmy.worldOPEnglish
      51·
      11 days ago

      Hi friend, this was just meant to be an introduction, as I get started blogging and sharing back some knowledge and lessons I learned along the way. I’ve never written a blog before (or much of anything!), and I’m sorry you didn’t find value in this.

      I wasn’t intending to boast, but I can see how it came across. I just meant to say, “companies are trying to tell you that you need ‘XYZ’ to scale,” and at least at the size of business I ran, you didn’t need any fancy tech at all – we could have made do with a dead-simple setup: a single server running Go and SQLite. It’s something I wish I had known when I started.

      I’ll take your feedback to heart and try to produce larger, more substantial posts to follow. Thanks for commenting.

      • just_another_person@lemmy.worldEnglish
        51·
        11 days ago

        Well, apologies for being bluntly critical. I can offer a few constructive tips to help with writing about technical topics:

        • Try starting with a simple topic flow: topic -> synopsis/purpose of why you’re writing -> background -> observations -> point<>counterpoint loop to elaborate on purpose -> closing
        • Stay away from asserting your point in the title unless you’re showing definitive proof of something
        • Find ways to speak to who may find your writing useful towards the beginning. “If you have a similar problem like X…”, “People who use X may find…”, or “Anyone having similar X experience may find…”. Something like that.
        • 1984@lemmy.todayEnglish
          31·
          10 days ago

          I can offer some constructive tips to you as well: dont speak for all of us.

          I liked the article. I didn’t interpret it as “everyone must do what I say”. It was simply a viewpoint from a person on the internet. That is what a blog is.

          The complexity of tech stacks have increased enormously in the last 10 years, and it’s only sane to see what tradeoffs we are making to be able to scale easily. Perhaps it’s not worth the trouble for 90% of us. If we follow best practice from cloud providers without thinking for ourselves, we will not learn to think for ourselves either.

          So let’s do that. Let’s think, wrote blogs, discuss, and allow for discussions. Don’t shut people down.

          • just_another_person@lemmy.worldEnglish
            12·
            10 days ago

            Wow, nobody should ever say anything then unless they’re on the right side of everyone else’s opinion or else they might be speaking for everyone. Great tip.

              • just_another_person@lemmy.worldEnglish
                12·
                10 days ago

                I criticized the WRITING of what turned out to be the OP for not having any content in their posted piece aside from a complaint about something. Never said he was wrong to feel how we feels about whatever.

      • TobberoneEnglish
        2·
        11 days ago

        I’ve been running that 40 dev/lots of costumers environment you mention, back in the days, and im looking forward to see how you do it:) i might just learn something useful. Some of the remedies were about as bad as the problems they were intended to solve.

        Anyways, it’s a long time ago by now. I’m curious to hear your take. Especially your take on hardening/security.

    • something_random_tho@lemmy.worldOPEnglish
      51·
      11 days ago

      Is it too late for, “I use nix btw”? I use it at home and for development.

      I planned to focus this blog series on ol’ faithful (Debian), but I could definitely see writing articles on how to use Nix and OpenBSD if people find it helpful.

    • fl42v@lemmy.mlEnglish
      31·
      11 days ago

      Well, I guess nixos itself isn’t too overcomplicated, but fun begins when you start layering abstractions over abstractions 😁

  • Shimitar@feddit.itEnglish
    2·
    9 days ago

    Not a bad take per-se, but a bit condescending. While I agree and like the KISS paradigm, remember that no solution works at a lower complexity level of the problem itself.

    So, define your problem clearly, find the simplest possible solution. Don’t overcomplicate, I agree, but don’t be fooled by false hopes.

    Is it really simpler? Yes. Will it scale if I need it to? Maybe not, but will I really need for it to scale?

    And so on.

  • rookbrood@lemmy.worldEnglish
    2·
    10 days ago

    I agree with the sentiment of your article and I’m looking forward to seeing your follow ups. I’m always trying to keep things as simple as the problem I solve allows for.