I just spun up Lemmy on my Kubernetes cluster with nginx-unprivileged and ingress-nginx. All is well so far! I’m thinking about posting the Kustomization manifests and continuing to maintain and publish OCI’s per version release of Lemmy.

    • gabe565@lemmy.cook.ggEnglish
      2·
      1 year ago

      Yep I’m still working on a helm chart. Currently, each service is deployed with the bjw-s app-template helm chart, but I’d like to combine it all into a single chart.

      The hardest part was getting ingress-nginx to pass ActivityPub requests to the backend, but we settled on a hack that seems to work well. We had to add the following configuration snippet to the frontend’s ingress annotations:

      nginx.ingress.kubernetes.io/configuration-snippet: |
  if ($http_accept = "application/activity+json") {
    set $proxy_upstream_name "lemmy-lemmy-8536";
  }
  if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
    set $proxy_upstream_name "lemmy-lemmy-8536";
  }
  if ($request_method = POST) {
    set $proxy_upstream_name "lemmy-lemmy-8536";
  }

      The value of the variable is $NAMESPACE-$SERVICE-$PORT.
      I tested this pretty thoroughly and haven’t been able to break it so far, but please let me know if anybody has a better solution!

      • seang96@exploding-heads.comEnglish
        1·
        1 year ago

        Doing this made / path with Lemmy ui break, but posts and comments were actually updating according to logs. i set it up with a nginx container behind my ingress now and it appears everything works besides my comments and posts not being federated even though I can curl the links for troubleshooting federation without issue. Got any ideas?

        • gabe565@lemmy.cook.ggEnglish
          2·
          1 year ago

          Hmm I’m not sure! That code snippet should only affect routing conditionally. When you added the configuration snippet, did your ingress logs show the requests to / going to the frontend or backend?

          An nginx container behind ingress seems cleaner, I just didn’t want to add another point that I could possibly break lol

          • seang96@exploding-heads.comEnglish
            2·
            1 year ago

            I just found out my posts are finally going through without any changes using the nginx proxy container to nginx ingress method! If you do have a way to do it all directly through nginx I’d love to see how it’s all done, maybe I was missing something outside of the snippet you posted.

            Since it’s currently working I’ll look into spinning a test instance up when I get a chance and play with that ingress annotation.

            • gabe565@lemmy.cook.ggEnglish
              2·
              1 year ago

              Awesome! A separate nginx container is fine, so if it’s working I’d probably leave it. I’ll look through and see if there’s anything I missed in my comment though for brevity.

      • anthr76@lemmy.kutara.ioOPEnglish
        1·
        1 year ago

        Firstly, awesome to hear you’re using bjw-s app-template helm chart. He’s my good friend and former coworker :)

        I’m also doing what @seang96@exploding-heads.com is doing.

        While I don’t consider this completed yet I have posted how I’m doing things so far here

        • gabe565@lemmy.cook.ggEnglish
          3·
          1 year ago

          That’s awesome! I love his Helm chart. It’s the most impressive Helm library I’ve ever seen. I maintain a bunch of charts and I exclusively use his library chart :)

          I just mentioned in a response to @seang96@exploding-heads.com, but I feel like deploying a separate nginx is probably cleaner, I just didn’t want another SPOF that I could break at some point in the future.