In today’s episode, Yud tries to predict the future of computer science.

    • dr2chase@ohai.social
      link
      fedilink
      arrow-up
      4
      ·
      8 months ago

      @datarama @corbin The Go compiler requires reproducible builds based on a small set of well-defined inputs, if the LLM cannot give the same answer for the same question each time it is asked, then it is not compatible with use in the Go compiler. This includes optimizations – the bits should be identical. #golang

      • corbin@awful.systemsOP
        link
        fedilink
        English
        arrow-up
        6
        ·
        8 months ago

        Indeed, this is also the case for anything packaged with #Nix; we have over 99% reproducibility and are not planning on giving that up. Also, Nix forbids network access during compilation; there will be no clandestine queries to OpenAI.

        • self@awful.systemsM
          link
          fedilink
          English
          arrow-up
          4
          ·
          8 months ago

          it very much fucks with me that there’s a nixpkgs working group dedicated to making NixOS an attractive platform for running LLM code. I’m guessing it comes down to two things: nix actually is very good at packaging the hundreds of specifically pinned dependencies your average garbage LLM project needs to run, and operating in the functional programming space makes these asshole grifters feel smarter (see also all the folks who contributed nothing but crypto miners to nixpkgs during that bubble)

          • corbin@awful.systemsOP
            link
            fedilink
            English
            arrow-up
            8
            ·
            8 months ago

            Let me know @corbin@defcon.social if you actually get LLMs to produce useful code locally. I’ve done maybe four or five experiments and they’ve all been grand disappointments. This is probably because I’m not asking questions easily answered by Stack Overflow or existing GitHub projects; LLMs can really only model the trite, not the novel.

        • dr2chase@ohai.social
          link
          fedilink
          arrow-up
          3
          ·
          8 months ago

          @corbin I’m curious how they deal with the Go builder (not compiler specifically) and all its signature verification hoo-hah. There’s ways around that (and those are consistent with “trust nobody”) but it’s not usual case, and not tested nearly as hard as the default path. You can use your own builder, too, that’s also an option (and now I wonder, could we get the Go builder to export a “build plan” for other-tool consumption?)

      • Evinceo@awful.systems
        cake
        link
        fedilink
        English
        arrow-up
        5
        ·
        8 months ago

        This reads like a PCJ comment, bravo. I’ll do one for rust:

        If an LLM cannot insult the user for having the tremerity to try and compile code, it’s not compatible for use with the Rust compiler.

        • dr2chase@ohai.social
          link
          fedilink
          arrow-up
          2
          ·
          8 months ago

          @Evinceo PCJ? And (lack of) reproducibility really would be a problem for Go, the LLM would need to expose all its random seeds and not have any run-to-run varying algorithms within it. This is not a joke or snark, the bits have to match on recompilation.

          • Evinceo@awful.systems
            cake
            link
            fedilink
            English
            arrow-up
            4
            ·
            8 months ago

            PCJ -> Programming Circlejerk.

            I was wasn’t expecting a serious treatment of this very silly idea, my mistake. I submit that it would cause enough difficult to diagnose bugs while just messing with it that you would never get into ‘but are the builds reproducible’ territory.

            • dr2chase@ohai.social
              link
              fedilink
              arrow-up
              2
              ·
              8 months ago

              @Evinceo there’s code generation, and there’s optimization decisions. Optimization problems often have the property that their correctness is easily checked, but choosing the best one is hard. Register allocation is the easy-to-understand example – if modeled as graph coloring, an incorrectly colored graph is trivial to detect.

              So, sadly, not silly enough.