• Hackworth@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    5
    arrow-down
    12
    ·
    26 days ago

    We follow the classic experimental paradigm reported in Power et al. (2022) for analyzing “grokking”, a poorly understood phenomenon in which validation accuracy dramatically improves long after the train loss saturates. Unlike the previous templates, this one is more amenable to open-ended empirical analysis (e.g. what conditions grokking occurs) rather than just trying to improve performance metrics

    • catloaf
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      3
      ·
      26 days ago

      Oh okay so they’re just redefining words that are already well-defined so they can make fancy claims.

      • Hackworth@lemmy.worldOP
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        6
        ·
        26 days ago

        Well-defined for casual use is very different than well-defined for scholarly research. It’s standard practice to take colloquial vocab and more narrowly define it for use within a scientific discipline. Sometimes different disciplines will narrowly define the same word two different ways, which makes interdisciplinary communication pretty funny.