A Telegram user who advertises their services on Twitter will create an AI-generated pornographic image of anyone in the world for as little as $10 if users send them pictures of that person. Like many other Telegram communities and users producing nonconsensual AI-generated sexual images, this user creates fake nude images of celebrities, including images of minors in swimsuits, but is particularly notable because it plainly and openly shows one of the most severe harms of generative AI tools: easily creating nonconsensual pornography of ordinary people.

  • flower3@feddit.de
    link
    fedilink
    English
    arrow-up
    60
    arrow-down
    7
    ·
    3 months ago

    I doubt tbh that this is the most severe harm of generative AI tools lol

    • Sanctus@lemmy.world
      cake
      link
      fedilink
      English
      arrow-up
      24
      arrow-down
      1
      ·
      3 months ago

      Pretty sure we will see fake political candidates that actually garner votes soon here.

      • CeeBee@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        arrow-down
        19
        ·
        3 months ago

        FR is not generative AI, and people need to stop crying about FR being the boogieman. The harm that FR can potentially cause has been covered and surpassed by other forms of monitoring, primarily smartphone and online tracking.

        • BleatingZombie@lemmy.world
          link
          fedilink
          English
          arrow-up
          22
          arrow-down
          2
          ·
          3 months ago

          I wholeheartedly disagree on it being surpassed

          If someone doesn’t have a phone and doesn’t go online then they can still be tracked by facial recognition. Someone who has never agreed to any Terms and Conditions can still be tracked by facial recognition

          I don’t think there’s anything as dubious as facial recognition due to its ability to track almost anyone regardless of involvement with technology

          • neatchee@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            arrow-down
            7
            ·
            3 months ago

            You don’t need to be online or use a digital device to be tracked by your metadata. Your credit card purchases, phone calls, vehicle license plate, and more can all be correlated.

            Additionally, saying “just don’t use a phone” is no different than saying “just wear a mask outside your house”. Both are impractical, if not functionally impossible, in modern society

            I’m not arguing which is “worse”, only speaking to the reality we live in

            • BleatingZombie@lemmy.world
              link
              fedilink
              English
              arrow-up
              9
              arrow-down
              3
              ·
              edit-2
              3 months ago

              I am arguing which is worse. There are people in Palestine who don’t have the internet, don’t have a phone, and don’t have a credit card. How are they being tracked without facial recognition?

              I also didn’t say don’t use a phone. I don’t know where you got that

              • neatchee@lemmy.world
                link
                fedilink
                English
                arrow-up
                3
                arrow-down
                2
                ·
                edit-2
                3 months ago

                I know what you’re arguing and why you’re arguing it and I’m not arguing against you.

                I’m simply adding what I consider to be important context

                And again, the things I listed specifically are far from the only ways to track people. Shit, we can identify people using only the interference their bodies create in a wifi signal, or their gait. There are a million ways to piece together enough details to fingerprint someone. Facial recognition doesn’t have a monopoly on that bit of horror

                FR is the buzzword boogieman of choice, and the one you are most aware of because people who make money from your clicks and views have shoved it in front of your face. But go ahead and tell me about what the “real threat” is 👍👍👍

                • BleatingZombie@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  4
                  ·
                  3 months ago

                  I didn’t say “real threat” either. I’m not sure where you’re getting these things I’m not saying

                  I think facial recognition isn’t as much of a “buzzword” as much as it is just the most prevalent issue that affects the most people. Yes there are other ways to track people, but none that allow you to easily track everybody regardless of their involvement with modern technology other than facial recognition

                  (Just to be clear I’m not downvoting you)

                  • neatchee@lemmy.world
                    link
                    fedilink
                    English
                    arrow-up
                    4
                    ·
                    edit-2
                    3 months ago

                    That’s why I put “real threat” in quotes ; I was paraphrasing what I consider to be the excessive focus on FR

                    I’m a security professional. FR is not the easiest way to track everybody/anybody. It’s just the most visible and easily grok’d by the general public because it’s been in movies and TV forever

                    To whit, FR itself isn’t what makes it “easy”, but rather the massive corpus of freely available data for training combined with the willingness of various entities to share resources (e.g. Sharing surveillance video with law enforcement).

                    What’s “easiest” entirely depends on the context, and usually it’s not FR. If I’m trying to identify the source of a particular set of communications, FR is mostly useless (unless I get lucky and identify, like, the mailbox they’re using or something silly like that). I’m much more interested in voice identification, fingerprinting, geolocation, etc in that scenario

                    Again, FR is just…known. And visible. And observable in its use for nefarious purposes by shitty governments and such.

                    It’s the stuff you don’t see on the news or in the movies that you should really be worried about

                    (and I’m not downvoting you either; that’s for when things don’t contribute, or deserve to be less visible because of disinformation; not for when you disagree with someone)