catculation@lemmy.zip to Technology@lemmy.worldEnglish · 2 months agoReddit changes have blocked all search engines except Google amid AI 'misuse' [U]9to5google.comexternal-linkmessage-square39fedilinkarrow-up1358arrow-down18
arrow-up1350arrow-down1external-linkReddit changes have blocked all search engines except Google amid AI 'misuse' [U]9to5google.comcatculation@lemmy.zip to Technology@lemmy.worldEnglish · 2 months agomessage-square39fedilink
minus-squareIphtashuFitz@lemmy.worldlinkfedilinkEnglisharrow-up3·2 months agoAsking for a friend… What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
minus-squareWilzax@lemmy.worldlinkfedilinkEnglisharrow-up7·2 months agoProbably a LOT of proxy IPs to act as different “Users” so you can overcome the rate limit that I expect they would be using to enforce such a deal
Asking for a friend…
What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
Probably a LOT of proxy IPs to act as different “Users” so you can overcome the rate limit that I expect they would be using to enforce such a deal