Anubis rarely ever annoys me (I still prefer it over cloudflare’s Mitm) but this specific visit feels almost vindictive. It used half my CPU cores at full blast for three straight minutes.

It seems to be effective since I can’t really “bypass” it with extensions, so AI crawlers probably can’t either, but damn there really should be an option to complete a manual captcha or email the administrators to complain.

As I’m typing this the challenge is still running xD

  • Jankatarch@lemmy.worldEnglish
    0·
    15 days ago

    Think of it like tipping culture. Get mad at AI industry scrapers that make anubis necessary. Sam Altman is the reason you are enduring this. Take a lighter and…

    • boonhet@sopuli.xyzEnglish
      0·
      15 days ago

      I suspect this is more about all the individual searches than mass scraping. We’ve had google et al scraping the entire web for a long time and nobody gave a fuck. AI makes it easy to launch a ton of web searches without putting much thought into it.

  • 1984@lemmy.todayEnglish
    0·
    15 days ago

    I wonder if this even is effective. If a bot can solve anubis challenges, why can’t it solve all difficulties?

    • Scrubbles@poptalk.scrubbles.techEnglish
      0·
      15 days ago

      I’m a server asking with it.

      I went from thousands of requests per minute down to 5. Five. Database was pegged constantly at 100% serving requests down to ~50% now. Huge difference.

    • balsoft@lemmy.mlEnglish
      0·
      15 days ago

      The point is to make scraping too expensive to be profitable. Even the easy level 4 anubis challenge multiplied by a few million pages becomes very expensive.

      • 1984@lemmy.todayEnglish
        0·
        15 days ago

        But once the challenge is completed, they have the token and can scrape until it’s expired right? So if it’s just taking them a bit more time to get the auth token, what is it solving?

        Example: the token lasts for a week and it takes 30 seconds to get the token.

        • OhNoMoreLemmy@lemmy.mlEnglish
          0·
          15 days ago

          I would assume the token is only good for a few page access.

          E.g. 500 would mean that a person would see this extremely rarely, and still block a bot from crashing the site.

          • 1984@lemmy.todayEnglish
            0·
            15 days ago

            No it actually lasts for a long time. We have it on my Lemmy instance for https://old.lemmy.today/ and it’s generated once and lasts for many days, probably a week. I guess it’s configurable but if you generate it too often, your visitors will also be annoyed.

        • Axolotl@feddit.itEnglish
          0·
          15 days ago

          It will still slow them down and cobsume a ton of processing power- for data centers, it gets less profitable for them to scrap, it also completly stop some shitty bots

      • draco_aeneus@mander.xyzEnglish
        0·
        15 days ago

        The idea is to pick some task that is very difficult to calculate, but easy to verify. E.g., confirming a crossword is correct is much faster than filling one in.

        Protein folding is one such example, and it’s useful to boot. The results can be used in medical research, for example.

        Donating compute via Folding@Home was quite popular for a while for this exact reason.

        • bountygiver [any]@lemmy.mlEnglish
          0·
          15 days ago

          The problem being the challenge needs to be unique and issued by the server on demand, otherwise a computer can have a bunch of pre-processed challenge answers ready to fire out a huge burst of requests. (Or re-use the solutions on other websites)

        • ggtdbz@lemmy.dbzer0.comEnglish
          0·
          15 days ago

          Folding@Home being built into the PS3 might have been one of the last tech-utopian hurrahs I can think of experiencing. As opposed to everything tech now feeling too dystopian for one of any number of reasons

        • Zarobi@aussie.zoneEnglish
          0·
          15 days ago

          I don’t know how protein folding works, but if it’s easy to check it’s correct, then doesn’t that violate that mathematical principle where verifying an answer is as difficult as computing the answer in the first place? I forget the name of it.

          I guess what I’m trying to say is, volunteers at home are already self-selecting to be mostly honest and donating correct answers for a good cause. Their output trustworthiness is moderate-high. Spammers, bots, A.I. swarms, ddos networks, random users, etc… not so much. You can’t really trust their output at all, you have to verify it very carefully and treat it as malicious or just incorrect junk by default.

          • AwesomeLowlander@sh.itjust.worksEnglish
            0·
            15 days ago

            verifying an answer is as difficult as computing the answer in the first place

            What the distributed computing stuff does is check many many potential answers. You’re only interested in the right answers, and they’re easy to verify. If there’s a concern about junk answers, the answers could probably be compared across multiple clients to verify.

          • JustAnotherKay@lemmy.worldEnglish
            0·
            15 days ago

            What you’re referring to is called the P vs NP problem, and you’ve got it backwards. The problem is basically “does having an easily verified answer mean that there is also an easy to FIND the answer” and all the evidence we have so far points to no, but we aren’t sure.

            This is the entire concept behind cryptology - it’s easy to verify you have the correct key, because if you have the correct key you can decode the message and it isn’t gibberish. It is not, however, easy to figure out what the key is from the encrypted text.

          • diverging@piefed.socialEnglish
            0·
            15 days ago

            that mathematical principle where verifying an answer is as difficult as computing the answer in the first place

            You’re thinking of the P vs NP problem, but you got the details wrong. There are known cases where the answer is easy to verify.

          • draco_aeneus@mander.xyzEnglish
            0·
            15 days ago

            You’re thinking, as someone else said, of P = NP. This is technically an unsolved problem. We don’t know if P = NP is true, there is no proof that guarantees it. However, it’s looking pretty unlikely.

            This problem would have huge consequences for a wide range of fields, and it would break most cryptography and encryption algorithms we have (assuming it is proven true).

            Anyway, this is what Anubis is already doing. That’s why OP complained about being stuck for 3 minutes with the CPU at full blast. Anubis gave the PC some problem that was difficult to compute. The problem itself was not important, it’s just to keep the computer busy for a bit. (The idea is that this significantly slows down spam bots and other automated access, I believe).

            Of course Anubis has to verify the answer is correct, and that the client didn’t just send garbage. So the problem Anubis already uses is one that is easier to verify than to compute.

            Btw, it is possible to verify solutions where checking is as expensive as solving. You provide each client with at least 2 problems. One where you already know the answer, and one where you don’t. You have the client solve both. If the known one is good, you let them through, and add the unknown one to the known set. You can even verify it by sending it to two other clients, see if they all get the same answer. But I believe that this solution wouldn’t be required, because protein folding ‘violates’ the P=NP problem.

  • ferret@sh.itjust.worksEnglish
    0·
    16 days ago

    Unfortunately yeah, people with exceptionally shit computers get caught in the cross-fire of these things

  • Apeman42@lemmy.worldEnglish
    0·
    16 days ago

    Wait, I’m not mining crypto trash for strangers whenever I see that annoying fox girl, am I?

    • Doctor MoodMood@lemmy.mlEnglish
      0·
      15 days ago

      Try logging in or creating an account! I don’t get challenges when just looking through the wiki either.

  • Mwa@thelemmy.clubEnglish
    0·
    15 days ago

    I wish AI scrapes didn’t make the Internet unusable to the point of them using Cloudflare MITM or Anubis

  • MonkderVierte@lemmy.zipEnglish
    0·
    15 days ago

    Also this instance; can’t browse it on Desktop with a Librewolf-alike policies.json. Can the sensibility be adjusted server-side? No problems on Archwiki & co.

      • Cethin@lemmy.zipEnglish
        0·
        15 days ago

        That’s a great resource, but I don’t recommend doing that. Their operations cost is already high. If everyone started using it to bypass not checks then it’d break down. That’s not what it’s there for.

  • Stefen Auris@pawb.socialEnglish
    0·
    16 days ago

    You could try hitting up the Internet archive for a quick workaround.

    I think you can also download a copy of the entire arch wiki as a zim file too

  • sudoer777@lemmy.mlEnglish
    0·
    15 days ago

    I’ve had websites like Nix Hydra randomly give me a webpage full of gibberish which is also annoying and a lot moreso than the Anibus ones because you can’t get past it, I thought they got hacked at first also