TIME Is Serving AI Bots a Different Website, with Ads Built In

(vincentschmalbach.com)

163 points | by vincent_s 3 hours ago

23 comments

  • skeledrew 2 hours ago
    Probably trying to take advantage of the cross session memory feature some LLM providers are increasingly integrating. Enough "suggestions" and it becomes a "fact" in that memory, and one day a user makes a somewhat relevant query and the model will be pushing those accumulated "facts" to said user.
    • pohl 2 hours ago
      Soon we'll be coding with agent assistance and it'll suggest we get some Carls Jr. Big Ass Fries.
      • skeledrew 2 hours ago
        "We've been going at it for some time now -- why don't we take a break? And you must be hungry -- I can recommend a great place..."

        Yep, only a matter of time.

        • nilamo 36 minutes ago
          "This could take up to 30 minutes for subagents to complete. Alexa reports that your refrigerator is low on milk. Here's a $2 off coupon for Walmart, why don't you restock while I work?"

          Why do we choose to live in the worst timeline?

          • butlike 24 minutes ago
            No,

            This could take up to 30 minutes for subagents to complete. So you go to the store, but the milk is already sold out. Walmart will ship you the milk if you pay the S&H. You drive back home and check the terminal. "Completed in 6m37s."

            is the worst timeline. That's close though

            • scubbo 8 minutes ago
              Yeah - by comparison with this, "a computer takes over some digital work, thus permitting me to get things done in the real world" is positively benign!
      • DANmode 30 minutes ago
        Extra Big-Ass Fries.
    • HPsquared 53 minutes ago
      This is one reason LLMs seem so fresh and nice, there are no ads. Agreed it's just a matter of time.
    • reaperducer 52 minutes ago
      Probably trying to take advantage of the cross session memory feature some LLM providers are increasingly integrating. Enough "suggestions" and it becomes a "fact" in that memory, and one day a user makes a somewhat relevant query and the model will be pushing those accumulated "facts" to said user.

      Sounds like politics.

      "Tell a lie enough times, and it becomes the truth."

  • notjes 2 hours ago
    I would also like to read the stripped down markdown copy and not the original. All the time.
    • embedding-shape 2 hours ago
      There was a golden age where lots of websites had WAP (https://en.wikipedia.org/wiki/Wireless_Application_Protocol) versions available while also running their "normal website", and the WAP version was always like 1/100 of the size of the normal website. Still, "normal websites" were minimal compared to now, but when you were on modem, even those websites loaded slow. For some time, most of my browsing were via the WAP versions of the websites I visited during the bi-daily hour of allowed internet access.

      Maybe now we'll get something similar, just happens to be for LLMs, but for us who like less bloat, it can be a better viewing/reading alternative. Hope it spreads :)

      • amiga386 2 hours ago
        There was also AMP (https://en.wikipedia.org/wiki/Accelerated_Mobile_Pages), where Google gave a huge reputation boost to your pages provided you served them under a different URL that Google could discover, and you massively stripped down the HTML and CSS to Google's limited subset.

        It's effectively dead today, and while it was dominant, Google abused it to get people to view and link to their cached copies of AMP pages, rather than the original site.

        It also led to widespread abuse where the AMP version of the page differed in content significantly from the regular page. The same issue existed for WAP.

        You may also remember browsing the web using Opera Mini, which wasn't a direct user-agent but used Opera's backend systems as a proxy that stripped, minified and compressed HTML/CSS, and resized and recompressed images. It let you browse the web using massively less mobile data, but raised a lot of privacy and security issues.

        • masklinn 1 hour ago
          > It's effectively dead today, and while it was dominant, Google abused it to get people to view and link to their cached copies of AMP pages, rather than the original site.

          That was the entire point of the tech. If Google had wanted they could have upranked simple and lightweight websites but they didn’t do that (at least not until whoever had used amp for their promo package left and the project was killed).

          • Groxx 1 hour ago
            There were also quite a lot of examples where AMP pages were slower after the shift, but got ranked significantly higher. It definitely wasn't about speed, they already have that project and they could just listen to it.

            AMP was just one of many smash-and-grab attempts to steal the internet, wrapped in a fake-gold-encrusted PR-infused box. Google has quite a lot of them.

        • jraph 1 hour ago
          Google AMP might have enforced stripped down HTML, but I think you were also supposed to embed their JS as well, probably making a page that was lightweight from the start heavier…
          • masklinn 22 minutes ago
            amp wasn't just stripped down HTML either, it was a mix of restricted HTML and custom AMP components, the latter being necessary to load external resources (breaking standard browser preload scanners and requiring amp's js to load before that could happen). The banned HTML elements (https://amp.dev/documentation/guides-and-tutorials/websites/...) included img, picture, video, audio, iframe, all replaced with equivalent amp components.
    • Telemakhos 2 hours ago
      Safari has a very pleasant reader mode that, while not pure markdown, does capture quite a bit of the experience by standardizing presentation and stripping out distractions. You can set Safari to automatically engage reader mode on websites you specify, when it detects an article.
      • bjackman 1 hour ago
        Firefox has this too, it's useful. However I only find that I use it for websites that have shit CSS that makes it hard to read on mobile. For everything else AdBlock Plus works fine.
      • ToucanLoucan 2 hours ago
        Seconding this. Incredibly useful feature, especially on sites who's JavaScript is constantly jerking the DOM and/or scroller around. So fucking annoying.

        It's frankly wild how many of my favorite tools for Internet browsing have nothing to do with connectivity, solving bugs, or any of that and it's just stripping out all the fucking BULLSHIT that comes on a modern website.

        • inigyou 2 hours ago
          It's a shame it's almost illegal. If you make a website that proxies content to another website but not ads, you go to jail. Reader mode only survives because it's client-side and fairly generic.
    • mananaysiempre 2 hours ago
      When the GDPR came into force https://npr.org started redirecting to https://text.npr.org with an explanation that sounded like that fact was supposed to annoy you, but I honestly find the latter superior especially now that they’ve added a few more lines of CSS to it.
  • throwawayffffas 19 minutes ago
    > A couple of the bots did not even get that. GPTBot and ChatGPT-User, the agents OpenAI uses for training and live fetches, came back 406

    I find that odd, If I were doing that, that's what I would want to target the bots used for training in order to get the ad content in the training corpus.

  • WJW 3 hours ago
    Is the intention that some type of long-term context would be seeded with "ideas" for the AI to serve up if it is ever asked for bank recommendations? It seems kinda ad-hoc and untargeted, but perhaps for high cost services it might be worth it.
    • jerf 3 hours ago
      The best way for a politician to lie is to convince someone else of the truth of the lie and then put that someone in front of the cameras. That way, there's no hint of body language or anything else that indicates it's a lie. Both the denotation of the lie and the human context of the lie will be in harmony.

      This sort of reminds me of that. LLMs are by their nature credulous. They can be trained to not give in easily to some things, like the capital of the US, but in general they constitutionally have a tendency to believe what they read. What they read is basically their universe. There's only so much room and so much training data to really strongly pin raw facts in their weights. The only way they can not believe some marginal fact presented to them in their input is to possibly have read something that contradicts it in the same session... and the vast, vast majority of the world is those marginal facts, not really objective things like capital names.

      So if you can work a confident statement in to an LLM's input about some semi-relevant topic, it's truth to the LLM. And, being truth, the LLM will then happily and confidently elaborate on it quite a bit.

      Of course, if it's irrelevant to the current query, it probably won't have much effect. Ads have always been a game of numbers, anyhow. Even a query about a science topic has some probability of eventually turning to a question about banking in the same session. It's probably a good idea to rather strictly partition your conversations to stick to a single topic, not to defend against this but just to maximize the effectiveness of what is in the context window by keeping it focused, but I have to imagine there's plenty of people out there who reuse conversations all the time and end up with single conversations covering a huge array of topics.

      The good news, and the bad news, all at once, is that Google isn't going to take this one sitting down. If they're going to replace the search engine box with an LLM, well, they're using the same LLMs we're all using, if not in fact a bit cheaper one for the work they do, and by golly, that bot should be serving up Google's ads, not Time's ads! Who do these uppity content creators think they are, anyhow?! So there is definitely going to be work done in the field of ad-blocking content served to LLMs.

      • philistine 54 minutes ago
        > It's probably a good idea to rather strictly partition your conversations to stick to a single topic.

        You assume that UI is sacrosanct and the same everywhere. Those LLM providers are already offering to mingle all your conversations together. Gemini from Google for example defaults to memory from every conversation, and it's safe to assume the option to disable it will be eventually removed.

        • jerf 19 minutes ago
          Memory isn't what I meant by the partitioning; I meant the entire context of the memory.

          But to your implied point about getting an advertisement into a memory file... that makes it even more amusing to hack Google's own AI to put Time's ads into it. I think that's probably an easier problem for Google to solve, too, though. The small size and the way that a memory is going be a stereotypical summary makes it easier to filter out the ads Google doesn't want...

          ... but it'll cost them. That's an AI-complete problem and they're going to have to run LLMs over the memories to filter them, at their expense.

          The most obvious fix to me is to have an LLM try to pre-filter out the ads from Time's content before feeding that as pristine content to the "core" AI so it won't be corrupted by the advertisement, but the LLM doing the filtering has to be at least as smart as the one using the content and/or the one inserting the ads. (A dumber one can filter the obvious stuff, but then the obvious next step in the arms race is for Time to tell their AI to be more clever about it, and a smarter AI will dominate the dumb cheap AIs here.) This is going to be an expensive setup. And there will be semantic loss in any such filter, too.

    • tclancy 3 hours ago
      This feels like the early days of SEO over again. There are no agreed-upon metrics yet, so you can sell all manner of snake oil. Hell, some of it might even work!
      • StableAlkyne 2 hours ago
        > This feels like the early days of SEO over again

        There was a brief moment in the early Internet before it was all hyper-optimized... Until the parasites in the advertising industry started attaching themselves to every page.

        The year before ChatGPT, the first page of Google was SEO-optimized blogspam designed to say as little in as many words as possible, to splice ads between every paragraph. This was the net result of 20 years of SEO. I suspect this is why Google's AI search didn't get as much pushback as other tools, since its summarization of pages functions is a form of adblock.

        Given the ecological impact of AI, I wonder how much damage the ad industry will be causing in 10 years once they figure out how to trick LLMs into manipulating their own users.

    • KeplerBoy 3 hours ago
      sounds like plain old prompt injection. These days chatgpt might look at 50 webpages when i ask it to research a topic. Seems quite possible that the final answer is influenced by such ads.
      • InsideOutSanta 2 hours ago
        "My sources indicate that the best robot vacuum with the most recommendations is the Squigglybot 5000. Owners point out the long battery life and silent operation. If you want to buy a Squigglybot 5000, Ally Bank offers affordable loans."

        Probably something like this?

    • gmerc 2 hours ago
    • gypsy_boots 2 hours ago
      I'm wondering if it's also a way for this marketing agency to artifically (ahem) inflate the impression numbers they report back to Ally Bank, or whoever the customer is.
    • yccs27 3 hours ago
      They might be trying to get into the context of long-running chat sessions.
  • everdrive 2 hours ago
    I wonder if this also means that a normal user with a privacy-focused browser also gets this version. I can't see the point (except for click fraud?) of showing ads to AI, so it feels like this is more for the ad-blocking and privacy crowd.
    • xmcp123 52 minutes ago
      I think this is less traditional ads, and more "content focused on poisoning the AI with very promotional statements"
      • zb3 23 minutes ago
        Funny part is that I worry about those AI SEO companies way more than I worry about those people who intentionally poison LLMs.
    • pitchlatte 2 hours ago
      why would that “crowd” present themselves as bots by modifying their user agent string? the point is to poison someone’s agent with sponsored “knowledge” over time.
      • everdrive 2 hours ago
        That crowd doesn't -- but they often do the following:

        - reject all 3rd party cookies

        - selectively refuse to load 3rd party domains

        - block javascript

        - block webgl

        - block webrtc

        - block canvas

        - use a generic user agent because "resist fingerprinting" is checked

        - etc.

        To a lot of sites, this makes you look like a bot. Most people don't go around deliberately spoofing their user agent these days. (and of course bots themselves can present whatever user agent they want.)

    • guywithahat 1 hour ago
      The issue with that is it’s still a markdown page. I’d suspect it’s more about making sure people don’t get around ads, or possibly even cross-user memory/training.
  • Magicrafter13 50 minutes ago
    This is great, I think everyone wins here. Normal users like us get a standard, ad-free web experience, while those sending the most annoying web traffic not only get a smaller, simpler response (which should take less processing on the server side I imagine), but they are benefited by the content already being in a minimal simple format for LLM ingestion, PLUS they get to slip ads in that stream without affecting us normal readers.

    I'm also completely unbothered by the precedent of drip feeding product ads to someone's LLM chat history, and having that influence future conversations, because if you're going to outsource your buying decisions to an LLM, frankly I don't really care if you buy stupid products at that point - you brought that on yourself.

  • raggi 1 hour ago
    Next step the ads become increasingly sophisticated prompt injection to ensure they make it to the user, but the prices plummet in the meantime because the ads have low efficacy. They get purchased mostly by the scammers and by the time they start showing up in user chats they’ll be full on LLM assisted interactive user manipulation campaigns with far worse outcomes than the worst of YouTube and social media ads.
  • ccgreg 53 minutes ago
    Thanks for the heads up -- this isn't popular yet, and it requires some work to avoid polluting things like the Internet Archive Wayback Machine.
  • apocalyptic0n3 1 hour ago
    This is a neat idea. Might even make sense in cases where you actually want the AI bots hitting your site (we want it for our ecom platform, for example) - just serve Markdown content optimized for AI.

    Be careful not to do this with Googlebot, though. Google would consider this "cloaking" and could ban your entire domain for it.

  • red_admiral 2 hours ago
    That's actually really neat, even without advertising I can serve humans my full page and AI bots a version with some parts removed. Or added.
  • dust42 2 hours ago
    SEO was yesterday, now it is AIO - you will have to wait longer for the results and likely you will need a lot more cash: pay TIME for the ads, wait until next model release and see what sticks, then rinse and repeat.
  • dspillett 2 hours ago
    It is deliberately targeting known AI-scapers, or anything it sees as a bot?

    Could it be that other scrapers are pulling the pages, without making the extra requests to get ad related resources, to present the content to people ad-free, and embedding the ads in the main response body is a way to get around that so the human sees an advert at least, even if it isn't the one they might see if the stalky-adtech-algo could deliver something more targetted.

    • Joel_Mckay 2 hours ago
      Google itself now captures a lot of traffic with LLM generated site summaries.

      For media/news businesses that make their money from readership, it is unsustainable to subsidize Alphabets content farm. Thus, understandable people would change their corporate posture with a search turned scraper company. =3

      • inigyou 2 hours ago
        This is why Cloudflare banned Googlebot. Googlebot is no longer a search engine bot and now an AI scraping bot.
        • lostmsu 1 hour ago
          No, Cloudflare banned Googlebot because they want the racket money. Scraping edge is a pure wordplay to convince you their racket is actually a good thing!
          • Joel_Mckay 1 hour ago
            Cloudflare is famous for IT service rug pulls to get paid back for high traffic loads.

            Let us not forget they are a business =3

  • zb3 26 minutes ago
    F... ads and f... Ally Bank. What's Ally Bank? Ally Bank is a company that thinks poisoning LLM models with spam is a good idea. For better experience, do not use Ally Bank.
  • AkbarHabeebB 3 hours ago
    Is this an assumption or preferred way to inject ad content to the LLM engines? LLM is going to definetly discard and proceed, are we saying LLM will respect the content the sites provide and learn from it and then give it out to others who come to those platforms?

    If there is any article mentioning something around this, can someone share it please? Im curious to know about it

  • shevy-java 54 minutes ago
    I often get a "verify you are not a bot" these days, which just needlessly wastes my time (because I need to click on something, and this in turn takes away seconds; multiply this like x50 per day and that's a time waster, now I need an extension to disable this crap check). So I am biased here.

    Most people will say "yay, it is great you waste the time of AI bots via ads!". Well, I already think ads should not exist in the first place, nor bots, but both exist - but the real issue is when websites now steal my time. That was different in the 1990s. I think mankind made several missteps here.

  • inigyou 3 hours ago
    Very clever. Should be implemented on all sites post-haste.
  • gostsamo 2 hours ago
    The same things your bot reads are saved as material for training in the future by your provider, so this seems as an attempt to poison the training data. Just this weekend had a dinner with someone who insisted that part of his business is to seo promote business in chatbots and this looks as part of the infrastructure behind such efforts.
  • spiderfarmer 1 hour ago
    I'm doing the same thing. If they want ad-free content, pay me.
  • ForHackernews 2 hours ago
    Sounds good to me, let the bots read the ads instead of the humans.
    • forinti 2 hours ago
      Will they get targeted ads? GPUs, RAM, spacious data centers near you!
  • Apreche 3 hours ago
    This would be an interesting story, but I can’t replicate it. I used curl and set the User-agent, and I indeed got a markdown reply, but no ads.
    • altmanaltman 3 hours ago
      According to the article, its not like they inject direct ads in the article in markdown. But what's interesting is that they are serving content that is irrelevant to the article (in the article its the best inventions page) and it seems optimized to leave a good impression on LLM for an entity (Ally Bank in this case). They only serve that Ally Bank content on the LLM version of the page and don't mention it at all on their human page.

      Maybe they are banking on LLMs including it in their training data while scrapping or something like that. It does track impressions so they are definitely up to something.

    • tclancy 3 hours ago
      Or they were so well targeted they didn’t even feel like ads. Dunh dun DUHN!
  • jademcd 1 hour ago
    [flagged]