tech

Max Knoblauch7/29/24

AI companies' content crawlers are saddling website owners with unexpected, crushing bandwidth charges

AI bots that crawl websites and scrape their content are becoming more difficult to block and it's costing websites thousands of dollars.

By looting large amounts of data without permission, AI bot crawlers can leave smaller sites (that make and write things) on the hook for big unexpected bandwidth charges.

One coding documentation site said a crawler accessed 73 terabytes of files in May, including 10 terabytes in a single day, costing it more than $5,000 in bandwidth fees.

The problem: AI-focused companies keep making new crawlers that companies haven't blocked yet, and some appear to be bypassing attempts to block their bots altogether. Last month, OpenAI and Anthropic were accused of ignoring attempts to block their crawlers. Wired reporting found that Jeff Bezos and Nvidia-backed AI search startup Perplexity AI has similarly ignored scraping blockers, prompting an investigation from Amazon's cloud division.

Websites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones)

Websites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones)

One coding documentation site said a crawler accessed 73 terabytes of files in May, including 10 terabytes in a single day, costing it more than $5,000 in bandwidth fees.

The problem: AI-focused companies keep making new crawlers that companies haven't blocked yet, and some appear to be bypassing attempts to block their bots altogether. Last month, OpenAI and Anthropic were accused of ignoring attempts to block their crawlers. Wired reporting found that Jeff Bezos and Nvidia-backed AI search startup Perplexity AI has similarly ignored scraping blockers, prompting an investigation from Amazon's cloud division.

More Tech

Stanford AI report: Model capability accelerating, China has closed the gap with the US

Stanford’s annual AI Index Report says “leading models are now nearly indistinguishable” from each other, and benchmarks used to evaluate them are falling behind their accelerating capabilities.

Cube in hands

tech

Meta will surpass Google in ad revenue this year, new industry data shows

In a world supported by digital ad dollars, Meta may soon be king. The Instagram owner’s net digital ad revenues are expected to hit $243.5 billion in 2026, surpassing Google’s projected $239.5 billion, according to new data from eMarketer.

The shift is happening as Big Tech companies, including Meta and Google, are increasing their spending on AI in hopes that AI will grow their top and bottom lines.

On the company’s last earnings call, Meta CFO Susan Li credited AI with driving performance gains, and said that growth will continue: “We expect the set of investments we’re making in 2026 will enable us to drive further gains as we continue to integrate AI across all layers of the marketing and customer engagement funnel.”

“In surpassing Google, Meta has essentially had many of its core strategies validated,” said Max Willens, principal analyst at eMarketer. “Meta has long understood that scale, network effects, and habits are more important than anything else in digital media. It has carefully built and defended the advantages it has in all three areas.”

JAPAN-FOOD-DRINK-SCIENCE-REASEARCH-MSG-AJINOMOTO

What does delicious Asian food seasoning have to do with a potential bottleneck for AI chips?

Japanese food flavoring company Ajinomoto, which commercialized MSG, also makes a key component in AI chips. It’s having trouble scaling to meet demand.

tech

Report: Microsoft looks to remake Copilot in the image of OpenClaw

Microsoft is feeling the heat from all corners of the tech world as it tries to infuse its productivity apps with useful AI tools.

OpenAI, Anthropic, and now open-source OpenClaw are enabling powerful agentic AI that can do work on your computer for you — including productivity functions like managing emails, spreadsheets, and slide decks.

This is obviously an area where Microsoft needs to compete, or it will be left in the dust by AI startups.

The Information reports that Microsoft is indeed realizing this, and is now trying to reboot its many Copilot tools to act more like the extremely popular DIY agentic AI tool OpenClaw.

OpenClaw is usually set up running on a dedicated personal computer, and given access to all of a user’s permissions and logins. The user issues orders to OpenClaw through messaging apps like Telegram or WhatsApp, and the agent goes off and completes tasks in the background, notifying you when they’re done. But many users have had security disasters with the setup, so Microsoft is looking to borrow the popular concept but implement the strict security controls needed for use in enterprise environments.

According to the report, Microsoft CEO Satya Nadella has made revamping 365 Copilot a top priority.

Microsoft Plots New Copilot Features Inspired by OpenClaw

Microsoft Plots New Copilot Features Inspired by OpenClaw

This is obviously an area where Microsoft needs to compete, or it will be left in the dust by AI startups.

The Information reports that Microsoft is indeed realizing this, and is now trying to reboot its many Copilot tools to act more like the extremely popular DIY agentic AI tool OpenClaw.

OpenClaw is usually set up running on a dedicated personal computer, and given access to all of a user’s permissions and logins. The user issues orders to OpenClaw through messaging apps like Telegram or WhatsApp, and the agent goes off and completes tasks in the background, notifying you when they’re done. But many users have had security disasters with the setup, so Microsoft is looking to borrow the popular concept but implement the strict security controls needed for use in enterprise environments.

According to the report, Microsoft CEO Satya Nadella has made revamping 365 Copilot a top priority.

tech

Rani Molla4/13/26

Tesla competitor Slate closes $650 million funding round and says 2026 production is “on time and on budget”

Tesla competitor Slate Auto said it closed a $650 million Series C funding round led by TWG Global, giving it the “operating capital to reach the next stage of development.” Slate’s new CEO, Peter Faricy, says it has more than 160,000 reservations, up from 150,000 in December, and is “on time and on budget” to deliver its first mid-$20,000 electric trucks to customers by the end of 2026.

Latest Stories

Sherwood Media, LLC produces fresh and unique perspectives on topical financial news and is a fully owned subsidiary of Robinhood Markets, Inc., and any views expressed here do not necessarily reflect the views of any other Robinhood affiliate, including Robinhood Markets, Inc., Robinhood Financial LLC, Robinhood Securities, LLC, Robinhood Crypto, LLC, Robinhood Derivatives, LLC, or Robinhood Money, LLC. Futures and event contracts are offered through Robinhood Derivatives, LLC.

©2026 Sherwood Media, LLC