Why do new artificial intelligence agents choose to reduce navigation on HTML?

Artificial intelligence agents take care of the world, and put the next big step in the development of artificial intelligence 🦖. So, what does all these factors share? They use Markdown instead of raw html when processing content on web pages ⛓. My curiosity to find out why?

This blog post will show you how this simple trick can provide you with up to 99 % of symbols and money!

Artificial intelligence agents and data processing: Introduction

Artificial intelligence agents They are software systems that harness the strength of artificial intelligence to accomplish tasks and follow -up goals on behalf of users. Equipped with thinking, planning and memory, these agents can make decisions, learning and adaptation – all of them on their own. 🤯

In recent months, artificial intelligence agents have launched, especially in the browser automation world. These AI’s AI’s agent browsers enable you to use LLMS to control software browsers, and to automate tasks such as adding products to the Amazon 🛒.

Have you ever wondered about AI’s libraries and frames from AI, such as Crawl4ai, ScrapeGRAPHAI and Langchain?

When processing data from web pages, These solutions often turn HTML into an automatic reduction– Or display ways to do this – before sending data to LLMS. But why do artificial intelligence agents prefer discounts on HTML? 🧐

The short answer is: To save the symbols and accelerate the treatment! ⏩

It is time to dig deeper! But first, let’s take a look at another approach used by artificial intelligence agents to reduce data download. 👀

From excessive data to clarity: The first step for artificial intelligence customers

Imagine that you want your artificial intelligence agent:

1. Contact the e -commerce site (for example Amazon)

2. Find a product (for example PlayStation 5)

3. Extract data from this specific product page

This is a common scenario for the artificial intelligence agent, as the renewal of e -commerce is a wild trip. After all, the product pages are chaos chaos for constantly changing changes, making software data a nightmare analysis. This is the place where the factors of artificial intelligence praise their great powers, and benefit from LLMS to extract data smoothly – it does not matter how the page structure is chasing!

Now, let’s say that you are on a mission to seize all the modern details of PlayStation 5 page page On the Amazon 🎮:

Here’s how to drive AI’s agent browser to achieve this:

Navigate to Amazon's homepage. Search for 'PlayStation 5' and select the top result. 
Extract the product title, price, availability, and customer ratings. 
Return the data in a structured JSON format.

This is what the artificial intelligence agent should do (hope 🤞):

1. Open the Amazon in the browser 🌍

2. Look for “PlayStation 5” 🔍

3. Determine the correct product 🎯

4. Extracting the product details from the page and returning it in JSON 📄

But this is the real challenge –Step 4. Amazon PlayStation 5 page is a monster! HTML is full of tons of information, which you don’t even need.

Want a guide? Full HTML copy of the page from the DOM page to the browser and drop it into a tool like the distinctive code calculator LLM:

🚨 Take yourself …

896,871 symbols?! 😱 Yes, I have read this correctly-ninety-six thousand, and eight hundred and seventy of the distinctive symbols!

This huge load of data – AKA tons of money! 💸 (more than $ 2 per request on GPT-4O! 😬)

As you can imagine, passing all these data to the artificial intelligence agent comes with great restrictions:

1. You may require installment/supporter plans to support the use of a highly distinctive symbol 💰
2. It costs a fortune – especially if you are running frequent inquiries 🤑
3. It slows down responses because artificial intelligence must address a ridiculous amount of information ⏳

Reform: fat trimming

Most artificial intelligence agents allow you to determine the CSS specific to extract the relevant departments only from the web page. Others use the algorithms of affairs for the automatic candidate content-such as stripping heads and appetite (which usually does not add any value). ✂

For example, if you check the Amazon PlayStation 5 product page, you will notice that most useful content lives within the HTML element that you have selected #ppd CSS specific:

Now, what if you told the artificial intelligence agent only focus on #ppd An element instead of the entire page? Will this make a difference? 🤔

Let’s put it on the test in the confrontation face to the face below! 🔥

Markdown Vs HTML in the processing of artificial intelligence data: comparison face to face

Compare the use of the distinctive symbol when processing a portion of the web page directly in exchange for converting it into a reduction.

Html

In your browser, copy HTML from #ppd The element, and drop it into a symbolic calculator:

From 896,871 symbols to only 309,951 –Nearly 65 % save!

This is a significant decrease, for sure, but let’s be real – still a lot of distinctive symbols! 😵‍💸

Price reduction

Now, let’s repeat the trick used by artificial intelligence agents by taking advantage of the HTML converting tool to Markdown online. But first, remember that artificial intelligence agents perform some pre -treatment to remove important signs of content such as and and tags const scriptRegex = /

Next, copy HTML, which was cleaned and converted into discounts using the HTML converter to Markdown online:

{
  "product_title": "PlayStation®5 console (slim)",
  "price": "$499.00",
  "availability": "In stock",
  "customer_ratings": {
    "rating": 4.6,
    "total_ratings": 5814
  }
}