What gets lost when you turn a web page into text for AI

When you give a web page to an AI tool, the tool rarely reads the page you see. It reads text taken out of the HTML. Menus, scripts and styles go, because they are noise. But the same rules that remove noise can also remove a fact that matters, such as an exception to your refund policy that sits in a side box. This post shows what a typical conversion keeps and drops, on one real example, and how to write pages so that the important parts survive.

Why tools convert pages to text first

A help article of a few hundred words can come inside many times more HTML: navigation, scripts, tracking code, cookie notices and class names. An AI model that reads all of it spends most of its input on markup, and the markup can pull answers off topic. So tools that work from web pages, such as summarizers, AI assistants and support chatbots, first reduce the page to its main text. Markdown is a common target, because headings, lists and tables keep their structure in a few plain characters.

The conversion is a set of rules, and every tool has its own. The example below uses the rules of our own converter, which the website text extractor runs, and which Paperlink AI also uses when it imports a website for a support chatbot. Other tools differ in the details, but most make the same kinds of choices.

One page, before and after

Here is a short returns page, as HTML:

<html><head><title>Returns | Example Store</title></head><body>
<header><a href="/">Example Store</a><nav><a href="/shop">Shop</a> <a href="/help">Help</a></nav></header>
<main>
<div class="breadcrumb"><a href="/help">Help</a> / Returns</div>
<h1>Returns and refunds</h1>
<p>You can return most items within <strong>30 days</strong> of delivery. Read the <a href="/help/exceptions">list of exceptions</a> before you send an item back.</p>
<aside class="note"><p>Sale items can be exchanged but not refunded.</p></aside>
<h2>How long a refund takes</h2>
<table>
<tr><th>Payment method</th><th>Refund time</th></tr>
<tr><td>Card</td><td>5 to 10 business days</td></tr>
<tr><td>Store credit</td><td>1 business day</td></tr>
</table>
<details><summary>Can I return a gift?</summary><p>Yes. Use the gift receipt, and we refund the gift card.</p></details>
<img src="/img/label.png" alt="Where to find the return label">
<div class="cookie-banner"><p>We use cookies.</p><button>Accept</button></div>
</main>
<footer>Contact us at help@example.com</footer>
</body></html>

And here is the complete text that the converter keeps:

# Returns and refunds

You can return most items within 30 days of delivery. Read the list of exceptions before you send an item back.

## How long a refund takes

| Payment method | Refund time |
|---|---|
| Card | 5 to 10 business days |
| Store credit | 1 business day |

### Can I return a gift?

Yes. Use the gift receipt, and we refund the gift card.

Where to find the return label

The heading, the paragraph, the table and the gift question are all there. The menu, the breadcrumb and the cookie banner are gone, as they should be. But five pieces of real content are gone or changed too, and two of them change the answer a customer gets:

• The sale-items rule. It sits in an <aside>, which the converter treats as a sidebar and removes. An AI tool that reads this text will say that "most items" can be returned and never mention that sale items cannot be refunded.
• The contact email. It is in the <footer>, which is removed with the header and the navigation.
• The link to the exceptions. The words "list of exceptions" stay, but the address is gone, so the reader cannot follow it.
• The bold. "30 days" is still there, but the emphasis is not. That is fine for a model, as long as the sentence states the fact on its own.
• The image. Only its alt text remains. If the label instructions were inside the image, they are lost.

What the conversion keeps and drops

The full rule set of our converter, in its default mode:

Two rules surprise people. The first is the name rule: a box with a class such as promo-terms or cookie-policy is removed even when it holds real content, because the converter cannot tell it from a banner. The second is the main-content rule: if your page has a <main> element, anything outside it is ignored, even text that looks like part of the article in the browser.

Text that never reaches the HTML

A converter can only keep text that is in the HTML the server sends. Some pages build their text in the browser with JavaScript. A tool that reads the raw HTML gets an empty frame. Our extractor says so instead of returning an empty result. When Paperlink AI imports a website, it renders a limited number of these pages each day in a browser, and imports the rest in a later update. Many AI crawlers do not render pages at all, so do not count on it.

Tabs and "show more" panels have the same risk. If the hidden text is in the HTML, as it is with a <details> element, it survives. If a script loads it when someone selects the tab, it is not there to convert.

How to write pages that survive conversion

• Put rules in the body text. Exceptions, limits and conditions belong in a paragraph or a list under the heading they apply to, not in a side box, a tooltip or a banner.
• State contact details on the page that needs them. A footer email is invisible to most converters. If a help article tells customers to contact you, write the address or the page name in the article.
• Make link text say where it goes. "See the list of exceptions" still makes sense when the address is gone. "Click here" does not.
• Never put text only in an image. Write the steps as text, and give the image alt text that describes it.
• Give tables a header row. The first row becomes the header, so a table without one gets a data row as its header and reads wrongly.
• Keep content out of boxes named like banners. If a theme puts your promotion terms in a box called promo, rename the class or move the terms into the article.
• Use real headings. Bold text styled to look like a heading becomes a normal paragraph, and the structure is lost.

Test your own pages in five minutes

1. Pick three pages that customers ask about most: usually pricing, returns or billing, and one long help article.
2. Run each through the website text extractor.
3. Read the result with the page open next to it. Look for a rule, a number or a contact detail that you can see in the browser but not in the text.
4. Move each missing fact into the body text, publish, and run the page again.

Then check the meaning as well as the text. The website summarizer writes a summary from the same extracted text. If the summary of your returns page does not mention the main exception, a support chatbot or an AI assistant may miss it too.

What this means for a support chatbot

Paperlink AI is a customer support chatbot that answers only from the documents you upload and the website pages you import. It converts each imported page with the rules above, so the text in the extractor is close to what the bot can answer from. A fact that the extractor drops is a fact the bot cannot use. When the bot cannot answer a question from your content, it says so and shows your support contact, and the question appears in the Unanswered report. A question that keeps coming back there, about something your page does say, often points to a fact in a side box or a footer.

Fixing the page fixes it for every tool that reads your site this way at once: the bot, AI assistants and summarizers. It also helps the customer who reads only the article and never scrolls to the footer.