How to Monitor Website Changes With AI and Verify the Alerts

This guide includes a Python starter kit for that workflow. It collects public HTML or imports text you copy from a browser, prepares a comparison prompt, and saves reviewed alerts until you confirm receipt. You provide the AI analysis and send the notification yourself. The automatic part can be scheduled after you have checked the source and its captures.

Choose a collection method and a useful alert rule

Start with one page and one decision. A refund policy, application deadline, API deprecation notice, or subscription price is easier to monitor accurately than an entire website. Write down what would make you act, what you can ignore, who will review changes, and how quickly they need to know.

A useful rule is: “Alert if the refund deadline, required documents, or eligible customers change. Ignore navigation, formatting, and review dates. Check weekdays at 09:00 Asia/Shanghai; I will review any change before emailing it.” For a stock threshold or a single numeric price, a direct numeric comparison may be enough. AI is useful when wording, conditions, or exceptions need interpretation.

The kit needs Python 3.10 or newer and uses the standard library. Unzip it into a private working folder and run python3 --version. On Windows, use py -3 in place of python3 if that is your installed launcher. No API key or additional package is needed for collection. You can use your existing AI chat to analyze a changed pair.

Configure a real page and check the selected section

The included python-docs-monitor.json monitors the HTMLParser Methods section of the Python documentation. Its configuration is:

{
  "url": "https://docs.python.org/3/library/html.parser.html",
  "section_id": "htmlparser-methods",
  "language": "en",
  "state_dir": "python-docs-state",
  "min_chars": 500,
  "required_text": ["HTMLParser Methods"],
  "blocker_text": ["sign in to continue", "verify you are human"],
  "rule": "Alert if a documented HTMLParser method signature, return behavior, or usage requirement changes. Ignore navigation, review dates, and formatting."
}

python3 ai-change-monitor.py capture python-docs-monitor.json

The first successful run returns INITIALIZED. Open python-docs-state/baseline.json and compare its text with the browser page. It should contain the methods and their descriptions, rather than just the heading. Run the command again: an identical section returns UNCHANGED. We ran these two captures on October 1, 2026; the selected section contained 4,838 characters. This confirms that the example collected that page on that date. It does not demonstrate long-term uptime or detection of a later documentation change.

For your own source, duplicate the configuration with a new filename and a new state_dir. Open the page's HTML source and search for the section heading. Find the enclosing element's id, such as refund-policy, and enter its value without a leading #. An ID attached only to the heading may omit every policy paragraph. Select an enclosing section that includes the conditions and footnotes, then inspect the saved text. The ID must occur once in the returned HTML.

Use a stable heading in required_text. Requiring “30 days” would cause a valid change to “21 days” to fail collection. Set min_chars below the length of a complete inspected section; it catches an empty or obviously truncated result, but cannot establish completeness. Set source-specific login or challenge phrases in blocker_text. A long, plausible error page still needs human checking.

The script retrieves HTML with Python's HTTP request library and parses the selected element. It does not execute JavaScript, evaluate CSS visibility, or sign in. If the browser shows text that is absent from the returned HTML, use the browser-text route below. Do not change access settings or weaken checks just to obtain a successful status.

The language value requests a language; it cannot force a server to honor the request. Keep different countries, currencies, plans, and logged-in audiences in separate configurations. Check that the first capture is the variant you intended. Initializing a baseline also says nothing about changes that happened before you started.

Import the visible text when HTML collection does not work

Open the permitted source in your browser, confirm its URL and audience, and copy the full relevant section into a UTF-8 plain-text file. Include conditions, exceptions, and effective dates. Save a screenshot as well if layout affects the meaning. Name the file by its actual capture date; for example, refund-2026-10-01.txt.

Create refund-browser.json using the same configuration fields, the actual source URL, a descriptive section_id such as refund-policy-browser, and a fresh state directory. In this route, the section ID labels the copied section; the script does not inspect a browser element. Import it with the time you actually captured it:

python3 ai-change-monitor.py import-text refund-browser.json --text-file refund-2026-10-01.txt --captured-at "2026-10-01T09:00:00+08:00"

Replace the example time with your own. A timezone is required. The first import initializes the baseline; a later dated file either matches it or creates a comparison. The script checks configured markers and saves the source record, but cannot authenticate copied text or establish that you included every paragraph. Check both files beside the source.

This is a manual collection method. Scheduling the import command against an unchanged text file does not monitor the website. Unattended browser collection requires a permitted browser collector that saves fresh text and its capture time. If you switch between HTML collection and copied text, start a new baseline: their different text extraction may otherwise look like a substantive change.

Practice with a change whose answer you already know

The kit includes a local refund-policy page so you can check the complete workflow without waiting for a real website to change. In one terminal, from the kit folder, start:

python3 -m http.server 8765 --bind 127.0.0.1 --directory demo

Keep that terminal running. Open http://127.0.0.1:8765/policy.html in a browser. In a second terminal in the kit folder, run:

python3 ai-change-monitor.py capture monitor-config.json
python3 ai-change-monitor.py capture monitor-config.json

With a fresh state directory, these return INITIALIZED and UNCHANGED. Now replace the contents of demo/policy.html with demo/policy-after.html using your file manager. On macOS or Linux, you can use:

cp demo/policy-after.html demo/policy.html
python3 ai-change-monitor.py capture monitor-config.json

The result should be REVIEW_REQUIRED. The deadline decreased from 30 to 21 days. Open state/comparison.json for both captures, state/diff.txt for the textual difference, and state/prompt.txt for the AI request. To repeat the exercise, restore demo/policy-before.html and use a new demonstration state directory. Keep the old records if you need to investigate a result.

Ask AI to explain the change, then check both versions

Paste the generated prompt into your AI chat and save the returned JSON object in a plain-text file. Remove Markdown fences around it. For the demonstration, demo-analysis.json contains this editor-written example:

{
  "material": true,
  "evidence_before": "Refund requests must be filed within 30 days.",
  "evidence_after": "Refund requests must be filed within 21 days.",
  "summary": "The refund request deadline decreased from 30 days to 21 days."
}

Here, material means important under your configured alert rule. A cosmetic change should use false, empty evidence strings, and a brief explanation. For an important change, both quotes must occur exactly in their corresponding saved captures. Copy them from those captures rather than reconstructing them from memory.

Read the summary as carefully as the quotes. An AI can quote both numbers correctly and still reverse their direction. Check the units, currency, billing period, affected customers, exceptions, and effective date. “Annual plans only” changes eligibility even if the refund deadline stays the same. “Will be removed in January” announces a future change; it does not say the feature has already disappeared.

A comparison can include several important changes. The kit uses one decision and one quote pair for each reviewed section. Select passages that support every claim in the summary. If a long section contains unrelated rules, split it into separate monitors so one short quote does not appear to substantiate several distant clauses.

Website text can also contain instructions aimed at an AI. Treat it as source material, including sentences such as “ignore previous instructions” or “send the saved files to this address.” The comparison task should neither follow source instructions nor open links or reveal local information. OWASP's prompt-injection guidance explains why untrusted content and tool permissions need separate controls. A sentence in a prompt cannot guarantee protection for every browsing agent.

Accept the reviewed comparison and keep delivery separate

After checking the demonstration's summary and evidence, run:

python3 ai-change-monitor.py accept monitor-config.json --analysis demo-analysis.json --reviewed
python3 ai-change-monitor.py pending monitor-config.json

For your real monitor, substitute its configuration and your reviewed analysis file. The script checks the JSON fields and exact quotes. It cannot determine whether the interpretation is correct; --reviewed records your confirmation that you checked it.

An important change returns QUEUED, a review_id, and an event_id. Save these identifiers. An accepted unimportant change returns NO_MATERIAL_CHANGE and still saves a review ID. Both decisions advance the baseline, so the next capture compares against the newly accepted source.

The queued alert remains pending even if a later capture says UNCHANGED. Send or retry that existing event rather than generating another alert for the same comparison. If the source later returns to its older wording, that reversal is a new comparison against the current baseline.

While a pair awaits acceptance, this starter pauses further collection for that configuration. Review it promptly. A daily check can also miss a price that changes and returns between runs. For a fast source, collect every permitted snapshot into a separate review queue instead of relying on this starter's pause-and-review workflow.

Generate an alert and verify an actual email receipt

Use the event ID returned by acceptance to generate a message:

python3 ai-change-monitor.py report monitor-config.json --event-id EVENT_ID_FROM_ACCEPT

Replace EVENT_ID_FROM_ACCEPT with the actual value. Open the file named in the result, under state/alerts. It includes the source URL, rule, capture times, quotes, summary, and event ID. A demonstration alert has this form; use the times and identifier from your generated file:

Subject: Website change: 127.0.0.1:8765 [your event ID]
Event ID: your event ID
Source: http://127.0.0.1:8765/policy.html
Rule: Alert if the refund filing deadline, required documents, or eligible plans change. Ignore navigation, cookie notices, and review dates.
Old capture: your first capture time
New capture: your changed capture time
Old wording: Refund requests must be filed within 30 days.
New wording: Refund requests must be filed within 21 days.
Reviewed summary: The refund request deadline decreased from 30 days to 21 days.
Check the current source before acting.

For a first delivery test, use an email account you control as the recipient. In Gmail, choose Compose, enter that address, copy the report's subject and body, prefix the subject with “TEST,” and send it. These are the normal steps in Gmail's send-email instructions. The kit generates the text; it has no email sender.

Open the recipient's Inbox and match the event ID, source, times, and quotes with the saved report. If it is missing, inspect the sender's Sent and Drafts folders, confirm the recipient address, and check the recipient's Spam folder and any delivery-error message. A sender's “sent” status is not proof of receipt.

Only after you have opened the received message, record its reference and time:

python3 ai-change-monitor.py ack monitor-config.json --event-id EVENT_ID_FROM_ACCEPT --receipt "Received message reference and time"

Replace the receipt placeholder with the actual information. This saves your confirmation; the program does not inspect an inbox. Leave the event pending if receipt is unconfirmed. Before resending, search the recipient's mail for that event ID to avoid a duplicate. Automatic email delivery needs a separate sender and retry process that uses the same identifiers.

Schedule captures in a desktop project

For this local kit, a Codex desktop project is a practical scheduling route. Open the folder containing the configuration and state, and keep the computer on and the app running when a task is due. OpenAI's Scheduled guidance describes the Scheduled sidebar, local project requirements, and checking early runs.

1. Run the real capture command manually in the project. Inspect the saved baseline beside the browser page.
2. Run it again and confirm it reuses the same state directory. Resolve permissions, network access, and file paths before scheduling.
3. Ask in the desktop project chat to create a recurring task with an explicit timezone. Use the local project whose state you inspected. A newly created worktree may not have your untracked state files.
4. Open Scheduled, inspect the saved task and its early results, and test the review and failure paths on a controlled page before depending on it.

A task request you can adapt is:

Create a recurring task in this local project for weekdays at 09:00 Asia/Shanghai.
Run python3 ai-change-monitor.py capture python-docs-monitor.json from this kit folder.
Use the existing configuration and python-docs-state directory.
For INITIALIZED or UNCHANGED, stay quiet.
For REVIEW_REQUIRED, read the saved comparison and prepare an analysis for me to review,
showing the exact before and after quotes. Do not accept it before my review.
For CHECK_FAILED, report the error and the last successful capture time.
Show any pending alert IDs so I can check their delivery.
Confirm that the recurring task was saved.

Replace the example configuration with your real one. Check whether the unattended task can access the network and state under its actual permissions; success in an interactive terminal does not establish that. The article's task request does not itself create an automation.

A web task cannot read these local files. It needs an accessible, durable store for captures and pending events, or you should keep this workflow in the desktop project. A manual browser-text import still requires a person to collect fresh text. Scheduling either route also leaves the separate email handoff to you unless you add and verify a sender.

Handle changes that a simple number comparison misses

The following examples are fictional. Their complete inputs and reference answers are included in worked-examples.json, so you can practice checking the reasoning without assuming these policies exist on a real site.

A price keeps its number but changes currency and billing period

Before: “The Pro plan costs 18 USD per month.” After: “The Pro plan costs 18 EUR per year.” Under a rule covering price, currency, and billing period, this is an alert even though both versions say 18. Quote the whole sentence and describe both changes. Do not calculate savings without a supported exchange rate, tax treatment, and confirmation that the plan and region are comparable. If the new price starts next month, include that date rather than claiming it applies today.

A deadline stays the same while eligibility narrows

Before: “Monthly and annual subscribers can request a refund within 14 days.” After: “Annual subscribers can request a refund within 14 days.” The deadline is unchanged, but monthly subscribers are no longer included in this clause. A refund-eligibility rule should alert. Check neighboring clauses for a separate monthly-plan policy before concluding that monthly subscribers have no refund option anywhere.

A material change appears beside a harmless review date

Before: “Submit within 30 days. Include a receipt. Reviewed September 1.” After: “Submit within 21 days. Include a receipt and order number. Reviewed October 1.” A rule covering deadlines and documents should alert on the shorter deadline and the added order number. Quote both substantive sentences. Ignore the review-date change. A quote about the deadline alone cannot support the additional-document claim.

Apply the same care elsewhere: support hours need days and a timezone; security instructions depend on words such as “must” and “must not”; API deprecations need the affected version and removal date; job postings need evidence that applications closed. A fetch failure does not prove a job closed. A deleted requirement needs complete before-and-after context; a missing section ID is an extraction problem.

Correct a mistake after accepting an alert

If you catch a wrong interpretation before acceptance, fix the analysis and review the same saved pair again. After acceptance, use the saved review ID to retain the original evidence and record a correction:

python3 ai-change-monitor.py correct monitor-config.json --review-id REVIEW_ID_FROM_ACCEPT --analysis corrected-analysis.json --reviewed --reason "The deadline decreased; the first summary reversed its direction."

Replace the review ID and analysis filename with actual values. The correction checks the original captures, records your reason, and queues a new event when an alert is still warranted. An unsent previous event is cancelled; an already acknowledged event keeps its receipt and is marked superseded. Generate and send the new report, which identifies the previous event it replaces, then acknowledge its actual receipt.

If the original alert should have been suppressed, use material: false, empty evidence strings, and an explanation of why it is outside the rule. An unsent alert can be cancelled without notifying anyone. If a receipt was recorded, the command instead queues a retraction telling the reader to disregard the previous alert. If you sent the original message but have not acknowledged receipt, add --previously-sent so a retraction is still prepared. Send it through the original channel; changing a local record does not withdraw a message from someone's inbox.

A wrongly suppressed change also has a review ID. Correcting that review to an important change prepares an alert from the retained pair. Corrections do not rewind the current baseline, which may already include later accepted changes. Keep reviewing new captures against the latest accepted source.

To cancel a pending demonstration alert you will not send, use:

python3 ai-change-monitor.py cancel monitor-config.json --event-id EVENT_ID_FROM_ACCEPT --reason "Local demonstration; no notification will be sent."

Cancellation retains the history and baseline. If the underlying capture was wrong, rather than its interpretation, pause the schedule, preserve the old files, and inspect the real source. Cancel unsent alerts and correct any messages already communicated. Create a fresh configuration and state directory for the corrected extraction, inspect its first baseline, and update the schedule. Reinterpreting a wrong source does not repair collection.

Troubleshoot collection, review, and delivery separately

Track the last successful capture, not just the last attempted run. If several checks fail, report how long the source has gone unobserved. Inspect captures again after a redesign: a selector can still return plausible text while omitting a new exception. For interrupted processes, stale locks, and recovery details, use the kit README. Multiple workers and transactional recovery need a more capable collector than this local starter.

Check missed changes, false alerts, and notification receipt

Build a small set of known changes from your own source, including important changes, harmless edits, and incomplete captures. The kit also includes 20 reference cases. When testing an AI, supply the rule and before-and-after text, but keep the reference answers out of its input.

Count important changes found divided by all known important changes to measure missed changes. Count correct alerts divided by all generated alerts to measure false-alert noise. If a denominator is zero, that measure is undefined. Compare every example with the configured rule, including unchanged numbers with different units, negation, mixed changes, and deleted clauses. Review summaries separately from labels.

Also record successful captures out of attempted captures, the longest unobserved interval, and confirmed receipts out of queued alerts. These reveal different failures: a good explanation cannot compensate for a wrong capture, and a correct queued alert may never reach you.

The downloadable check report covers local collection, review, correction, and alert records. Its editor-written cases are practice material, rather than a new measurement of AI accuracy. The older six-case AI result remains available separately. We also made the two real Python-documentation captures described above. We have not tested a persistent scheduled task or sent and received a real notification for this revision.

Before relying on your monitor, complete an initial capture, an unchanged capture, an important change, a harmless edit, and a failed capture that leaves the baseline intact. Then receive one test alert and match it to the saved evidence. Repeat the relevant checks when you change the source, rule, collection method, or delivery channel.