Building a website change detection tool involves a cycle of fetching, comparing, and notifying. Depending on your needs—whether you want to track simple text changes or complex JavaScript-rendered updates—you can choose between a lightweight script or a more robust headless browser setup.
1. The Core Logic
The simplest way to detect a change is to compare the “fingerprint” of a page’s content at two different times.
* Fetch: Download the HTML of the target URL.
* Store/Hash: Create a unique hash (like MD5 or SHA-1) of the content and save it.
* Compare: On the next run, fetch the page again, hash it, and compare it to the stored version.
* Notify: If the hashes differ, send an alert (email, Slack, Discord, etc.).
2. Simple Implementation (PHP)
For static sites, a basic PHP script running via a Cron Job is often sufficient.
<?php
$url = “https://example.com”;
$storageFile = “last_hash.txt”;
// Fetch current content
$currentContent = file_get_contents($url);
if ($currentContent === false) {
die(“Failed to fetch page.”);
}
// Generate hash of the content
$currentHash = sha1($currentContent);
// Compare with previous hash
if (file_exists($storageFile)) {
$oldHash = file_get_contents($storageFile);
if ($currentHash !== $oldHash) {
// Change detected!
mail(“you@example.com”, “Change Detected!”, “The page at $url has changed.”);
}
}
// Update the stored hash
file_put_contents($storageFile, $currentHash);
?>
3. Advanced Considerations
* Handling Dynamic Content: Many modern sites load data via JavaScript. file_get_contents won’t see this. You may need a headless browser like Playwright or Puppeteer to render the page before checking.
* False Positives: Things like timestamps, “Current Time” widgets, or rotating ads will trigger a “change” alert every time. To fix this, use a DOM parser (like DOMDocument in PHP) to target a specific HTML ID or class (e.g., only the #product-price div).
* Rate Limiting: Websites may block your IP if you check too frequently. It is best to set your intervals reasonably (e.g., every hour) and use a proper User-Agent header.
4. Recommendation: changedetection.io
If you want a production-ready solution without building everything from scratch, I highly recommend checking out changedetection.io. It is a powerful open-source tool that you can self-host using Docker. It includes:
* A visual selector to pick exactly which part of the page to watch.
* Support for Chrome/Playwright to handle JavaScript-heavy sites.
* Built-in notification support for over 80 services.
Would you like me to help you write a more specific script that targets a particular HTML element using a DOM parser?
I want to check for changes on this link https://member.expireddomains.net/domains/combinedexpired/#listing
Monitoring expireddomains.net is a bit more complex than a standard website because that specific URL is behind a member login and the site has strong protections against automated scraping (like Cloudflare).
To build a tool for this, you need to handle three main things: Authentication, Session Persistence, and DOM Parsing.
1. The Challenge: Authentication
If you simply use file_get_contents or a basic cURL request, the site will redirect you to the login page or block you. To check that specific link, your tool must:
* Log in automatically OR
* Use “Cookies” from your browser session so the site thinks the script is you.
2. PHP Implementation (Using cURL and Cookies)
Since you work with PHP, you can use cURL with a “cookie jar” to maintain your logged-in state.
Step A: Get your Session Cookie
* Log in to ExpiredDomains.net in your browser.
* Open Developer Tools (F12) -> Application -> Cookies.
* Look for the session cookie (usually named sessionid or similar).
Step B: The PHP Script
Instead of just hashing the whole page (which contains changing ads and timestamps), this script targets the specific domain list.
<?php
$url = “https://member.expireddomains.net/domains/combinedexpired/#listing”;
$cookieString = “sessionid=YOUR_SESSION_ID_HERE;”; // Manually provide your session
$storageFile = “domain_snapshot.txt”;
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIE, $cookieString);
curl_setopt($ch, CURLOPT_USERAGENT, “Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.0.0”);
$html = curl_exec($ch);
curl_close($ch);
if ($html) {
// Use DOMDocument to target only the table or the list
$dom = new DOMDocument();
@$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
// Target the specific table class or ID used by ExpiredDomains
// This prevents false positives from sidebar/header changes
$nodes = $xpath->query(‘//table[contains(@class, “base1”)]’);
if ($nodes->length > 0) {
$tableContent = $dom->saveHTML($nodes->item(0));
$currentHash = md5($tableContent);
if (file_exists($storageFile)) {
$oldHash = file_get_contents($storageFile);
if ($currentHash !== $oldHash) {
// Change detected – Send notification logic here
echo “The domain list has updated!”;
} else {
echo “No changes.”;
}
}
file_put_contents($storageFile, $currentHash);
}
}
?>
3. Better Alternatives for this Site
Because ExpiredDomains.net frequently updates its layout and security, a custom PHP script might break often. You might consider these methods:
* Browser Extensions (Distill Web Monitor): This is the easiest way. You can set it to monitor only the “Domain” column of that table. Since it runs in your browser, it’s already logged in.
* Python + Playwright: If you want to automate it on a server, Playwright is better than PHP for this because it can solve “Turnstile” or CAPTCHAs and can mimic a real human clicking the “Login” button.
Helpful Resources for Building Scrapers
How to use PHP cURL with Cookies
Introduction to PHP Simple HTML DOM Parser
Web Scraping with PHP and XPath
How to avoid being blocked while scraping
Note: Be careful with the frequency of your checks. If you check every minute, ExpiredDomains will likely ban your IP address. A check every 15 to 30 minutes is usually safer for this specific platform.
Leave a Reply