WebSec - A Web Secretary

What is WebSec?

Web Secretary is a web page monitoring software. However, it goes beyond the normal functionalities offered by such software. It will detect changes based on content analysis, making sure that it's not just HTML that changed, but actual content. You can tell it what to ignore in the page (hit counters and such), and it can mail you the document with the changes highlighted or load the highlighted page in a browser.

Web Secretary is actually a suite of two Perl scripts called websec and webdiff. websec retrieves web pages and email them to you based on a URL list that you provide. webdiff compares two web pages (current and archive) and creates a new page based on the current page but with all the differences highlighted using a predefined color.

Personally, I put Web Secretary on crontab to monitor a large number of web pages. When the highlighted pages are delivered to me, I use procmail to sort them out and file them into another folder. Sometimes, when I am busy, I will not have time to accessing the web for a few days. However, with Web Secretary, I can always access the "archive" that it has created for me at my own leisure.


In the Savannah project file section.

The latest versions are:


Are there any dependencies?

Only Perl 5 and LWP module which should be standard with all Perl distributions.

How to install?

Simply unpack the archive and modify the config file to your hearts content. There is no GUI to configure this program, it's all in the text files.

How to use?

Just run the program and it will do its magic, the best mode would be to put it in a cron job for automatic daily work, this is great if you are connected all the time.

If you are connected by dialup, you may want to make it run automatically upon connection, how to do this is different between OSes & Distributions so exact instructions you will need to find on your own.

How do I get help? How do I help?

You can subscribe to the mailing list. Post messages to request help and offer help. You can suggest ideas and even provide patches to implement them :-)

Please share with us the web pages that you have monitored using Web Secretary, as well as tips and tricks for maximizing the signal-to-noise ratio.

There are several other facilities to help each other:


The original author is Chew Wei Yih (also known as Victor Chew), with the help of several contributors (see the README file).

Baruch Even picked up the program when it was mostly unmaintained to give it a new public home on Savannah.


Last modified by Baruch Even on 2003-05-11