Webmirror 2.0

You can define that maximum size of downloads during the process.
size 2000
size 100K
size 10M
You can define no more than 2000 bytes, 100Kbytes or ten megs in the examples above. Bare numbers mean bytes, the postfix K means a multiplicator of 1024 and the postfix M means a multiplicator 1024*1024.

Note that this command is not to be used as a limiting factor to stop the download a regular way. It is rather a safe heaven to stop the program consuming your disk and network bandwith.

Assume that you want to mirror a site that you estimate to be 20MB. Then set this parameter to

size 30M
This setting will have no effect if your estimation was right, but stops donwloading if the site is going to be much larger than expected.

When the download reaches this limit, it aborts downloading and many pages may not be saved to disk, which were already retrieved, but not fully processed. No leafs will be generated for pages scheduled but not retrieved.

Also note that this size parameter does not limit the log file which can grow fairly large in case there is a repeated soft error downloading pages. A soft error can be 502 Service temporarily overloaded that will cause webmirror to retry download.

The default limit for the total size of files retrieved is one megabyte in case no size parameter exists in the RDF file.

TOC