Wednesday, August 05, 2009

Linux: Avoid scp when copying a lot of files

When you are copying a lot of files files it may take a long time with
scp. This is because scp spawns a new process for each file and can be
resource intensive.

When using the -r switch, scp does not care about symbolic links and
will follow them, even if it has already copied the file. This can
lead to scp copying an infinite amount of data and can easily fill up
your hard disk.

Therefore, when you think of backups or remote copy of directories,
think of rsync.

Linux: Download whole website with wget

Command to download whole website

wget -r -p -np -k http://www.sonomamgmt.com