Monthly Archives: May 2008

how to: using lftp to mirror an ftp site

I encountered this problem before and back then, I tried to solve it… and I did, using wget. I totally forgot about this until it came back to haunt me… hahahaha!

Back then, I used wget with the -c option, means to continue/resume a partially downloaded file, or skip if the file is already downloaded. Note that it’s using the file size as basis… This can lead to a disaster if the file on the receiving end has the same file size but with a different content…

And so my quest to find a better solution begins… (again!)

Anyway, I stumbled on this command, lftp, which has a good mirroring support. And so, after a quick read, I came up with this script:


dir_log="$HOME/log/$(date +%y%m)"

mkdir -p $dir_log

deb_file="$dir_log/$(date +%y%m%d).mirror.log"

lftp << EOC
debug -o $deb_file
user ftp_user ftp_password
mirror -e dir_to_mirror "$HOME/mirror_dir"

And so, just another bash script… hopefully, this one won’t haunt me 🙂