I encountered this problem before and back then, I tried to solve it… and I did, using wget. I totally forgot about this until it came back to haunt me… hahahaha!
Back then, I used wget with the -c option, means to continue/resume a partially downloaded file, or skip if the file is already downloaded. Note that it’s using the file size as basis… This can lead to a disaster if the file on the receiving end has the same file size but with a different content…
And so my quest to find a better solution begins… (again!)
Anyway, I stumbled on this command, lftp, which has a good mirroring support. And so, after a quick read, I came up with this script:
#!/bin/bash dir_log="$HOME/log/$(date +%y%m)" mkdir -p $dir_log deb_file="$dir_log/$(date +%y%m%d).mirror.log" lftp << EOC debug -o $deb_file open your.ftp.site.here user ftp_user ftp_password mirror -e dir_to_mirror "$HOME/mirror_dir" quit EOC
And so, just another bash script… hopefully, this one won’t haunt me🙂