There
are different ways of downloading files from remote locations. While
all web browsers support downloading files via http or ftp, I have
found that in the case of Firefox (at least), it doesn't support
resuming downloads. While this is not a problem for files of small
size, when it comes to downloading huge files such as ISO images of Linux distributions,
this lack of support for resuming downloads from where you had left off
earlier would mean you have to start downloading the file again from
the beginning.
The most common and fail safe method of downloading huge files in Linux is to use the wget tool. By using the -c option, you can resume the download of the file at a later stage in the event the downloading fails due to connection time out.
I usually use the following wget command to download Linux distributions.
$ wget -c full_path_to_the_linux_isoUsing Curl to speed up your downloads
But
here is a nice tip to speed up your download of files by a significant
factor. This tip is applicable for people who have a large bandwidth
internet connection in the lines of 4 MBps and above. Usually different
mirrors have different speeds. A Linux distribution mirror in say Japan
may have a 100MBps connection but another mirror in a different
location will be connected only to a 10 MBps pipe. More over these
mirrors throttle the bandwidth made available for each individual
connection thus providing a upper limit of download speeds.
What
you can do is split the file (in our case the ISO image) you are
downloading into a number of pieces and download each piece from a
different mirror simultaneously. At the end of the download, you can
combine all the pieces together to get your file in one piece. This is
known as download acceleration where the software connects to more than
one location simultaneously and splits the downloads among the
locations. This feature is commonly found in many download managers for
Windows but is hard to find in those available for Linux.
You
can get the same feature in Linux too by using the program curl to
download your files. Let's say I want to download the Ubuntu ISO image
of size 700MB. I split the download into 4 parts as follows :
$ curl --range 0-199999999 -o ubuntu-iso.part1 $url1 &Where url1, url2, url3 and url4 are as follows ...
$ curl --range 200000000-399999999 -o ubuntu-iso.part2 $url2 &
$ curl --range 400000000-599999999 -o ubuntu-iso.part3 $url3 &
$ curl --range 600000000- -o ubuntu-iso.part4 $url4 &
url1=http://ubuntu.intergenia.de/releases/feisty/ubuntu-7.04-desktop-i386.iso
url2=http://ubuntu.virginmedia.com/releases/7.04/ubuntu-7.04-desktop-i386.iso
url3=http://es.releases.ubuntu.com/7.04/ubuntu-7.04-desktop-i386.iso
url4=http://ftp-mirror.internap.com/pub/ubuntu-releases/7.04/ubuntu-7.04-desktop-i386.iso
This creates four background download processes, each transferring a different part of the ISO image from a different server. The --range option specifies a subrange of bytes to extract from the ISO file. The -o
option provides the name of the file to which the data is to be saved.
So once all the four curl processes finish their download, you will
have four files namely ubuntu-iso.part1, ubuntu-iso.part2,
ubuntu-iso.part3 and ubuntu-iso.part4 in your current directory.
To get the original Ubuntu ISO file, I just combine the files using the cat command as follows :
$ cat ubuntu-iso.part? > ubuntu-7.04-desktop-i386.isoAs an aside, I have entered all the commands in a file as follows :
As I said earlier, if you have a high speed Internet connection then this method will considerably shorten the time taken to download huge files.
#FILE NAME : da.sh (Download accelerator)... set the executable bit of the file and then I run it.
#!/bin/sh
url1=http://ubuntu.intergenia.de/releases/feisty/ubuntu-7.04-desktop-i386.iso
url2=http://ubuntu.virginmedia.com/releases/7.04/ubuntu-7.04-desktop-i386.iso
url3=http://es.releases.ubuntu.com/7.04/ubuntu-7.04-desktop-i386.iso
url4=http://ftp-mirror.internap.com/pub/ubuntu-releases/7.04/ubuntu-7.04-desktop-i386.iso
curl --range 0-199999999 -o ubuntu-iso.part1 $url1 &
curl --range 200000000-399999999 -o ubuntu-iso.part2 $url2 &
curl --range 400000000-599999999 -o ubuntu-iso.part3 $url3 &
curl --range 600000000- -o ubuntu-iso.part4 $url4 &
$ chmod u+x da.sh
$ ./da.sh