Page 1 of 1

can you set up bulk downloads from internet archive?

Posted: Wed Aug 11, 2021 8:16 am
by pizzacube
can you set up bulk downloads from the internet archive?

some large collections will not download in bulk and it gives you pages like this one example

https://archive.org/download/Entex_Adve ... 2012_04_23

that list links to download the file?

can the FDM get the files from the pages automatic so i dont have to click on each one and download them 1 at a time manually?

some of them are like 100 download links it will take forever.

Re: can you set up bulk downloads from internet archive?

Posted: Wed Aug 11, 2021 4:45 pm
by Usher
It depend on your needs. You can:
* Download zip archives from there.
* Download torrent which contains zip files together with images and some additional info/metadata.
* Use "Download all" from FDM context menu to download all files even if you don't need them (metadata) or they are duplicated (some folders may contain also unzipped files). With this option you can also select only needed file extensions.

For me torrent is the best choice.

Oh, and your example is a very small - only one zip and less than 1 megabyte in total.

Re: can you set up bulk downloads from internet archive?

Posted: Thu Aug 12, 2021 2:15 am
by pizzacube
none of that is an option for the large collections

https://archive.org/download/redump.sega_saturn

Re: can you set up bulk downloads from internet archive?

Posted: Thu Aug 12, 2021 3:40 pm
by Alex
Hello,
Why is "Download all" not an option? Works fine for me. Yes, it downloads lot of useless HTML files, but it's still usable.

Re: can you set up bulk downloads from internet archive?

Posted: Thu Aug 12, 2021 8:49 pm
by Usher
pizzacube wrote:none of that is an option for the large collections
https://archive.org/download/redump.sega_saturn
"Download all" should work OK. Do you have enough space for that download? If you are afraid of high CPU load, use moderate (medium load) mode and change (lower) traffic limits for that mode before starting download. You may even use snail mode for the start…