Home arrow News arrow Latest arrow Having fun downloading with Wget and Rapidshare

Subscribe

No Spam just the latest tips from reviewingit.com






Article listings

Last comments

Windows Vista Missing Hibernat...
Thanks
It works. :) Thanks alot.
04/08/09 18:04 More...
By Levent Kaya

Windows Vista Missing Hibernat...
Thanks
It works. :) Thanks alot.
04/08/09 17:56 More...
By Levent Kaya

Create Invisible Registry Entr...
can anybody creat a key for Atomic Email Hunter 5.44 ??!! ...
31/07/09 16:08 More...
By benny

Vista DHCP and Router Problems...
The texts close to Clothing are published by the writing ser...
24/07/09 22:44 More...
By vfvd

Having fun downloading with Wg...
mast :( :x :x :zzz 8) :roll :grin
24/07/09 08:30 More...
By mast

Having fun downloading with Wget and Rapidshare Print E-mail
User Rating: / 9
PoorBest 
Written by Administrator   
Tuesday, 12 June 2007

Image 

This nifty tip requires you to have a linux boxor cygwin... but Windows users don't fret I have a little something for you too... well anyway... if you download from rapidshare... wouldn't it be great if you could download from rapid share using your headless linux box and script those downloads.... those of you without a linux box check out the commands you can use with wget to download from websites, also its nice to know that you can get a copy of wget that runs in windows.. how about that... learn 1 command and use it on 2 platforms... neat... read on...

 

Well basically when I got this to work on my headless debian linux box... it waz fantastic..

Kudos to Mark Suillivan and Beowulf for the script and updates.. I got this tip from a gentoo forum

OK lets get organised... Let me divide this article up so that we can get going

If you just wanna fool around with wget skip on to item 6... For the rest of you just move on to item 1.

 1. To get this going firstly you need to prepare some of this stuff...

You need - sed,list_urls.sed and wget

Make sure the version of wget you use is 1.10.2 and above need the --no-check-certificate option

A rapidshare account with direct downloads set to on in your profile.

 Next get these scripts here at my website click below

list_urls.sed, BashGet.sh, RapDownload.sh

2. Modify the script RapDownload.sh

replace <userid> and <password> with your rapidshare userid and password. And you are good to go....

3. Make sure you chmod +x Getbash.sh, chmod +x RapDownload.sh, chmod +x list_urls.sed

4. Now to Download any rapidshare files just type

./RapDownload.sh <urltodownload>

(e.g. ./RapDownload.sh http://rapidshare.com/files/somebloatedstuff.zip)

or

./RapDownload.sh <filename of list with urls to download>

(e.g. ./RapDownload.sh downloadlist)

5. Ok some extra tips:

You might want to use lynx -source <url of rapidshare page with links> to get the html

(e.g. lynx -source http://www.rapidshare.com/XXYZ2 source.html) 

then

list_urls.sed <name of html file you got from above> > <name of list file you want to create>

(e.g. list_urls.sed source.html > downloadlist)

6. Ok here are some general commands for Wget use in windows I found useful, wget is great for downloading stuff quickly from webpages without too much messing around...

Ok first you might want to get wget for windows HERE 

Look at wget in action, ok i know this pic is tiny but can you see.. the ascii progress bars in action.... ahhh... the devil is in the details... or should I say deteriorating eyesight.... tsk tsk..

Image

a) wget -r http://www.somewebsite.com

The -r command tells wget to recursively download everthing from the listed url 

 b) wget -r -l2 http://www.somewebsite.com

Now the -r does the same as above the -l tells wget to limit to that number of levels here 2 levels deep

c) wget -r -A.zip -R.htm http://www.somewebsite.com

This one tells wget to do a recursive get and Accept all files with .zip extension and reject all files with .htm extension

d) wget -H -http://www.spanning.com

Ok you might want to do a wget with a -H instead of -r if the files you want to get spanns across other websites

e) wget -r ftp://username:password@site/path/to/leech

 You can use wget to download sites with userid and password

These tips have made me a really happy leecher.. and WGET is just so powerful one tiny command can do so much. I just have to give this tip a five cool smilies!!!!! CoolCoolCoolCoolCool


Add as favourites (10) | Quote this article on your site | Views: 35945

Comments (28)
RSS comments
1. 24-07-2009 08:30
 
mast :( :x :x :zzz 8) :roll :grin
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
2. 10-12-2008 10:06
 
Thank You Very Much
Thank You Very Much for sharing this script.
Guest
 
Yossi
3. 11-06-2008 18:20
 
Thank You Very Much
Looks like a great way to download rapidshare files in the background. 
 
Does it work on OSX ? 
 
rapidshare account from rapidownload. 
 
 
Keep up the good work, if i find bugs i'll let you know! 
 
Nick
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
4. 24-12-2007 17:47
 
thanks
thanks for this script, after one or two small modifications it does exactly what I was looking for :) merry christmas
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
5. 07-12-2007 23:33
 
Monitor Status (cont.)
Hello Guest, 
 
I removed the -q option, on both the wget commands. In the section where it gets the cookie information from Rapidshare.com, it displayed a status bar, but on section where it actually downloads the file, it still does not display any status information. Any further support that you can provide would be very much appreciated. 
 
Thank You, 
Funtime
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
6. 04-12-2007 03:07
 
Monitor Status
Hi Funtime... edit Bashget.sh and for the wget commands remove the -q parameter. Wget will cough out its regular progress bar
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
7. 03-12-2007 15:00
 
Montior Status?
I just got Linux re-installed on my laptop, and had spent hours trying to get wget to work with my Rapidshare.com Premium Account. These scripts work wonderfully. I was just wondering if there was a way to modify it so that it will tell me how far into the download the current file is...so that I know how much longer there is before it is completed. Any help you can provide on this would be very much appreciated. 
 
Thanks, 
Funtime
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
8. 28-11-2007 07:36
 
Glitch!!
I have been using this script for quite some time. Thanks a lot!! 
 
Well I found a serious glitch in it. Its was partly due to carelessness on my part. 
 
When the script is used to download a long set of urls and at some point the disk gets full, wget keeps retrying to download again and again. Due to this my rapidshare account was locked. The following was the message I got when I logged in to rapidshare: 
 
“Fraud-Detection, Account xxxxxxxx: This account was used to do harmful actions against the RapidShare infrastructure.” 
 
Here is the reply the help desk sent me: 
 
“This account was used with a script or program that caused damage to our infrastructure. If you use a program that repeatedly accesses RapidShare in any way, please stop that program.” 
 
So here is my question: 
Will using –-tries in wget solve this problem?
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
9. 27-11-2007 14:19
 
Check this url and enjoy
http://www.g-loaded.eu/2007/09/15/use-wget-or-curl-to-download-from-rapidshare-premium/
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
10. 14-11-2007 09:56
 
any help with windows?
can someone tell me how can I work with the wget on windows? what the syntax? 
 
cheers
Guest
 
donatello
11. 14-11-2007 06:03
 
any help with windows?
They are enabled. I checked thrice. Also if I put the link displayed in the output into a browser it downloads the 100mb file as it should. I just don't know why it wont in wget.
Guest
 
Michael
12. 13-11-2007 15:15
 
Micheal
Could it be that your direct download is not enabled in your profile? :x
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
13. 13-11-2007 12:00
 
Micheal
I have direct downloads enabled and I'm sure my username and password are correct but when trying to download I just get 5.7KB files. Any idea as to what is wrong?
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
14. 28-07-2007 12:06
 
Thanks...
Hi Alexandros thanks for the pointer...
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
15. 26-07-2007 22:02
 
Great Tutorial !! But ...
... you should mention that in the File BashGet.sh (line 53) the path for the files is set to "./downloads". 
That means that if you don't have a folder named "downloads" in the place the Script is executed, it simply won't download and abort with an error message. 
Just replace "./downloads" with the place you want to download the files to.  
For example : /media/sdb1/RapidDown 
or /home/Username/downloads
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
16. 15-07-2007 20:44
 
Great Tutorial !! But ...
for parallel downloads one could try: http://code.google.com/p/grapid
Guest
 
anony
17. 15-07-2007 07:20
 
Thanks
Hi Bob, 
 
Thanks for the confirmation :)
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
18. 15-07-2007 06:17
 
Thanks
confirmed... adding --background to wget commands in the script will make it download all files at once :)
Guest
 
bob
19. 15-07-2007 05:50
 
install wget on shared host (re: first p
regardless of being a root user, you can most likely install the latest wget to your home dir.  
 
1. download latest wget 
2. tar -zxvpf wget.x.x.x 
3. ./configure --prefix=$HOME 
4. make 
5. make install 
 
now you must run the correct executable (i.e.): 
/home/XXX/bin/wget -V 
 
edit this in the script supplied here to reflect the correct executable (two instances in the script) 
 
have fun!
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
20. 29-06-2007 01:24
 
Did some thinking
Hi Chop, 
 
I think one way for the script to work with multiple parallel downloads would probably be to wget --background up to x number of downloads, also using trickle to control the bandwidth... Will try this when I got the time... or hopefully someone reads this and beats me to it... hehe... :zzz
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
21. 29-06-2007 01:09
 
Sorry
Hi Chop, 
 
Sorry about that... I misunderstood. Hmmm... I think that will be a tough one. Parallel download will eat up alot of bandwidth. :p
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
22. 28-06-2007 16:46
 
multiple download cont.
Hi again, 
I think you got me wrong. I read the code carefully and sure that it only downloads 1 file in the list then when it finishes, it goes for the next one until the end of the list. What i meant is to download 4 files at the same time. The work around i did is to have 4 shells running but i think it's ugly :( . I'm working on that for a better one. 
 
chop
Guest
 
chop
23. 26-06-2007 06:04
 
Multiple files
Hi Chop, 
 
Your question is answered in the same article step 4 and step 5. 
 
You can list the URLs for the downloads into a text file (e.g. textfile.txt). Then execute ./RapidDownload.sh textfile.txt 
 
Hope this helps ;)
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
24. 26-06-2007 05:07
 
multiple download
Hi there, 
I have a question that how can i download multiple files in one go cuz i think it'll be faster. Or if possible 4 files at a time 
btw, good job :)
Guest
 
chop
25. 25-06-2007 01:27
 
Options
At this point I can only think of this few solutions: 
 
1. If you are using paid hosting then you should ask them to update wget. If they won't its a good time to change ISP. 
 
2. Since its Debian flavor(My favorite) then they are probably using Debian Sarge 3.1. Etch has been released so they should update soon. As far as I know the Etch distributions has the updated Wget. 
 
Can't think of anything else at the moment.  
 
:x  
 
Hope this helps!!
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
26. 24-06-2007 17:18
 
Options
Thank you for answering. But I can not use aptitude for update wget because I want to use your script at my hosting. So, I haven't any admin privileges. Can you recommend for me any solution to solve this problem, please? 
P.S. Debian is current distributive at my hosting.
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
27. 24-06-2007 16:23
 
Sadly yes
Hi Nikita, 
Sadly yes it is required. But good news is... its pretty easy to update to a new version of wget. First update your aptitude to point to update sources. Personally I use debian so... just run: 
 
apt-get update 
apt-get install wget 
 
What distro do you have?
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
28. 24-06-2007 13:08
 
Question
Hello! 
Thank a lot for this article. 
But I have problem with wget (it doesn't support --no-check-certificate). If it is possible to download without this option?
Guest
 
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it

Write Comment
  • Please keep the topic of messages relevant to the subject of the article.
  • Personal verbal attacks will be deleted.
  • Please don't use comments to plug your web site. Such material will be removed.
  • Just ensure to *Refresh* your browser for a new security code to be displayed prior to clicking on the 'Send' button.
  • Keep in mind that the above process only applies if you simply entered the wrong security code.
Name:
E-mail
Homepage
Title:
BBCode:Web AddressEmail AddressBold TextItalic TextUnderlined TextQuoteCodeOpen ListList ItemClose List
Comment:



Code:* Code
I wish to be contacted by email regarding additional comments

Powered by AkoComment Tweaked Special Edition v.1.4.6





Reddit!Del.icio.us!Facebook!Slashdot!Netscape!Technorati!StumbleUpon!Newsvine!Furl!Yahoo!Ma.gnolia!Add this social bookmarking functionality to your website! title=
Last Updated ( Wednesday, 13 June 2007 )
 
< Prev   Next >

Site Search

Sponsors

Google