1

I have a dedicated server running CentOS 5, and I have a computer at home that runs windows 7.

For a projects I need to download a large set of large files on the server. Now the problem I am facing is that I have only a SSH connection to my server, it has no GUI. The files are several gigs and there are a lot.

First downloading them to my pc and them uploading them to the server will cost me ages, as I have a slow download, and a horrible upload speed.

Options

Now the first thing that comes to mind is using wget and then just insert the link.

This is not an option for me, because the files I need to download are downloaded from a website where there is some sort of authentication.

  • I have to log in to the website
  • browse through the pages
  • select the properties for the download, using dropdowns etc
  • hit a download button and the file downloads

The second thing that comes to my mind is using a text-based browser, I used Lynx for some downloads, but this does not work for all the downloads. The website I am downloading from is using heavy Ajax in some parts, so this is not ideal.

Ideal solution

The ideal solution for me with be to use a browser in my windows 7 machine, but the downloads are stored on my dedicated server. Maybe there is some client server solution for browsers.

Anyone have any ideas on the subject?

Saif Bechan
  • 11,000

3 Answers3

5

You can install Firefox on your server (or any other broswer) and use a it remotly.
On Unix the display can be exported over the network. So you can export it to your windows desktop (X11 can be exported inside SSH, makeing it simple to work) into an application such as xming. The result will be Firefox GUI on your windows computer but actualy running on the linux server (so downloaded file are downloaded on the server).

Another similar solution could be to use VNC instead of X11 forwarding.

radius
  • 9,653
  • Wow that is great, this is just what I need. What do you think will be better to use for a server that runs several websites. I guess to use VNC I have to install the GNOME(or other) desktop envoirment. Will this not be booted everytime the server tuns. – Saif Bechan Jun 22 '10 at 19:34
  • @Dennis Williamson you should use SENotify - http://stackapps.com/questions/817 ;) – radius Jun 22 '10 at 19:41
  • @Saif Bechan I've never use VNC for this but using X11 forwarding is probably lighter as you can run Firefox without running any desktop enviroment. VNC will be better in term of performance, X11 forwarding can be a bit slow and give weird color but for browsing a website to download file it should be enough. Give it a try. – radius Jun 22 '10 at 19:45
  • I will take a look at that, having already seen your posting of it, but how is that going to help with my typing? ;-) – Dennis Williamson Jun 22 '10 at 20:11
  • May be you type faster than me, you just have to start typing first ;) – radius Jun 22 '10 at 20:20
  • Ok guys, I have searched google but I can not find any good information on setting up x11 forwarding. I installed firefox on my server. I have also installed xming, but it just sits in my taskbar, I can not figure out how to work with it. Can you maybe give me some basic guidelines on what to do, maybe in another answer post? – Saif Bechan Jun 22 '10 at 20:54
  • How do you connect using SSH ? If you use proxy make a new connexion to you server and check enable X11 Forwarding in Connection->SSH->X11, when logged on, type firefow, it should start – radius Jun 22 '10 at 21:06
  • Ah damn it just popped up out of nowhere, lol. But I can not work in it. It is just frozen. Should I open up certain ports or something, or maybe adjust a setting. – Saif Bechan Jun 22 '10 at 21:08
  • You mean firefox appears but is frozen ? Just wait a while, X11 is a bit slow, especially when displaying new windows – radius Jun 22 '10 at 21:10
  • Yeah i typed firefox and it just poped up out of nowhere. Ok I will give it a few minutes to load. It looks pretty frozen tho. I can see the whole interface of firefox, only the tabs are empty. – Saif Bechan Jun 22 '10 at 21:14
  • Wow cool it is working, it is quite slowly but I thing I can get my work done using this. Awesome man, thanks!! – Saif Bechan Jun 22 '10 at 21:18
2

You can also use text-based browsers, this is the geek's way to surf the web!

Note that javascript isn't supported though.

Here are three text-based browsers available on most distros (either by package or already installed):

w3m-js is an experimental version of w3m integrating some Javascript implementation. It can be worth giving a try.

Weboide
  • 3,365
  • Seems you miss a part of the question ;) – radius Jun 22 '10 at 21:07
  • I had the same problem with Magento. I wanted to download their relatively large package directly onto the shared hosting server I was using but only past releases were accessible from lynx. IMO this is completely unacceptible. There needs to be some kinda of non-JavaScript-based navigation on all websites, but most of all sites that distribute server software--which people often prefer to download from the command-line. – Lèse majesté Jun 22 '10 at 21:14
  • Woops my bad... – Weboide Jun 22 '10 at 21:20
0

Why is the absence of a GUI such a problem? If you really can't manage the command line you can use WinSCP, which gives you a pretty decent file browser over SSH. However, you would be far better off setting up rsync on your Windows machine and using that, as it will minimise the amount of traffic once you have your base set of files downloaded.

John Gardeniers
  • 27,558
  • I am using WinSCP. I don't think you understand my problem. I need to download files on my dedicated server using a web interface that support ajax. I do not need the files on my PC, just on the server. – Saif Bechan Jun 23 '10 at 00:22
  • @Saif - You're right, I didn't understand the question. Perhaps you should clarify it. In particular, leave out any reference to your home computer, which has no relevance to the question. – John Gardeniers Jun 23 '10 at 00:46
  • I want to use my PC, that is the most important part of the question. Other people understand the question just fine. That's why it's answered already. Try to read the answer I accepted, maybe that way you can understand the question. – Saif Bechan Jun 26 '10 at 23:45
  • @Saif, perhaps other people are better mind readers than I am. – John Gardeniers Jun 27 '10 at 07:32
  • Ah yes I know, Sorry for the rude comment and thank you for at least giving it a try. Thanks for mentioning rsync, I need such a program for another project. cheers. – Saif Bechan Jun 30 '10 at 17:06