0

I'm a noob at this so can anyone tell me how to login to a website and fill forms and retrieve results that can be parsed into say .csv. For instance, a website where you enter certain parameters and the server returns products that best match your input parameters. I need to retrieve the list of products with their specifications and parse them into .csv. Doing that requires me to select certain buttons on the webpage which seem to be javascript objects. I tried mechanize but it doesn't seem to work for javascript objects. I would prefer to post my queries in python. Thanks!

haran kumar
  • 327
  • 1
  • 2
  • 9

1 Answers1

0

You have two options I can think of:

1) Figure out what the fields generated by the javascript are named, or their naming scheme, and submit the values directly to the form handler -- eliminating the need to even deal with the input page.

2) Use a framework that emulates a "real" browser and is capable of processing javascript. This question has some suggestions for frameworks, but having never used one, I can't suggest any myself.

Community
  • 1
  • 1
jedwards
  • 29,432
  • 3
  • 65
  • 92
  • Jedwards, thanks for your response, but I just wanted to clarify that the javascript objects are buttons. I need to press them to proceed. I'm not sure if mechanize can actually handle such navigation. The form filling part, I can find the form names and fill them from mechanize but it's the navigation thats tripping me. – haran kumar Jun 20 '13 at 17:11
  • It depends what the javascript buttons are actually doing. If they're just submitting the form, then you can do that with mechanize. If they're doing some sort of processing, you'll need to do that yourself and then submit the form. – jedwards Jun 20 '13 at 17:13
  • So the object is an onclick button. Would you suggest using Selenium then? I'm sorry for asking such silly questions, I'm very new to this. Again thank you for your help – haran kumar Jun 20 '13 at 17:23
  • It depends on what the onclick function does. Sometimes it's just something like `form.submit()`, in which case you don't need to do anything. You should be able to look at the .js source to see what it does. – jedwards Jun 21 '13 at 05:19
  • So the button is not a form.submit(), it navigates to another place so I guess its not a form submission. Using selenium in python is much easier though as it allows you to click these buttons by referencing their id. Thanks for your help anyway jedwards – haran kumar Jun 24 '13 at 16:47