The website (https) I want to copy requires username and password to access the website. I tried Biterscripting, but it only copies the login page and not the actual web page. I am not sure if there is a way to input the username and password using a script.
"http:// username:password @ address" still gives me the login page and not the actual web page.
Given a list of URLs, I want to copy the contents to text files.
I think you need to use biterscripting commands for IS (Internet Session) like this. (To POST to a page, you need to use the 'ispost' command. To GET a page, you can just use the 'cat' command.)
# Declare variables
var str page
# Start internet session, named s, user agent Mozilla.
isstart s "" "Mozilla/5.0"
# If the site is https://www.abc.def, connect to site.
isconnect s "https://www.abc.def" > $page
# The site's index page is in variable $page.
# Suppose the login form is in this format -
# <form name="login" action="login.php" method="post">
# Account: <input type="text" name="login"><
# Password: <input type="password" name="pswd">
# <input type="submit" value="submit">
# </form>
# Login by posting "login=me&pswd=mypassword&submit=submit" to login.php.
ispost s "login.php" "login=me&pswd=mypassword&submit=submit" > $page
# "me" and "mypassword" are values of login and password.
The page after the login is now in variable $page. I thought this is the page you are looking for.
Hope this helps.
What language do you use?
First you have to login, and then retrieve the result page.
You can use cURL to achieve this. Here you have a simple example using cURL with PHP:
php curl: I need a simple post request and retrival of page example
