Android: How to login into webpage programmatically, using HttpsURLConnection

Question

I'm new in Android (and in Java too), so sorry if my problem is a basic proposition! I have to write an Android app, whitch signs into an aspx webpage in the background, get some data from it, and after that logs out form the webpage. (and do that all programmatically)

Basicly, the procedure likes getting email-list from Gmail:
1. go to 'https://mail.google.com', and signs in
2. click to the "Contacts" (== go to "https://mail.google.com/mail/?shva=1&zx=dzi4xmuko5nz#contacts")
3. fetch the page using HttpsURLConnection (or something like this), and get emails in an (e.g. Map or String) object
4. click to the "Sign out" link

I hope, it's understandable. Looking the internet, I find the solution for only the "fetching part", so that's not a problem. But I don't have any idea about the "clicking part".

  ......
    // Get the connection
    URL myurl = new URL("https://mail.google.com");
    HttpsURLConnection con = (HttpsURLConnection) myurl.openConnection();

    // complete the fields
    con.setRequestProperty("Email","myacc");
    con.setRequestProperty("Passwd","mypass");

    /* 
     * in this part, should make sign in, and go directly to contacts... 
     * I don't have any idea how to do it...
     */

    // for the present, just write out the data
    InputStream ins = con.getInputStream();
    BufferedReader in = new BufferedReader(new InputStreamReader(ins));

    String inputLine;
    while ((inputLine = in.readLine()) != null) {
        Log.d("Page:"," "+inputLine);
    }

    in.close();

    /*
     * And here should be the "Sign out" part
     */
  ......

Any help would be great, Thank You for it! (and sorry, if my english isn't so well...)

EDIT: problem solved. Thank You!

 .......    
    String GMAIL_CONTACTS = "https://mail.google.com/mail/?shva=1#contacts";
    String GMAIL_LOGIN = "https://mail.google.com";

        DefaultHttpClient httpClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost(GMAIL_LOGIN);

        List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(3);
        nameValuePairs.add(new BasicNameValuePair("Email", MY_ACC));
        nameValuePairs.add(new BasicNameValuePair("Passwd", MY_PASS));
        nameValuePairs.add(new BasicNameValuePair("signIn", "Sign In"));

        httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));

        // Execute HTTP Post Request 
        HttpResponse response = httpClient.execute(httpPost);
        Log.d(TAG, "response stat code " + response.getStatusLine().getStatusCode());

        if (response.getStatusLine().getStatusCode() < 400) {

            String cookie = response.getFirstHeader("Set-Cookie")
                    .getValue();
            Log.d(TAG, "cookie: " + cookie);

            // get the contacts page 
            HttpGet getContacts = new HttpGet(GMAIL_CONTACTS);
            getContacts.setHeader("Cookie", cookie);
            response = httpClient.execute(getContacts);

            InputStream ins = response.getEntity().getContent();
            BufferedReader in = new BufferedReader(new InputStreamReader(
                    ins));

            String inputLine;
            while ((inputLine = in.readLine()) != null) {
                Log.d(TAG, " " + inputLine);
            }

            in.close();
        } else {
            Log.d(TAG, "Response error: "
                    + response.getStatusLine().getStatusCode());
        }
 .......

Answer 1

"Clicking" is basically sending a request to a server and displaying the return informations.

1/ find out what url to call for that request (if it is a web page, see firebug for example)

2/ find out what the parameters are, find out if the method is GET or POST

3/ reproduce programmatically.

4/ a "login" phase probably imply the use of a cookie, which the server gives you and that you must send back afterward for each request

However, your approach is wrong. You should not try to login directly to google via url connections. (Also you should use HttpClient). Moreover, request properties are not parameters. They are headers.

I strongly recommend you start with something simpler in order to get comfortable with HTTP in java, GETs, POSTs, parameters, headers, responses, cookies ...

edit

Once you receive the response, you'll want to check that

response.getStatusLine().getStatusCode() < 400

It will tell you that login was successful. (2xx are success, 3xx are moved and such. 4xx are errors in the request, 5xx are server side errors ; Gmail responds 302 to login to suggest redirection to inbox). Then, you'll notice that there is a particular header in the response "Set-Cookie" that contains the cookie you want for further connections so :

String cookie = response.getFistHeader("Set-Cookie");

Then, you should be able to call the request to get the contacts :

HttpGet getContacts = new HttpGet(GMAIL_CONTACTS);
getContacts.setHeader("Cookie", cookie);
response = httpClient.execute(getContacts);
InputStream ins = response.getEntity().getContent();

It should be something like that.

Answer 2

What you are trying to do is parse the Gmail html page. This is wrongn approach as Gmail uses javascript to build the page. Your code would have to emulate browser (execute javascript) for this to work.

If you only need read access to Gmail then use Gmail inbox feed API. This gives you access to unread messages in inbox.

If you need full access then see the Gmail IMAP access. As IMAP is a different protocol then HTTP you'd need to use separate IMAP library for java. See this tutorial.

Answer 3

You should consider using a post request to pass data to the server : Sending POST data in Android

Chaging the properties of a connection has nothing to do with what you want to achieve.

Regards, Stéphane