How to post login data using jsoup

Question

I want to simulate a login and then scrape some content while maintaining the logged-in session. I've got it working using HtmlUnit, but was looking into Jsoup since I want it to work on Android as well.

loginUrl = https://www.internetvas.slt.lk/SLTVasPortal-war/login/j_security_check

usageUrl = https://www.internetvas.slt.lk/SLTVasPortal-war/application/index.jsp?page=usage ; this is the page that I need extract information from.

[UPDATE -- ]

Thanks to greenapps, I've finally been able to login successfully. Just had to change the loginUrl, from ../login/login.jsp to ../login/j_security_check

Here's the working code:

Connection.Response response = Jsoup.connect(loginUrl)
                                  .data("j_username", myUserName)
                                  .data("j_password", myPassword)
                                  .method(Method.POST).execute();

Document usageDoc = Jsoup.connect(usageUrl).cookies(response.cookies()).get();

System.out.print(usageDoc.toString());

But now the problem is, once logged in, some content that are generated through JS are shown blank or empty. Is there a workaround to get those values show up?

@greenapps [HtmlUnit. Java GUI-Less browser](http://htmlunit.sourceforge.net/) — Gayan Weerakutti, Apr 22 '15 at 09:51
It does not seem to load the desired page. Well you are trying to load two pages. Please explain better what happens and not happens. What kind of 'response' do you get from first page? — greenapps, Apr 22 '15 at 09:55
@greenapps [loginUrl](https://www.internetvas.slt.lk/SLTVasPortal-war/login/login.jsp) is the login page. Once logged in I should be able to load the usage page through the [usageUrl](https://www.internetvas.slt.lk/SLTVasPortal-war/application/index.jsp?page=usage) (which in case the desired page) — Gayan Weerakutti, Apr 22 '15 at 09:58
But you did not answer my question: `What kind of 'response' do you get from first page?`. So what is 'best'? — greenapps, Apr 22 '15 at 10:46
What you need to do is save the cookie it gives you, and send it with the next request. See http://stackoverflow.com/questions/6432970/jsoup-posting-and-cookie. Tell me if that works. — Jonas Czech, Apr 22 '15 at 10:54
@JonasCz I'm passing the cookies using `response.cookies()` as you could see in the second statement. — Gayan Weerakutti, Apr 22 '15 at 11:11
Sorry, did't see that. Did you try http://stackoverflow.com/a/10533408/4428462 ? — Jonas Czech, Apr 22 '15 at 11:14
@JonasCz yup I did. Please have a look on source the code for the submit button. I updated the question btw. — Gayan Weerakutti, Apr 22 '15 at 11:18
You should post the complete
of course so we can see all the parameters. — greenapps, Apr 22 '15 at 12:17
Your loginUrl should be `https://www.internetvas.slt.lk/SLTVasPortal-war/login/j_security_check`. As you can see in `
`.Maybe you have to add a .jsp. I don't know. — greenapps, Apr 22 '15 at 12:22
@greenapps yeah, that does work. but .. please see the edited post — Gayan Weerakutti, Apr 22 '15 at 14:32
`content that are generated through JS`. What is JS? And where do we see content here? — greenapps, Apr 22 '15 at 14:36
Yes of course the java scipt will not be executed by Jsoup. You better use HtmlUnit ;-). — greenapps, Apr 22 '15 at 14:39
HtmlUnit does not support Android. That's why I'm checking into Jsoup — Gayan Weerakutti, Apr 22 '15 at 14:40

score 1 · Answer 1 · answered Apr 22 '15 at 14:34

1

Your loginUrl should be https://www.internetvas.slt.lk/SLTVasPortal-war/login/j_security_check1. As you can see in <form id="login_form" action="j_security_check" method="post">.

answered Apr 22 '15 at 14:34

greenapps

11,154
2
16
19

Url should be `j_security_check` and not `_check1`, right? – Gayan Weerakutti Apr 22 '15 at 14:42

How to post login data using jsoup

1 Answers1