I've logged into the website and got to the page i want to crawl to. Copied the URL from address bar, then connected to that using jsoup as below
public Document getDoc() throws IOException {
String url = "https://.......";
Connection connection = Jsoup.connect(url);
for (Entry<String, String> cookie : cookies.entrySet()) {
connection.cookie(cookie.getKey(), cookie.getValue());
}
Response response = connection.execute();
cookies.putAll(response.cookies());
return response.parse();
}
But this document pulls a login page and what i expect is the dom of page I want to crawl (as i already have active session in chrome). So i believe when I connect using jSoup I need to pass active session data as parameter. Is there a way I can pic active session cookie from windows directory C:\Users\\AppData\Local\Google\Chrome\User Data\Default\Session Storage and use that to fetch active session and hence the page i want and not the login page