0

I'm trying to navigate to the next page after my successful login with the script below. 

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

url = 'https://login.morganstanleyclientserv.com/'
chromedriver = 'path to chromedriver.exe'
driver = webdriver.Chrome(chromedriver)
driver.get(url)

username = driver.find_element_by_name('userid')
username.send_keys('xxxxx')
password = driver.find_element_by_name('password')
password.send_keys('xxxxx')
password.send_keys(Keys.RETURN)

After it hits 'enter' the browser shows the spinning wheel on the login page but never goes away and nothing happens. I've tried using various WebDriverWait's ie: WebDriverWait(driver,10).until(EC.presence_of_element_located((By.ID,"firstSecCode"))) but still have not made any progress.

This scrape has been causing me headaches when using casperjs as well: Script is timing out and not finding selector

EDIT:

I've found a temporary workaround for this, but I'm still not satisfied as it involves a manual step still. What I have to do is clear my cache & cookies, then go to the page and login and wait for the page to load. After this, the script runs no problem with my imported chrome profile that contains the cookie(s) collected from the manual login.

Aaron
  • 99
  • 1
  • 9
  • What happens if you click the "login / submit" button instead of hitting the enter key? – Goralight Dec 19 '17 at 16:33
  • does it login successfully? – Stack Dec 19 '17 at 16:38
  • Are there any `iframes` in page source, if it logs in successfully? – Stack Dec 19 '17 at 16:39
  • @Goralight I get the same result with `driver.find_element_by_id('btnLogin').click()` – Aaron Dec 19 '17 at 16:48
  • @Stack I'm using the exact same parameters as I do when I manually log in. Same loading wheel comes up but then the next page loads when I manually log in. I see there are iframes on the login screen but I do not seen any on the first page after a successful login – Aaron Dec 19 '17 at 16:50
  • @Stack Well.. at least when I view the page source on the second page and ctrl-f for 'frame' nothing comes up. However when I just inspect an element on the page it has the same frame as the login page. – Aaron Dec 19 '17 at 16:55
  • Put a breakpoint after `driver.get`, start test in debug mode and wait for breakpoint; when test paused at breakpoint, open developer tools and go to Net tab. Let script to run further and review Net tab after it's done. Compare it to manual login. You should see some difference in requests issued to a server. – timbre timbre Dec 19 '17 at 17:24
  • @KirilS. After monitoring the network tab I can see that the script is getting hung up on an 'authenticate' XHR object after the login button is clicked. Also I see a XSRF token when manually logging in that does not come up when the script is run – Aaron Dec 19 '17 at 18:37
  • when you login manually, is it on the same **instance** of the browser? or an opposite: try to start selenium chrome with existing default profile (like explained here: https://stackoverflow.com/questions/14480717/load-chrome-profile-using-selenium-webdriver-using-java). It could be that you have some cors/certificate/cross-site exception with new profile, which is not happening with default one – timbre timbre Dec 19 '17 at 19:27
  • @KirilS. That did the trick! I am still curious as to what it actually needed though. – Aaron Dec 19 '17 at 19:57
  • I think you either manually accepted some certificate, or default profile has some security exception, but not on new profile selenium creates. You can try opening chrome incognito window, navigate to site and login, but pay attention to any warnings/confirmations you receive. You may need to add them to selenium profile too. – timbre timbre Dec 19 '17 at 20:57
  • @KirilS. Opened in icognito window and logged in with no cert messages or anything. Also logged in from a non-work related machine that has never been to the site and logged in with no visible cert/auth. – Aaron Dec 20 '17 at 13:00
  • @KirilS.came in this morning and code would not work with selenium until after I manually opened chrome and logged in.. which means my chrome profile is definitely getting authenticated somehow behind the scenes. Before I manually log in, the script gets hung up on a an Post request XHR object – Aaron Dec 21 '17 at 13:06
  • Back to working on this...I've figured out that I have a cookie that authenticates me upon logging in. I have been able to login with and without importing a chrome profile. When not using a Chrome profile I have to inject the cookie (using pickle), but after a while I'm guessing it expires because with both methods I have to go manually log in to refresh it for the script to work. – Aaron Jan 05 '18 at 18:24

0 Answers0