1

I'm learning Scraping with python + selenium.

Wish

I want to login google and chrome when I use selenium. and popele say It's detecting a automation mode for bot detection. So you should hide you are using selenium.

That's why I wrote this code.

Here is my code

import time
from selenium import webdriver
from selenium.webdriver.support.select import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from requests.exceptions import Timeout
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By

options = webdriver.ChromeOptions()
options.add_argument('--window-size=1280,800')
options.add_argument('--user-data-dir=/Users/username/Library/Application Support/Google/Chrome/')
options.add_argument('--profile-directory=Profile 3')
options.add_argument('--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36')
options.add_experimental_option('useAutomationExtension', False)
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_experimental_option("excludeSwitches", ['enable-automation'])
driver = webdriver.Chrome("selenium/chromedriver", options=options)
driver.get('https://www.google.co.jp/')
time.sleep(1000)

My user profile is not working. When using selenium, It doesn't show me as signed in.

chrome://version

enter image description here

Tried answering code

I got this error

selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: unrecognized chrome option: excludeSwitches

Changed My code

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.select import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from requests.exceptions import Timeout
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By

options = webdriver.ChromeOptions()
options.add_argument('--window-size=1280,800')
options.add_argument('--user-data-dir=/Users/wakasugiwataru/Library/Application Support/Google/Chrome/')
options.add_argument('--profile-directory=Profile 3')
options.add_experimental_option("debuggerAddress", "127.0.0.1:1557")
options.add_argument('--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36')
options.add_experimental_option('useAutomationExtension', False)
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_experimental_option("excludeSwitches", ['enable-automation'])
driver = webdriver.Chrome("selenium/chromedriver", options=options)
driver.get('https://www.google.co.jp/')
time.sleep(1000)
wataru
  • 121
  • 6

2 Answers2

0

There are two ways:

use the userdata folder that you used with normal browser, when you copy the profile to a new folder copy the entire user data folder not only the profile. Your login informations will be in the user data not in profile

options.add_argument('user-data-dir=/Users/username/Library/Application Support/Google/Chrome')
options.add_argument('profile-directory=Profile 3')

you don't need to provide -- with add_argument above code will use the exiting profile 3

But this will not allow to login to google in your case because you copied only the profile not the user data folder

Another way to connect is to connect to a already logged in browser session:

follow below steps :

  1. open chrome.exe as :

Connecting to an existing chrome browser from selenium: close all chrome browsers and then, start the Chrome browser in debug mode by opening cmd and running chrome.exe with — remote-debugging-port argument

"<path>\chrome.exe" --remote-debugging-port=1557

Now open any other browser and navigate to localhost:1557 to check that the browser was indeed opened with that port Now in selenium code use chrome experimental option to connect to this chrome option

ChromeOptions options = new ChromeOptions();
options.add_experimental_option("debuggerAddress", "127.0.0.1:1557");
browser=new ChromeDriver(options);
PDHide
  • 18,113
  • 2
  • 31
  • 46
  • `"\chrome.exe" --remote-debugging-port=1559` and `options.add_experimental_option("debuggerAddress", "127.0.0.1:1557");` is that same meaning of both code? `"\chrome.exe" --remote-debugging-port=1559` code is for using in terminal?? – wataru Dec 06 '20 at 00:10
  • I tried your answer but It didn't work. I got error. I added changed code and error. please check that something wrong. – wataru Dec 06 '20 at 00:20
  • First code is to open chrome with the specified debug portal that is run in terminal , and second code is python code to connect to that already opened browser – PDHide Dec 06 '20 at 02:20
  • That means First should I open a chrome from terminal like this `open -a '/Applications/Google Chrome.app' --remote-debugging-port=1559` and second try to run a code right?? – wataru Dec 06 '20 at 08:51
  • I run above your code but nothing just happen. it was like a server listening. – wataru Dec 06 '20 at 08:52
  • /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222& – PDHide Dec 06 '20 at 08:54
  • Change it to 1559 , after opening try opening localhost:1559 in browser and see you can view something – PDHide Dec 06 '20 at 08:55
  • Now login to google and then run the code , now what ever you do in selenium will happen in the browser which you just logged in. Also login to google – PDHide Dec 06 '20 at 08:57
  • Incod winstead 1559 I had mentioned 1557 , I have changed everything to 1557 – PDHide Dec 06 '20 at 12:25
0

When you access Google Chrome Browser for the first time after installation, initially Chrome offers you a Guest Login and Chrome would ask you to Sign in with your Google Account credentials to access your bookmarks, history, passwords, and other settings synced up within your Google Account.

ChromeGuest

At this stage you are accessing Google Chrome Browser as a guest user without any access to bookmarks, history, passwords, and other settings. To access your bookmarks, history, passwords, and other settings you have to login with your Google Account credentials. Once you successfully login you will be able to see the logged in icon at the top right hand corner.

ChromeLoggedIn

Now, each time you would access Google Chrome Browser, all your bookmarks, history, passwords, and other settings would be synced up within your Google Account. Hence you find yourself logged in.


Accessing Chrome with Selenium

When you spin up a Selenium driven ChromeDriver initiated Browsing Context the Browsing Context is opened with a Guest Login. Hence you are not shown as signed in.

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • I didn't know about guest mode thank you so much good infomation. – wataru Dec 06 '20 at 14:03
  • @wataru Actually, I tried to answer your initial query within your first version of your question. No doubt you updated your question now without understanding the basic ideas. Good luck :) – undetected Selenium Dec 06 '20 at 14:33
  • Sorry some one edit my question and I agreed that editing. What do you mean? There is another solution?? – wataru Dec 07 '20 at 02:52
  • @wataru You agreed to a bad edit to your question and accepted the answer which doesn't address your actual query. So question is what have you gained? – undetected Selenium Dec 07 '20 at 06:44
  • I can use the browser which logged into chrome with selenium. – wataru Dec 07 '20 at 06:46
  • @wataru You seem to be double lucky. Your initial version of your question was, why aren't you logged in using Selenium and in return you got an answer how login through logged in chrome. All the best. – undetected Selenium Dec 07 '20 at 07:00
  • Yeah you're right I'm so sorry. Your answer is correct first my question. – wataru Dec 07 '20 at 07:16