How to handle 406 response in Selenium?

0

Issue

I’m trying to scrape a website using Selenium. I’m able to open most of the websites. But there are some websites that give me the response 406 Not Acceptable. Instead of receiving the actual website I only get a 406 Not Acceptable message in the browser.

I’m currently using the following Python code (using a Mac):

from selenium import webdriver
DRIVER_PATH = '/Users/User123/Downloads/chromedriver 2'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get('https://www.WEBSITE.com/')

I tried to open the same website with the requests and beautiful soup libraries just out of curiosity and there it worked (I didn’t receive that error). Nevertheless, I need selenium for my use case due to dynamic content on the page.

I googled it but didn’t find any resources that cover the selenium vs. 406 issue.

Any ideas?

Solution

They might be blocking Selenium? It might not be the case but try using this driver: https://github.com/ultrafunkamsterdam/undetected-chromedriver

Answered By – bushcat69

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave A Reply

Your email address will not be published.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More