Selenium How to scrape website table when website pagination relies on Javascript?



I’m pretty new to webscraping and As of Now I only came across scraping website that had pagination link in their next button. But I came to situation where website pagination button link doesn’t change when I’m clicking on the next button.

driver = webdriver.Chrome(ChromeDriverManager().install())

but this website doesnot change link on loading the next button.
Any help or hint I can do to grab these pagination table data?

Ps: Right click is disabled on above website (I used ctrl+shift+c) to enable it


I just tried the below-mentioned java code for the above-mentioned website and I am able to click the next button and also able to get the data.

int rowSize=driver.findElements(By.xpath("//tbody//tr")).size();
int pageSize=Integer.parseInt(driver.findElement(By.xpath("//li[@class='ellipsis']/following-sibling::li//span[@class='show-for-sr']/following-sibling::span")).getText());
for(int page=1;page<pageSize;page++)

for (int i=1;i<rowSize;i++)
    for(int col=1;col<=8;col++)
    System.out.print(driver.findElement(By.xpath("//tbody//tr["+i+"]/td["+col+"]")).getText()+" | ");

Answered By – Yosuva Arulanthu

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave A Reply

Your email address will not be published.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More