I'm using RSelenium to automatically scroll down a social media website and save posts. Sometimes I get to the bottom of the webpage and no more posts can be loaded as no more data is available. I just want to be able to check if this is the case so I can stop trying to scroll.
How can I tell if it's possible to continue scrolling in RSelenium? The code below illustrates what I'm trying to do - I think I just need help with the "if" statement.
FYI there's a solution for doing this in Python here (essentially checking if page height changes between iterations), but I can't figure out how to implement it (or any another solution) in R.
# Open webpage
library(RSelenium)
rD = rsDriver(browser = "firefox")
remDr = rD[["client"]]
url = "http://ift.tt/2gMtIdz"
remDr$navigate(url)
# Keep scrolling down page, loading new content each time.
ptm = proc.time()
repeat {
remDr$executeScript("window.scrollTo(0,document.body.scrollHeight);")
Sys.sleep(3) #delay by 3sec to give chance to load.
# Here's where i need help
if([INSERT CONDITION TO CHECK IF SCROLL DOWN IS POSSIBLE]) {
break
}
}
Aucun commentaire:
Enregistrer un commentaire