使用动态鼠标悬停事件抓取网站

问题描述：

我正在尝试抓取鼠标悬停事件动态生成的数据。我想从

https://slushpool.com/stats/?c=btc的哈希率分布图中获取信息，该图是滚动每个圆圈时生成的。

下面的代码从网站获取 html 数据，并返回表格，鼠标经过圆圈时表格会被填充。但是，我一直没能弄清楚如何触发每个圆圈的 mouseover 事件来填充表格。

from lxml import etree
from xml.etree import ElementTree
from selenium import webdriver

driver_path = "#Firefox web driver"
browser = webdriver.Firefox(executable_path=driver_path)
browser.get("https://slushpool.com/stats/?c=btc") 


page = browser.page_source #Get page html 
tree = etree.HTML(page) #create etree

table_Xpath = '/html/body/div[1]/div/div/div/div/div[5]/div[1]/div/div/div[2]/div[2]/div[2]/div/table'

table =tree.xpath(table_Xpath) #get table using Xpath

print(ElementTree.tostring(table[0])) #Returns empty table. 
#Should return data from each mouseover event

有没有办法触发每个圆圈的鼠标悬停事件，然后提取生成的数据。

提前感谢您的帮助！

解决方案 1：

要触发每个圆圈的鼠标悬停事件，您必须引发WebDriverWait，visibility_of_all_elements_located()并且可以使用以下定位器策略：

• 代码块：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
chrome_options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')
driver.get("https://slushpool.com/stats/?c=btc")
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h1//span[text()='Distribution']"))))
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//h1//span[text()='Distribution']//following::div[1]/*[name()='svg']//*[name()='g']//*[name()='g' and @class='paper']//*[name()='circle']")))
for element in elements:
    ActionChains(driver).move_to_element(element).perform()

• 浏览器快照：

解决方案 2：

这是circle你所说的定位器：

.find_element_by_css_selector('._1p0PmxVw._3GzjmWLG')

但它会因为鼠标悬停效果而改变，如下：

.find_element_by_css_selector('._1p0PmxVw._3GzjmWLG._1suU9Mx1')

因此，您需要等到每次移动时元素都发生变化。

最重要的是如何检查悬停元素，然后您可以得到以下内容：

并导致您想要获取数据的元素出现：

xpath: //div[@class="_3jGHi0co _1zbokARu" and contains(@style,"display: block")]

您可以使用ActionChains来移动元素。

最后你可以尝试下面的代码：

browser.get('https://slushpool.com/stats/?c=btc')
browser.maximize_window()

#wait all circle
elements = WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, '._1p0PmxVw._3GzjmWLG')))
table = browser.find_element_by_class_name('paper')

#move perform -> to table
browser.execute_script("arguments[0].scrollIntoView(true);", table)

data = []
for circle in elements:
    #move perform -> to each circle
    ActionChains(browser).move_to_element(circle).perform()
    # wait change mouseover effect
    mouseover = WebDriverWait(browser, 5).until(EC.visibility_of_element_located((By.XPATH, '//div[@class="_3jGHi0co _1zbokARu" and contains(@style,"display: block")]')))
    data.append(mouseover.text)

print(data[0])
print(data)

导入后：

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains

控制台输出：

• 第一条数据 > data[0]

536.9 Ph/s - 1.074 Eh/s

用户数 2

平均哈希率 546.1 Ph/s

群组哈希率 1.092 Eh/s

• 所有数据 >data

[u'536.9 Ph/s - 1.074 Eh/s
User Count 2
Average Hash Rate 546.9 Ph/s
Group Hash Rate 1.094 Eh/s', u'67.11 Ph/s - 134.2 Ph/s
User Count 14
Average Hash Rate 91.27 Ph/s
Group Hash Rate 1.278 Eh/s', u'67.11 Ph/s - 134.2 Ph/s
User Count 14
Average Hash Rate 91.27 Ph/s
Group Hash Rate 1.278 Eh/s', u'16.78 Ph/s - 33.55 Ph/s
User Count 23
Average Hash Rate 23.36 Ph/s
Group Hash Rate 537.2 Ph/s', u'8.389 Ph/s - 16.78 Ph/s
User Count 33
Average Hash Rate 11.80 Ph/s
Group Hash Rate 389.4 Ph/s', u'4.194 Ph/s - 8.389 Ph/s
User Count 67
Average Hash Rate 5.704 Ph/s
Group Hash Rate 382.2 Ph/s', u'2.097 Ph/s - 4.194 Ph/s
User Count 137
Average Hash Rate 2.959 Ph/s
Group Hash Rate 405.3 Ph/s', u'1.049 Ph/s - 2.097 Ph/s
User Count 233
Average Hash Rate 1.475 Ph/s
Group Hash Rate 343.7 Ph/s', u'1.049 Ph/s - 2.097 Ph/s
User Count 233
Average Hash Rate 1.475 Ph/s
Group Hash Rate 343.7 Ph/s', u'524.3 Th/s - 1.049 Ph/s
User Count 397
Average Hash Rate 731.4 Th/s
Group Hash Rate 290.4 Ph/s', u'262.1 Th/s - 524.3 Th/s
User Count 745
Average Hash Rate 360.3 Th/s
Group Hash Rate 268.4 Ph/s', u'131.1 Th/s - 262.1 Th/s
User Count 1479
Average Hash Rate 182.7 Th/s
Group Hash Rate 270.1 Ph/s', u'65.54 Th/s - 131.1 Th/s
User Count 2351
Average Hash Rate 92.47 Th/s
Group Hash Rate 217.4 Ph/s', u'32.77 Th/s - 65.54 Th/s
User Count 3107
Average Hash Rate 47.23 Th/s
Group Hash Rate 146.8 Ph/s', u'16.38 Th/s - 32.77 Th/s
User Count 3380
Average Hash Rate 25.24 Th/s
Group Hash Rate 85.30 Ph/s', u'8.192 Th/s - 16.38 Th/s
User Count 4276
Average Hash Rate 13.00 Th/s
Group Hash Rate 55.57 Ph/s', u'4.096 Th/s - 8.192 Th/s
User Count 540
Average Hash Rate 5.953 Th/s
Group Hash Rate 3.215 Ph/s', u'2.048 Th/s - 4.096 Th/s
User Count 284
Average Hash Rate 3.193 Th/s
Group Hash Rate 906.8 Th/s', u'1.024 Th/s - 2.048 Th/s
User Count 226
Average Hash Rate 1.368 Th/s
Group Hash Rate 309.1 Th/s', u'512.0 Gh/s - 1.024 Th/s
User Count 136
Average Hash Rate 774.4 Gh/s
Group Hash Rate 105.3 Th/s', u'256.0 Gh/s - 512.0 Gh/s
User Count 116
Average Hash Rate 401.5 Gh/s
Group Hash Rate 46.57 Th/s', u'128.0 Gh/s - 256.0 Gh/s
User Count 75
Average Hash Rate 186.4 Gh/s
Group Hash Rate 13.98 Th/s', u'64.00 Gh/s - 128.0 Gh/s
User Count 78
Average Hash Rate 96.39 Gh/s
Group Hash Rate 7.518 Th/s', u'32.00 Gh/s - 64.00 Gh/s
User Count 70
Average Hash Rate 45.68 Gh/s
Group Hash Rate 3.198 Th/s', u'16.00 Gh/s - 32.00 Gh/s
User Count 48
Average Hash Rate 23.37 Gh/s
Group Hash Rate 1.122 Th/s', u'8.000 Gh/s - 16.00 Gh/s
User Count 62
Average Hash Rate 11.91 Gh/s
Group Hash Rate 738.5 Gh/s', u'4.000 Gh/s - 8.000 Gh/s
User Count 153
Average Hash Rate 3.078 Gh/s
Group Hash Rate 471.0 Gh/s']