无法获取跨度类文本

问题描述

请找到附件图片。我想获取图像中突出显示的部分， 我想获取此属性 1 · Trending,#tuesdaymotivations,7,750 Tweets .

请指教。 获取的 URL=https://twitter.com/explore/tabs/trending

from selenium import webdriver

url = 'https://twitter.com/explore/tabs/trending'

# scrolling and scraping tweets

driver = webdriver.Chrome('/chromedriver')
driver.get(url)

trends = driver.find_element_by_xpath('//div[@data-testid="trends"]')
trend = trends[0]
trend.find_element_by_xpath('.//span').text

输出：

Traceback (most recent call last):
  File "/home/PycharmProject/Twitter_trending/ss.py",line 13,in <module>
    trends = driver.find_element_by_xpath('//div[@data-testid="trends"]')
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py",line 394,in find_element_by_xpath
    return self.find_element(by=By.XPATH,value=xpath)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py",line 976,in find_element
    return self.execute(Command.FIND_ELEMENT,{
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py",line 321,in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/errorhandler.py",line 242,in check_response
    raise exception_class(message,screen,stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[@data-testid="trends"]"}
  (Session info: chrome=90.0.4430.72)

预期输出：

1
·
Trending
#tuesdaymotivations
7,750 Tweets

2
·
Politics · Trending
#ModiResignorRepeal
37.6K Tweets
3
·
Trending
#ThankfulTuesday
12.8K Tweets...etc

解决方法

尝试使用 JavaScript 获取文本。使用此代码。

span = trend.find_element_by_xpath('.//span')
text = driver.execute_script("return arguments[0].innerText",span)

此方法还可以提供来自嵌套元素的文本。仍然无法正常工作，然后尝试使用 css 选择器而不是 xpath。

trends = driver.find_elements_by_css_selector('div[data-testid="trends"]')
trend = trends[0]
span = trend.find_element_by_xpath('.//span')
text = driver.execute_script("return arguments[0].innerText",span)