用于抓取文章的报纸API

问题描述

我已使用python的报纸3k api抓取文章。我无法取消“印度时报”的文章，而从响应其余文章中获得的发布日期为null则会提供适当的文章。

article = Article(url)
article.download()
article.parse()
result=vars(article)
print(result['publish_date']) 

解决方法

报纸的当前版本无法从“印度时报” HTML代码中提取“发布日期”，因为该日期在 script 标记内。您可以使用请求和 BeautifulSoup 提取此日期。后者嵌入在报纸中。我还指出，关键字位于meta标记中，因此 Newspaper 无法提取这些关键字。我还添加了一些代码来提取关键字。希望下面的代码可以帮助您查询《印度时报》上的文章。如有任何问题，请告诉我。

import requests
import re as regex
from newspaper import Article
from newspaper.utils import BeautifulSoup

base_url = 'https://timesofindia.indiatimes.com/business/india-business/govt-working-to-reduce-e-vehicle-tax-niti-aayog-ceo/articleshow/78210495.cms'

raw_html = requests.get(base_url)
soup = BeautifulSoup(raw_html.text,'html.parser')

# parse date published
data = soup.findAll('script')[1]
find_date = regex.search(r'datePublished.{3}\d{4}-\d{2}-\d{2}',data.string)
date_published = find_date.group().split('"')[2]

# parse other elements using Newspaper
article = Article('')
article.download(raw_html.content)
article.parse()
article_tags = article.tags
article_content = article.text
article_title = article.title

# parse keywords
article_meta_data = article.meta_data
article_keywords = sorted({value for (key,value) in article_meta_data.items() if key == 'keywords'})