如何使用python和美丽的汤来刮标题和描述？

编程问答 2022-05-26

问题描述

问题概述：

链接：https://www.bobfinancial.com/eterna.jsp

在详细信息部分：基本上我想要所有积分。

详情：

[ #This is an array of Strings... 
    
"Milestone Rewards: Earn 10,000 bonus reward points on spending ₹ 50,000 within 60 days & 20,000 bonus reward points on spending ₹ 5,00,000 in a year.","Fuel Surcharge Waiver*: 1% fuel surcharge waiver at all fuel stations across India on transactions between Rs.400 and Rs.5,000 (Max. Rs. 250 per statement cycle). Note -No Reward Points are earned on fuel transactions.","Core Reward Points:  3 reward points for every ₹ 100 spent on any other category.","Redeem reward points for cashback: Redeem your reward points as cashback and other exciting options.All your accumulated reward points can be redeemed for cashback @ 1 reward point = ₹ 0.25","In-built insurance cover: Get free Personal Accidental Death Cover to ensure financial protection of your family (Air: 1 Crs,Non-Air: 10 Lakhs) ","Zero liability on lost card. Report loss of card immediately to ensure zero liability on any fraudulent transactions","Easy EMI option. Convert purchase of > 2,500/- on your card into easy EMIs of 6/12 months"
]

实际问题：

抓取链接：https://www.online.citibank.co.in/credit-card/rewards/citi-rewards-credit-card?eOfferCode=INCCCCTWAFCTRELM

我从该链接中想要的项目在以下图片中：

citi_1

citi_2

from urllib.request import urlopen
from bs4 import BeautifulSoup
import json,requests

details = []

url = ['https://www.online.citibank.co.in/credit-card/rewards/citi-rewards-credit-card?eOfferCode=INCCCCTWAFCTRELM']
html = urlopen(url[0])
soup = BeautifulSoup(html,'lxml')
addr= soup.find_all('span',class_ = 'm-bottom-0 header-4 font-weight-bold display-text')
print(addr)

我写了上面的代码，得到这个输出后卡住了：

[]

我不知道如何进一步获取我想要的信息，非常感谢任何帮助。

解决方法

您可以只搜索具有 display-text 类的标题。正文就是下面的 <span> 条目。这避免了任何可能会破坏其他类似页面的硬编码偏移。例如：

from bs4 import BeautifulSoup
import requests
    
url = ['https://www.online.citibank.co.in/credit-card/rewards/citi-rewards-credit-card?eOfferCode=INCCCCTWAFCTRELM']
html = requests.get(url[0])
soup = BeautifulSoup(html.content,'lxml')
data = []

for span in soup.select('span.m-bottom-0.display-text.font-weight-bold'):
    data.append([span.get_text(strip=True),span.find_next('span').get_text(strip=True)])

print(data)

这将为您提供一个包含标题和描述的数据结构，如下所示：

[
    ['Citi Rewards Credit Card','Make your shopping more rewarding'],['Make your shopping more rewarding','Get up to 2500 welcome reward points*'],['Accelerated rewards','Earn minimum 1 reward point for every 125 on all purchases.\nEarn 10X reward points at online and physical department and apparel stores.'],['Bonus rewards','Get 300 bonus points on card purchase of INR 30,000 or more in a month'],['Evergreen reward points',"Redeem now or keep collecting – it's a choice you have because your reward points will never expire."],['Tap n Pay','Now pay the easy way by enabling contactless payments on your Citi credit card.Click hereto see how.'],['Rewards',"Redeem rewards: Redeem now or keep collecting – it's a choice you have.Click hereto see how"],['Travel and Lifestyle Services','Contact the Travel and Lifestyle specialist to create and plan an experience that will help you enjoy the best in life. Simply call 0008 004 407 027 for Mastercard®cardholders or 1800-114-999 for Visa cardholders.']
]

您可以使用 print(data[2:])

跳过前两个结果（如果不需要）

它可以扩展到多个 URL 并且可以合并标题和描述：

from bs4 import BeautifulSoup
import requests
    
urls = [
    'https://www.online.citibank.co.in/credit-card/fuel-card/citi-indianoil-card?eOfferCode=INCCCCTWAFCTIOPLM','https://www.online.citibank.co.in/credit-card/rewards/citi-rewards-credit-card?eOfferCode=INCCCCTWAFCTRELM',]

for url in urls:
    html = requests.get(url)
    soup = BeautifulSoup(html.content,'lxml')
    data = []

    for span in soup.select('span.m-bottom-0.display-text.font-weight-bold'):
        data.append(f'{span.get_text(strip=True)}: {span.find_next("span").get_text(strip=True)}')

    # Display the list of strings
    print('\n'.join(data[1:]))
    print()

为您提供以下两个 URL 的输出：

Accelerated Rewards on Fuel spends: 4 Turbo points / Rs. 150 spent &1% fuel surcharge reversalon fuel purchase atauthorized IndianOil outlets^
Earn on all Daily Spends: 2 Turbo points / Rs. 150 spent on groceries and supermarkets#1 Turbo point / Rs. 150 on all other spends.#Clickherefor the details
Redeem Instantly: 1 Turbo Point = Re. 1 of free fuelRedeem your Turbo points instantly via SMS for freeFuel atauthorized IndianOil outlets^
Tap n Pay: Now pay the easy way by enabling contactless payments on your Citi credit card.Click hereto see how.
Redeem Rewards!: Choose from a range of redeeming options including fuel,holidays,air miles,cash back and more.Click hereto see how
Dining Privileges: Up to 20% savings across participating restaurants. To find a Citibank partner restaurant near you,click here
Travel and Lifestyle Services: Contact the Travel and Lifestyle specialist to create and plan an experience that will help you enjoy the best in life. Simply call 0008 004 407 027 for Mastercard cardholders or 1800-114-999 for Visa cardholders.

Accelerated rewards: Earn minimum 1 reward point for every 125 on all purchases.
Earn 10X reward points at online and physical department and apparel stores.
Bonus rewards: Get 300 bonus points on card purchase of INR 30,000 or more in a month
Evergreen reward points: Redeem now or keep collecting – it's a choice you have because your reward points will never expire.
Tap n Pay: Now pay the easy way by enabling contactless payments on your Citi credit card.Click hereto see how.
Rewards: Redeem rewards: Redeem now or keep collecting – it's a choice you have.Click hereto see how
Travel and Lifestyle Services: Contact the Travel and Lifestyle specialist to create and plan an experience that will help you enjoy the best in life. Simply call 0008 004 407 027 for Mastercard®cardholders or 1800-114-999 for Visa cardholders.
,

试试这个，希望有帮助

from urllib.request import urlopen
from bs4 import BeautifulSoup
import json,requests

details = []

url = ['https://www.online.citibank.co.in/credit-card/rewards/citi-rewards-credit-card?eOfferCode=INCCCCTWAFCTRELM']
html = requests.get(url[0])
print(html.status_code)
soup = BeautifulSoup(html.content,'lxml')
x = soup.select('span.m-bottom-0')

addr= soup.select('span.m-bottom-0')[12:20] # number of span
for d in addr:
    print(d.get_text())

addr= soup.select('span.m-bottom-0')[58:70]
for d in addr:
    print(d.get_text()) # get_text() method for inner tag text
beautifulsouppythonpython-requestsscrapinghubweb-scraping