问题描述
carner_list = ['<a href="/lyric/34808442/Loyle+Carner/damselfly">damselfly</a>','<a href="/lyric/37311114/Loyle+Carner/damselfly">damselfly</a>','<a href="/lyric/37360958/Loyle+Carner/damselfly">damselfly</a>','<a href="/lyric/33661937/Loyle+Carner/The+Isle+of+Arran">The Isle of Arran</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>']
现在我想摆脱那些重复的物品。问题是，双精度项仅在字符串i [38：]的特定点彼此不同。
我的想法是创建一个for循环：
new_list = []
for i in carner_list:
if i[38:] in new_list:
print("found")
else:
new_list = new_list + [i]
print("not")
但这不起作用。
语法是否有误或者我完全走错了轨道？
最佳罗素
解决方法
我键入了一个名为listContains的小函数，我认为它可以解决您的问题。您的代码无法正常工作，因为您在i[38:]中搜索了值new_list，而在new_list中添加了i的整个值。
因此，您还应该对列表的每个值应用 [38：] 规则。
我认为以下代码可以更好地解释我的意思：
carner_list = ['<a href="/lyric/34808442/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/37311114/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/37360958/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/33661937/Loyle+Carner/The+Isle+of+Arran">The Isle of Arran</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>']
new_list = []
def listContains(myList,toSearch):
for val in myList:
if val[38:] == toSearch:
return True
return False
for i in carner_list:
if listContains(new_list,i[38:]):
print("found")
else:
new_list.append(i)
print("not")
print(new_list)
如果要测试，可以从here
开始 ,您用来确定重复的字符串部分（从索引38到末尾）不是您实际存储在列表中的部分，因此in运算符将不起作用。
您可以改用字典来存储重复数据删除的字符串，并将您关心的部分字符串作为密钥，以便in运算符可以工作：
new = {}
for i in carner_list:
key = i[38:]
if key not in new:
new[key] = i
print(list(new.values()))
这将输出：
['<a href="/lyric/34808442/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>']
因此，当前搜索的方式是查看子字符串是否等于new_list中的任何内容。因为它是子字符串，所以永远不会如此。
您可以使用lambda，然后对其进行过滤以获得真实结果，以查看该项目是否在新列表中。然后将其转换为列表，并检查该列表的长度是否不等于0。
len(list(filter(lambda x: i[38:] in x,new_list))) != 0
最终密码
carner_list = ['<a href="/lyric/34808442/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>']
new_list = []
for i in carner_list:
if len(list(filter(lambda x: i[38:] in x,new_list))) != 0:
print("found")
else:
new_list.append(i)
print("not")
使用BeautifulSoup解析html，然后检查
例如：
from bs4 import BeautifulSoup
carner_list = ['<a href="/lyric/34808442/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>']
new_list = []
check_val = set()
for i in carner_list:
s = BeautifulSoup(i,"html.parser")
if s.text not in check_val: #check for text
new_list.append(i)
check_val.add(s.text)
print(new_list)
输出：
['<a href="/lyric/34808442/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/33661937/Loyle+Carner/The+Isle+of+Arran">The Isle of '
'Arran</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the '
'Morning</a>']
为什么不使用正则表达式
import re
carner_list = ['<a href="/lyric/34808442/Loyle+Carner/Damselfly">Damselfly</a>','<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>']
print({re.findall(r'"([^"]*)"',x)[0].split("/")[4]: x for x in carner_list })
#Below is the output generated
'''
{'Damselfly': '<a href="/lyric/37360958/Loyle+Carner/Damselfly">Damselfly</a>','The+Isle+of+Arran': '<a href="/lyric/33661937/Loyle+Carner/The+Isle+of+Arran">The Isle of Arran</a>','Mean+It+in+the+Morning': '<a href="/lyric/33661936/Loyle+Carner/Mean+It+in+the+Morning">Mean It in the Morning</a>'}
'''