问题描述
<script type='text/javascript'>
var gaProperty = 'UA-00000000-1';
var disableStr = 'ga-disable-' + gaProperty;
if ( document.cookie.indexOf( disableStr + '=true' ) > -1 ) {
window[disableStr] = true;
}
function gaOptout() {
document.cookie = disableStr + '=true; expires=Thu,31 Dec 2099 23:59:59 UTC; path=/';
window[disableStr] = true;
}
</script>
我正在尝试使用 python 从 csv 文件中的 url 列表中的每个页面(即 UA-00000000-1)中提取 var gaProperty。我是 python 的新手,并从我看到的一些脚本中组合了一个脚本,但它不起作用:
from requests_html import HTMLSession
from urllib.request import urlopen
from bs4 import BeautifulSoup
import csv
import re
list = []
with open('list.csv','r') as csvf: # Open file in read mode
urls = csv.reader(csvf)
for url in urls:
list.append(url) # Add each url to list contents
for url in list:
page = urlopen(url[0]).read()
path = " ".join(url)
soup = BeautifulSoup(page,"lxml")
data = soup.find_all('script',type='text/javascript')
gaid = re.search(r'UA-[0-9]+-[0-9]+',data[0].text)
print(path,gaid)
我得到的错误结果是:
https:www.example.com/contact-us/ None
我需要为每个 url 实现所需的输出:
https:www.example.com/contact-us/ UA-00000000-1
知道如何在 Python 中实现这一点吗?
解决方法
更具体地说,我会在模式中包含 var gaProperty,然后确保捕获组延迟捕获 '
之间的所有内容,即包装 gaid 值。
import re
html ='''
<script type='text/javascript'>
var gaProperty = 'UA-00000000-1';
var disableStr = 'ga-disable-' + gaProperty;
if ( document.cookie.indexOf( disableStr + '=true' ) > -1 ) {
window[disableStr] = true;
}
function gaOptout() {
document.cookie = disableStr + '=true; expires=Thu,31 Dec 2099 23:59:59 UTC; path=/';
window[disableStr] = true;
}
</script>'''
gaid = re.search(r"var gaProperty = '(.*?)'",html).group(1)
print(f'https:www.example.com/contact-us/{gaid}')