问题描述
当我尝试将响应保存到文件时,即使实际响应显示在控制台中,也不会保存。保存在文件中的结果为“无”。参见下面的示例
from concurrent.futures import ThreadPoolExecutor
import requests
#from timer import timer
######### create test file
URLsTest = '''
https://en.wikipedia.org/wiki/NBA
https://en.wikipedia.org/wiki/NFL
'''.strip()
with open('input.txt','w') as f:
f.write(URLsTest)
####################
with open('input.txt','r') as f:
urls=f.read().split('\n') # url list
def fetch(tt): # received tuple
session,url = tt
print('Processing')
with session.get(url) as response:
print(response.text)
#@timer(1,5)
def main():
with ThreadPoolExecutor(max_workers=100) as executor:
with requests.Session() as session: # for Now,just one session
results = executor.map(fetch,[(session,u) for u in urls]) # tuple list (session,url),each tuple passed to function
executor.shutdown(wait=True)
# write all results to text file
with open('output.txt','w') as f2:
for r in results: # tuple (url,html)
f2.write("%s\n" % r)
main()
响应文件-output.txt
None
None
解决方法
首先,由于将输出保存到文件中,因此可以避免打印html。这样,您可以避免使用资源来打印结果。
然后,您的提取操作未为results
返回任何内容。因此,您应该将print
更改为return
因此,不打印而是返回response.text
# print(response.text)
return response.text
,
理想的做法是不打印html,因为这样就不得不将工作保存或输出到文件中,从而使您无法以其原始形状打印整个结果。