问题描述
这是对Can we send data from Google cloud storage to SFTP server using GCP Cloud function?
的扩展问题with pysftp.Connection(host=myHostName,username=myUsername,password=myPassword,cnopts=cnopts) as sftp:
storage_client = storage.Client()
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(filename) #source_blob_name
with sftp.open(remotepath,'w+',32768) as f:
blob.download_to_file(f)
W+
/w
命令不会截断前一个文件并抛出错误。这是当 SFTP 服务器中已经存在同名文件时!对此的最佳解决方案应该是什么?
gcs-to-sftp
4wz44jq3oe2g
Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/google/cloud/storage/client.py",line 728,in download_blob_to_file
checksum=checksum,File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py",line 986,in _do_download
response = download.consume(transport,timeout=timeout)
File "/env/local/lib/python3.7/site-packages/google/resumable_media/requests/download.py",line 168,in consume
self._process_response(result)
File "/env/local/lib/python3.7/site-packages/google/resumable_media/_download.py",line 186,in _process_response
response,_ACCEPTABLE_STATUS_CODES,self._get_status_code
File "/env/local/lib/python3.7/site-packages/google/resumable_media/_helpers.py",line 104,in require_status_code
*status_codes
google.resumable_media.common.InvalidResponse: ('Request Failed with status code',404,'Expected one of',<HTTPStatus.OK: 200>,<HTTPStatus.PARTIAL_CONTENT: 206>)
During handling of the above exception,another exception occurred:
Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py",line 449,in run_background_function
_function_handler.invoke_user_function(event_object)
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py",line 268,in invoke_user_function
return call_user_function(request_or_event)
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py",line 265,in call_user_function
event_context.Context(**request_or_event.context))
File "/user_code/main.py",line 20,in hello_sftp
copy_file_on_ftp(myHostName,myUsername,myPassword,bucket_name,filename)
File "/user_code/main.py",line 42,in copy_file_on_ftp
file_to_export.download_to_file(f)
File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py",line 1128,in download_to_file
checksum=checksum,File "/env/local/lib/python3.7/site-packages/google/cloud/storage/client.py",line 731,in download_blob_to_file
_raise_from_invalid_response(exc)
File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py",line 4100,in _raise_from_invalid_response
raise exceptions.from_http_status(response.status_code,message,response=response)
google.api_core.exceptions.NotFound: 404 GET https://storage.googleapis.com/download/storage/v1/b/sftp__test/o/test_file.csv?alt=media: No such object: sftp__test/test_file.csv: ('Request Failed with status code',<HTTPStatus.PARTIAL_CONTENT: 206>)
解决方法
根据@Martin-Prikryl 的建议回答
替换
with sftp.open(remotepath,'w+',32768) as f:
与
sftp.open(remotepath,"wb")
或使用
from io import BytesIO
flo = BytesIO()
blob.download_to_file(flo)
flo.seek(0)
sftp.putfo(flo,remotepath)
,
+
模式下的 w+
不应该在那里。它打开文件进行读取和写入,这在您的场景中是不需要的。
尽管 w
应该截断文件(并且它在常见的 OpenSSH Linux 服务器上会截断),但当与 +
结合使用时,它似乎不适用于您的服务器。