AWS Batch 将 tar.gz 文件提取到 /opt/ml/model 失败并显示 OSError：[Errno 30] 只读文件系统

编程问答 2022-05-26

问题描述

我们有 python 代码，它在 docker 容器中执行以下操作

import boto3
import tarfile


s3 = boto3.client('s3')

s3.download_file("dev-bucket","test/model.tar.gz","/opt/ml/model/model.tar.gz")

tar = tarfile.open("/opt/ml/model/model.tar.gz",'r:gz')
tar.extractall(path="/opt/ml/model")

但是，作业在提取时失败，并显示“OSError: [Errno 30] Read-only file system”。完整的堆栈跟踪是：

回溯（最近一次调用最后一次）：

>   File "inference.py",line 6
>     tar.extractall(path="/opt/ml/model")   File "/opt/conda/lib/python3.7/tarfile.py",line 2002,in extractall
>     numeric_owner=numeric_owner)   File "/opt/conda/lib/python3.7/tarfile.py",line 2044,in extract
>     numeric_owner=numeric_owner)   File "/opt/conda/lib/python3.7/tarfile.py",line 2114,in _extract_member
>     self.makefile(tarinfo,targetpath)   File "/opt/conda/lib/python3.7/tarfile.py",line 2163,in makefile
>     copyfileobj(source,target,tarinfo.size,ReadError,bufsize)   File "/opt/conda/lib/python3.7/tarfile.py",line 250,in copyfileobj
>     dst.write(buf) OSError: [Errno 30] Read-only file system

Dockerfile 如下：

FROM continuumio/miniconda3

# use python3.7
RUN /opt/conda/bin/conda install python=3.7

# Update conda
RUN /opt/conda/bin/conda update -n base conda

# Install build-essential
RUN apt-get update && apt-get install -y build-essential \
    wget \
    nginx \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies
RUN conda install -y pandas==0.25.1 scikit-learn==0.21.2 s3fs==0.4.2
RUN pip install pyarrow==1.0.0 mxnet joblib==0.13.2 boto3

CMD [ "/bin/bash" ]

ENV PYTHONUNBUFFERED=TRUE
ENV PYTHONDONTWRITEBYTECODE=TRUE
ENV PATH="/opt/program:${PATH}"


RUN mkdir -p /opt/ml/model
RUN chmod -R +w /opt/ml/model
RUN mkdir -p /opt/ml/input/data
# Set up the program in the image
COPY helloworld /opt/program
WORKDIR /opt/program

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

aws-batchdockeroperating-systemtarfile