busboy收到文件后出现ENOENT错误

编程问答 2022-08-15

问题描述

我正在处理一个旧项目，其中在EC2实例上，将MP4文件从具有axios的React应用程序逐个上传到具有busboy的nestJS API。然后，将文件上传到S3存储桶。

当AWS S3库尝试加载文件时，有时会出现明显的“随机”错误：

 warn
 module: videoservice
 method: savetostream
 message: Error on Upload Success callback
 { [Error: ENOENT: no such file or directory,open 'file.mp4'] errno: -2,code: 'ENOENT',syscall: 'open',path: 'file.mp4' }

 (node:26886) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory,open 'file.mp4'
 (node:26886) UnhandledPromiseRejectionwarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block,or by rejecting a promise which was not handled with .catc h(). (rejection id: 20)

以下是摘要：

API上传端点 

  @Post('upload/:id')
  @ApiOperation({ ... })
  @ApiResponse({ status: 201,type: Video })
  @ApiConsumes('multipart/form-data')
  @ApiImplicitFile({ name: 'file',required: true })
  @ApiCreatedResponse({ type: UploadResponseDto })
  async upload(
    @Decoded() decoded: any,@Param('id') videoId: number,@Query('fileSize') fileSize :number,@Req() req: Request,@Res() res: Response,@Body() uploadDto: UploadDto,): Promise<any> {
    try {
      const busboy = new Busboy({
        headers: req.headers,});

      const data = new Map();
      const writtenFiles = [];
      let upload;

      busboy.on('field',(fieldname,val) => {
        data.set(fieldname,val);
      });

      busboy.on(
        'file',async (fieldname,file,filename,encoding,mimetype) => {
          try {
            data.set('filename',filename);
            data.set('mimetype',mimetype);
            data.set('encoding',encoding);

            const filepath = path.join(fieldname + path.extname(filename));
            upload = filepath;

            const writeStream = fs.createWriteStream(filepath);
            file.pipe(writeStream);

            const promise = new Promise((resolve,reject) => {
              file.on('end',() => {
                writeStream.end();
              });
              writeStream.on('finish',resolve);
              writeStream.on('error',reject);
            });
    
            writtenFiles.push(promise);
          } catch (err) {
            this.logger.error(log('busboy on file',err.message));
            return res.status(HttpStatus.INTERNAL_SERVER_ERROR).send({
              statusCode: HttpStatus.INTERNAL_SERVER_ERROR,message: err.message,});
          }
        },);

      busboy.on('error',err => {
        this.logger.warn(log('busboy on error',err.message));
      });

      busboy.on('finish',async () => {
        await Promise.all(writtenFiles);
        res.status(HttpStatus.CREATED).send('OK');

        await this.bucketService.saveStream( // Next snippet
          fs.createReadStream(upload),data.get('filename'),data.get('mimetype'),data.get('fileSize'),+data.get('programId'),+data.get('videoId')
        );
        return;
        fs.unlinkSync(upload);
      });

      req.pipe(busboy);
    } catch (err) {
      this.logger.error(log('catch',err.message));
      return res.status(HttpStatus.INTERNAL_SERVER_ERROR).send({
        statusCode: HttpStatus.INTERNAL_SERVER_ERROR,});
    }
  }

BucketService saveStream方法 

public async saveStream(
    body: any
    filename: string,mimeType: string,fileSize: number,programId: number,videoId: number
  ): Promise<any> {
    try {
      const callback = async (err: any,data: any) => {
        if (err) {
          this.logger.warn(log('Error on Upload Success callback',err));
          throw err; // Here is where the error is raised
        }
      };

      return this.s3
        .upload(
          this.setParams(body,mimeType,programId,videoId),(err,data) => callback(err,data),)
    } catch (err) {
      this.logger.error(log('Error on S3 Upload',err));
      throw err;
    }
  }

  private setParams(
    file: any,filename: string,videoId: number
  ): any {
    return {
      ...awsBucketConfig,Key: `${AWS_UPLOAD_DIR}${programId}/${videoId}/${Date.Now()}${path.extname(
        filename,)}`,Body: file,ContentType: mimeType,};
  }

我有时认为这可能是因为EC2上的临时文件的名称始终相同：file.mp4，并且当同时上传两个文件时，第一个完成的文件会删除该文件（端点上的fs.unlinkSync(upload);）没有其他正在进行的进程，因此，当尝试上载该进程时，找不到该进程。但这不是真的，因为我进行了测试，确保我的文件被一张一张地上传。但是，我还通过更改控制器来确保名称总是不同的：

const filepath = path.join(fieldname + path.extname(filename));

作者

const filepath = path.join(Math.floor(Math.random() * 10000) + path.extname(filename));

，但错误仍在发生。发生的另一件奇怪的事情是，在我的计算机上同时可以看到（ls文件，而在EC2中看不到。

事实：

  • EC2：t2.xlarge（大约免费4GB）

  • OS：Ubuntu 18

  • 节点版本：10.21.0

  • 平均文件大小：2GB

  • 依赖项

     "archiver": "^3.1.1","async-busboy": "^0.7.0","aws-sdk": "^2.553.0","axios": "^0.19.0","body-parser": "^1.19.0","busboy": "^0.3.1",

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

busboyenoentnestjs