启动多个线程进程来处理队列

问题描述

使用下面的代码，我开始对进程进行线程化，write_process写入队列，read_process从队列中读取：

import time
from multiprocessing import Process,Queue,Pool
class QueueFun():

    def writing_queue(self,work_tasks):
        while True:
            print("Writing to queue")
            work_tasks.put(1)
            time.sleep(1)

    def read_queue(self,work_tasks):
        while True:
            print('Reading from queue')
            work_tasks.get()
            time.sleep(2)


if __name__ == '__main__':
    q = QueueFun()
    work_tasks = Queue()

    write_process = Process(target=q.writing_queue,args=(work_tasks,))
    write_process.start()

    read_process = Process(target=q.read_queue,))
    read_process.start()

    write_process.join()
    read_process.join()

在上面的代码中运行：

Writing to queue
Reading from queue
Writing to queue
Reading from queue
Writing to queue
Writing to queue
Reading from queue
Writing to queue

如何启动N个进程从队列中读取？

我尝试使用下面的代码启动3个进程，但仅启动了1个进程，这是因为.join（）阻止了第二个进程启动？：

for i in range(0,3):
    read_process = Process(target=q.read_queue,))
    print('Starting read_process',i)
    read_process.start()
    read_process.join()

我也考虑过使用https://docs.python.org/2/library/multiprocessing.html中所述的Pool，但这似乎与转换现有集合有关：

print pool.map(f,range(10))

如何在每个线程处理一个共享队列的地方启动n个线程？

解决方法

您可以将其放入列表中，并在创建循环之外加入它：

from sqlalchemy import Column,String,Integer,Date

from base import Base


class Actor(Base):
    __tablename__ = 'actors'

    id = Column(Integer,primary_key=True)
    name = Column(String)
    birthday = Column(Date)

    def __init__(self,name,birthday):
        self.name = name
        self.birthday = birthday