问题描述
我正在构建一个为其定义模型的 RL 代理:
def build_model(height,width,channels,actions):
model = Sequential()
model.add(Conv2D(32,(8,8),strides=(4,4),activation='relu',input_shape=(3,height,channels)))
model.add(Conv2D(64,(4,strides=(2,2),activation='relu'))
model.add(Dense(512,activation='relu'))
model.add(Dense(256,activation='relu'))
model.add(Dense(actions,activation='linear'))
return model
然后创建一个模型:
env = gym.make('SpaceInvaders-v0')
height,channels = env.observation_space.shape
actions = env.action_space.n
model = build_model(height,actions)
然后定义代理:
def build_agent(model,actions):
policy = LinearannealedPolicy(epsgreedyQPolicy(),attr='eps',value_max=1,value_min=.1,value_test=-.2,nb_steps=10000)
memory = SequentialMemory(limit=2000,window_length=3)
dqn = DQNAgent(model=model,memory=memory,policy=policy,dueling_type='avg',nb_actions=actions,nb_steps_warmup=1000)
抱怨我尝试构建代理:
dqn = build_agent(model,actions)
我收到此错误:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-100-320eef5d391d> in <module>
----> 1 dqn = build_agent(model,actions)
<ipython-input-95-97801a497208> in build_agent(model,actions)
2 policy = LinearannealedPolicy(epsgreedyQPolicy(),nb_steps=10000)
3 memory = SequentialMemory(limit=2000,window_length=3)
----> 4 dqn = DQNAgent(model=model,nb_steps_warmup=1000)
~/Documents/RL/pRL/lib/python3.8/site-packages/rl/agents/dqn.py in __init__(self,model,policy,test_policy,enable_double_dqn,enable_dueling_network,dueling_type,*args,**kwargs)
105 # Validate (important) input.
106 if list(model.output.shape) != list((None,self.nb_actions)):
--> 107 raise ValueError(f'Model output "{model.output}" has invalid shape. DQN expects a model that has one dimension for each action,in this case {self.nb_actions}.')
108
109 # Parameters.
ValueError: Model output "Tensor("dense_30/BiasAdd:0",shape=(None,3,24,18,6),dtype=float32)" has invalid shape. DQN expects a model that has one dimension for each action,in this case 6.
我检查了 model.output.shape 和 self.nb_actions 的值:
print(model.output_shape)
# (None,6)
print(actions) # This is self.nb_action
# 6
我不知道“model.output”的形状应该是什么。在这个错误中,它看起来应该是 '(None,6)' 但我不知道我必须做出什么改变才能得到这个形状。
解决方法
将 model.output.shape 转换为元组,然后比较 if tuple(model.output.shape) != (None,4)