问题描述
我遇到了多模式问题的错误。 输入形状： img输入：-（3740，150，150，3）， 文字输入：-（3740，260） 其中3740是样本数。 这里附带了作为功能的模型，其中build_img_encoder描述IMG编码器模型，build_wrd_conv描述word_encoder部分，这是在Model Plot图像中可见的2个输入分支
def build_merged_model_Concatenate(embedding_matrix,input_length):#input_length-->word length
#call encoders
#define input 2 channels
img_input = Input(shape= build_img_encoder_model().input_shape[1:],name="img_input")
wrd_input = Input(shape= build_wrd_conv(embedding_matrix,input_length).input_shape[1:],name="wrd_input")
#call img and word encoders
img_encoder = build_img_encoder_model()(img_input)
wrd_encoder = build_wrd_conv(embedding_matrix,input_length)(wrd_input)
#merge 2 models
merge = concatenate([img_encoder,wrd_encoder],name="concat_merge1")
merged_model1_concat = keras.models.Model(inputs=[img_input,wrd_input],outputs = merge)## model assign
dense_merge0 = Dense(1024,activation='relu',name="dense_merge0")(merge)
dropout_merge0 = Dropout(0.5,name='dropout_merge0')(dense_merge0)
dense_merge1 = Dense(256,name='dense_merge1')(dropout_merge0)
dropout_3 = Dropout(0.2,name="dropout_3")(dense_merge1)
dense_merge2 = Dense(128,name='dense_merge_2')(dropout_3)
output_classify = Dense( 2,activation='sigmoid')(dense_merge2)
merged_classify = keras.models.Model(inputs=[img_input,outputs= output_classify)## model assign
OPTIMIZER = Adam(learning_rate=0.0001,beta_1=0.9,beta_2=0.999)
METRICS = ['accuracy',f1_m,precision_m,recall_m]
merged_model1_concat.compile(loss = 'binary_crossentropy',optimizer=OPTIMIZER,metrics=METRICS)
merged_classify.compile(loss = 'binary_crossentropy',metrics=METRICS)
return merged_model1_concat,merged_classify
正在运行的模型拟合模型开始训练并运行几乎一个完整的时期，然后弹出此错误：-
WARNING:tensorflow:`input_shape` is undefined or non-square,or `rows` is not in [128,160,192,224]. Weights for input shape (224,224) will be loaded as the default.
WARNING:tensorflow:`input_shape` is undefined or non-square,224) will be loaded as the default.
Epoch 1/100
2/53 [>.............................] - ETA: 6s - loss: 1.0590 - accuracy: 0.6094 - f1_m: 0.5766 - precision_m: 0.6011 - recall_m: 0.5547WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0632s vs `on_train_batch_end` time: 0.1851s). Check your callbacks.
53/53 [==============================] - ETA: 0s - loss: 0.8733 - accuracy: 0.6548 - f1_m: 0.6309 - precision_m: 0.6285 - recall_m: 0.6362WARNING:tensorflow:Model was constructed with shape (None,150,3) for input Tensor("img_input_15:0",shape=(None,3),dtype=float32),but it was called on an input with incompatible shape (None,260).
WARNING:tensorflow:Model was constructed with shape (None,3) for input Tensor("input_54:0",3) for input Tensor("input_53:0",260).
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-84-23200389ca70> in <module>()
26 x,concat_merge_classify = build_merged_model_Concatenate(embedding_matrix_full,input_length)
27 ##plot_model(Classify_Model1,to_file="MergedClassify_Model_2_multiply.png",show_shapes=True)
---> 28 hist2= concat_merge_classify.fit(x=[X_TRAIN_,padded_docs_train],y=Y_train,epochs=100,batch_size =64,callbacks=[reducelr,checkpointer,earlyStopping],validation_data = ([padded_docs_test],Y_test))
29 print('\n# Evaluate on test data')
30 loss,accuracy,f1_score,precision,recall= concat_merge_classify.evaluate([X_TEST_,padded_docs_test],Y_test,verbose=1,batch_size=64)
12 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args,**kwargs)
971 except Exception as e: # pylint:disable=broad-except
972 if hasattr(e,"ag_error_Metadata"):
--> 973 raise e.ag_error_Metadata.to_exception(e)
974 else:
975 raise
ValueError: in user code:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1224 test_function *
return step_function(self,iterator)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1215 step_function **
outputs = model.distribute_strategy.run(run_step,args=(data,))
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run
return self._extended.call_for_each_replica(fn,args=args,kwargs=kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica
return self._call_for_each_replica(fn,args,kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica
return fn(*args,**kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1208 run_step **
outputs = model.test_step(data)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1174 test_step
y_pred = self(x,training=False)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:985 __call__
outputs = call_fn(inputs,*args,**kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:386 call
inputs,training=training,mask=mask)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:508 _run_internal_graph
outputs = node.layer(*args,**kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:985 __call__
outputs = call_fn(inputs,**kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:976 __call__
self.name)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py:180 assert_input_compatibility
str(x.shape.as_list()))
ValueError: Input 0 of layer conv1_pad is incompatible with the layer: expected ndim=4,found ndim=2. Full shape received: [None,260]
那是怎么回事？，看来我错过了某个地方的形状。
解决方法
检查此内容：
validation_data = ([padded_docs_test],Y_test)).
看起来不适合训练数据