我正在Mac OSX上使用Keras在虚拟数据上培训MobileNet架构.我设置了nump.random和tensorflow.set_random_seed,但由于某些原因我无法获得可重现的结果:每次重新运行代码时,我都会得到不同的结果.为什么?这不是因为GPU,因为我在拥有Radeon显卡的MacBook Pro 2017上运行,因此Tensorflow没有利用它.代码运行
python Keras_test.py
所以这不是状态问题(我没有使用Jupyter或IPython:每次运行代码时都应该重置环境).
编辑:我在导入Keras之前通过移动所有种子的设置来更改我的代码.结果仍然不确定,但结果的方差比以前小得多.这非常离奇.
目前的模型非常小(就深度神经网络而言)并不是微不足道的,它不需要GPU运行,它可以在几分钟内在现代笔记本电脑上进行训练,因此重复我的实验是在任何人的能力范围内.我邀请你这样做:我对了解从系统到另一个系统的变化程度非常感兴趣.
import numpy as np # random seeds must be set before importing keras & tensorflow my_seed = 512 np.random.seed(my_seed) import random random.seed(my_seed) import tensorflow as tf tf.set_random_seed(my_seed) # Now we can import keras import keras.utils from keras.applications import MobileNet from keras.callbacks import ModelCheckpoint from keras.optimizers import Adam import os height = 224 width = 224 channels = 3 epochs = 10 num_classes = 10 # Generate dummy data batch_size = 32 n_train = 256 n_test = 64 x_train = np.random.random((n_train,height,width,channels)) y_train = keras.utils.to_categorical(np.random.randint(num_classes,size=(n_train,1)),num_classes=num_classes) x_test = np.random.random((n_test,channels)) y_test = keras.utils.to_categorical(np.random.randint(num_classes,size=(n_test,num_classes=num_classes) # Get input shape input_shape = x_train.shape[1:] # Instantiate model model = MobileNet(weights=None,input_shape=input_shape,classes=num_classes) model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy']) # Viewing Model Configuration model.summary() # Model file name filepath = 'model_epoch_{epoch:02d}_loss_{loss:0.2f}_val_{val_loss:.2f}.hdf5' # Define save_best_only checkpointer checkpointer = ModelCheckpoint(filepath=filepath,monitor='val_acc',verbose=1,save_best_only=True) # Let's fit! model.fit(x_train,y_train,batch_size=batch_size,epochs=epochs,validation_data=(x_test,y_test),callbacks=[checkpointer])
和往常一样,这是我的Python,Keras& Tensorflow版本:
python -c 'import keras; import tensorflow; import sys; print(sys.version,'keras.__version__','tensorflow.__version__')' /anaconda2/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future,it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters Using TensorFlow backend. ('2.7.15 |Anaconda,Inc.| (default,May 1 2018,18:37:05) \n[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]','2.1.6','1.8.0')
以下是多次运行此代码所获得的一些结果:如您所见,代码使用描述性文件名保存10个时期中的最佳模型(最佳验证准确性),因此比较不同运行中的文件名可以判断结果的可变性.
model_epoch_01_loss_2.39_val_3.28.hdf5 model_epoch_01_loss_2.39_val_3.54.hdf5 model_epoch_01_loss_2.40_val_3.47.hdf5 model_epoch_01_loss_2.41_val_3.08.hdf5
解决方法
您可以在Keras docs:
https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development找到答案.
简而言之,为了确保您可以在一台计算机/笔记本电脑的cpu上使用python脚本获得可重现的结果,那么您将不得不执行以下操作:
>将PYTHONHASHSEED环境变量设置为固定值
>将python内置伪随机生成器设置为固定值
>将numpy伪随机生成器设置为固定值
>将tensorflow伪随机生成器设置为固定值
>配置新的全局张量流会话
# Seed value # Apparently you may use different seed values at each stage seed_value= 0 # 1. Set `PYTHONHASHSEED` environment variable at a fixed value import os os.environ['PYTHONHASHSEED']=str(seed_value) # 2. Set `python` built-in pseudo-random generator at a fixed value import random random.seed(seed_value) # 3. Set `numpy` pseudo-random generator at a fixed value import numpy as np np.random.seed(seed_value) # 4. Set `tensorflow` pseudo-random generator at a fixed value import tensorflow as tf tf.set_random_seed(seed_value) # 5. Configure a new global `tensorflow` session from keras import backend as K session_conf = tf.ConfigProto(intra_op_parallelism_threads=1,inter_op_parallelism_threads=1) sess = tf.Session(graph=tf.get_default_graph(),config=session_conf) K.set_session(sess)
毋庸置疑,您不必在python脚本中使用的numpy,scikit-learn或tensorflow / keras函数中指定任何种子或random_state,因为上面的源代码我们全局设置了伪 – 固定值的随机发电机.