tensorflow.python.framework.errors_impl.InternalError：GPU同步失败

编程问答 2022-08-12

问题描述

我已经安装了以下软件：

  • 窗口10
  • Python 3.8
  • tensorflow-gpu 2.3
  • CUDA 10.1
  • CudNN 7.6.5
  • Nvidia gtx 1080
  • 驱动程序版本：451.48
  • 内存：8192MiB

在训练过程中会出现以下错误：

enter image description here

Traceback (most recent call last):
 File "training.py",line 519,in <module>
   history = model.fit(X_train,y_train,epochs=n_epochs,batch_size=batch_size,\
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\keras\engine\training.py",line 108,in _method_wrapper
   return method(self,*args,**kwargs)
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\keras\engine\training.py",line 1103,in fit
  callbacks.on_train_batch_end(end_step,logs)
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\keras\callbacks.py",line 440,in on_train_batch_end
  self._call_batch_hook(ModeKeys.TRAIN,'end',batch,logs=logs)
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\keras\callbacks.py",line 289,in _call_batch_hook
  self._call_batch_end_hook(mode,line 309,in _call_batch_end_hook
  self._call_batch_hook_helper(hook_name,line 342,in _call_batch_hook_helper
  hook(batch,line 961,in on_train_batch_end
   self._batch_update_progbar(batch,line 1016,in _batch_update_progbar
   logs = tf_utils.to_numpy_or_python_type(logs)
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\keras\utils\tf_utils.py",line 537,in to_numpy_or_python_type
  return nest.map_structure(_to_single_numpy_or_python_type,tensors)
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\util\nest.py",line 635,in map_structure
  structure[0],[func(*x) for x in entries],File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\util\nest.py",in <listcomp>
  structure[0],File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\keras\utils\tf_utils.py",line 533,in _to_single_numpy_or_python_type
   x = t.numpy()
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\framework\ops.py",line 1063,in numpy
  maybe_arr = self._numpy()  # pylint: disable=protected-access
 File "C:\Anaconda3_64\lib\site-packages\tensorflow\python\framework\ops.py",line 1031,in _numpy
  six.raise_from(core._status_to_exception(e.code,e.message),None)  # pylint: disable=protected-access
 File "<string>",line 3,in raise_from
tensorflow.python.framework.errors_impl.InternalError: GPU sync Failed

内部错误：GPU同步失败

有线索吗？

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

cudnnnvidiapythontensorflow