logo标签列表

无法创建CUBLAS句柄。Tensorflow与OpenCV的交互。

pythonopencvtensorflowcublas
6
我正在尝试使用PlayStation Eye Camera进行深度强化学习项目。网络、TensorFlow安装（0.11）和CUDA（8.0）是可用的，因为我已经能够在仿真中训练网络。现在，当我尝试从真实相机读取图像时，网络代码会崩溃，并出现下面的错误。我的OpenCV安装（3.2.0）有问题吗，还是有其他问题？我将不胜感激，因为我没有找到任何关于这个问题的信息。
E tensorflow/stream_executor/cuda/cuda_blas.cc:367] failed to create      cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
W tensorflow/stream_executor/stream.cc:1390] attempting to perform BLAS operation using StreamExecutor without BLAS support


Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "main.py", line 48, in worker
    action = dqn.getAction()
  File "../network/evaluation.py", line 141, in getAction
    Q_value = self.Q_value.eval(feed_dict= {self.input_state:[self.currentState]})[0]
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 559, in eval
    return _eval_using_default_session(self, feed_dict, self.graph, session)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3761, in _eval_using_default_session
    return session.run(tensors, feed_dict)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 717, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 915, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 965, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 985, in _do_call
    raise type(e)(node_def, op, message)
InternalError: Blas SGEMM launch failed : a.shape=(1, 1600), b.shape=(1600, 4), m=1, n=4, k=1600
     [[Node: MatMul = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/gpu:0"](Reshape, Variable_6/read)]]

来自相机类的相关代码：

# OpenCV
import numpy as np
import cv2

# scipy
from scipy.misc import imresize

# Time
from time import time

# Clean exit
import sys
import os

# Max value for the gray values
MAX_GRAY = 255.0
INPUT_SIZE = 75

class Camera:


    # Initialization method
    def __init__(self, duration, exchanger, framesPerAction = 10, width = 640, height = 480, show = True):

        # Create the video capture
        self.cap = cv2.VideoCapture(1)

        # Set the parameters of the capture
        self.cap.set(3, width)
        self.cap.set(4, height)
        self.cap.set(5, 30)

        # Get the properties of the capture
        self.width = int(self.cap.get(3))
        self.height = int(self.cap.get(4))
        self.fps = int(self.cap.get(5))

        # Print these properties
        print 'Width:', self.width, '| Height:', self.height, '| FPS:', self.fps

        # Duration that the camera should be running
        self.duration = duration

        # Number of frames that should be between every extracted frame
        self.framesPerAction = framesPerAction

        # Exchanges the frames with the network
        self.exchanger = exchanger

        # Display the frames on the monitor
        self.show = show

        # Counter for the number of frames since the last action
        self.frameCounter = 0

    # Starts the loop for the camera
    def run(self):

        startTime = time()

        # Loop for a certain time
        while(self.duration > time() - startTime):

            # Check frames per second
#           print 'Start of this frame', time()-startTime

            # Capture frame-by-frame
            ret, frame = self.cap.read()

            # Close when user types ESCAPE(27)
            if cv2.waitKey(1) & 0xFF == 27:
                break

            # Increment framecounter
            if(self.frameCounter != self.framesPerAction):
                self.frameCounter += 1

            # Extract the resulting frame
            else:

                # Crop to square
                step = int((640 - 480) / 2)
                result = frame[0 : 480, step : step + 480]

                # Downsample the image
#               result = cv2.resize(gray, (75, 75))
                result = imresize(result, size=(75, 75, 3))

                # Transform to grayscale
#               gray = cv2.cvtColor(input, cv2.COLOR_BGR2GRAY)
                result = self.rgb2gray(result)

                # Change range of image from [0,255] --> [0, 1]
                result = result / 255.0

                # Store the frame on the exchanger
                self.exchanger.store(0, False, result)

                # reset framecounter
                self.frameCounter = 0

            # Display the frame on the monitor
            if(self.show):
                    cv2.imshow('frame', frame)

        # When everything done, release the capture
        self.cap.release()
        cv2.destroyAllWindows()

        # Exit so that the network thread also stops running
        os._exit(0)
可能是Tensorflow崩溃，出现CUBLAS_STATUS_ALLOC_FAILED错误的重复问题。 - talonmies
你在TensorFlow 1.0上仍然遇到这个问题吗？ - Neal
@talonmies，那个修复方法在这里不起作用。我认为问题与网络有关，因为它在使用cublas时遇到了麻烦，因为opencv已经在使用它了。 - RandomEngineer
@Neal 谢谢，这实际上解决了问题。本来没有计划更新，因为会有向后兼容性问题，但实际上比预期的要容易得多。感谢您的提示。 - RandomEngineer
太棒了！作为参考，在将代码从旧版TensorFlow升级到1.0时，您可以使用此升级脚本：https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/compatibility - Neal
3个回答
7
也许以下命令可以帮助您：
sudo rm -rf .nv/

祝你好运。

1
这很有帮助——在Ubuntu的自动更新搞乱了GPU驱动程序之后。谢谢您。 - toobee
这应该是被接受的答案。非常感谢你，Haozhe！ - John Jiang
5

我也遇到过同样的问题，不过我的解决方法更简单。我的情况是在命令行中运行脚本时同时打开了相同脚本的 Idle shell。关闭 shell 后问题得以解决。

2
显然，这个错误可能有各种原因。我通过在官方仓库上关注此问题来解决此问题。Tensorflow GPU 2.2的PyPi版本使用CUDA 10.1和libcublas10.2.1.243，但我安装了cublas10.2.2.89。为了解决这个问题：

Centos:

yum remove libcublas
yum install libcublas10-10.2.1.243-1.x86_64

Ubuntu:

sudo apt remove libcublas10
sudo apt install libcublas10=10.2.1.243-1

然后我删除了nvidia缓存：

rm -rf ~/.nv/

它奏效了。

简而言之，nvidia的闭源策略造成了版本不匹配的迷宫，你要么使用自己版本的CUDA、cudnn和cublas构建tensorflow分布式系统，这并不像听起来那么容易，要么确保你安装了完全正确的所有版本，但是由于nvidia与Linux基金会和开源项目几乎没有合作，所以这也不像本应该那么容易。

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接