python+opencv+caffe+攝像頭做目標(biāo)檢測的實(shí)例代碼

更新時(shí)間：2018年08月03日 09:39:46 作者：IT勤雜工

今天小編就為大家分享一篇python+opencv+caffe+攝像頭做目標(biāo)檢測的實(shí)例代碼，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。一起跟隨小編過來看看吧

首先之前已經(jīng)成功的使用Python做圖像的目標(biāo)檢測，這回因?yàn)轫?xiàng)目最終是需要用攝像頭的，

所以實(shí)現(xiàn)攝像頭獲取圖像，并且用Python調(diào)用CAFFE接口來實(shí)現(xiàn)目標(biāo)識(shí)別

首先是攝像頭請選擇支持Linux萬能驅(qū)動(dòng)兼容V4L2的攝像頭，

因?yàn)橹坝脤W(xué)ARM的時(shí)候使用的Smart210，我已經(jīng)確認(rèn)我的攝像頭是支持的，

我把攝像頭插上之後自然就在 /dev 目錄下看到多了一個(gè)video0的文件，

這個(gè)就是攝像頭的設(shè)備文件了，所以我就沒有額外處理驅(qū)動(dòng)的部分

一、檢測環(huán)境

再來在開始前因?yàn)橹鞍粗鴩兜闹笇?dǎo)手冊安裝的opencv3.2當(dāng)時(shí)沒有開啟V4L2及GTK_2.x的支持，

所以後面遇到了一連串的問題，請大家如下面方法檢測

$ python

1.檢測Python的V4L2支持及攝像頭驅(qū)動(dòng)是否正常

進(jìn)入Python之後如下命令

Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cap = cv2.VideoCapture(0)
>>> print cap.isOpened()
True
>>>

如果返回True就代表攝像頭及你的opencv的V4L2支持就已經(jīng)完全正常了

如果返回False就代表opencv或是攝像頭有問題叁考後面的修改方式

2.再來因?yàn)槲覀円褦z像頭的影像生成窗口，所以我們需要檢測Python的gtk支持如下

>>> import cv2
>>> cv2.namedWindow('test',cv2.WINDOW_AUTOSIZE)
>>>

如果沒有報(bào)任何錯(cuò)誤就代表gtk也是正常的

如如果出現(xiàn)下面提示

OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows,
 GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, 
then re-run cmake or configure script) in cvNamedWindow,

那就代表opencv不支持gtk

如果上面兩個(gè)測試都是好使的你可以跳過本步驟

首先我發(fā)現(xiàn)我不論如何重新編譯opencv發(fā)現(xiàn)python一直都是有問題的，

最後發(fā)現(xiàn)是因?yàn)椴恢朗颤N時(shí)候已經(jīng)裝過 python-opencv的包了

可以用命令

$ pip list |grep opencv

如果發(fā)現(xiàn)有任何跟opencv有關(guān)的包都可以利用 pip uninstall xxx 來移除

假設(shè)看到 pythom-opencv的包那就 pip uninstall opencv-python 來移除

還有檢查dpkg -i |grep opencv 如果跟python的opencv有關(guān)的包也得移除

$ sudo apt-get remove python-opencv

然後到你之前安裝opencv3.2的那個(gè)源碼目錄，

$ cd build
$ make uninstall

這樣就會(huì)卸載之前安裝的opencv

再來V4L的頭文件已經(jīng)改名了，但是opencv會(huì)默認(rèn)使用linux/videodev.h所以要做個(gè)軟鍊接

$ ln -s /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h

opencv安裝過程中會(huì)自動(dòng)的檢測相關(guān)的包，以及一些依賴，

先列出我安裝的包，但是因?yàn)榄h(huán)境多少有點(diǎn)不同，下面會(huì)教大家如何看缺少的包

$ sudo apt-get install libgphoto2-dev v4l2ucp libv4l-dev dv4l libwebcam0-dev libgtkglext1-dev libunicap2-dev

再來執(zhí)行cmake

$ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=ON -D WITH_V4L=ON -D WITH_GTK=ON -D WITH_GTK_2_X -D WITH_OPENGL=ON \
-D WITH_CUDA=ON -D WITH_CUBLAS=ON -D BUILD_PYTHON_SUPPORT=ON \
-D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-3.2.0/modules/ ../opencv-3.2.0

其中如果V4L2使用share庫也可以把-D WITH_V4L: = ON 換成

-D WITH_LIBV4L=ON

在cmake的過程中會(huì)有類似如下的提示

Detected version of GNU GCC: 54 (504)
FP16: Feature disabled
Found OpenEXR: /usr/lib/x86_64-linux-gnu/libIlmImf.so
Checking for module 'libucil'
 No package 'libucil' found
Looking for linux/videodev.h
Looking for linux/videodev.h - found
Looking for linux/videodev2.h
Looking for linux/videodev2.h - found
Looking for sys/videoio.h
Looking for sys/videoio.h - not found
Checking for module 'libavresample'
 No package 'libavresample' found
Found TBB: build
found IPP (ICV version): 9.0.1 [9.0.1]
at: /mnt/sdb/ubuntu/install/opencv/build/3rdparty/ippicv/ippicv_lnx
CUDA detected: 8.0
CUDA NVCC target flags: -gencode;arch=compute_20,code=sm_20;-gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-D_FORCE_INLINES
LAPACK_IMPL: Atlas, LAPACK_LIBRARIES: /usr/lib/liblapack.so;/usr/lib/libcblas.so;/usr/lib/libatlas.so
Could NOT find JNI (missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH) 
Could NOT find Matlab (missing: MATLAB_MEX_SCRIPT MATLAB_INCLUDE_DIRS MATLAB_ROOT_DIR MATLAB_LIBRARIES MATLAB_LIBRARY_DIRS MATLAB_MEXEXT MATLAB_ARCH MATLAB_BIN) 
VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
 
General configuration for OpenCV 3.2.0 =====================================
 Version control:    unknown
 
 Platform:
 Timestamp:     2017-08-14T00:31:55Z
 Host:      Linux 4.10.0-30-generic x86_64
 CMake:      3.5.1
 CMake generator:    Unix Makefiles
 CMake build tool:   /usr/bin/make
 Configuration:    Release
 
 C/C++:
 Built as dynamic libs?:  YES
 C++ Compiler:    /usr/bin/c++ (ver 5.4.0)
 C++ flags (Release):   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
 C++ flags (Debug):   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
 C Compiler:     /usr/bin/cc
 C flags (Release):   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
 C flags (Debug):    -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
 Linker flags (Release):
 Linker flags (Debug):
 ccache:      NO
 Precompiled headers:   YES
 Extra dependencies:   /usr/lib/x86_64-linux-gnu/libpng.so /usr/lib/x86_64-linux-gnu/libtiff.so /usr/lib/x86_64-linux-gnu/libjasper.so /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so gtk-x11-2.0 gdk-x11-2.0 pangocairo-1.0 atk-1.0 cairo gdk_pixbuf-2.0 gio-2.0 pangoft2-1.0 pango-1.0 gobject-2.0 glib-2.0 fontconfig freetype gthread-2.0 gtkglext-x11-1.0 gdkglext-x11-1.0 GLU GL Xmu Xt SM ICE pangox-1.0 X11 gmodule-2.0 gstbase-1.0 gstreamer-1.0 gstvideo-1.0 gstapp-1.0 gstriff-1.0 gstpbutils-1.0 unicap dc1394 avcodec-ffmpeg avformat-ffmpeg avutil-ffmpeg swscale-ffmpeg gphoto2 gphoto2_port exif dl m pthread rt /usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so /usr/lib/liblapack.so /usr/lib/libcblas.so /usr/lib/libatlas.so cudart nppc nppi npps cublas cufft -L/usr/local/cuda-8.0/lib64
 3rdparty dependencies:  libwebp zlib libjpeg tbb
 
 OpenCV modules:
 To be built:     cudev core cudaarithm flann imgproc ml video cudabgsegm cudafilters cudaimgproc cudawarping imgcodecs photo shape videoio cudacodec highgui objdetect ts features2d calib3d cudafeatures2d cudalegacy cudaobjdetect cudaoptflow cudastereo stitching superres videostab python2
 Disabled:     world
 Disabled by dependency:  -
 Unavailable:     java python3 viz
 
 GUI: 
 QT:       NO
 GTK+ 2.x:     YES (ver 2.24.30)
 GThread :     YES (ver 2.48.2)
 GtkGlExt:     YES (ver 1.2.0)
 OpenGL support:    YES (/usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so)
 VTK support:     NO
 
 Media I/O: 
 ZLib:      build (ver 1.2.8)
 JPEG:      libjpeg (ver 90)
 WEBP:      build (ver 0.3.1)
 PNG:       /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.2.54)
 TIFF:      /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 - 4.0.6)
 JPEG 2000:     /usr/lib/x86_64-linux-gnu/libjasper.so (ver 1.900.1)
 OpenEXR:      /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so (ver 2.2.0)
 GDAL:      NO
 GDCM:      NO
 
 Video I/O:
 DC1394 1.x:     NO
 DC1394 2.x:     YES (ver 2.2.4)
 FFMPEG:      YES
  avcodec:     YES (ver 56.60.100)
  avformat:     YES (ver 56.40.101)
  avutil:     YES (ver 54.31.100)
  swscale:     YES (ver 3.1.101)
  avresample:    NO
 GStreamer:     
  base:      YES (ver 1.8.3)
  video:      YES (ver 1.8.3)
  app:      YES (ver 1.8.3)
  riff:      YES (ver 1.8.3)
  pbutils:     YES (ver 1.8.3)
 OpenNI:      NO
 OpenNI PrimeSensor Modules: NO
 OpenNI2:      NO
 PvAPI:      NO
 GigEVisionSDK:    NO
 Aravis SDK:     NO
 UniCap:      NO
 UniCap ucil:     NO
 V4L/V4L2:     YES/YES
 XIMEA:      NO
 Xine:      NO
 gPhoto2:      YES
 
 Parallel framework:   TBB (ver 4.4 interface 9002)
 
 Other third-party libraries:
 Use IPP:      9.0.1 [9.0.1]
   at:      /mnt/sdb/ubuntu/install/opencv/build/3rdparty/ippicv/ippicv_lnx
 Use IPP Async:    NO
 Use VA:      NO
 Use Intel VA-API/OpenCL:  NO
 Use Lapack:     YES (/usr/lib/liblapack.so /usr/lib/libcblas.so /usr/lib/libatlas.so)
 Use Eigen:     NO
 Use Cuda:     YES (ver 8.0)
 Use OpenCL:     YES
 Use OpenVX:     NO
 Use custom HAL:    NO
 
 NVIDIA CUDA
 Use CUFFT:     YES
 Use CUBLAS:     YES
 USE NVCUVID:     NO
 NVIDIA GPU arch:    20 30 35 37 50 52 60 61
 NVIDIA PTX archs:
 Use fast math:    NO
 
 OpenCL:      <Dynamic loading of OpenCL library>
 Include path:    /mnt/sdb/ubuntu/install/opencv/opencv-3.2.0/3rdparty/include/opencl/1.2
 Use AMDFFT:     NO
 Use AMDBLAS:     NO
 
 Python 2:
 Interpreter:     /usr/bin/python2.7 (ver 2.7.12)
 Libraries:     /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.12)
 numpy:      /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.1)
 packages path:    lib/python2.7/dist-packages
 
 Python 3:
 Interpreter:     /usr/bin/python3 (ver 3.5.2)
 
 Python (for build):   /usr/bin/python2.7
 
 Java:
 ant:       NO
 JNI:       NO
 Java wrappers:    NO
 Java tests:     NO
 
 Matlab:      Matlab not found or implicitly disabled
 
 Documentation:
 Doxygen:      /usr/bin/doxygen (ver 1.8.11)
 
 Tests and samples:
 Tests:      YES
 Performance tests:   YES
 C/C++ Examples:    NO
 
 Install path:     /usr/local
 
 cvconfig.h is in:    /mnt/sdb/ubuntu/install/opencv/build
-----------------------------------------------------------------
 
Configuring done

開始的部份會(huì)檢查一堆依賴，請仔細(xì)看如可有缺包，可以百度一下怎么安裝，因?yàn)槲乙呀?jīng)裝完了就不提示了，

不過有些包我也找不到我就沒解決了如上面的

Checking for module 'libucil'
 No package 'libucil' found

這個(gè)我還真沒找到怎么解決，不過反正問題不大

再來下面的部份一定要注意幾個(gè)部分一定要有

 GUI: 
 GTK+ 2.x:     YES (ver 2.24.30)

Video I/O:
 V4L/V4L2:     YES/YES
 FFMPEG:      YES
 Python 2:
 Interpreter:     /usr/bin/python2.7 (ver 2.7.12)
 Libraries:     /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.12)
 numpy:      /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.1)
 packages path:    lib/python2.7/dist-packages

主要就是上面幾個(gè)個(gè)非常重要，必須要裝上

再來就正常 make

$ make -j8
$ make install

設(shè)置nccl的ld環(huán)境

$ vi /etc/ld.so.conf.d/nccl.conf

加上下面目錄

/usr/local/nccl/lib/

然后執(zhí)行

$ sudo ldconfig

安裝完后回python按步驟一再次檢查是否環(huán)境都好使了

二、撰寫Python測試程序

# -*- coding:utf-8 -*-
# 用于模型的單張圖像分類操作
import os
os.environ['GLOG_minloglevel'] = '2' # 將caffe的輸出log信息不顯示，必須放到import caffe前
import caffe # caffe 模塊
from caffe.proto import caffe_pb2
from google.protobuf import text_format
import numpy as np
import cv2
import matplotlib.pyplot as plt
import time
import skimage.io
 
global num
num = 0
 
 
 
def detect(image1,net):
 # 傳進(jìn)來的image1的dtype為uint8
 # print image1.shape
 # print image1.dtype
 # print image1.size
 
 # image = np.array(image1, dtype=np.float32)
 # image = caffe.io.resize_image(image1, (480, 640))
 image = skimage.img_as_float(image1).astype(np.float32)
 # image = caffe.io.resize_image(image2, (300, 300))
 
 # skimage.io.imsave("photo.png", image)
 # cv2.imwrite("photo.png", image)
 # image = caffe.io.load_image(caffe_root + 'examples/images/bird.jpg')
 # 以下方式讀取的imaged的dtype為float32
 # image = caffe.io.load_image(caffe_root + 'photo.png')
 # image = caffe.io.load_image(image1)
 
 # 改變dtype
 # image.dtype = 'float32'
 # print 'mode:'+image.mode
 # print image.shape
 # print image.dtype
 # print image.size
 
 # plt.imshow(image)
 
 # * Run the net and examine the top_k results
 # In[5]:
 global num
 num += 1
 print 'image num:' + str(num)
 
 transformed_image = transformer.preprocess('data', image)
 net.blobs['data'].data[...] = transformed_image
 
 time_start=time.time()
 # Forward pass.
 net.forward()
	
 time_end=time.time() 
 print 'time:' + str(time_end-time_start) + ' s'
 
 
 
 loc = net.blobs['bbox-list'].data[0]
 print(loc)
 #查看了結(jié)構(gòu)文件發(fā)現(xiàn)在CAFFE一開始圖像輸入的時(shí)候就已經(jīng)將圖片縮小了，寬度1248高度384
 #然后我們在net.blobs['bbox-list'].data得到的是偵測到的目標(biāo)座標(biāo)，但是是相對(duì)于1248*384的
 #所以我們要把座標(biāo)轉(zhuǎn)換回相對(duì)原大小的位置，下面im.shape是保存在原尺寸的寬高，
 for l in range(len(loc)):
		xmin = int(loc[l][0] * image.shape[1] / 1248)
		ymin = int(loc[l][1] * image.shape[0] / 384)
		xmax = int(loc[l][2] * image.shape[1] /1248)
		ymax = int(loc[l][3] * image.shape[0] / 384)
		#在該座標(biāo)位置畫一個(gè)方框
		cv2.rectangle(image, (xmin, ymin), (xmax, ymax), (55 / 255.0, 255 / 255.0, 155 / 255.0), 2)
 # 顯示結(jié)果
 
 #plt.imshow(image, 'brg')
 #plt.show()
 cv2.imshow('img', image)
 
 
 
def show_info(cam):
 print 'POS_FRAMES:'+str(cam.get(1))
 print 'FRAME_COUNT:'+str(cam.get(7))
 print 'FORMAT:'+str(cam.get(8))
 print 'MODE:'+str(cam.get(9))
 print 'SATURATION:'+str(cam.get(12))
 print 'FPS:'+str(cam.get(5))
 
#CPU或GPU模型轉(zhuǎn)換
caffe.set_mode_gpu()
#caffe.set_mode_cpu()
#caffe.set_device(0)
 
caffe_root = '/var/smb/work/mycode/'
# 網(wǎng)絡(luò)參數(shù)（權(quán)重）文件
caffemodel = caffe_root + 'module/detectnet/snapshot_iter_2391.caffemodel'
# 網(wǎng)絡(luò)實(shí)施結(jié)構(gòu)配置文件
deploy = caffe_root + 'module/detectnet/deploy.prototxt'
 
 
img_root = caffe_root + 'data/'
 
# 網(wǎng)絡(luò)實(shí)施分類
net = caffe.Net(deploy, # 定義模型結(jié)構(gòu)
    caffemodel, # 包含了模型的訓(xùn)練權(quán)值
    caffe.TEST) # 使用測試模式(不執(zhí)行dropout)
 
# 加載ImageNet圖像均值 (隨著Caffe一起發(fā)布的)
print(os.environ['PYTHONPATH'])
#mu = np.load(os.environ['PYTHONPATH'] + '/caffe/imagenet/ilsvrc_2012_mean.npy')
#mu = mu.mean(1).mean(1) # 對(duì)所有像素值取平均以此獲取BGR的均值像素值
 
# 圖像預(yù)處理
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2,0,1))
#transformer.set_mean('data', mu)
transformer.set_raw_scale('data', 255)
transformer.set_channel_swap('data', (2,1,0))
 
# 處理圖像
cam = cv2.VideoCapture(0) 
if cam.isOpened():
 cam.set(3, 400)
 cam.set(4, 300)
 cam.set(5, 3)
 time.sleep(6)
 cam.set(15, -8.0)
 size = (int(cam.get(3)), int(cam.get(4)))
 print 'size:'
 print size
 
cv2.namedWindow('img', cv2.WINDOW_NORMAL)
 
# cnt=2
# while cnt:
#  cnt -= 1
while cam.isOpened():
 ret, img = cam.read()
 if ret:
  #show_info(cam)
  detect(img,net)
 
 if 0xFF == ord('q') & cv2.waitKey(5) == 27:
  break
 # time.sleep(0.033)
cam.release()
cv2.destroyAllWindows()

介面上會(huì)打印bbox也就是偵測到的目標(biāo)在圖像的座標(biāo)，另外請自行修改python代碼里的相關(guān)目錄，