應用機器人/感測器和感知/OpenCV/基本OpenCV教程
< 應用機器人 | 感測器和感知/Open CV
這篇文章解釋了使用OpenCV進行Python計算機視覺的基礎知識。
在Ubuntu（以及其他Debian衍生版本）上，你可以透過執行以下命令安裝OpenCV、Python和OpenCV-Python繫結所需的所有軟體包：sudo apt-get install python-opencv 這將安裝一個稍微過時的OpenCV版本（2.3.1），但這對於本課程中你所需的操作來說應該足夠了。如果你沒有使用Debian衍生版本，請諮詢你的包管理器或Google。
這是一個非常簡單的OpenCV程式。它開啟你的攝像頭，讀取影像，顯示它，並重復直到你按下鍵盤上的一個鍵。這是一個驗證你的OpenCV安裝是否有效的良好測試。
#!/usr/bin/python
import cv2
# Open video device
capture1 = cv2.VideoCapture(0)
while True:
ret, img = capture1.read() # Read an image
cv2.imshow("ImageWindow", img) # Display the image
if (cv2.waitKey(2) >= 0): # If the user presses a key, exit while loop
break
cv2.destroyAllWindows() # Close window
cv2.VideoCapture(0).release() # Release video device
該程式實現了任何OpenCV應用程式的基本功能。
#!/usr/bin/python
#
# ENGR421 -- Applied Robotics, Spring 2013
# OpenCV Python Demo
# Taj Morton <mortont@onid.orst.edu>
#
import sys
import cv2
import time
import numpy
import os
##
# Opens a video capture device with a resolution of 800x600
# at 30 FPS.
##
def open_camera(cam_id = 0):
cap = cv2.VideoCapture(cam_id)
cap.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT, 600);
cap.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH, 800);
cap.set(cv2.cv.CV_CAP_PROP_FPS, 30);
return cap
##
# Gets a frame from an open video device, or returns None
# if the capture could not be made.
##
def get_frame(device):
ret, img = device.read()
if (ret == False): # failed to capture
print >> sys.stderr, "Error capturing from video device."
return None
return img
##
# Closes all OpenCV windows and releases video capture device
# before exit.
##
def cleanup(cam_id = 0):
cv2.destroyAllWindows()
cv2.VideoCapture(cam_id).release()
##
# Creates a new RGB image of the specified size, initially
# filled with black.
##
def new_rgb_image(width, height):
image = numpy.zeros( (height, width, 3), numpy.uint8)
return image
########### Main Program ###########
if __name__ == "__main__":
# Camera ID to read video from (numbered from 0)
camera_id = 0
dev = open_camera(camera_id) # open the camera as a video capture device
while True:
img_orig = get_frame(dev) # Get a frame from the camera
if img_orig is not None: # if we did get an image
cv2.imshow("video", img_orig) # display the image in a window named "video"
else: # if we failed to capture (camera disconnected?), then quit
break
if (cv2.waitKey(2) >= 0): # If the user presses any key, exit the loop
break
cleanup(camera_id) # close video device and windows before we exit
簡化計算機視覺問題的一種常見方法是將影像轉換為黑白（“二進位制”）影像。用於決定什麼變為黑色和什麼變為白色的閾值決定了哪些特徵被忽略，哪些特徵對你的演算法很重要。許多閾值演算法期望影像輸入為灰度。以下函式將接收影像，將其轉換為灰度，然後將其在指定的閾值處轉換為二進位制影像。請確保在此處也包含模板程式（如上所述）中的函式。
##
# Converts an RGB image to grayscale, where each pixel
# now represents the intensity of the original image.
##
def rgb_to_gray(img):
return cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
##
# Converts an image into a binary image at the specified threshold.
# All pixels with a value <= threshold become 0, while
# pixels > threshold become 1
def do_threshold(image, threshold = 170):
(thresh, im_bw) = cv2.threshold(image, threshold, 255, cv2.THRESH_BINARY)
return (thresh, im_bw)
#####################################################
# If you have captured a frame from your camera like in the template program above,
# you can create a bitmap from it as follows:
img_gray = rgb_to_gray(img_orig) # Convert img_orig from video camera from RGB to Grayscale
# Converts grayscale image to a binary image with a threshold value of 220. Any pixel with an
# intensity of <= 220 will be black, while any pixel with an intensity > 220 will be white:
(thresh, img_threshold) = do_threshold(img_gray, 220)
cv2.imshow("Grayscale", img_gray)
cv2.imshow("Threshold", img_threshold)
一種簡單的目標檢測技術是使用輪廓查詢，它找到影像中形狀的輪廓。OpenCV可以返回它找到的輪廓的不同資訊，包括外輪廓、層次結構和簡單的列表。有關更多資訊，請參閱OpenCV文件中的findContours()函式（https://docs.opencv.tw/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#findcontours）。
在這個例子中，我們找到了每個形狀的最外輪廓（冰球的輪廓），並返回了物件的形狀近似值。
##
# Finds the outer contours of a binary image and returns a shape-approximation
# of them. Because we are only finding the outer contours, there is no object
# hierarchy returned.
##
def find_contours(image):
(contours, hierarchy) = cv2.findContours(image, mode=cv2.cv.CV_RETR_EXTERNAL, method=cv2.cv.CV_CHAIN_APPROX_SIMPLE)
return contours
#####################################################
# If you have created a binary image as above and stored it in "img_threshold"
# the following code will find the contours of your image:
contours = find_contours(img_threshold)
# Here, we are creating a new RBB image to display our results on
results_image = new_rgb_image(img_threshold.shape[1], img_threshold.shape[0])
cv2.drawContours(results_image, contours, -1, cv2.cv.RGB(255,0,0), 2)
# Display Results
cv2.imshow("results", results_image)
找到輪廓後，你可能想要找到形狀的中心。OpenCV可以透過找到輪廓的矩 (http://en.wikipedia.org/wiki/Image_moment) 來實現。以下函式可以用來查詢並顯示輪廓列表的質心。
##
# Finds the centroids of a list of contours returned by
# the find_contours (or cv2.findContours) function.
# If any moment of the contour is 0, the centroid is not computed. Therefore
# the number of centroids returned by this function may be smaller than
# the number of contours passed in.
#
# The return value from this function is a list of (x,y) pairs, where each
# (x,y) pair denotes the center of a contour.
##
def find_centers(contours):
centers = []
for contour in contours:
moments = cv2.moments(contour, True)
# If any moment is 0, discard the entire contour. This is
# to prevent division by zero.
if (len(filter(lambda x: x==0, moments.values())) > 0):
continue
center = (moments['m10']/moments['m00'] , moments['m01']/moments['m00'])
# Convert floating point contour center into an integer so that
# we can display it later.
center = map(lambda x: int(round(x)), center)
centers.append(center)
return centers
##
# Draws circles on an image from a list of (x,y) tuples
# (like those returned from find_centers()). Circles are
# drawn with a radius of 20 px and a line width of 2 px.
##
def draw_centers(centers, image):
for center in centers:
cv2.circle(image, tuple(center), 20, cv2.cv.RGB(0,255,255), 2)
#####################################################
# If you have computed the contours of your binary image as above
# and created an RGB image to show your algorithm's output, this will
# draw circles around each detected contour:
centers = find_centers(contours)
cv2.drawContours(results_image, contours, -1, cv2.cv.RGB(255,0,0), 2)
draw_centers(centers, results_image)
cv2.imshow("results", results_image)
由於你的攝像頭（可能）沒有直接安裝在球場中心的正上方，因此你需要對傳入的影片幀進行透視變換，以校正影像中的傾斜。通常情況下，這是透過選擇矩形中的4個已知位置，並找到旋轉/傾斜影像的矩陣來完成的，使原始影像上選擇的4個點被放置在正確的位置。本示例要求使用者在初始化過程中點選球場角落的4個點。一個更高階的演算法可以使用位於球場4個角落的基準點 (http://en.wikipedia.org/wiki/Fiduciary_marker)。
# Global variable containing the 4 points selected by the user in the corners of the board
corner_point_list = []
##
# This function is called by OpenCV when the user clicks
# anywhere in a window displaying an image.
##
def mouse_click_callback(event, x, y, flags, param):
if event == cv2.EVENT_LBUTTONDOWN:
print "Click at (%d,%d)" % (x,y)
corner_point_list.append( (x,y) )
##
# Computes a perspective transform matrix by capturing a single
# frame from a video source and displaying it to the user for
# corner selection.
#
# Parameters:
# * dev: Video Device (from open_camera())
# * board_size: A tuple/list with 2 elements containing the width and height (respectively) of the gameboard (in arbitrary units, like inches)
# * dpi: Scaling factor for elements of board_size
# * calib_file: Optional. If specified, the perspective transform matrix is saved under this filename.
# This file can be loaded later to bypass the calibration step (assuming nothing has moved).
##
def get_transform_matrix(dev, board_size, dpi, calib_file = None):
# Read a frame from the video device
img = get_frame(dev)
# Displace image to user
cv2.imshow("Calibrate", img)
# Register the mouse callback on this window. When
# the user clicks anywhere in the "Calibrate" window,
# the function mouse_click_callback() is called (defined above)
cv2.setMouseCallback("Calibrate", mouse_click_callback)
# Wait until the user has selected 4 points
while True:
# If the user has selected all 4 points, exit loop.
if (len(corner_point_list) >= 4):
print "Got 4 points: "+str(corner_point_list)
break
# If the user hits a key, exit loop, otherwise remain.
if (cv2.waitKey(10) >= 0):
break;
# Close the calibration window:
cv2.destroyWindow("Calibrate")
# If the user selected 4 points
if (len(corner_point_list) >= 4):
# Do calibration
# src is a list of 4 points on the original image selected by the user
# in the order [TOP_LEFT, BOTTOM_LEFT, TOP_RIGHT, BOTTOM_RIGHT]
src = numpy.array(corner_point_list, numpy.float32)
# dest is a list of where these 4 points should be located on the
# rectangular board (in the same order):
dest = numpy.array( [ (0, 0), (0, board_size[1]*dpi), (board_size[0]*dpi, 0), (board_size[0]*dpi, board_size[1]*dpi) ], numpy.float32)
# Calculate the perspective transform matrix
trans = cv2.getPerspectiveTransform(src, dest)
# If we were given a calibration filename, save this matrix to a file
if calib_file:
numpy.savetxt(calib_file, trans)
return trans
else:
return None
#####################################################
### Calibration Example ###
if __name__ == "__main__":
cam_id = 0
dev = open_camera(cam_id)
# The size of the board in inches, measured between the two
# robot boundaries:
board_size = [22.3125, 45]
# Number of pixels to display per inch in the final transformed image. This
# was selected somewhat arbitrarily (I chose 17 because it fit on my screen):
dpi = 17
# Calculate the perspective transform matrix
transform = get_transform_matrix(dev, board_size, dpi)
# Size (in pixels) of the transformed image
transform_size = (int(board_size[0]*dpi), int(board_size[1]*dpi))
while True:
img_orig = get_frame(dev)
if img_orig is not None: # if we did get an image
# Show the original (untransformed) image
cv2.imshow("video", img_orig)
# Apply the transformation matrix to skew the image and display it
img = cv2.warpPerspective(img_orig, transform, dsize=transform_size)
cv2.imshow("warped", img)
else: # if we failed to capture (camera disconnected?), then quit
break
if (cv2.waitKey(2) >= 0):
break
cleanup(cam_id)
將以上所有內容放在一起，我們可以從影片流中找到冰球及其中心。完整的原始碼可以從 http://classes.engr.oregonstate.edu/engr/spring2013/engr421-001/code/opencv/puck_detector.py 下載。
從左到右：原始影像 透視變換後的影像 二進位制影像（閾值化） 影像的輪廓（紅色）和質心（藍色）
如果你有一個需要更改設定才能在Linux上正常工作的攝像頭，請嘗試使用“uvcdynctrl”工具。為了使羅技網路攝像頭正常工作，我必須更改它們的曝光設定，以防止它們不斷調整設定。不幸的是，OpenCV無法控制網路攝像頭的所有方面，因此你需要使用外部工具。以下是如何使用“uvcdynctrl”來顯示可用裝置、可用控制元件，以及如何查詢和設定控制元件值的一個示例。
# Available Devices
$ uvcdynctrl --list
Listing available devices:
video1 Camera
Media controller device /dev/media1 doesn't exist
ERROR: Unable to list device entities: Invalid device or device cannot be opened. (Code: 5)
video0 Integrated Camera
Media controller device /dev/media0 doesn't exist
ERROR: Unable to list device entities: Invalid device or device cannot be opened. (Code: 5)
# Available Controls
$ uvcdynctrl -d video1 --clist
Listing available controls for device video1:
Hue
Exposure
Gain
# Get a value
$ uvcdynctrl -d video1 -g 'Exposure'
60
# Set a Value
$ uvcdynctrl -d video1 -s 'Exposure' -- 30
$ uvcdynctrl -d video1 -g 'Exposure'
30