基于FPGA的實時雙目立體視覺系統(tǒng)的設(shè)計與實現(xiàn)
發(fā)布時間：2019-05-10 20:39
【摘要】：立體視覺技術(shù)是計算機視覺領(lǐng)域中非常重要的研究方向,其通過匹配一對不同視角的圖像來提取現(xiàn)實場景中的深度信息,該技術(shù)在無人駕駛、無人機、虛擬現(xiàn)實、人機交互以及3DTV等領(lǐng)域廣泛應(yīng)用。在過去幾十年中,提出了許多算法以及設(shè)計平臺來提高系統(tǒng)的精確度與實時性,然而由于在獲取立體視覺的深度信息時,其運算量巨大且算法復雜度較高,所以實時獲得高質(zhì)量的深度信息仍然具有較大的挑戰(zhàn)性。本文主要提出了基于FPGA+ARM架構(gòu)的實時雙目立體視覺的軟、硬件系統(tǒng)設(shè)計方案。根據(jù)系統(tǒng)方案要求,系統(tǒng)主要包括圖像傳感器視頻的高速接收、極線幾何校正、Census變換、半全局立體匹配算法、左右一致性校驗以及中值濾波等關(guān)鍵模塊。整個立體視覺的系統(tǒng)在單一的Xilinx ZC706開發(fā)板上實現(xiàn),硬件平臺以XC7Z045芯片為處理器核心,根據(jù)對系統(tǒng)軟、硬件功能劃分,PS(ARM)端主要是實現(xiàn)軟件控制與算法的調(diào)試,PL(FPGA)端主要實現(xiàn)視覺算法的并行處理的硬件加速。根據(jù)系統(tǒng)支持不同的分辨率輸出以及算法的配置模式,系統(tǒng)采用了模塊化、參數(shù)化的設(shè)計思想。在系統(tǒng)設(shè)計過程中,攝像頭采用APTINA的MT9V034,接口數(shù)據(jù)為高速LVDS串行傳輸,可以減少噪聲等干擾;極線幾何校正采用二元回歸多項式方程來模擬矩陣相乘,減少兩幅圖像坐標存儲,大大減少了片上硬件邏輯資源。多項式計算采用流水線技術(shù),明顯提高系統(tǒng)的工作頻率;由于半全局立體匹配算法的復雜度以及對存儲器的要求很高,本文采用有效的計算代價函數(shù)的方法,并簡化為4個方向進行代價聚合,設(shè)計并行緩存器,降低對存儲器的要求。中值濾波可以濾除誤匹配點以及動態(tài)規(guī)劃帶來的條紋等問題。實驗結(jié)果表明,本設(shè)計的系統(tǒng)可以達到視頻的實時處理要求,圖像的分辨率為640×480,最大視差搜索范圍為64個像素,幀率60 fps。對于匹配算法模塊的工作頻率可以達到130 MHz,可以實現(xiàn)1280×1024@72.2 fps的實時視頻處理。
[Abstract]:Stereo vision technology is a very important research direction in the field of computer vision. It extracts depth information from real scene by matching a pair of images with different angles of view. This technology is used in unmanned, unmanned aerial vehicles, virtual reality, and so on. Human-computer interaction and 3DTV are widely used. In the past decades, many algorithms and design platforms have been proposed to improve the accuracy and real-time performance of the system. However, because of the huge computational complexity and the high complexity of the algorithm, the depth information of stereo vision is obtained. Therefore, it is still challenging to obtain high quality depth information in real time. In this paper, the software and hardware design scheme of real-time binocular stereo vision based on FPGA ARM architecture is proposed. According to the requirements of the system scheme, the system mainly includes high-speed video reception of image sensor, polar geometric correction, Census transform, semi-global stereo matching algorithm, left and right consistency check and median filtering and other key modules. The whole stereo vision system is implemented on a single Xilinx ZC706 development board. The hardware platform is based on the XC7Z045 chip as the processor core. According to the software and hardware functions of the system, the, PS (ARM) terminal is divided into two parts: software control and algorithm debugging. The PL (FPGA) side mainly realizes the hardware acceleration of parallel processing of visual algorithm. According to the different resolution output and the configuration mode of the algorithm, the system adopts the design idea of modularization and parametrization. In the process of system design, the camera uses APTINA MT9V034, interface data for high-speed LVDS serial transmission, which can reduce noise and other interference. The polar line geometry correction uses the binary regression polynomial equation to simulate the matrix multiplication, reduces the coordinate storage of two images, and greatly reduces the on-chip hardware logic resources. The pipeline technology is used to calculate the multinomial, which obviously improves the working frequency of the system. Because of the complexity of semi-global stereo matching algorithm and the high requirement for memory, this paper adopts an effective method of computing cost function, and simplifies it to four directions for cost aggregation, designs parallel buffer, and reduces the requirement of memory. Median filtering can filter out mismatching points and stripes caused by dynamic programming. The experimental results show that the system can meet the requirements of real-time video processing. The resolution of the image is 640x480, the maximum disparity search range is 64 pixels, and the frame rate is 60 fps.. For the matching algorithm module, the working frequency of the matching algorithm module can reach 130 MHz, and the real-time video processing of 1280 脳 104 鈮，
本文編號：2473972
[Abstract]:Stereo vision technology is a very important research direction in the field of computer vision. It extracts depth information from real scene by matching a pair of images with different angles of view. This technology is used in unmanned, unmanned aerial vehicles, virtual reality, and so on. Human-computer interaction and 3DTV are widely used. In the past decades, many algorithms and design platforms have been proposed to improve the accuracy and real-time performance of the system. However, because of the huge computational complexity and the high complexity of the algorithm, the depth information of stereo vision is obtained. Therefore, it is still challenging to obtain high quality depth information in real time. In this paper, the software and hardware design scheme of real-time binocular stereo vision based on FPGA ARM architecture is proposed. According to the requirements of the system scheme, the system mainly includes high-speed video reception of image sensor, polar geometric correction, Census transform, semi-global stereo matching algorithm, left and right consistency check and median filtering and other key modules. The whole stereo vision system is implemented on a single Xilinx ZC706 development board. The hardware platform is based on the XC7Z045 chip as the processor core. According to the software and hardware functions of the system, the, PS (ARM) terminal is divided into two parts: software control and algorithm debugging. The PL (FPGA) side mainly realizes the hardware acceleration of parallel processing of visual algorithm. According to the different resolution output and the configuration mode of the algorithm, the system adopts the design idea of modularization and parametrization. In the process of system design, the camera uses APTINA MT9V034, interface data for high-speed LVDS serial transmission, which can reduce noise and other interference. The polar line geometry correction uses the binary regression polynomial equation to simulate the matrix multiplication, reduces the coordinate storage of two images, and greatly reduces the on-chip hardware logic resources. The pipeline technology is used to calculate the multinomial, which obviously improves the working frequency of the system. Because of the complexity of semi-global stereo matching algorithm and the high requirement for memory, this paper adopts an effective method of computing cost function, and simplifies it to four directions for cost aggregation, designs parallel buffer, and reduces the requirement of memory. Median filtering can filter out mismatching points and stripes caused by dynamic programming. The experimental results show that the system can meet the requirements of real-time video processing. The resolution of the image is 640x480, the maximum disparity search range is 64 pixels, and the frame rate is 60 fps.. For the matching algorithm module, the working frequency of the matching algorithm module can reach 130 MHz, and the real-time video processing of 1280 脳 104 鈮，
本文編號：2473972
本文鏈接：http://www.sikaile.net/kejilunwen/ruanjiangongchenglunwen/2473972.html
最近更新
教材專著
