【实战】yolov8 tensorrt ubuntu部署
<h1 id="ubuntu1804-yolov8-tensorrt模型加速部署实战">ubuntu18.04 yolov8 tensorrt模型加速部署【实战】</h1><p><strong>TensorRT-Alpha</strong>基于tensorrt+cuda c++实现模型end2end的gpu加速，支持win10、linux，在2023年已经更新模型：YOLOv8, YOLOv7, YOLOv6, YOLOv5, YOLOv4, YOLOv3, YOLOX, YOLOR,pphumanseg,u2net,EfficientDet。<br>
Windows10教程正在制作，可以关注仓库：https://github.com/FeiYull/TensorRT-Alpha</p>
<p>【Win10下yolov8 tensorrt模型加速部署【实战】】：https://www.cnblogs.com/feiyull/p/17092046.html</p>
<h1 id="一加速结果展示">一、加速结果展示</h1>
<h2 id="11-性能速览">1.1 性能速览</h2>
<p>🚀快速看看yolov8n 在移动端RTX2070m(8G)的新能表现：</p>
<div align="center">
<table>
<thead>
<tr>
<th style="text-align: center">model</th>
<th style="text-align: center">video resolution</th>
<th style="text-align: center">model input size</th>
<th style="text-align: center">GPU Memory-Usage</th>
<th style="text-align: center">GPU-Util</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: center">yolov8n</td>
<td style="text-align: center">1920x1080</td>
<td style="text-align: center">8x3x640x640</td>
<td style="text-align: center">1093MiB/7982MiB</td>
<td style="text-align: center">14%</td>
</tr>
</tbody>
</table>
<p>下图是yolov8n的运行时间开销，单位是ms：<br>
<img src="https://img-blog.csdnimg.cn/51ed467668e24b8c982008903733a6e2.jpeg#pic_center"></p>
</div>
<p>更多TensorRT-Alpha测试录像在B站视频：<br>
<strong>B站：YOLOv8n</strong><br>
<strong>B站：YOLOv8s</strong></p>
<p><img src="https://img-blog.csdnimg.cn/78afd19e93464dd8bf8c4d12cf129147.jpeg#pic_center"></p>
<h2 id="12精度对齐">1.2精度对齐</h2>
<p>下面是左边是python框架推理结果，右边是TensorRT-Alpha推理结果。<br>
<img src="https://img-blog.csdnimg.cn/56466c1923f84f43a5ef52f21cd74e99.jpeg#pic_center"></p>
<div align="center">
yolov8n : Offical( left ) vs Ours( right )
</div>
<br>
<p><img src="https://img-blog.csdnimg.cn/0ea125d5f6754251a44c913ade381905.jpeg#pic_center"></p>
<div align="center">
yolov7-tiny : Offical( left ) vs Ours( right )
</div>
<br>
<p><img src="https://img-blog.csdnimg.cn/8f718f04409b44c6b1cb00d93bbd6fb3.jpeg#pic_center"></p>
<div align="center">
yolov6s : Offical( left ) vs Ours( right )
</div>
<br>
<p><img src="https://img-blog.csdnimg.cn/cd2e2ed1e1554aa8a35795541ef08058.jpeg#pic_center"></p>
<div align="center">
yolov5s : Offical( left ) vs Ours( right )
</div>
<br>
<p>YOLOv4 YOLOv3 YOLOR YOLOX略。</p>
<h1 id="二ubuntu1804环境配置">二、Ubuntu18.04环境配置</h1>
<p>如果您对tensorrt不是很熟悉，请务必保持下面库版本一致。</p>
<h2 id="21-安装工具链和opencv">2.1 安装工具链和opencv</h2>
<pre><code class="language-bash">sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install git
sudo apt-get install gdb
sudo apt-get install cmake
</code></pre>
<pre><code class="language-bash">sudo apt-get install libopencv-dev
# pkg-config --modversion opencv
</code></pre>
<h2 id="22-安装nvidia相关库">2.2 安装Nvidia相关库</h2>
<p>注：Nvidia相关网站需要注册账号。</p>
<h3 id="221-安装nvidia显卡驱动">2.2.1 安装Nvidia显卡驱动</h3>
<pre><code class="language-bash">ubuntu-drivers devices
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt install nvidia-driver-470-server # for ubuntu18.04
nvidia-smi
</code></pre>
<h3 id="222-安装-cuda113">2.2.2 安装 cuda11.3</h3>
<ul>
<li>进入链接: https://developer.nvidia.com/cuda-toolkit-archive</li>
<li>选择：CUDA Toolkit 11.3.0(April 2021)</li>
<li>选择： -> -> -> -> <br><br>
在网页你能看到下面安装命令，我这里已经拷贝下来：</li>
</ul>
<pre><code class="language-bash">wget https://developer.download.nvidia.com/compute/cuda/11.3.0/local_installers/cuda_11.3.0_465.19.01_linux.run
sudo sh cuda_11.3.0_465.19.01_linux.run
</code></pre>
<p>cuda的安装过程中，需要你在bash窗口手动作一些选择，这里选择如下：</p>
<ul>
<li>select： -> -> 接着按下回车键取消Driver和465.19.01这个选项，如下图(<font color="#dd0000"><strong>it is important!</strong></font>) -> <br><br>
<img src="https://img-blog.csdnimg.cn/68a9b9252ef648e2b90ba48a59dd9137.jpeg#pic_center"><br>
bash窗口提示如下表示安装完成</li>
</ul>
<pre><code class="language-bash">#===========
#= Summary =
#===========
#Driver: Not Selected
#Toolkit:Installed in /usr/local/cuda-11.3/
#......
</code></pre>
<p>把cuda添加到环境变量：</p>
<pre><code class="language-bash">vim ~/.bashrc
</code></pre>
<p>把下面拷贝到 .bashrc里面</p>
<pre><code class="language-bash"># cuda v11.3
export PATH=/usr/local/cuda-11.3/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.3/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export CUDA_HOME=/usr/local/cuda-11.3
</code></pre>
<p>刷新环境变量和验证</p>
<pre><code class="language-bash">source ~/.bashrc
nvcc -V
</code></pre>
<p>bash窗口打印如下信息表示cuda11.3安装正常<br></p>
<pre><code class="language-bash">nvcc: NVIDIA (R) Cuda compiler driver<br>
Copyright (c) 2005-2021 NVIDIA Corporation<br>
Built on Sun_Mar_21_19:15:46_PDT_2021<br>
Cuda compilation tools, release 11.3, V11.3.58<br>
Build cuda_11.3.r11.3/compiler.29745058_0<br>
</code></pre>
<h3 id="223-安装-cudnn82">2.2.3 安装 cudnn8.2</h3>
<ul>
<li>进入网站：https://developer.nvidia.com/rdp/cudnn-archive</li>
<li>选择： Download cuDNN v8.2.0 (April 23rd, 2021), for CUDA 11.x</li>
<li>选择： cuDNN Library for Linux (x86_64)</li>
<li>你将会下载这个压缩包:"cudnn-11.3-linux-x64-v8.2.0.53.tgz"</li>
</ul>
<pre><code class="language-bash"># 解压
tar -zxvf cudnn-11.3-linux-x64-v8.2.0.53.tgz
</code></pre>
<p>将cudnn的头文件和lib拷贝到cuda11.3的安装目录下：</p>
<pre><code class="language-bash">sudo cp cuda/include/cudnn.h /usr/local/cuda/include/
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64/
sudo chmod a+r /usr/local/cuda/include/cudnn.h
sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
</code></pre>
<h3 id="224-下载-tensorrt8424">2.2.4 下载 tensorrt8.4.2.4</h3>
<p>本教程中，tensorrt只需要下载\、解压即可，不需要安装。</p>
<ul>
<li>进入网站： https://developer.nvidia.cn/nvidia-tensorrt-8x-download</li>
<li>把这个打勾： I Agree To the Terms of the NVIDIA TensorRT License Agreement</li>
<li>选择: TensorRT 8.4 GA Update 1</li>
<li>选择: TensorRT 8.4 GA Update 1 for Linux x86_64 and CUDA 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6 and 11.7 TAR Package</li>
<li>你将会下载这个压缩包:"TensorRT-8.4.2.4.Linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz"</li>
</ul>
<pre><code class="language-bash"># 解压
tar -zxvf TensorRT-8.4.2.4.Linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz
# 快速验证一下tensorrt+cuda+cudnn是否安装正常
cd TensorRT-8.4.2.4/samples/sampleMNIST
make
cd ../../bin/
</code></pre>
<p>导出tensorrt环境变量(<font color="#dd0000"><strong>it is important!</strong></font>)，注：将LD_LIBRARY_PATH:后面的路径换成你自己的！后续编译onnx模型的时候也需要执行下面第一行命令</p>
<pre><code class="language-bash">export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/xxx/temp/TensorRT-8.4.2.4/lib
./sample_mnist
</code></pre>
<p>bash窗口打印类似如下图的手写数字识别表明cuda+cudnn+tensorrt安装正常<br>
<img src="https://img-blog.csdnimg.cn/928d777b0490452cad3e2814eaf4d6da.png#pic_center"></p>
<h3 id="225-下载仓库tensorrt-alpha并设置">2.2.5 下载仓库TensorRT-Alpha并设置</h3>
<pre><code class="language-bash">git clone https://github.com/FeiYull/tensorrt-alpha
</code></pre>
<p>设置您自己TensorRT根目录:</p>
<pre><code class="language-bash">git clone https://github.com/FeiYull/tensorrt-alpha
cd tensorrt-alpha/cmake
vim common.cmake
# 在文件common.cmake中的第20行中，设置成你自己的目录，别和我设置一样的路径eg:
# set(TensorRT_ROOT /root/TensorRT-8.4.2.4)
</code></pre>
<h1 id="三yolov8模型部署">三、YOLOv8模型部署</h1>
<h2 id="31-获取onnx文件">3.1 获取onnx文件</h2>
<p>直接在网盘下载 weiyun or google driver 或者使用如下命令导出onnx:</p>
<pre><code class="language-bash"># 🔥 yolov8 官方仓库: https://github.com/ultralytics/ultralytics
# 🔥 yolov8 官方教程: https://docs.ultralytics.com/quickstart/
# 🚀TensorRT-Alpha will be updated synchronously as soon as possible!
# 安装 yolov8
conda create -n yolov8 python==3.8 -y
conda activate yolov8
pip install ultralytics==8.0.5
pip install onnx==1.12.0
# 下载官方权重(".pt" file)
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8l.pt
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x.pt
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x6.pt
</code></pre>
<p>导出 onnx:</p>
<pre><code class="language-bash"># 640
yolo mode=export model=yolov8n.pt format=onnx dynamic=True #simplify=True
yolo mode=export model=yolov8s.pt format=onnx dynamic=True #simplify=True
yolo mode=export model=yolov8m.pt format=onnx dynamic=True #simplify=True
yolo mode=export model=yolov8l.pt format=onnx dynamic=True #simplify=True
yolo mode=export model=yolov8x.pt format=onnx dynamic=True #simplify=True
# 1280
yolo mode=export model=yolov8x6.pt format=onnx dynamic=True #simplify=True
</code></pre>
<h2 id="32-编译-onnx">3.2 编译 onnx</h2>
<pre><code class="language-bash"># 把你的onnx文件放到这个路径:tensorrt-alpha/data/yolov8
cd tensorrt-alpha/data/yolov8
# 请把LD_LIBRARY_PATH:换成您自己的路径。
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/TensorRT-8.4.2.4/lib
# 640
../../../../TensorRT-8.4.2.4/bin/trtexec --onnx=yolov8n.onnx--saveEngine=yolov8n.trt--buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
../../../../TensorRT-8.4.2.4/bin/trtexec --onnx=yolov8s.onnx--saveEngine=yolov8s.trt--buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
../../../../TensorRT-8.4.2.4/bin/trtexec --onnx=yolov8m.onnx--saveEngine=yolov8m.trt--buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
../../../../TensorRT-8.4.2.4/bin/trtexec --onnx=yolov8l.onnx--saveEngine=yolov8l.trt--buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
../../../../TensorRT-8.4.2.4/bin/trtexec --onnx=yolov8x.onnx--saveEngine=yolov8x.trt--buildOnly --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640
# 1280
../../../../TensorRT-8.4.2.4/bin/trtexec --onnx=yolov8x6.onnx--saveEngine=yolov8x6.trt--buildOnly --minShapes=images:1x3x1280x1280 --optShapes=images:4x3x1280x1280 --maxShapes=images:8x3x1280x1280
</code></pre>
<p>你将会的到例如：yolov8n.trt、yolov8s.trt、yolov8m.trt等文件。</p>
<h2 id="33-编译运行">3.3 编译运行</h2>
<pre><code class="language-bash">git clone https://github.com/FeiYull/tensorrt-alpha
cd tensorrt-alpha/yolov8
mkdir build
cd build
cmake ..
make -j10
# 注: 效果图默认保存在路径 tensorrt-alpha/yolov8/build
# 下面参数解释
# --show 表示可视化结果
# --savePath 表示保存，默认保存在build目录
# --savePath=../ 保存在上一级目录
## 640
# 推理图片
./app_yolov8--model=../../data/yolov8/yolov8n.trt --size=640 --batch_size=1--img=../../data/6406407.jpg --show --savePath
./app_yolov8--model=../../data/yolov8/yolov8n.trt --size=640 --batch_size=8--video=../../data/people.mp4--show --savePath
# 推理视频
./app_yolov8--model=../../data/yolov8/yolov8n.trt --size=640 --batch_size=8--video=../../data/people.mp4--show --savePath=../
# 在线推理相机视频
./app_yolov8--model=../../data/yolov8/yolov8n.trt --size=640 --batch_size=2--cam_id=0--show
## 1280
# infer camera
./app_yolov8--model=../../data/yolov8/yolov8x6.trt --size=1280 --batch_size=2--cam_id=0--show
</code></pre>
<h1 id="四参考">四、参考</h1>
<p>https://github.com/FeiYull/TensorRT-Alpha</p>
</div>
<div id="MySignature" role="contentinfo">
CV&DL<br><br>
来源：https://www.cnblogs.com/feiyull/p/17066486.html
頁:
[1]