【TensorRT】問題解決:The engine plan file is generated on an incompatible device, expecting compute 6.1 got compute 7.5, please rebuild. (Pytorch TensorRT)

➣ Reading Time: 5 minutes

前言

這篇是我在使用 TensorRT 的時候碰到的問題,出現了以下的訊息:

The engine plan file is generated on an incompatible device, expecting compute 6.1 got compute 7.5, please rebuild.

解決方法

此為 model 讀取 TensorRT 時,讀到了「不同 GPU 轉換而成的 trt weight」,
導致於 GPU cuda 無法解析。

請將對應的錯誤 GPU 版本 trt weight 刪除,並「重新轉換出符合此 GPU 的 trt weight」!

TensorRT recconvert code 範例

print('Start loading weight.')
self.net = MLModel(cfg=self.cfg, phase = 'test')
trained_model = "./core/weights/weight.pth"
self.net = self.load_model(self.net, trained_model, cpu)
self.net.eval().cuda()

print('Finished loading model!')
cudnn.benchmark = True
self.device = torch.device("cpu" if cpu else "cuda")

self.net = self.net.to(self.device)
x = torch.ones((1, 3, 720, 1280)).cuda()

tic = time.time()
print("Start Convert to TensorRT")
self.trt_net = torch2trt(self.net, [x])
# print(os.getcwd())
torch.save(self.trt_net.state_dict(), './core/weights/weight_trt.pth')
print(f"Conver to TensorRT cost {time.time() -tic} sec")
print("Convert to TensorRT Finish")

Reference

預設圖片
Howard Weng
我是 Howard Weng,很多人叫我嗡嗡。這個網站放了我的各種筆記。希望這些筆記也能順便幫助到有需要的人們!如果文章有幫助到你的話,歡迎幫我點讚哦!
文章: 391