Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why performance of Mask_rcn tensorrt-fp16_dynamic-320x320-1344x1344 is bad #126

Open
azuryl opened this issue Jul 26, 2023 · 1 comment
Open

Comments

@azuryl
Copy link

azuryl commented Jul 26, 2023

python ./tools/test.py configs/mmdet/instance-seg/instance-seg_tensorrt-fp16_dynamic-320x320-1344x1344.py /data/azuryl/mmdetection_2.27.0/configs/mask_rcnn/mask_rcnn_r50_fpn_2x_coco.py --model /data/azuryl/mmdeploy_model/maskrcnn_f16_d320_1344/end2end.engine --metrics segm --device cuda:0
/data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
loading annotations into memory...
Done (t=2.09s)
creating index...
index created!
2021-07-23 02:54:25,221 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
2021-07-23 02:54:25,222 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 4952/4952, 4.0 task/s, elapsed: 1242s, ETA: 0s
Evaluating segm...
/data/azuryl/mmdetection_2.27.0/mmdet/datasets/coco.py:474: UserWarning: The key "bbox" is deleted for more accurate mask AP of small/medium/large instances since v2.12.0. This does not change the overall mAP calculation.
UserWarning)
Loading and preparing results...
DONE (t=8.15s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type segm
DONE (t=154.80s).
Accumulating evaluation results...
DONE (t=21.92s).

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.196
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.394
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.173
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.009
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.160
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.466
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.027
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.253
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.586

2021-07-23 03:19:01,735 - test - INFO - OrderedDict([('segm_mAP', 0.196), ('segm_mAP_50', 0.394), ('segm_mAP_75', 0.173), ('segm_mAP_s', 0.009), ('segm_mAP_m', 0.16), ('segm_mAP_l', 0.466), ('segm_mAP_copypaste', '0.196 0.394 0.173 0.009 0.160 0.466')])

@azuryl azuryl changed the title why performance of tensorrt-fp16_dynamic-320x320-1344x1344 is bad why performance of /mask_rcn tensorrt-fp16_dynamic-320x320-1344x1344 is bad Jul 26, 2023
@azuryl azuryl changed the title why performance of /mask_rcn tensorrt-fp16_dynamic-320x320-1344x1344 is bad why performance of Mask_rcn tensorrt-fp16_dynamic-320x320-1344x1344 is bad Jul 26, 2023
@grimoire
Copy link
Owner

grimoire commented Aug 2, 2023

This repo is developed on dependency

torch=1.8.1
tensorrt=8.0.1.6
mmdetection=2.18.0
cuda=11.1

which are relatively old versions.
I do not have much time to maintain this repo. Please move to mmdeploy which provide latest library support and more backends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants