Improve your Object Detection and Instance Segmentation Results for Detection of Small Objects

One of the important tasks of computer vision is to identify objects in real-time, what if we have large image of large size and the objects we need to identify are small? Now one of the methods we can can rely is SAHI.

They call it Slicing Aided Hyper Inference (SAHI) , it provides a generic slicing aided inference and finetuning pipeline for small object detection.

Try SAHI on Huggingface: https://huggingface.co/spaces/fcakyon/sahi-yolox

Installation of SAHI :

pip install -U sahi

Dependencies :

pip install torch torchvision mmdet mmcv-full yolov5 pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/index.html

• On Windows, Shapely needs to be installed via Conda:

conda install -c conda-forge shapely 

That’s it, now you can import and use any SAHI function in Python:

from sahi import get_sliced_prediction

Detecting Smaller Objects using Sliced Inference with SAHI:

Concept of sliced inference is basically; performing inference over smaller slices of the original image and then merging the sliced predictions on the original image.

# Using SAHI CLI 
sahi predict --source image/file/or/folder --model_path path/to/model --model_config_path path/to/config

will perform sliced inference on default parameters and export the prediction visuals to runs/predict/exp folder.

# To provide additional sliced parameters
sahi predict --slice_width 256 --slice_height 256 --overlap_height_ratio 0.1 --overlap_width_ratio 0.1 --model_confidence_threshold 0.25 --source image/file/or/folder --model_path path/to/model --model_config_path path/to/config

• Specify detection framework as --model_type mmdet for MMDetection or --model_type yolov5 for YOLOv5, to match with your model weight
• Specify postprocess type as --postprocess_type GREEDYNMM or --postprocess_type NMS to be applied over sliced predictions
• Specify postprocess match metric as --postprocess_match_metric IOS for intersection over smaller area or --postprocess_match_metric IOU for intersection over union
• Specify postprocess match threshold as --postprocess_match_threshold 0.5
• Add --postprocess_class_agnostic argument to ignore category ids of the predictions during postprocess (merging/nms)
• If you want to export prediction pickles and cropped predictions add --export_pickle and --export_crop arguments. If you want to change crop extension type, set it as --visual_export_format JPG.
• If you want to export prediction visuals, add --export_visual argument.
• By default, scripts apply both standard and sliced prediction (multi-stage inference). If you don’t want to perform sliced prediction add --no_sliced_prediction argument. If you don't want to perform standard prediction add --no_standard_prediction argument.
• If you want to perform prediction using a COCO annotation file, provide COCO json path as add --dataset_json_path dataset.json and coco image folder as --source path/to/coco/image/folder, predictions will be exported as a coco json file to runs/predict/exp/results.json. Then you can use coco_evaluation command to calculate COCO evaluation results or coco_error_analysis command to calculate detailed COCO error plots.

check here more SAHI CLI commands : https://github.com/obss/sahi/blob/main/docs/cli.md

You can have the flexibility to choose different models to inference with SAHI:

• YOLOv5 + SAHI walkthrough: https://colab.research.google.com/github/obss/sahi/blob/main/demo/inference_for_yolov5.ipynb
• MMDetection + SAHI walkthrough: https://colab.research.google.com/github/obss/sahi/blob/main/demo/inference_for_mmdetection.ipynb
• Detectron2 + SAHI walkthrough: https://colab.research.google.com/github/obss/sahi/blob/main/demo/inference_for_detectron2.ipynb

Slicing Operation

• Slice an image:

from sahi.slicing import slice_image
slice_image_result = slice_image(
    image=image_path,
    output_file_name=output_file_name,
    output_dir=output_dir,
    slice_height=256,
    slice_width=256,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2,
)

• Slice a COCO formatted dataset:

from sahi.slicing import slice_coco
coco_dict, coco_path = slice_coco(
    coco_annotation_file_path=coco_annotation_file_path,
    image_dir=image_dir,
    slice_height=256,
    slice_width=256,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2,
)

Inference between Normal and Sliced Inference:

Instance Segmentation result with SAHI :

We could spot some far away objects been detected. That is awesome !

For Error Analysis Plot and Interactive Result Visualization , refer below links:

https://github.com/obss/sahi/issues/357
https://github.com/obss/sahi/issues/356

Reference : https://github.com/obss/sahi

Paper : https://arxiv.org/pdf/2202.06934v2.pdf

Hope you learned something new today, Happy Learning!