Segmentation/Detection Methods
This post was migrated from Tistory. You can find the original here.
Semantic Segmentation
“What is each pixel?”
Image
→ CNN Encoder
→ Feature Map (C×h×w)
→ 1×1 Conv (class projection)
→ Upsampling / Decoder
→ Segmentation Map (H×W×Class)
We remove the FC layers used in classification networks and turn the network into a fully convolutional architecture.
In this structure, the output stays in the form of a spatial feature map, so positional information is preserved.
And since there are no FC layers, the network can handle inputs of various resolutions without any constraint on input size.
(An FC layer is connected to every neuron in the previous layer, so if the input size changes, the number of weights changes too — which is why the input size has to stay fixed.)
As a result, we can perform classification while retaining positional information, so we know what object is present in each pixel region.
However, since the spatial dimensions shrink as the input passes through the conv layers,
we need an upsampling step — deconvolution, unpooling, etc. — to expand them back.
Once that’s done, we get the semantic segmentation of the image.
There are also many other segmentation methods and variants beyond this.
Detection
“What is where?”
There are 2 tasks that need to be performed:
- Estimate bounding boxes, either with an algorithm or a neural network.
- Predict a class for each bounding box.
If both tasks are performed simultaneously, it’s a One-Stage-Detector model.
If the two are done sequentially, one stage at a time, it’s a Two-Stage-Detector model.
e.g. One-Stage-Detector - YOLO, SSD, RetinaNet
e.g. Two-Stage-Detector - R-CNN, Mask R-CNN, Cascade R-CNN
Left - Faster R-CNN, Right - YOLO
Left - Faster R-CNN, Right - YOLO
Depending on the detection model, there are various approaches and methods for carrying out these two tasks.