Mask RCNN Code Reading

BackBone :

• ResNet and FPN FPN的code其實作者寫在了model.py的Class MaskRCNN裡面並沒有獨立出來, 而其餘ResNet有

Region Proposal Network:

• Anchor Boxes Generate 產生anchor boxes的code
• RPN RPN網路架構的code
• Proposal Layer 將Anchor Boxes Generate與RPN結合的部分並會經過NMS

TASK_HEAD:

• FPN Classifier Graph 有關於ROI Align Layer還有分類的網路架構的部分
• Build FPN Mask Graph 有關於ROI Align Layer還有Mask的網路架構的部分
• Detection Layer FPN Classifier Graph後,按照confidence還有數量篩選, 算是後處理了
• Detection Target Layer Proposal Layer後,Training的code
• Mask RCNN Loss有關於所有的loss

如果是training的話

if mode == "training":
   # Class ID mask to mark class IDs supported by the dataset the image
   # came from.
   active_class_ids = KL.Lambda(
    lambda x: parse_image_meta_graph(x)["active_class_ids"]
    )(input_image_meta)
   
   if not config.USE_RPN_ROIS:
    # Ignore predicted ROIs and use ROIs provided as an input.
    input_rois = KL.Input(shape=[config.POST_NMS_ROIS_TRAINING, 4],
                          name="input_roi", dtype=np.int32)
    # Normalize coordinates
    target_rois = KL.Lambda(lambda x: norm_boxes_graph(
        x, K.shape(input_image)[1:3]))(input_rois)
   else:
    target_rois = rpn_rois
   
   # Generate detection targets
   # Subsamples proposals and generates target outputs for training
   # Note that proposal class IDs, gt_boxes, and gt_masks are zero
   # padded. Equally, returned rois and targets are zero padded.
   rois, target_class_ids, target_bbox, target_mask =\
    DetectionTargetLayer(config, name="proposal_targets")([
        target_rois, input_gt_class_ids, gt_boxes, input_gt_masks])
   
   # Network Heads
   # TODO: verify that this handles zero padded ROIs
   mrcnn_class_logits, mrcnn_class, mrcnn_bbox =\
    fpn_classifier_graph(rois, mrcnn_feature_maps, input_image_meta,
                         config.POOL_SIZE, config.NUM_CLASSES,
                         train_bn=config.TRAIN_BN)
   
   mrcnn_mask = build_fpn_mask_graph(rois, mrcnn_feature_maps,
                                  input_image_meta,
                                  config.MASK_POOL_SIZE,
                                  config.NUM_CLASSES,
                                  train_bn=config.TRAIN_BN)
   
   # TODO: clean up (use tf.identify if necessary)
   output_rois = KL.Lambda(lambda x: x * 1, name="output_rois")(rois)
   
   # Losses
   rpn_class_loss = KL.Lambda(lambda x: rpn_class_loss_graph(*x), name="rpn_class_loss")(
    [input_rpn_match, rpn_class_logits])
   rpn_bbox_loss = KL.Lambda(lambda x: rpn_bbox_loss_graph(config, *x), name="rpn_bbox_loss")(
    [input_rpn_bbox, input_rpn_match, rpn_bbox])
   class_loss = KL.Lambda(lambda x: mrcnn_class_loss_graph(*x), name="mrcnn_class_loss")(
    [target_class_ids, mrcnn_class_logits, active_class_ids])
   bbox_loss = KL.Lambda(lambda x: mrcnn_bbox_loss_graph(*x), name="mrcnn_bbox_loss")(
    [target_bbox, target_class_ids, mrcnn_bbox])
   mask_loss = KL.Lambda(lambda x: mrcnn_mask_loss_graph(*x), name="mrcnn_mask_loss")(
    [target_mask, target_class_ids, mrcnn_mask])
   
   # Model
   inputs = [input_image, input_image_meta,
          input_rpn_match, input_rpn_bbox, input_gt_class_ids, input_gt_boxes, input_gt_masks]
   if not config.USE_RPN_ROIS:
    inputs.append(input_rois)
   outputs = [rpn_class_logits, rpn_class, rpn_bbox,
           mrcnn_class_logits, mrcnn_class, mrcnn_bbox, mrcnn_mask,
           rpn_rois, output_rois,
           rpn_class_loss, rpn_bbox_loss, class_loss, bbox_loss, mask_loss]
   model = KM.Model(inputs, outputs, name='mask_rcnn')

如果是測試的話

else:
    # Network Heads
    # Proposal classifier and BBox regressor heads
    mrcnn_class_logits, mrcnn_class, mrcnn_bbox =\
     fpn_classifier_graph(rpn_rois, mrcnn_feature_maps, input_image_meta,
                          config.POOL_SIZE, config.NUM_CLASSES,
                          train_bn=config.TRAIN_BN)

    # Detections
    # output is [batch, num_detections, (y1, x1, y2, x2, class_id, score)] in 
    # normalized coordinates
    detections = DetectionLayer(config, name="mrcnn_detection")(
     [rpn_rois, mrcnn_class, mrcnn_bbox, input_image_meta])

    # Create masks for detections
    detection_boxes = KL.Lambda(lambda x: x[..., :4])(detections)
    mrcnn_mask = build_fpn_mask_graph(detection_boxes, mrcnn_feature_maps,
                                   input_image_meta,
                                   config.MASK_POOL_SIZE,
                                   config.NUM_CLASSES,
                                   train_bn=config.TRAIN_BN)

    model = KM.Model([input_image, input_image_meta, input_anchors],
                  [detections, mrcnn_class, mrcnn_bbox,
                      mrcnn_mask, rpn_rois, rpn_class, rpn_bbox],
                  name='mask_rcnn')

以下是別人網誌上的舊圖 不太依樣了 build_classfier_grpah拿掉了 上圖是inference下圖是train