Comparison to default CenterNet

README.md

@@ -1,3 +1,41 @@
+# Cycle-CenterNet
+Table structure parsing (TSP), Wired Table in the Wild (WTW).
+
+The unofficial Cycle-CenterNet architecture repository, based on the MMDetection fork.
+![](demo/schema.jpg)
+> [**Parsing table structures in the wild**]([http://openaccess.thecvf.com/content/ICCV2021/papers/Long_Parsing_Table_Structures_in_the_Wild_ICCV_2021_paper.pdf]),       
+> Long, R., Wang, W., Xue, N., Gao, F., Yang, Z., Wang, Y., & Xia, G. S.      
+
+Contact: [arkhipov.ai@phystech.edu](mailto:arkhipov.ai@phystech.edu). Any questions or discussions are welcome! 
+
+## Abstract 
+This paper tackles the problem of table structure parsing (TSP) from images in the wild. In contrast to existing studies that mainly focus on  parsing well-aligned tabular images with simple layouts from scanned PDF documents, we aim to establish a practical table structure parsing system for real-world scenarios where tabular input images are taken or scanned with severe deformation, bending or occlusions. For designing such a system, we propose an approach named Cycle-CenterNet on the top of CenterNet with a novel cycle-pairing module to simultaneously detect and group tabular cells into structured tables. In the cycle-pairing module, a new pairing loss function is proposed for the network training. Alongside with our Cycle-CenterNet, we also present a large-scale dataset, named Wired Table in the Wild (WTW), which includes well-annotated structure parsing of multiple style tables in several scenes like photo, scanning files, web pages, etc.. In experiments, we demonstrate that our Cycle-CenterNet consistently achieves the best accuracy of table structure parsing on the new WTW dataset by 24.6% absolute improvement evaluated by the TEDS metric. A more comprehensive experimental analysis also validates the advantages of our proposed methods for the TSP task.
+
+## Models
+
+[**Link**](https://drive.google.com/file/d/1ZQZycYwWXMlfZnvC9hFUCtlqd4eOwP8v/view?usp=share_link) to download model CenterNet(ResNet backbone). [**Config**](https://github.com/ArchieAlexArkhipov/Cycle-CenterNet/blob/master/configs/centernet/wtw_centernet_resnet34_dcnv2_150e_coco.py) 
+
+[**Link**](https://drive.google.com/file/d/1aZ2IF0tQq1Ino4QAsmrk8jcJ85urqsFE/view?usp=share_link) to download model CenterNet(DLA backbone). [**Config**](https://github.com/ArchieAlexArkhipov/Cycle-CenterNet/blob/master/configs/centernet/wtw_centernet_dla34_dcnv2_150e_coco.py) 
+
+[**Link**](https://drive.google.com/file/d/1oeqtA84eF_KJi953f8qX8cggDBjOVS3A/view?usp=share_link) to download model Cycle-CenterNet(DLA backbone) trained on bounding boxes. [**Config**](https://github.com/ArchieAlexArkhipov/Cycle-CenterNet/blob/master/configs/centernet/wtw_bbox_cycle_centernet_dla34_dcnv2_150e_coco.py) 
+
+[**Link**](https://drive.google.com/file/d/1taFK_co-9ofbL1pivhKrqHf8-icb1ZaK/view?usp=share_link) **to download model Cycle-CenterNet(DLA backbone) trained on bounding quadrangles**. [**Config**](https://github.com/ArchieAlexArkhipov/Cycle-CenterNet/blob/master/configs/centernet/wtw_quad_cycle_centernet_dla34_dcnv2_150e_coco.py) 
+
+## Installation
+
+```conda create --name openmmlab python=3.8 -y```
+
+```conda activate openmmlab```
+
+```pip install torch==1.8.1+cu101 torchvision==0.9.1+cu101 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html```
+
+```pip install mmcv-full==1.6.2 -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.8/index.html```
+
+```pip install -e .```
+
+```conda install -n openmmlab ipykernel --update-deps --force-reinstall```
+
+# MMDetection README
 <div align="center">
   <img src="resources/mmdet-logo.png" width="600"/>
   <div>&nbsp;</div>

configs/centernet/centernet_resnet18_dcnv2_140e_coco.py

@@ -1,124 +1,145 @@
 _base_ = [
-    '../_base_/datasets/coco_detection.py',
-    '../_base_/schedules/schedule_1x.py', '../_base_/default_runtime.py'
+    "../_base_/datasets/coco_detection.py",
+    "../_base_/schedules/schedule_1x.py",
+    "../_base_/default_runtime.py",
 ]
 
 model = dict(
-    type='CenterNet',
+    type="CenterNet",
     backbone=dict(
-        type='ResNet',
+        type="ResNet",
         depth=18,
         norm_eval=False,
-        norm_cfg=dict(type='BN'),
-        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet18')),
+        norm_cfg=dict(type="BN"),
+        init_cfg=dict(type="Pretrained", checkpoint="torchvision://resnet18"),
+    ),
     neck=dict(
-        type='CTResNetNeck',
+        type="CTResNetNeck",
         in_channel=512,
         num_deconv_filters=(256, 128, 64),
         num_deconv_kernels=(4, 4, 4),
-        use_dcn=True),
+        use_dcn=True,
+    ),
     bbox_head=dict(
-        type='CenterNetHead',
+        type="CenterNetHead",
         num_classes=80,
         in_channel=64,
         feat_channel=64,
-        loss_center_heatmap=dict(type='GaussianFocalLoss', loss_weight=1.0),
-        loss_wh=dict(type='L1Loss', loss_weight=0.1),
-        loss_offset=dict(type='L1Loss', loss_weight=1.0)),
+        loss_center_heatmap=dict(type="GaussianFocalLoss", loss_weight=1.0),
+        loss_wh=dict(type="L1Loss", loss_weight=0.1),
+        loss_offset=dict(type="L1Loss", loss_weight=1.0),
+    ),
     train_cfg=None,
-    test_cfg=dict(topk=100, local_maximum_kernel=3, max_per_img=100))
+    test_cfg=dict(topk=100, local_maximum_kernel=3, max_per_img=100),
+)
 
 # We fixed the incorrect img_norm_cfg problem in the source code.
-img_norm_cfg = dict(
-    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+img_norm_cfg = dict(mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
 
 train_pipeline = [
-    dict(type='LoadImageFromFile', to_float32=True, color_type='color'),
-    dict(type='LoadAnnotations', with_bbox=True),
+    dict(type="LoadImageFromFile", to_float32=True, color_type="color"),
+    dict(type="LoadAnnotations", with_bbox=True),
     dict(
-        type='PhotoMetricDistortion',
+        type="PhotoMetricDistortion",
         brightness_delta=32,
         contrast_range=(0.5, 1.5),
         saturation_range=(0.5, 1.5),
-        hue_delta=18),
+        hue_delta=18,
+    ),
     dict(
-        type='RandomCenterCropPad',
+        type="RandomCenterCropPad",
         crop_size=(512, 512),
         ratios=(0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3),
         mean=[0, 0, 0],
         std=[1, 1, 1],
         to_rgb=True,
-        test_pad_mode=None),
-    dict(type='Resize', img_scale=(512, 512), keep_ratio=True),
-    dict(type='RandomFlip', flip_ratio=0.5),
-    dict(type='Normalize', **img_norm_cfg),
-    dict(type='DefaultFormatBundle'),
-    dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
+        test_pad_mode=None,
+    ),
+    dict(type="Resize", img_scale=(512, 512), keep_ratio=True),
+    dict(type="RandomFlip", flip_ratio=0.5),
+    dict(type="Normalize", **img_norm_cfg),
+    dict(type="DefaultFormatBundle"),
+    dict(type="Collect", keys=["img", "gt_bboxes", "gt_labels"]),
 ]
 test_pipeline = [
-    dict(type='LoadImageFromFile', to_float32=True),
+    dict(type="LoadImageFromFile", to_float32=True),
     dict(
-        type='MultiScaleFlipAug',
+        type="MultiScaleFlipAug",
         scale_factor=1.0,
         flip=False,
         transforms=[
-            dict(type='Resize', keep_ratio=True),
+            dict(type="Resize", keep_ratio=True),
             dict(
-                type='RandomCenterCropPad',
+                type="RandomCenterCropPad",
                 ratios=None,
                 border=None,
                 mean=[0, 0, 0],
                 std=[1, 1, 1],
                 to_rgb=True,
                 test_mode=True,
-                test_pad_mode=['logical_or', 31],
-                test_pad_add_pix=1),
-            dict(type='RandomFlip'),
-            dict(type='Normalize', **img_norm_cfg),
-            dict(type='DefaultFormatBundle'),
+                test_pad_mode=["logical_or", 31],
+                test_pad_add_pix=1,
+            ),
+            dict(type="RandomFlip"),
+            dict(type="Normalize", **img_norm_cfg),
+            dict(type="DefaultFormatBundle"),
             dict(
-                type='Collect',
-                meta_keys=('filename', 'ori_filename', 'ori_shape',
-                           'img_shape', 'pad_shape', 'scale_factor', 'flip',
-                           'flip_direction', 'img_norm_cfg', 'border'),
-                keys=['img'])
-        ])
+                type="Collect",
+                meta_keys=(
+                    "filename",
+                    "ori_filename",
+                    "ori_shape",
+                    "img_shape",
+                    "pad_shape",
+                    "scale_factor",
+                    "flip",
+                    "flip_direction",
+                    "img_norm_cfg",
+                    "border",
+                ),
+                keys=["img"],
+            ),
+        ],
+    ),
 ]
 
-dataset_type = 'CocoDataset'
-data_root = 'data/coco/'
+dataset_type = "CocoDataset"
+data_root = "data/coco/"
 
 # Use RepeatDataset to speed up training
 data = dict(
     samples_per_gpu=16,
     workers_per_gpu=4,
     train=dict(
         _delete_=True,
-        type='RepeatDataset',
+        type="RepeatDataset",
         times=5,
         dataset=dict(
             type=dataset_type,
-            ann_file=data_root + 'annotations/instances_train2017.json',
-            img_prefix=data_root + 'train2017/',
-            pipeline=train_pipeline)),
+            ann_file=data_root + "annotations/instances_train2017.json",
+            img_prefix=data_root + "train2017/",
+            pipeline=train_pipeline,
+        ),
+    ),
     val=dict(pipeline=test_pipeline),
-    test=dict(pipeline=test_pipeline))
+    test=dict(pipeline=test_pipeline),
+)
 
 # optimizer
 # Based on the default settings of modern detectors, the SGD effect is better
 # than the Adam in the source code, so we use SGD default settings and
 # if you use adam+lr5e-4, the map is 29.1.
-optimizer_config = dict(
-    _delete_=True, grad_clip=dict(max_norm=35, norm_type=2))
+optimizer_config = dict(_delete_=True, grad_clip=dict(max_norm=35, norm_type=2))
 
 # learning policy
 # Based on the default settings of modern detectors, we added warmup settings.
 lr_config = dict(
-    policy='step',
-    warmup='linear',
+    policy="step",
+    warmup="linear",
     warmup_iters=1000,
     warmup_ratio=1.0 / 1000,
-    step=[18, 24])  # the real step is [18*5, 24*5]
+    step=[18, 24],
+)  # the real step is [18*5, 24*5]
 runner = dict(max_epochs=28)  # the real epoch is 28*5=140
 
 # NOTE: `auto_scale_lr` is for automatically scaling LR,