Exploit CAM by itself: Complementary Learning System for Weakly Supervised Semantic Segmentation
Abstract: Weakly Supervised Semantic Segmentation (WSSS) with image-level labels has long been suffering from fragmentary object regions led by Class Activation Map (CAM), which is incapable of generating fine-grained masks for semantic segmentation. To guide CAM to find more non-discriminating object patterns, this paper turns to an interesting working mechanism in agent learning named Complementary Learning System (CLS). CLS holds that the neocortex builds a sensation of general knowledge, while the hippocampus specially learns specific details, completing the learned patterns. Motivated by this simple but effective learning pattern, we propose a General-Specific Learning Mechanism (GSLM) to explicitly drive a coarse-grained CAM to a fine-grained pseudo mask. Specifically, GSLM develops a General Learning Module (GLM) and a Specific Learning Module (SLM). The GLM is trained with image-level supervision to extract coarse and general localization representations from CAM. Based on the general knowledge in the GLM, the SLM progressively exploits the specific spatial knowledge from the localization representations, expanding the CAM in an explicit way. To this end, we propose the Seed Reactivation to help SLM reactivate non-discriminating regions by setting a boundary for activation values, which successively identifies more regions of CAM. Without extra refinement processes, our method is able to achieve improvements for CAM of over 20.0% mIoU on PASCAL VOC 2012 and 10.0% mIoU on MS COCO 2014 datasets, representing a new state-of-the-art among existing WSSS methods. The code is publicly available at: https://github.com/tmlr-group/GSLM.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: **1. Clarify the motivation for seed reactivation**
Updated at Section 4.3 "Seed Reactivation" paragraph.
**2. Incorporate the comparison to DRS**
- Updated at Section 4.3 "Seed Reactivation" paragraph.
- Added Section 5.3 "Comparison of Seed Reactivation and DRS" paragraph
**3. discuss and compare with the missing related works mentioned by reviewers oBpv and hhvb**
- Updated Table3 and Table4 to include the missing realted works
- Discussed and compared with the missing related works at Section 5.2 "Improvements on segmentation results" paragraph.
**4. Clarify the activation loss in Eq. 5**
Updated at Section 4.3 "Activation Loss" paragraph 2.
**5. Discuss the hyper-parameter settings and sensitivity**
Updated at the last item of 5.3 "Effect of hyper-parameters" paragraph
**6. Clarify Figure 4**
Updated Figure 4
**7. clarify the image-level experimental setting**
Updated at Section 5.1 "Dataset and Evaluation Metrics" paragraph line 7~8.
**8. Fix the minor typos mentioned by reviewer oBpv**
- Added page 2 missing parentheses.
- Fixed typo "Artificial Intelligent" -> Artificial Intelligence.
**9. Clarify the comparison with L2G**
Updated at Section 5.2 "Improvements on segmentation results" paragraph line 7~9.
**10. Provide color versions of Figs. 8 and 9**
Updated Figure 8 and 9.
**11. Add author information and acknowledgements for camera ready version.**
Code: https://github.com/tmlr-group/GSLM
Assigned Action Editor: ~Mathieu_Salzmann1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 1714
Loading