Abstract: Highlights•An image-level attention module is created to better differentiate lesion types.•A fine-grained disease attention module is used to improve local representations.•DualAttNet outperforms existing attention models on lesion detection from CXRs.