Research Output

Cross-modality interactive attention network for multispectral pedestrian detection

  Multispectral pedestrian detection is an emerging solution with great promise in many around-the-clock applications, such as automotive driving and security surveillance. To exploit the complementary nature and remedy contradictory appearance between modalities, in this paper, we propose a novel cross-modality interactive attention network that takes full advantage of the interactive properties of multispectral input sources. Specifically, we first utilize the color (RGB) and thermal streams to build up two detached feature hierarchy for each modality, then by taking the global features, correlations between two modalities are encoded in the attention module. Next, the channel responses of halfway feature maps are recalibrated adaptively for subsequent fusion operation. Our architecture is constructed in the multi-scale format to better deal with different scales of pedestrians, and the whole network is trained in an end-to-end way. The proposed method is extensively evaluated on the challenging KAIST multispectral pedestrian dataset and achieves state-of-the-art performance with high efficiency.

  • Type:

    Article

  • Date:

    26 September 2018

  • Publication Status:

    Published

  • Publisher

    Elsevier BV

  • DOI:

    10.1016/j.inffus.2018.09.015

  • Cross Ref:

    S1566253518304111

  • ISSN:

    1566-2535

  • Library of Congress:

    TK Electrical engineering. Electronics Nuclear engineering

  • Dewey Decimal Classification:

    621.3822 Signal processing

  • Funders:

    National Natural Science Foundation of China; Ministry of Science and Technology of the People's Republic of China; Guangdong Science and Technology Department

Citation

Zhang, L., Liu, Z., Zhang, S., Yang, X., Qiao, H., Huang, K., & Hussain, A. (2019). Cross-modality interactive attention network for multispectral pedestrian detection. Information Fusion, 50, 20-29. https://doi.org/10.1016/j.inffus.2018.09.015

Authors

Keywords

Signal Processing; Hardware and Architecture; Software; Information Systems

Monthly Views:

Available Documents