Objective

2

Pedestrian detection involves locating all pedestrians in images or videos by using rectangular boxes with confidence scores. Traditional pedestrian detection methods cannot handle situations with different postures and mutual occlusion. In recent years

deep neural networks have performed well in object detection

but they are still unable to solve some challenging issues in pedestrian detection. In this study

we propose a method called DC-CSP(density map and classifier modules with center and scale prediction) to enhance pedestrian detection by combining pedestrian density and score refinement. Under an anchor-free architecture

our method first refines the classification to obtain more accurate confidence scores and then uses different IoU (intersection over union) thresholds to handle varying pedestrian densities with the objective of reducing the omission of occluded pedestrians and the false detection of a single pedestrian.

Method

2

First

our DC-CSP network is primarily composed of a center and scale prediction(CSP) subnetwork

a density map module (DMM)

and a classifier module (CM). The CSP subnetwork includes a feature extraction module and a detection head module. The feature extraction module uses ResNet-50 as its backbone

in which output feature maps are down-sampled by 4

8

16

and 16 with respect to the input image. The shallower features provide more precise localization information

and the deeper features contain more semantic information with larger receptive fields. Thus

we fuse the multi-scale feature maps from all the stages into a single one with a deconvolution layer. Upon the concatenation of feature maps

the detection head module first uses a 3×3 convolutional layer to reduce channel dimension to 256 and then two sibling 1×1 convolutional layers to produce the center heat map and scale map. On the basis of the CSP subnetwork

we design a density estimation module that first utilizes the concatenated feature maps to generate the features of 128 channels via a 1×1 convolutional layer

and then concatenates them with the center heat map and scale map to predict a pedestrian density map with a convolutional kernel of 5×5. The density estimation module integrates diverse features and applies a large kernel to consider surrounding information

generating accurate density maps. Moreover

a CM is designed to use the bounding boxes transformed from the center heat map and the scale map as input. This module utilizes the concatenated feature maps to produce 256-channel features via a 3×3 convolutional layer and then classifies the produced features by using a convolutional layer with a 1×1 kernel. The majority of the confidence scores of the background are below a certain threshold; thus

we can obtain a threshold for easily distinguishing pedestrians from the background. Second

the detection scores in CSP are relatively low and the CM can better discriminate between pedestrians and the background. Therefore

to increase the confidence scores of pedestrians and simultaneously decrease that of the background in the final decision

we design a stage score fusion (SSF) rule to update the detection scores by utilizing the complementarity of the detection head module and CM. In particular

when the classifier judges a sample as a pedestrian

the SSF rule will slightly boost the detection scores. By contrast

when the classifier judges a sample as the background

the SSF rule will slightly decrease. In other cases

a comprehensive judgment will be made by averaging the scores from both modules. Third

an improved adaptive non-maximum suppression (NMS)

called the improved adaptive NMS(IAN) post-processing method

based on the estimated pedestrian density map is also proposed to improve the detection results further. In particular

a high IoU threshold will be used for mutually occluded pedestrians to reduce missed detection

and a low IoU threshold will be used for a single pedestrian to reduce false detection. In contrast with adaptive NMS

our IAN method fully considers various scenes. In addition

IAN is based on NMS rather than on soft NMS

and thus

it involves lower computational cost.

Result

2

To verify the effectiveness of the proposed modules

we conduct a series of ablation experiments in which C-CSP

D-CSP

and DC-CSP respectively represent the addition of the CM

DMM

and both modules to the CSP subnetwork. We conduct quantitative and qualitative analyses on two widely used public datasets

i.e.

Citypersons and Caltech

for each setting. The experimental results of C-CSP verify the rationality of the SSF rule and demonstrate that the confidence scores of pedestrians can be increased while that of the background can be decreased. Simultaneously

the experimental results of D-CSP demonstrate the effectiveness of the IAN method

which can considerably reduce missed detection and false detection. For the quantitative analyses of DC-CSP

its log-average miss rate decreases by 0.8%

1.3%

1.0%

and 0.8% in the Reasonable

Heavy

Partial

and Bare subsets of Citypersons

respectively

and decreases by 0.3% and 0.7% in the Reasonable and Allsubsets of Caltech

respectively

compared with those of other methods. For the qualitative analyses of DC-CSP

the visualization results show that our method can work well in various scenes

such as pedestrians occluded by other objects

smaller pedestrians

vertical structures

and false reflection. Pedestrians in different scenes can be detected more accurately

and the confidence scores are more convincing. Furthermore

our method can avoid numerous false detections in situations with a complex background.

Conclusion

2

In this study

we propose a deep convolutional neural network with multiple novel modules for pedestrian detection. In particular

the IAN method and the SSF rule are designed to utilize density and classification features

respectively. Our DC-CSP method can considerably alleviate issues in pedestrian detection

such as missed detection

false detection

and inaccurate confidence scores. Its effectiveness and robustness are verified on multiple benchmark datasets.