Objective

2

Video-based intelligent action recognition has been developing for computer vision analysis nowadays. It is required to recognize action in a specific scene of video due to such multiple video types. To appreciate sports leisure for users like the meta-video set of various badminton stroke

it can assist coaches to analyze stroke better if badminton strokes can be accurately located and recognized in a badminton video. Sports video analysis like the approach of the badminton stroke recognition can be transferred to tennis and table tennis via similar sports features. For a long time span of video based action recognition method

it is necessary to locate the action time domain. Badminton-oriented video can be as this kind of videos to locate stroke time domains. For the time domain localization of video actions

current research is focused on a clear action switching boundary between adjacent actions in a video

and the foreground or background features of adjacent actions are quite different

such as the action video dataset 50Salads and dataset Breakfast. However

there is no obvious boundary information between foreground and background of adjacent strokes in a badminton video. Therefore

the action recognition based long time span video is not suitable for the localization of badminton strokes. In addition

most existing researches on badminton stroke recognition are based on a static image of a stroke derived from a badminton video

and the stroke recognition of badminton-relevant meta-video is lacking. Our method is focused on an approach for locating and classifying the strokes of ball-control player in an extracted badminton video highlight.

Method

2

First

the pose estimation model regional multi-person pose estimation(RMPE) is used to detect human poses in a badminton video highlight. The pose of the targeted player is located via adding prediction scores and position constraints to shield other irrelevant factors of human bones. For the detected pose of targeted player

the node constraints are added to locate arms of the player. The holding arm and the non-holding arm are distinguished according to the difference of the swinging amplitude

and the time domain localization of badminton stroke is carried out by the swinging amplitude variation of the holding arm for extracting the meta-video of badminton stroke. The swing amplitude of the player's arm in a frame is defined as the linear weighted sum of the square of the upper and lower limbs swing vector modulus. Then

the dataset of badminton meta-videos is applied to train convolutional block attention module-temporal segment networks (CBAM-TSN) for predicting badminton strokes in meta-videos

which add convolutional block attention module in temporal segment networks. It is necessary to extract two-stream of meta-videos from dataset beforehand through training CBAM-TSN because temporal segment network (TSN) inherited the structure of two-stream convolutional neural network(CNN). The two-stream is composed of spatial stream (RGB frames) and temporal stream (optical frames). The predicted stroke from the model of CBAM-TSN contains four familiar types: forehand

backhand

overhead and lob. Finally

we classify the overhead scenario into clear or smash by morphology processing

the clear-oriented meta-videos tend to continuous dynamic mask in the background area at the end of the stroke

but the smash-oriented meta-videos have no continuous dynamic mask information in the background area. Our badminton mask in a meta-video is captured based on the result of images morphological processing. The strokes of clear and smash can be distinguished based on position-relevant features of the badminton mask.

Result

2

In a highlighted badminton video

it shows that the segmentation is correct if a meta video segmented by the method of strokes localization and a meta video extracted manually both contain the same badminton stroke. Our indicator of intersection over union (IoU) is used to evaluate the performance of strokes localization. Furthermore

the performance of badminton strokes classification is evaluated via using machine learning based indicator ROC-AUC

recall and precision. The experiment results show that our IoU of stroke localization in badminton video highlights is reached to 82.6%. The indicator AUC about four kinds of badminton strokes (forehand

backhand

overhead and lob) predicted by the model of CBAM-TSN is all over 0.98

the micro-AUC

macro-AUC

average recall and precision is reached to 0.990 8

0.990 3

93.5% and 94.3%

respectively. In addition

the CBAM-TSN is compared to the three popular approaches of action recognition in the context of badminton strokes recognition

gets the highest result on precision

micro-AUC and macro-AUC. The final average recall and precision is reached to 91.2% and 91.6% of each. Therefore

it can effectively locate and classify major player's strokes in a badminton video highlight.

Conclusion

2

We facilitate a novel badminton strokes recognizing method in badminton video highlights

which is in combination with badminton stroke localization and badminton stroke classification. The potential sports video analysis is developed further.