On parameterizing higher-order motion for behaviour recognition

Yan Sun, Jonathon S. Hare, Mark S. Nixon

Abstract: Highlights • Optical flow field is disambiguated systematically into acceleration, jerk and snap fields and their components. Our acceleration algorithm achieves superior performance to the state of art on efficiency and preciseness. • We show generalised application of higher-order motion detection in a richer selection of imagery for not only the new approach but also the new basis, on synthetic and on real image sequences. • Jerk and snap flow are applied for detecting heel strikes and the results demonstrates that jerk can be adapted in gait analysis or other applications on real images. • We demonstrate the capability to discriminate violence using velocity and acceleration flow and show the advantages of using analysis of acceleration. Abstract Human behaviours consist different types of motion; we show how they can be disambiguated into their components in a richer way than that currently possible. Studies on optical flow have concentrated on motion alone without the higher order components: snap, jerk and acceleration. We are the first to show how the acceleration, jerk, snap and their constituent parts can be obtained from image sequences, and can be deployed for analysis, especially of behaviour. We demonstrate the estimation of acceleration in sport, human motion, traffic and in scenes of violent behaviour to demonstrate the wide potential for application of analysis of acceleration. Determining higher order components is suited to the analysis of scenes which contain them: higher order motion is innate to scenes containing acts of violent behaviour, but it is not just for behaviour which contains quickly changing movement: human gait contains acceleration though approaches have yet to consider radial and tangential acceleration, since they concentrate on motion alone. The analysis of synthetic and real-world images illustrates the ability of higher order motion to discriminate different objects under different motion. Then the new approaches are applied in heel strike detection in the analysis of human gait. These results demonstrate that the new approach is ready for developing new applications in behaviour recognition and provides a new basis for future research and applications of higher-order motion analysis.