MMAct: A Large-Scale Dataset for Cross Modal Learning on Human Action Understanding



  • -2022.08 MMAct V2 is released for spatial-temporal localization! details will be updated soon and now avaliable as request.

  • -2022.08 Updated the download manner, MMAct dataset link will be sent to the user after request directly.

  • -2021.04 MMAct Challenge 2021 will be hosted with ActivityNet at CVPR'21, check here for details!

  • -2019.11 MMAct introduction video was released, check here.

  • -2019.10 MMAct is avaliable now, check publication and download.

  • -2019.08 MMAct will be presented for ICCV 2019 in Korea.


MMAct is a new large-scale multi modal dataset for action understanding based on diverse modalities

7 modalities

RGB, Keypoints, Acceleration, Gyroscope, Orientation, Wi-Fi, Pressure.

1900+ Videos

Untrimmed videos with 1920x1080@30FPS

40k+ Instances

Average length ranges from 3-8 seconds.

37 Classes

Daily, Abnormal, Desk work actions

5 Scenes

Free space, Occlusion, Station Entrance, Desk work, Outdoor

4 + 1 Views

4 survillence views + 1 egocentric view

40 Subjects

20 female, 20 male


Collected using a semi-naturalistic collection protocol.


Dataset publicly available for research purposes


For the entire dataset with annotations, please follow this link for request.



    author = {Kong, Quan and Wu, Ziming and Deng, Ziwei and Klinkigt, Martin and Tong, Bin and Murakami, Tomokazu},
    title = {MMAct: A Large-Scale Dataset for Cross Modal Human Action Understanding},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    month = {October},
    year = {2019}

Terms of Use

1) The usage of this dataset is limited to the academic research ONLY, any direct or indirect commercial use is NOT allowed.

2) The disclosure of this dataset in your research is only allowed for the demonstration in academic publications and presentations. And the original property holder‘s name of this dataset is necessary to be indicated in your publications or presentations.

3) Any distribution of this dataset is prohibited, if it is original, modified or combinated with other dataset. Any commercial usage of this dataset is prohibited, if as it is, or if it is assembled with any models or programs.

4) Hitachi, Ltd. does not bear any responsibility with respect to, any losses, expenses, and damages caused by user of this dataset. All users agree to indemnify, defend and hold harmless, the data collection participants, Hitachi, Ltd. and it’s officers, employees, and agents, individually and collectively.

5) Any way out of use of the above terms, will be canceled the usage license and should permanently remove the contents related with this dataset.