AI + Baduanjin

The AI & Baduanjin Program is an innovative project that combines traditional health preservation exercises with AI motion capture technology. The program takes Zhou Tsz Shan Sam (Jeffrey)’s standard Baduanjin movements as its data blueprint and, through multi-view cameras and deep learning algorithms, captures the body’s key skeletal points in real time to generate a precise digital motion model. As users practice, the AI system compares their movements against the model in real time, corrects errors, and outputs quantitative feedback such as joint angles and center of gravity deviation, working like a personal AI coach. The program aims to use technology to lower the learning threshold of Baduanjin so that more people can practice this Chinese health preservation exercise, passed down for over 1,000 years, correctly and well, anytime and anywhere.

Purposes

(1) Balancing Tradition and Innovation

Baduanjin is a national intangible cultural heritage. Its traditional teaching relies on oral teaching and physical demonstration from master to disciple, and learners often put in twice the effort for half the result because they cannot confirm the details of each movement. This project applies AI motion capture technology to preserve the standard movements of Zhou Tsz Shan Sam (Jeffrey), an outstanding young inheritor of martial arts, as reusable digital assets, giving traditional exercises new life in the digital age.

(2) Solving the “No Feedback” Pain Point of General Practitioners

When ordinary users follow along with a video, they have no way to tell whether their movements are correct. The AI & Baduanjin Program provides visual, data-based movement evaluation through real-time comparison and correction — the equivalent of having a professional coach right in your phone— so that exercising at home can achieve the results of an in-person class.

(3) Building a Replicable Young Martial Artist IP

As founder of this committee, Zhou Tsz Shan Sam (Jeffrey) has a youthful, international, tech-forward image. This project digitizes his martial arts skills and personal influence, accumulating experience and data for the AI transformation of more traditional martial arts projects in the future.

Development Process

(1) Motion Capture

In a professional motion capture studio, 16 optical motion capture cameras and 4 RGB cameras were set up. Wearing a motion capture suit, Zhou Tsz Shan Sam (Jeffrey) performed the complete standard Baduanjin sequence (ready position → Holding Up the Sky → Drawing the Bow → ... → Rising on the Toes), capturing 8 core movements in total, with each movement repeated 10 times to ensure data stability.

(2) Data Processing and Modeling

The motion capture software calculated the 3D coordinates of 33 key skeletal points on the human body.

Abnormal frames were smoothed to remove jitter.

A standard motion library was built: the average of Zhou Tsz Shan Sam (Jeffrey)’s repeated performances serves as the golden template, with a reasonable tolerance range of ±5% retained.

(3) AI Algorithm Development

Based on MediaPipe and a self-developed lightweight pose estimation model, the system achieves real-time 2D to 3D pose reconstruction from a phone camera.

A comparison engine was developed: it aligns the user’s real-time posture with the standard template frame by frame and calculates metrics including joint angle difference, center of gravity trajectory, and force timing.

Error correction feedback was designed as voice prompts plus a visual skeleton overlay (the user’s skeleton shown in blue, the standard template in red, with deviations highlighted).

(4) Application Rollout

A WeChat Mini Program prototype was developed, featuring:

Follow-along Practice Mode: real-time AI scoring and correction

Free Practice Mode: records past scores and generates progress curves

Challenge Mode: compete in motion consistency against the digital avatar of Zhou Tsz Shan Sam (Jeffrey)

Future Outlook

(1) Technology Iteration

The program plans to introduce multimodal feedback (for example, monitoring heart rate and breathing rhythm through smart wearable devices to determine whether the exertion of force is coordinated with the breath), and to optimize the algorithms so that the program runs smoothly even on low-end phones.

(2) Content Expansion

Starting from Baduanjin, the program will gradually expand to cover more traditional martial arts, such as the Five Animals Exercise and basic Tai Chi single forms, forming an “AI Traditional Martial Arts Training Matrix.” Zhou Tsz Shan Sam (Jeffrey) will continue to serve as the motion capture performer and martial arts advisor.

(3) Commercialization and Social Value

The program will cooperate with school PE classes and community elderly care service centers to provide a free public welfare version.

It will launch a paid advanced version (including Zhou Tsz Shan Sam (Jeffrey)’s exclusive commentary and training plans).

It will explore overseas markets: translating the program into multiple language versions and sharing China’s health preservation wisdom with the world.

(4) Data Accumulation and Research

De-identified practice data can be fed back to sports research institutions to study the rehabilitation effects of Baduanjin on different groups of people, advancing an evidence-based approach for traditional exercises.

The AI & Baduanjin Program does not use technology to replace tradition; it uses technology to protect tradition. Through this project, Zhou Tsz Shan Sam (Jeffrey) hopes that an elderly person living in a remote mountain area, an office worker, and an overseas Chinese can all practice Baduanjin correctly with a real young martial artist through a phone screen. This is the original intention and the ultimate destination of the program.