CATDOLL: Why can't Chinese and Japanese animations match the lip movements and dubbing of Hollywood animation characters?

The most important factor of animation compared to comics is that the pictures are moving. Then, with the animation of the pictures, various problems also come along. If you want the animation to look smoother, the movements to look coordinated, etc., the number of frames becomes an important factor. However, the more troublesome problem is the degree of fit between the lip shape of the anime character and the dubbing.

Friends who like to watch animated movies may have noticed a problem: the lip movements of the characters in Hollywood animated blockbusters are consistent with the lines, while the lip movements of the characters in most Chinese and Japanese animations do not match the lines. Why is this?

In fact, this is caused by the differences in animation production processes between Asia and Europe and the United States.

European and American animation production process:

Pre-production planning, script, storyboard, design draft → character actor recording → original animation and scene production → shot synthesis → full film editing → film final mixing

Sino-Japanese animation production process:

Pre-production planning, script, storyboard, design draft → original animation and scene production → shot synthesis → full film editing → dubbing + mixing → final film mixing

Simply put, it is the difference between dubbing first and dubbing later. In Hollywood animated films, actors start dubbing after the script, storyboards and design drafts are out. After that, the original animation and scene production will refer to the dubbing to make accurate matching lip movements, so that the character's voice and lip movements can basically match. In most domestic and Japanese animations, when the whole film is almost completed, the dubbing actors will make lip movements that match the emotions of the shot according to the director's requirements. Therefore, the lip movements cannot match well.

European and American Animation

For European and American cartoons, represented by Hollywood animations, the film has not yet taken shape when the dubbing actors are dubbing. They only see the animation characters and need to interpret the characters themselves by figuring out the script. After the dubbing is completed, the artist team will complete the animation according to the script, dubbing and director's requirements. Therefore, the lip movements of the animation characters match the dubbing.

Zhang Guoli recalled that he mentioned such a scene when he dubbed "Mr. Ping" in "Kung Fu Panda 3".

"What we saw was not the finished film. I have not seen it yet. What we saw were its sketches. Sometimes it was a few drawings, sometimes it was a few pictures... They pointed a camera at me and filmed my lip movements. Then I looked at a few frames of the pictures, or the unfinished drawings, and said the lines. They then followed our lip movements." (Exclusive interview with Time.com)

There is a saying that the perfect match between lip movements and dialogue in Hollywood animation is to facilitate the reading of "lip language" by the hearing-impaired. In fact, as early as the mid-1920s, when sound films were just invented, the pioneer of animation Max Fleischer had already paid attention to lip matching. He invented the "rotoscope technology" to produce rotoscoped animation.

This is an animation technique used by animators to track real movement frame by frame. They project pre-filmed movies onto frosted glass, and then the animators trace it. Frame-by-frame drawing provides primitive but good conditions for lip-syncing of animated characters. The world's first feature-length animated film, Disney's "Snow White" released in 1937, used rotoscoping technology.

In fact, there are rules for drawing lip sync in 2D hand-drawn animation. The animator's control of this rule greatly affects the matching degree between the character's lip sync and the dubbing. For example, the physiological differences between humans and different animal characters will affect the way the mouth moves.

Since the development of computer graphics technology, the development of CG animation industry has far exceeded the imagination of the pioneers of animation. Nowadays, there are few 2D hand-drawn animations, and most of them are 3D modeling. The specific operation for lip shape is to set the lip shape of the main phonemes during the modeling process. After the voice of the voice actor is obtained, the phonemes can be analyzed in real time during the playback of the sound, and then the lip shape corresponding to the phonemes can be found from the material library, so that the animated characters can make the lip shape corresponding to the sound. In this way, the picture presented is smoother and more realistic.

"At the Tim Burton Annual at the French Film Archive, the Corpse Bride section had different replacement heads for each character. The director made different mouth shapes for each of the 26 letters of the English alphabet, so that he could easily replace the head for each frame based on the character's recording." (Zhihu user MorphZh)

The increasing maturity of motion capture technology has made the lip movements of CG animation characters and dubbing more perfectly matched. By attaching markers to the key points of the performer's facial expressions and performing expression capture, the movement trajectory of the mouth muscles can be reproduced in real time.

From this, it can be seen that the production of Hollywood animated films has been very meticulous in matching lip movements and lines since the beginning of animation. In the European and American animation industry, this is not only a tradition, but also a set of established standards. It is these well-made details that make the audience feel very "natural" when watching and it is easy to get into the play.

Japanese Animation

Japanese animation is derived from comics. The main creators create it independently or lead a team of painters to create it. Basically, the animation part is completed first, and then the "voice actors" dub according to the lip movements of the animation characters. In most cases (except for close-ups), the lip movements of Japanese animation characters only seem to open and close, so for voice actors, there is no matching of lip movements and lines, but more of a time control. However, due to the characteristics of the Japanese language, as long as the opening and closing of the "key frames" correspond, the picture will not look inconsistent.

Of course, there are a few Japanese works that are dubbed first and then drawn, but they are rare. For example, the first Japanese animation that was popular in the United States, "AKIRA". However, the production time of ordinary theater versions and TV animations is so tight that the production time for each episode is barely enough for the production team to finish drawing, and they often draw and record at the same time.

"How tight is the production schedule for Japanese animation? The opening of this episode has already been broadcast on TV, but the second half is not yet ready!! The most "famous" incident was the 13th episode of "Attack on Titan", which was released before it was finished, and two still pictures replaced 15 seconds of animation, which was criticized by the entire Japanese audience." (Zhihu netizen Meng Deer)

But in any case, as the only country that can compete with the United States in the field of animation, Japan can turn this "defect" of mismatched lip syncing into a unique "style" that is unique and domineering. Just imagine, when watching "Spirited Away", immersed in the magical and warm world created by Hayao Miyazaki, who would worry about whether the lip syncing and the lines are completely consistent?

Chinese Animation

Chinese domestic animations are similar to Japanese ones in handling lip movements, and also use the "keyframe lip syncing" method. In specific operations, just make the exaggerated lip movements in the lines and leave the other parts vague. This method is clever and cost-effective.

"For example, in the line "Take my stick, Old Sun", in "Havoc in Heaven", make a big and round mouth shape for the word "I", and tighten it a little for the word "Old". For the following two words "Sun Yi", just make a small mouth shape, whether it is round or flat, and close the mouth and teeth, it doesn't matter. Then make another big and round mouth shape for the word "stick." (Roc Lee, Zhihu user)

As for imported animated films, there are two ways to process the images in the Chinese dubbing version. One is to add subtitles. As you can imagine, although the characters in the film speak in Chinese, they still keep the lip shape in a foreign language, which will make the audience feel awkward. The other is to blur the lip shape, so that you can see the mouth moving everywhere, and the dubbing can match it, but this "matching" is also similar to the dubbing technique of Japanese animated films.

In recent years, both in Hollywood and China, when producing or dubbing animated films, for commercial considerations, more and more celebrities have been attracted to join the dubbing team. The audience's attitude towards this phenomenon is also mixed. Some people think that celebrity dubbing is a great attraction for movie fans, while others think that celebrities do not have the qualities of professional dubbing actors, and sacrificing the effect for commercial interests is a temporary solution and will never become a classic.

In fact, who does the dubbing and whether the lip movements match the voice cannot be used to judge the quality of an animation. A truly good animation film requires a strong production team, expensive equipment and a lot of money.

Hollywood animations produced by DreamWorks, Disney, and Pixar represent quality, Japanese anime leads the trend, and domestic animated blockbusters are also on the rise - "The Return of the Great Sage" and "Big Fish and Begonia" have received keen attention from the Chinese people.

All this is enough to show that the quality of animation is not determined by national boundaries, and only sincere works can touch people's hearts.