This article is full of so many interesting details. E.g. I found this passage on the technological progress in rendering snow from Frozen (2013) to Zootopia 2 fascinating:
"To give an example of the amazing amount of detail in Zootopia 2: at one point during production, our rendering team noticed some shots that had incredibly detailed snow with tons of tiny glints, so out of curiosity we opened up the shots to see how the artists had shaded the snow, and we found that they had constructed the snow out of zillions upon zillions of individual ice crystals. We were completely blown away; constructing snow this way was an idea that Disney Research had explored shortly after the first Frozen movie was made in 2013, but at the time it was purely a theoretical research idea, and a decade later our artists were just going ahead and actually doing it. The result in the final film looks absolutely amazing, and on top of that, instead of needing a specialized technology solution to make this approach feasible, in the past decade both our renderer and computers in general have gotten so much faster and our artists have improved their workflows so much that a brute-force solution was good enough to achieve this effect without much trouble at all."
A similar approach was taken by Pixar when making the beach environment for the Piper short that was previewed with Finding Dory. It was absolutely mind-blowing.
I was on the RenderMan team at the top and remember thinking it really neat that our system could stand up to that.
I remember find it mind blowing when I learned that in Brave, the artists weren't just using a texture/displacement mapped surface for the clothing and armor. They were using tori primitives for the chain mail, and curve primitives for the clothing. (I.e., the clothing actually woven out of curve primitives for the threads.)
The technology is amazing, but rendering lifelike animation seems such an antithesis to what the medium allows. You literally can do anything and for some reason the choice is to constrain it to reality.
The things that Don Hertzfeldt did with line drawings and a vacuum cleaner embrace the medium so much more.
The showcase of frames at the end really broke something through for me. It's easy to simply sit in a theater or on your couch and watch the movie as a movie. But while the theater screen is large, you don't get to pause it. So nearly all of the incredible detail gets blurred in a way that makes it easy to be immersed in the movement and story, but also forget the art of visuals. Seeing those specific frames laid out, each one of those would be an incredible art piece on their own! They would all be extremely difficult to create for an individual and take so so much time. I always wondered what those 1000+ people in the credits were actually doing, now i know! I never realized the incredibly depth and thought and time and art that goes into every frame of an animated movie.
There's an old, semi-retired YouTube video essay channel called "Every Frame a Painting". I disagreed with several of its essays, enjoyed many of them, but the biggest takeaway/agreement I got from that channel was that core spirit in the title itself. It is something I still find very useful reminder when thinking about films and/or criticizing them. The medium of a movie (or a TV show) is 12 or 24 (or more rarely 60) frames per second. We don't always reflect on how everyone of them in (even a "bad" movie) is essentially a painting. Art was involved to get that shot, that frame of the shot. Often art by lots of people, very few of them are the people you see on that screen, yet their fingerprints and hard work still shows through. "Every frame a painting" is a good sentiment to remember, I think. Especially for animation, but for any movie.
Thank you for sharing this - it reminds of the film adaption of "The Peasants" novel which uses a painted animation technique made up of thousands and thousands of paintings. Quite literally, nearly "Every Frame [is] a Painting".
For some reason, if I don't think about it, instinctually I would always describe Overwatch (to take a gaming example) and Zootopia as "simple" graphics. My mind recalls big swathes of primary colours in relatively flat yet cheerful lighting, rounded/smoothed shapes, relatively little complex texturing or surface detail.
It's when I pause overwatch that I start realizing 1. how much detail there is, and 2. How quickly and flawlessly it's rendered on relatively slow computers. And then I start truly appreciating the relentless optimizing work to make it "seem simple and fast" :).
Same thing with Zootopia - I've enjoyed the movies (doesn't hurt that I have two young kids), but they would not come to mind if I were asked to name breakthrough or particularly well animated movies. Yet the detail and work is clearly there once you pause and examine :)
> I would always describe Overwatch (to take a gaming example) and Zootopia as "simple" graphics.
I think an art director would describe them as "readable". When there's a lot of detail and quick motion, it's important that the audience can very quickly recognize what they're looking at and what's happen. Otherwise, it just turns into a big jumble of chaos that the viewer can't follow, like in Michael Bay's Transformer movies.
A big part of the art of movie making is telegraphic a sense of rich realism and complexity while still having everything clearly visually parsable. Doing that when cuts and action are fast is quite difficult.
Doing it well affects every level of the production: the colors assigned to characters so they are separated from the background, wardrobe choices to also keep characters distinct, lighting, set design, texture, animation, focus, the way the camera moves. It all works together to produce one coherent readable scene.
A nice example of this is shown in Figure 2 of the paper "Illustrative Rendering in Team Fortress 2" [1] from Valve. It shows how they tried to make the silhouettes of each character class distinct and readable. (And the paper also discusses the choices that went into the color palette.)
In case of games, that's pretty much optional. Many games (e.g. Battlefield) take the opposite approach where spotting the enemy in the chaos is intentionally hard and a skill to master. I'm sure there are also intentionally less readable movies or at least scenes, although no immediate example comes to mind.
I recently put together a system that trained a model to identify "background worthy frames" of TV shows. Animated shows scored often quite highly with a many frames being valid, I suppose an essayist would be able to explain why.
We are "just fine" with blurry details, on some level... but a lot of processing a movie holistically comes from that level of detail being present. Even if few people walking out of the theater could put their finger on why the world felt vibrant, it'll come down to the fact those details were there.
So much of movie making is like that. No normal person comes out of a theater saying "wow, the color grading on that movie really helped the drive the main themes along, I particularly appreciated the way it was used to amplify the alienation the main character felt at being betrayed by his life-long friend, and the lighting in that scene really sent that point home". That's all film nerd stuff. But it's the lighting, the color grading, the camera shots, all this subtle stuff that the casual consumer will never cite as their reason for liking or disliking the movie that results in the feelings that were experienced.
They aren't necessary. People still connect with the original Snow White, and while it may have been an absolute technical breakthrough masterstroke for the time, by modern standards it is simple. But used well the details we can muster for a modern production can still go into the general tone of the film; compare the two next to each other while looking for this effect and you may be able to "feel" what I'm talking about.
Wow these frames of forests are absolutely stunning - It’s like Princesses Monoke vibes met Disney rendering technology. I wish I knew about all the techniques mentioned in this article, it’s super cool how basically everything is backed up by really low level physical simulations.
Hair, water, and wind - its crazy how those elements are the things that continue to have levels of improvement for these animated films yet for my eyes i keep thinking "well thats looks pretty solid to me" only to be outdone by the next movie and go "oh yeah that looks better somehow".
20 years ago I was saying this about video games. However, when I go back and look at video games from 20 years ago, I now see why they kept pushing things forward. Graphics were serviceable, but they're so much better now. That said, the game (or movie) still needs to be the most important thing. When a studio tries to lean on graphics as the selling point, it always ends up poorly.
>That said, the game (or movie) still needs to be the most important thing. When a studio tries to lean on graphics as the selling point, it always ends up poorly.
Counterpoint: Avatar. I have never heard someone say they loved that movie because of the amazing story. They like it because it's very, very, very pretty to look at.
I've never seen Avatar for this reason. It seemed like all 3D hype. However, after 3 movies and 2 more planned, with the 3D hype long dead, are the visuals going to carry a 5 movie franchise from a director with over 20 Academy Awards under his belt?
I've been debating watching the movies now that there a few of them out, thinking there must be something there. However, I've never heard anyone actually talk about these movies in real life, which is still a concern.
Is that true though? Most quality animated features included quality animation. The visuals of the beautifully hand painted early Disney movies were part of the art form. Toy story looks basic to us now but it being the first totally computer animated movie and the visual style that came with that was part of it. Movies are a visual medium, so I guess saying that the visual part of the movie doesn't matter much to you is like saying that the words a writer uses doesn't matter much to your enjoyment of a book.
I totally agree. I love watching a small set of animated films over the holidays in time order to really see the technical progress. It's especially fun to go from something nostalgic I loved as a kid, to something I can really see the technical underpinnings of today.
This is so cool, I’m glad the company allowed the author to release this to the public. People like myself with knowledge of some of the art and technology involved but that stand outside the industry can get a little bit of a sense how the SOTA of animation evolves.
Secondly the bit about the evolution catching the unnamed studio, likely Pixar in production capability as of the first Zootopia certainly shows up on screen.
Yeah, I often cite Big Hero 6 (from 2014) as the place where as an animation fan I felt Hyperion (Disney's renderer) very visibly in the end product out-paced Renderman (Pixar's more mature renderer). Big Hero 6's wide shots get truly wide and the scale of its San Fransokyo is almost palpable in many sequences, showing off especially Hyperion's focus on being able to do dynamic crowd work and large crowd scenes and plenty of architecture surrounding that. (Zootopia in 2016 further cemented that Hyperion was great at crowd work.) It was one of my complaints with Pixar's Soul in 2020 that its New Orleans felt almost claustrophobic and isolated, no truly wide establishing shots, no real crowds to speak of, mostly just shots of one or two characters in close up on city streets that in the real world New Orleans would be bustling and busy. Real New Orleans has a different kind of claustrophia from big crowds and never feels isolating or as lonely. Some of that may have been the intended vibe for that particular movie, but some of that seems technical at this point from the different focuses Hyperion and Renderman have been given and how much I think Hyperion shows technical improvement and mastery of somethings that Renderman cannot seem to do.
It somewhat suggests that Disney is correct in having the two studios compete in renderers rather than share one (even as they unify other parts of the process, such as this article mentions moving to Pixar's Presto tool as one of the things that happened in Zootopia 2's production).
When I first started at Disney Animation, at one point I asked Ed Catmull what the rationale was for staffing two separate rendering teams, and he had an interesting answer. His answer was that it turns out that even when Disney Animation was using RenderMan, the high end needs of the studio still required enough rendering developers/TDs that in terms of cost it was essentially no different than staffing a team to build an in-house renderer, and from that perspective he liked the idea of having two separate teams with different focuses/perspectives so that for hard problems the wider company got two attempts at coming up with good solutions instead of one.
To this day the Hyperion and RenderMan teams work pretty closely together and share a lot of learnings/tech/R&D. The focuses are pretty different between the two renderers, and that’s actually been pretty beneficial to both.
The story with Presto is both a bit different and kind of similar. The two studios are now unified in using Presto, but Disney Animation now has an in-house Presto development team that co-develops Presto with Pixar. The two dev teams focus on the needs of their respective studios but move Presto forward together.
Yes, when people talk down on big corporations, it is stuff like this that gives me hope and I wish people could give them more credit for what they give back to society. People always ask "where is the modern Bell Labs" and I think the answer is simple: all around us.
I get the frustration of not seeing eg the next laser or transistor from a modern Bell Labs - but for a few hundred dollars an Indian teenager can build an app which could be used by a billion people using only free and open source technologies. There are so many little inventions and contributions that needed to happen for that (and many other miracles) to be possible. The physical hardware frontier has definitely matured since Bell was at their peak but we continue to make rapid progress on the software/digital front.
I mean not many private corps are publishing unbiased scientific research papers these days. I think that’s what people are referring to, an entire research division which is mainly or at least often working in the public good.
I always say "most companies today are IT companies, they just don't know it".
I would argue that Disney is definitily a big IT company and relies a lot on that tech.
Perfect storytelling might need flaweless execution to not distract. cirque du soleil for example are also experts in every single aspect relevant to their show/business. Check out the YT video from their sound manager "
Inside the Sound of Cirque du Soleil: Drawn to Life" this is so crazy but it explains so much especially how they control the audiance clapping.
For me, being an "IT Company" or "Tech Company" means the tech is what drives the business decisions.
Where I work, it's a b2b service company. We've had CIOs get up and say we're a tech company, but when push comes to shove, the IT org always loses to "the business". The business solutions are what are being sold, they really don't care what the tech is under the hood... even if the tech enables every product to exist at this point.
Very interesting and so many throwaway lines that could easily be entire blog posts by themselves!
I wonder how hamstrung Disney is by their chosen animation style (wide-eyed cutesy characters, rounded edges, bright colours). Given the technical prowess on display here, what could they create if they gave their artists free rein to experiment like in Love, Death, and Robots? Or is it more that these constraints provide structure to work inside?
What I like about this piece is how it shows the technical prowess underpinning the visual outputs in a film like what Disney puts out.
I only wish it went further! There are a ton of lessons those of us outside films/games could learn from working in that kind of deadline-consttrained innovative landscape. Tell about how you fought against the rendering deadlines and sped up the snowscape frames by 30% to get it in under the wire!
Most companies doing CGI work, both in games and movies, are quite open about their technical details. The whole industry relies on pooled research and development. While the actual code is typically confidential, publishing information serves multiple purposes for the work's publicity, the advancement of the field, the happiness of employees, and company prestige for recruiting people.
I wonder if there’s any collaboration between the technical side and writing side?
Specifically, I see incredible effort and time getting tiny details right on the technical side, but the storyline and dialog seems to not have had the same level of effort applied to it.
I like how you are trying to appear sophisticated by hanging your film taste off someone else's opinions. What other great things have people told you to like?
"To give an example of the amazing amount of detail in Zootopia 2: at one point during production, our rendering team noticed some shots that had incredibly detailed snow with tons of tiny glints, so out of curiosity we opened up the shots to see how the artists had shaded the snow, and we found that they had constructed the snow out of zillions upon zillions of individual ice crystals. We were completely blown away; constructing snow this way was an idea that Disney Research had explored shortly after the first Frozen movie was made in 2013, but at the time it was purely a theoretical research idea, and a decade later our artists were just going ahead and actually doing it. The result in the final film looks absolutely amazing, and on top of that, instead of needing a specialized technology solution to make this approach feasible, in the past decade both our renderer and computers in general have gotten so much faster and our artists have improved their workflows so much that a brute-force solution was good enough to achieve this effect without much trouble at all."
https://www.fxguide.com/fxfeatured/the-tech-of-pixar-part-1-...
Related but unlinked Part 2 on other aspects of Finding Dory:
https://www.fxguide.com/fxfeatured/the-tech-of-pixar-part-2-...
I remember find it mind blowing when I learned that in Brave, the artists weren't just using a texture/displacement mapped surface for the clothing and armor. They were using tori primitives for the chain mail, and curve primitives for the clothing. (I.e., the clothing actually woven out of curve primitives for the threads.)
The things that Don Hertzfeldt did with line drawings and a vacuum cleaner embrace the medium so much more.
https://youtube.com/@everyframeapainting
https://en.wikipedia.org/wiki/The_Peasants_(2023_film)#Produ...
For some reason, if I don't think about it, instinctually I would always describe Overwatch (to take a gaming example) and Zootopia as "simple" graphics. My mind recalls big swathes of primary colours in relatively flat yet cheerful lighting, rounded/smoothed shapes, relatively little complex texturing or surface detail.
It's when I pause overwatch that I start realizing 1. how much detail there is, and 2. How quickly and flawlessly it's rendered on relatively slow computers. And then I start truly appreciating the relentless optimizing work to make it "seem simple and fast" :).
Same thing with Zootopia - I've enjoyed the movies (doesn't hurt that I have two young kids), but they would not come to mind if I were asked to name breakthrough or particularly well animated movies. Yet the detail and work is clearly there once you pause and examine :)
I think an art director would describe them as "readable". When there's a lot of detail and quick motion, it's important that the audience can very quickly recognize what they're looking at and what's happen. Otherwise, it just turns into a big jumble of chaos that the viewer can't follow, like in Michael Bay's Transformer movies.
A big part of the art of movie making is telegraphic a sense of rich realism and complexity while still having everything clearly visually parsable. Doing that when cuts and action are fast is quite difficult.
Doing it well affects every level of the production: the colors assigned to characters so they are separated from the background, wardrobe choices to also keep characters distinct, lighting, set design, texture, animation, focus, the way the camera moves. It all works together to produce one coherent readable scene.
[1] https://steamcdn-a.akamaihd.net/apps/valve/2007/NPAR07_Illus...
So much of movie making is like that. No normal person comes out of a theater saying "wow, the color grading on that movie really helped the drive the main themes along, I particularly appreciated the way it was used to amplify the alienation the main character felt at being betrayed by his life-long friend, and the lighting in that scene really sent that point home". That's all film nerd stuff. But it's the lighting, the color grading, the camera shots, all this subtle stuff that the casual consumer will never cite as their reason for liking or disliking the movie that results in the feelings that were experienced.
They aren't necessary. People still connect with the original Snow White, and while it may have been an absolute technical breakthrough masterstroke for the time, by modern standards it is simple. But used well the details we can muster for a modern production can still go into the general tone of the film; compare the two next to each other while looking for this effect and you may be able to "feel" what I'm talking about.
Counterpoint: Avatar. I have never heard someone say they loved that movie because of the amazing story. They like it because it's very, very, very pretty to look at.
I've been debating watching the movies now that there a few of them out, thinking there must be something there. However, I've never heard anyone actually talk about these movies in real life, which is still a concern.
I don't really enjoy the stories all that much, but the experience is fun.
Although, I usually turn off the water while I'm eating my popcorn.
I don't really enjoy the stories all that much, but the experience is fun.
Secondly the bit about the evolution catching the unnamed studio, likely Pixar in production capability as of the first Zootopia certainly shows up on screen.
It somewhat suggests that Disney is correct in having the two studios compete in renderers rather than share one (even as they unify other parts of the process, such as this article mentions moving to Pixar's Presto tool as one of the things that happened in Zootopia 2's production).
When I first started at Disney Animation, at one point I asked Ed Catmull what the rationale was for staffing two separate rendering teams, and he had an interesting answer. His answer was that it turns out that even when Disney Animation was using RenderMan, the high end needs of the studio still required enough rendering developers/TDs that in terms of cost it was essentially no different than staffing a team to build an in-house renderer, and from that perspective he liked the idea of having two separate teams with different focuses/perspectives so that for hard problems the wider company got two attempts at coming up with good solutions instead of one.
To this day the Hyperion and RenderMan teams work pretty closely together and share a lot of learnings/tech/R&D. The focuses are pretty different between the two renderers, and that’s actually been pretty beneficial to both.
The story with Presto is both a bit different and kind of similar. The two studios are now unified in using Presto, but Disney Animation now has an in-house Presto development team that co-develops Presto with Pixar. The two dev teams focus on the needs of their respective studios but move Presto forward together.
I would argue that Disney is definitily a big IT company and relies a lot on that tech.
Perfect storytelling might need flaweless execution to not distract. cirque du soleil for example are also experts in every single aspect relevant to their show/business. Check out the YT video from their sound manager " Inside the Sound of Cirque du Soleil: Drawn to Life" this is so crazy but it explains so much especially how they control the audiance clapping.
Where I work, it's a b2b service company. We've had CIOs get up and say we're a tech company, but when push comes to shove, the IT org always loses to "the business". The business solutions are what are being sold, they really don't care what the tech is under the hood... even if the tech enables every product to exist at this point.
I wonder how hamstrung Disney is by their chosen animation style (wide-eyed cutesy characters, rounded edges, bright colours). Given the technical prowess on display here, what could they create if they gave their artists free rein to experiment like in Love, Death, and Robots? Or is it more that these constraints provide structure to work inside?
These inventions are represented at Siggraph and are often also research paper and land in the whole ecosystem (in software like hudini).
LDR live from their ideas and stories and experiments with styles. I don't think its relevant for technology it would just be a tech demo.
I only wish it went further! There are a ton of lessons those of us outside films/games could learn from working in that kind of deadline-consttrained innovative landscape. Tell about how you fought against the rendering deadlines and sped up the snowscape frames by 30% to get it in under the wire!
https://disneyanimation.com/publications/
Or, for a continuously updated list of publications specifically about Disney’s Hyperion Renderer:
https://blog.yiningkarlli.com/2019/07/hyperion-papers.html
Specifically, I see incredible effort and time getting tiny details right on the technical side, but the storyline and dialog seems to not have had the same level of effort applied to it.