What was the most difficult challenge for you guys with this project?
Also I'm a little intrigued on looking into efficiency techniques for video gen. What are current research directions in that field and promising directions you are particularly interested in? Any pointers for beginners?
Interesting. It does however seem to struggle with a lot of the typical t2i models struggle with.
Over smoothing, lots of uncanney shine, mangled limbs and such.
Also I'm a little intrigued on looking into efficiency techniques for video gen. What are current research directions in that field and promising directions you are particularly interested in? Any pointers for beginners?
If you swap the text encoder for something like T5-small or CLIP, this could be <2.5Gb at INT4; doable on a lot of machines.