Technologies and systems that generate, transform, or interact with media (video, audio, images, and immersive content) in real time or near-real time.
5 Mins - Shorts
https://www.youtube.com/playlist?list=PLMgr0xgBW16zRsuKGBrbO8NEMzEp_lsVZ
30 Mins - Relevant Almost Human Episodes
Multimodal diffusion / video generation that can run in real time, including on-device / mobile:
- “generate video in real time… on mobile”
- autoregressive generation for interactive use cases (gaming/virtual worlds), “every pixel is programmable”
- controllability + consistency as the next frontier (editable pixels, recurring characters/worlds)
- cost envelope and feasibility on consumer GPUs / edge devices
Dig Deeper - Readings