My Weird Prompts
Daniel Rosehill
Podcast
Episodes
Listen, download, subscribe
Video Tokens, Omni, and the Local VRAM Math
A listener's question about video understanding gets the full treatment: what actually changed in the past year, why Google's Omni models and agentic video navigation matter, and why uploading a clip to ChatGPT still feels like sneaking past a bouncer. The conversation works through the real arithmetic — roughly 100 tokens per second of video at default settings — and shows how that number drives every product decision, from Google's 88% token reduction to OpenAI's decision to leave video off the supported upload list entirely. Then it turns to the local question: what does it actually take to run Qwen3-VL on your own hardware for a short, high-stakes clinical clip? The answer isn't the model weights. It's the KV cache, and the math gets uncomfortable fast. Along the way: specialist token-compression research, a hand-perfusion study with a brutal generalization gap, and why the life-or-death use case turns out to be the affordable one. Key sources: • Gemini API — https://ai.google.dev/gemini-api/docs/video-understanding • Google blog — https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-agentic-video-in-gemini/ • Google blog — https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-omni/ Episode 5903 — myweirdprompts.com/n/5903 (permanent short link: myweirdprompts.com/537327)
My Weird Prompts RSS Feed
