After processing 20 two-hour videos, we learned that failing to dedupe leads to massive redundancy and performance issues in production.