Your product needs to understand video. Maybe it's a measurement platform that has to know where a sponsor appeared, a media platform packaging content for advertisers, or a brand-safety model that can't keep guessing from titles and tags. Whatever the use case, someone on your team is about to ask the same question: do we build this, or buy it?
This guide is for the product leaders and engineers making that call. It covers what building really involves, where a general-purpose model fits, a scorecard for evaluating vendors, and how to tell when a custom model is worth it.
The three paths
- Build: stand up your own pipeline and train your own models.
- Prompt a general model: send video to a large multimodal model and write the questions, the taxonomy and the rules yourself.
- Buy purpose-built models: call an API that returns the signals your product needs, already structured.
Each is the right answer for someone. The mistake is choosing on the first demo instead of on what it takes to run in production for years.
What building actually involves
The demo is the easy part. One clip through a notebook proves very little. Production is a pipeline, and every stage is something your team owns for good.
| Stage | What it takes | What keeps costing you |
|---|---|---|
| Ingest | Fetching video from URLs, feeds and libraries; handling formats, failures and retries | Volume growth, new sources, broken links |
| Preparation | Frame sampling or storyboards, audio extraction, transcription | Compute on every video, every time |
| Taxonomy | Choosing and mapping a category system such as the IAB Content Taxonomy | Taxonomy updates and edge cases |
| Training data | Labeled examples for every category and risk type, reviewed by people who know the industry | Relabeling as content and language change |
| Models | Training, evaluation and accuracy targets by category | Drift, retraining and regression testing |
| Brand safety and suitability | A risk rubric and suitability levels that hold up to advertiser scrutiny | Each new advertiser's standards and every disputed call |
| Serving | An API, batching, queues and monitoring at library scale | On-call, uptime and cost control |
Two costs are easy to miss. The first is expertise: knowing that "they're killing them" in a sports clip isn't violence, or that a car-accident story can be wrong for an automaker and exactly right for an insurer or a law firm. The second is time. Every month spent building the pipeline is a month your engineers aren't shipping the product it was meant to power.
Where a general model fits
"We already use Gemini. Why do we need anything else?" It's the question we hear most, and it deserves a straight answer.
A general multimodal model is a good tool for a one-off answer about a clip, a prototype, or a question nobody has asked before. Google's Gemini Decisions API, launched in November 2026, goes a step further: you send one input, including video, with questions you write as yes/no, multiple choice or a score, and it returns structured answers.
What you take on with that path:
- You write the intelligence. The taxonomy, the brand-safety rubric for each advertiser and the validation are yours to build and maintain.
- Consistency is your job. The output follows the questions you wrote, so keeping results comparable across millions of videos means owning your prompts as carefully as your code.
- Everyone has the same model. If your competitors can call the same model and get the same results, so can your clients. Then what does your product bring to the table?
- Your video goes to the model provider. For some content owners and contracts, that alone settles it.
A scorecard for evaluating vendors
Run this side by side on your own videos, not on the vendor's demo reel. Score each criterion from 1 to 5.
| Criterion | What good looks like | Ask the vendor |
|---|---|---|
| Output structure | The same fields for every video, ready for your database | Show me the JSON for 50 of my videos |
| Taxonomy | Standard categories (for example IAB) with the depth your use case needs | Which taxonomy and version, and how deep does it go? |
| Brand safety and suitability | Scores by risk category, a suitability level and the reasoning behind each | Can I see why a video was flagged? |
| Accuracy on your content | Results checked against videos your team has already labeled | Will you run a blind test on our sample? |
| Throughput | Libraries and feeds, not clip by clip | How many videos per hour in production? |
| Input | Send links in batches, with no re-hosting | What do I send, and how many at a time? |
| Data handling | Your video isn't passed to a third-party model | Whose model processes my video? |
| Customization | New signals and custom models for your taxonomy | What happens when I need a signal you don't have? |
| Track record | Models built and maintained in production over years | How long has this model run in production, and at what volume? |
| Cost predictability | A price you can forecast as volume grows | What happens to my bill at ten times the volume? |
When a custom model is worth it
A purpose-built or custom model earns its keep when at least one of these is true:
- The signal is part of what you sell, so its accuracy is your product's reputation.
- You run at library or feed scale, where small error rates become large numbers.
- Your taxonomy or your advertisers' standards are specific, and generic answers keep needing correction.
- Errors are expensive: a misplaced ad, a missed sponsor, a wrongly blocked video.
It probably isn't worth it when you're answering occasional one-off questions, exploring a prototype, or working with content no model has been trained for yet.
A simple way to decide
- Build if video understanding is your core IP and you're ready to fund the team, the data and the infrastructure for years.
- Prompt a general model for prototypes, low volume and questions that change every week.
- Buy purpose-built models when you need consistent, structured signals at scale and want your engineers shipping your product, not maintaining a pipeline.
Where Qortex fits
Qortex runs hosted video models built for media, advertising and entertainment. Send video URLs, choose your signals and get structured JSON back.
- Content Intelligence: content categories, transcripts, summaries, sentiment, risk profiles, brand suitability, brand and logo recognition, scene and on-screen understanding, and advertiser opportunities.
- Creative Intelligence: the messages, themes, products, tone and calls to action inside an ad.
- Built for scale: our infrastructure processes 100k+ videos per hour.
- Our own models: we've been working on them for over five years, taking in billions of data points and human-curating them. Your video isn't sent to a big LLM provider.
- New signals added regularly. Custom models on request.
Want to run the scorecard on your own videos? Get API access
