Wowza Moves Up the Stack with the Video Intelligence Framework (VIF)
Wowza has spent more than two decades serving as plumbing for applications as diverse as premium concerts and fights, high school sports, church services, security and surveillance, and smart cities. The Wowza Streaming Engine (WSE) is a media server that ingests, transcodes, packages, and delivers live video for something north of 35,000 deployments across 170-plus countries. On July 20, Wowza shipped a WSE module that embeds real-time AI analysis directly into the media server. This analysis can be performed in real time on local hardware, with no internet required.
Wowza calls the technology the Video Intelligence Framework, or VIF. To be clear, architecturally, VIF doesn’t enable anything impossible with external AI stacks. But it makes those workflows cheaper, lower-latency, and easier to operate by running inference where the video already flows, inside Wowza Streaming Engine.
Because it’s inexpensive and accessible, VIF can dramatically cut the cost of mission-critical analytics that organizations would run anyway, turning required workflows into far cheaper ones. For many others, these factors will enable producers to get more from their videos, like scene-level metadata to feed programmatic advertising and enable more personalized and higher-performing ads and higher CPMs. In short, if you’re streaming live video for mission-critical operations or revenue, VIF is a technology you need to know about.
What it is
VIF is an AI inference module that runs inside Wowza Streaming Engine, rather than in a separate NVR/VMS stack or a cloud video AI service. It samples frames from streams already flowing through the engine, runs them through computer-vision models on a local GPU, and converts the results into structured events. As described in more detail below, VIF comes with several models, but it’s model-agnostic by design, so customers can use their own models or add third-party models for specific verticals.
The detection could be a goal, a jersey number, a weapon, a smoldering fire, or a car crossing the centerline. The analysis runs where the video already sits, whether at the edge, on-premises, in a private cloud, or fully air-gapped with no outside dependency after setup. Wowza quotes analysis latency under 200 milliseconds, though that will vary by model, resolution, and load, and in practice depends on the available NVIDIA GPU memory and compute headroom alongside existing transcoding workloads.
For integration partners, this deployment model simplifies adding AI-related features to existing installations. Because VIF utilizes the current media server and camera footprint, integrators can deploy automated detection and alerting onto active installations, often without new hardware, moving the configuration focus from hardware provisioning to software and workflow integration.
How it works
Let's explore how VIF works within the context of a live sports deployment (Figure 1) assuming models supplied by the streaming publisher. The Streaming Engine component runs as normal. Arena cameras connect to encoders that send live video into Wowza Streaming Engine. The engine ingests those streams, transcodes them to multiple bitrates, and packages them as HLS or DASH with DRM and captions. It delivers the packaged streams to a CDN or origin, which distributes them to the players and apps where fans watch live. Nothing about this changes.
Figure 1. VIF works in parallel to the Wowza Streaming Engine (in green).
As the figure shows, VIF runs as a separate workload alongside delivery. The VIF Frame Extractor samples frames from the same live streams as they pass through the engine. Those frames go to the AI Inference Layer, which runs sports models against them for goal detection, player ID, and scoreboard OCR. The results pass to the VIF Event and Metadata Engine, which converts them into structured events and routes them.
The Event and Metadata Engine sends output in two directions. Some events go outbound to external systems: JSON over webhooks to a scouting and analytics database, auto-created clips and tags to a highlight CMS, and camera and quality alerts to an ops dashboard. Others go back into the live stream through Wowza Streaming Engine as timed ID3 metadata and overlays, reaching player and ad systems and rendering as live graphics. Downstream, those outputs can feed personalized highlight libraries, coaching and performance tools, and contextual ads placed around goals and key moments.
Running VIF Without Connectivity
Now let's analyze VIF within the context of a security application that must run without connectivity (Figure 2), again assuming models supplied by the integrator or user. The deployment runs the same pipeline. Cameras feed encoders, Wowza Streaming Engine ingests and packages the streams, and VIF branches off the engine to sample frames, run inference, and route the results.
Two aspects change from the sports example. The models detect humans and vehicles, intrusion and zone breaches, smoke and fire, PPE compliance, and license plates. And the outputs feed security systems, a SIEM, VMS bookmarks and search, an evidence clip service, an ops dashboard, and dispatch, instead of highlight and ad systems.
Figure 2. In a surveillance deployment, the same pipeline runs entirely inside the organization's network. No video leaves the building for analysis, which is the requirement for buyers under data-residency rules.
What's different is where it all happens. A sports stream sends videos out to fans through a CDN, so the video leaves the network by design. A security deployment keeps everything in place. The cameras, the streaming engine, the GPU running inference, and the systems consuming the output all sit inside the organization's own network. Frames are sampled and analyzed on local hardware, and the events, clips, and alerts go only to internal systems. No frame is sent to an outside service to be analyzed, and nothing has to cross the network boundary for VIF to work.
One framework, two markets
In the sports and security markets, VIF uses a similar workflow to solve two different problems. In surveillance, the core problem is sheer volume: most captured footage is never reviewed. Wowza's press release cites Gartner's estimate that 80 to 90 percent of stored video is never watched. VIF turns that storage into real-time alerts.
In broadcast, the problem is that scene-level metadata is expensive and slow, hard to capture live and costly to generate for VOD after the fact. VIF produces that metadata as a byproduct of the stream. One pipeline, two economics.
Three things to test
Every technology sounds great at the announcement, but the devil is always in the details. Early testers should focus their attention on three technology characteristics.
Hardware. Wowza positions VIF as running on existing NVIDIA GPU infrastructure, potentially without new datacenter hardware. The availability details qualify that as supported NVIDIA AI infrastructure. Whether a given site's current GPUs qualify, and whether they can carry inference on top of their existing transcoding load, is worth checking before assuming no new hardware. A more accurate reading is “may not require new hardware,” not a blanket “will never require new hardware.”
Sampling. VIF samples frames rather than reading all of them. That suits event and threshold detection but may rule out applications that require continuous tracking.
Pricing. Wowza had not set pricing in time to include with this article. Check the press release for pricing.
What ships with VIF
VIF is generally available July 22. It ships natively with production-grade models right, including an RF-DETR object-detection model and CLIP-based scene analysis. Additionally, the framework includes access to the NVIDIA Synthetic Video Detector for deployments running on supported NVIDIA AI infrastructure.
Because the framework is model-agnostic, it fully supports a Bring Your Own Model (BYOM) workflow. Operators and developers can integrate their own custom-trained computer vision models or bundle specialized third-party algorithms tailored for distinct verticals.
Production Alternatives
In essence, VIF competes against two alternative application paths. The first is a separate on-prem stack, typically configured with one or more NVIDIA-equipped GPU servers or AI appliances. These ingest camera or streaming feeds, run computer-vision models, and push events into SIEM, VMS, and operational systems. VIF should offer lower CapEx, lower OPpEx, and a simpler development path. It may also materially reduce latency by reusing existing streaming infrastructure and GPUs, avoiding separate analytics hardware and software, and keeping inference in the same process that already handles the live video.
The other alternative is pure cloud video AI services, where live or recorded streams are sent to cloud analytics APIs for object detection, scene analysis, and metadata extraction. VIF will require more CAPEX, because you buy and manage your own GPUs, but OpEx should be substantially cheaper since you avoid metered cloud compute and storage for every analyzed minute. Latency will also be much lower because you eliminate the cloud round trip. Obviously, this cloud model isn’t available for applications that must work without connectivity or that cannot send video off-premises under data-residency and security mandates.
Taken together, these advantages put Wowza at the top of a very short list for Wowza Streaming Engine users needing to add AI analysis to their workflows, especially when they need on-prem or air-gapped operation. It also puts Wowza on the short list for developers choosing an engine for live streaming applications that require AI analysis or may do so in the future.
Read Jan Ozer's full interview with Wowza CEO Krish Kumar on Streaming Learning Center.
See Wowza's VIF launch video for more detail.
Related Articles
Is the streaming industry on another pendulum swing in streaming infrastructure, away from embracing the "pure" cloud approach in vogue over the last few years toward a more hybrid setup integrating both cloud and on-prem elements? Wowza VP of marketing Alex Gammelgard explores this theory along with growing data security concerns and more in an interview with Streaming Media contributing editor Timothy Fore-Siglin at Streaming Media 2025.
26 Nov 2025
Today's streaming industry growth is being fueled by both increased demand for traditional OTT streaming as well as expansion into new corporate and creator use cases. These latter use cases in particular are driving providers like Wowza to adapt and make video streaming more accessible through easier, more affordable, and more flexible implementations.
06 Oct 2023