Measuring and Monitoring Streaming Media Quality

Time Profile Metrics
From an end-user perspective, a successful streaming experience is indicated most closely by the time profile. How long did the user have to wait before the stream began? Many studies have shown how quickly an audience member will give up on content if forced to wait too long. How often was the stream interrupted? Most users will agree that interruptions during playback for re-buffering are a major cause of frustration.

Connect Time
Connect time measures the amount of time elapsed between the initial request for data by the media player and the start of buffering. This time includes Domain Name System (DNS) lookup and resolution, metafile actions and resolution, server/player "handshakes," and the transport of the first byte of data to the player.

Buffer Time
Buffer time measures the amount of time used to build up the initial buffer of data for the stream. The time required for initial buffering is determined by the bit rate of the stream, the bandwidth available between server and client, the consistency of that bandwidth, and settings on the server and client.

Buffering can never be eliminated completely because it is an inherent part of the streaming protocol. However, a properly configured serving system can keep buffering time to well under ten seconds. (The current generation of streaming servers and players frequently keep this number under one second through the use of data bursting.)

Rebuffer Events
A rebuffer event is when playback is halted to rebuild the buffer. Rebuffering occurs when there is a decrease in the incoming bit rate of the stream. Therefore, rebuffering is an indication of an inconsistent connection.

Rebuffer Time
Rebuffer time equals the total amount of time spent during all rebuffer events. This time represents interruptions of the viewing/ listening experience. With a properly configured serving system, rebuffer time should be zero.

Advanced Metrics
One type of advanced metrics is packet metrics. Streaming media, like all data on the Internet, is delivered in the form of individual packets. Tracking the delivery of these packets can reveal the nature of problems with the streaming infrastructure. In this area, important numbers to know are how many packets arrived late, early, and on time based on the scheduled delivery time. It is also important to know if packets arrived in their correct order and if any packets were lost.

Another kind of advanced metrics is transport metrics. Like packet metrics, transport metrics are intended for the network engineer diagnosing the performance of streaming infrastructure.

Server Type
There are many flavors of streaming servers currently available on the market. Each has its own strengths and weaknesses. As well, specific versions of a streaming server may have peculiarities. (Note: if you are serving your streams from your own in-house facility, you will already know this information.)

Protocol
Streaming media can be served across different protocols such as UDP, TCP, or HTTP. Often, streaming servers will support multiple protocols and even switch between protocols in response to different client requests and changing Internet conditions.

Traceroute
Traceroute is the path that streamed packets travel between their streaming server and the end-user or agent. This information can be used to diagnose problems with router configurations or peering arrangements. Traceroutes are particularly interesting when measuring the performance of a CDN with a distributed serving architecture. In this case, one can see how effectively the CDN selects a nearby server for a particular audience member.