Where Do You Want to Go Today?

That was a very good advertising line dreamed up by Microsoft. It is full of adventure and potential excitement and is based on you, the consumer, making a choice. So now we have to make a choice about what kind of moving-image content we are going to compress. Will it be film, video, or computer-generated digital imagery? They are all different.

What Is a Moving Image?

Think about moving pictures. How are they captured? They are two-dimensional recordings of an array of pixels at an instant in time. Film and TV cameras both capture motion, but the resulting images have very little in common when they are examined from a technical standpoint.

What Sort of Compression Do You Need?

There are two basic kinds of compression. The first is utterly lossless, while the other sacrifices some fidelity in order to improve the compression ratio. Lossless compression is a format-conversion process that compresses the physical manifestation of the video. Lossy compression actually compresses the informational content of the video. This is a subtle and very important distinction. Lossy compression ranges from high quality with undetectable artifacts down to massively degraded viewing experiences. The developers of video-compression technology continually strive to improve the ratio of quality to bit rate or compression.

What Sort of User Are You?

Video compression is of interest to a variety of users. Professionals see it as a full-time occupation and may have significant funding to support building large and complex systems. At the other end of the scale is the enthusiastic amateur videographer who wants to create a DVD to record some family history. Compression systems are available for both extremes as well as for those whose needs are somewhere in between. We will consider the larger scale first. Then we will work our way down to something suitable for single users running a small operation.

Broadcasters

There are relatively few large broadcasters in the world compared with the number of small- to medium-sized potential compressionists. By large broadcasters, we mean organizations that are broadcasting multiple channels of content simultaneously. Examples might be the BBC; MTV; or News Corporation, which is the parent company of all the Sky brands and others such as Fox Television.

The BBC broadcasts more than 26 hours of content for every hour of the day. This includes multiple national and international TV services and many audio services that are going out in the form of AM, FM, and digital radio. Some of this content is also streamed on the Internet to augment a large-scale Web site operation.

The production processes for that content are now almost completely digital, from inputting the source material and passing it through the editing process, to putting it on the air. Some content still arrives in analog form, and the U.K. still has some analog broadcasting for the domestic user. That analog service is actually delivered using digital systems almost to the very last point in the chain before the transmitter.
Video compression is fundamental to the broadcasting workflow. Storing everything in a totally uncompressed form is unrealistic for both technical and financial reasons. The compression chosen must allow the footage to be edited and reused without a generational loss being introduced. High-quality master copies of content might be preserved using a 50-Mbps storage format and routine content would use 25 Mbps. Other intermediate formats are used that have lower bit rates.

Archives require a lossless or nearly lossless storage format. By the time the video reaches the TV set in your living room, it has been compressed by a factor of 10 to 1 and is no longer anything like archival quality.

Long-term archival storage will almost certainly be in a variety of different formats. Some of them are lower quality than current production and broadcast standards allow because the older legacy systems could not achieve the quality levels that are now commonplace.

You cannot put back what is not there. Repurposing old archival content presents some special problems, and finding ways to get the very best quality out of old material is an ongoing challenge. The BBC Research Centre develops technology solutions to this sort of problem from time to time. A recent innovation was a digital processing tool that samples the PAL recorded color sub-carrier more accurately in
order to restore old PAL footage, eliminating a lot of analog noise and cross-color artifacts. This sort of research requires considerable resources and it benefits the entire industry.

Independent production companies now produce a lot of their own programs. They don’t have the money available to invest in massive technical infrastructures that the broadcasters use, but they do use similar equipment to shoot and edit.

Programs are typically delivered on Sony Digital betacam-based systems or on a format called DVCAM and its derivatives such as DVCPRO. These correspond to the 50Mbps and 25-Mbps formats that broadcasters prefer. HDTV variants of these are also beginning to be used. These formats are slightly compressed. All formats in use sacrifice some small portion of the picture information, even if it is just recording color data at a lower resolution than brightness.

The original material might have been shot on film or on video. Today’s material is delivered to broadcasters in a digital format. New technologies such as Material eXchange Format (MXF) allow media to be delivered in files rather than on tapes, and this allows transfers to be networked through an information technology (IT) infrastructure.
Any long-term storage of programs must be in a form that is ready to be reprocessed for new platforms and markets. A production company that has been in business for a while is likely to have some format-conversion and storage problems similar to those experienced by the broadcaster due to the continued change in video formats.

Pro-Consumer

This is a category of consumer that buys high-end equipment that is near professional quality and is almost good enough for production companies and broadcasters to use. The gap is narrowing, and the difference between pro-consumer and professional equipment is likely to be in the quality of the imaging optics and the storage capacity. Many of the physical components in the camera are common to consumer and professional equipment—especially in the case of DV-format cameras.

The imaging chip in professional broadcast-standard equipment produces a higher resolution and has a greater dynamic range from black to white, resulting in high-quality images. Sometimes a professional system will use multiple CCD chips rather than a single
chip to get even better quality.

Broadcasters will require lenses with good depth-of-field control and little or no distortion of the image. The storage capacity of the videotape in a professional system will be higher than a consumer model in terms of bit rate. A professional format is also likely to record for less time because the amount of data recorded is larger. The consequence is that tape-based systems must transport the tape faster to record sufficient data. The same applies to solid-state storage because the bulk of the stored data is larger.

The pricing of digital handycams is reducing to the point where this market is opening up now. Analog video recorders can still be purchased, but most manufacturers are also shipping a range of digital camcorders. Sony, for instance, manufactures the DCR PC105E model, which will play back DVCAM tapes recorded in low-end studio digital VCRs such as the DSR 11. DVCAM is widely used in news-gathering operations, and the video is often manipulated using the 25-Mbps DV25 format.

Problems You May Encounter

If the video you are trying to compress is of particularly poor quality, you will find the results of compressing it to be disappointing. The encoder interprets any significant noise in the original as frame-to-frame changes. Even non-moving background content will compress badly if there is significant noise present. Either your compressor will be unable to reach the target bit rate, or if you cap the bit rate, quality of the video when played back will be very bad indeed.

Motion artifacts are particularly hard to deal with when your source is interlaced video. Removing the interlacing is a significant challenge.

Digitizing home movie film may be subject to unstable positioning of the film within the gate of the projector or telecine unit unless you are using a professional system. This is called gate weave, and it leads to tracking errors and unintended frame-to-frame differences that must be motion compensated. Since the camera introduced some gate weave as well, you have two film-weave artifacts to cope with.

Tools that track the frames and stabilize the video before compression are worth investing in. Stabilization is not vital to the compression process, of course, but the output quality will be improved. The bit rate is consumed by compressing movement within the frame and not by the movement of the frame.

This last point is very subtle, isn’t it? This is very typical of the sort of complexity you will encounter in the compression process.

Dealing With Difficult Content

Preprocessing your video through a noise filter will help the compression process. At the outset, you are unlikely to purchase products such as Final Cut Pro or Premier as a priority. Their superior color-correction and noise-reduction plug-ins will rescue footage that was unusable, and you can add them to your system when you can afford to.
Tracking errors due to gate movement could be corrected by using the motion-tracking capabilities of Adobe After Effects to gently coerce the frames back to their optimum position with respect to one another. A slight loss of resolution will happen because you must frame a portion of the screen that always encloses a picture. If the film shifts significantly in the gate, you will lose some of the picture around the edges. A neatly cropped final output will remedy this, and as long as you only crop the over-scanned part of the picture, it is unlikely to seriously affect your viewing enjoyment. There is always the option to scale or composite into a frame of some kind if the movement is significant and the necessary cropping becomes extreme.

Data Loss During Digital Transcoding

Compression is simply a data-reduction process. No compression is genuinely lossless except perhaps for some of the production techniques used at the high end. In order to compress video to usable sizes for DVD or domestic-video archiving, some considerable data loss has to take place. This data simply cannot be put back once it has been discarded.

More recent codecs provide additional compression tools to alleviate these effects, which will help a great deal. The H.264 codec, for example, offers some very sophisticated compression features, but the penalty is increased CPU power and time required to compress the content.

You have to decide for yourself what the acceptable level of data loss is going to be. For film, there is no point in trying to preserve detail that is finer than the film grain, because all you are doing is preserving noise in the original photographed image. Mathematicians are bound to argue about this, and it is similar to the situation regarding audio sample rates. But that is a topic for later on. For now, note the name Harry Nyquist,
who conceived the idea of digital sampling 60 years before the technology was available to realize his invention.

Trading off horizontal resolution for video is preferable to losing vertical resolution due to the artifacts that vertical interpolations will introduce. If there is significant fine detail such as text, that will become illegible after severe compression.

Variable-bit-rate compression is much preferred for content that is not intended to be broadcast within a fixed-bit-rate budget. If the bit-rate budget is constrained in some way, a cooperative compression process on several channels will deliver some benefits. Two or more channels can make room for each other and be transmitted using less bandwidth than they would require if they were compressed independently of each other. This technique is called statistical multiplexing.

Downsides to Video Compression

There are times when video compression causes more problems than it solves. Trying to deploy it in unsuitable circumstances leads to difficulty later in the production and consumption cycle.

Origination Stage

Video compression should be avoided at the origination stage. Some cameras compress the video as it is written to tape or disk inside the camera. These devices never yield an uncompressed output. The consequence is that any content you shoot with such a camera will probably turn out to be totally useless for anything other than home movies.
For example, suppose you are shooting some training video and you want to color key the background. Regardless of whether you use blue or green screens or even the newer retro-reflective materials to get a good hard edge for your matte, any compression that loses detail will immediately compromise your ability to pull a matte off that video. The jaggies and edge-coded artifacts are so severe that your footage is rendered useless. The solution is to get a better camera and shoot the footage at the best-quality camera settings available, and then write it to tape with a high-bit-rate storage format with a nearlossless compression method.

Not for Archiving Master-Quality Material

For much the same reasons that you would a) shoot footage using the best available quality of equipment, film stock, or video recording format and b) use lossless compression techniques (the intent is to preserve the maximum information), the archives must be maintained at the highest possible quality. A completely lossless coding format is desirable but expensive in terms of storage space. Experience shows that if content is compressed with poorly chosen coding techniques, the archives become unusable very quickly. Putting the tracking systems in place and logging the rushes as soon as they arrive from the field must be seen as a benefit and not a cost. In the long term it will save money.

I will describe a very painful episode based on a true story. Only the names and places have been changed to protect the innocent.

Let’s say it is 2 years after you’ve launched a major multimedia-based Web site, and through cost-reduction exercises, you gave up logging the incoming material and just encoded it for delivery via modem, and then discarded the originals. When your marketing people strike the deal with a broadband portal to deliver your site as a major broadband experience, just how are you going to re-encode all that content? That is when you say, “If only we had kept the originals and built ourselves a media database to track and manage it all.” But the situation gets worse than that. Much worse. You have important and very complicated rights issues with media. Maybe your rights to use some of the clips have expired and they ought not to be visible on your site. Rights are sometimes granted only for modem delivery and either specifically exclude broadband or omit it. That gives the legal experts an opportunity to argue for additional payment. Some rights are clear and the media can be repurposed immediately. But how would you know which is which? In the words of the Ghostbusters, “Who you gonna call?” Effectively, the only archives you have are that rather scratchy looking, modem-quality, postage stamp-sized, frame-dropped, pixelated mess on your streaming server, and
because your finance people talked you into saving money at the outset, you are going to reap the results of a very bad business decision.

Maybe your business will recover from that. But in all likelihood it won’t, not without spending an awful lot of time and money reworking some content over again. And that assumes you are still able to find original master tapes—and get the rights to use them again.

The moral of this story is to think ahead to what you might be doing in the future—not
just next year but 5 and 10 years from now.

Getting Out of the Mire

The solution is to institute a media audit and try to trace where the originals of some of that footage are. The avoidance strategy in the first place would have been to build a media database even if it was manually operated. Keeping track of the location of every master copy of a clip and any compressed instances of it requires a reliable record-keeping system. Diligence is the only way to track the provenance of your media. Given that the originals are stored online somewhere, it is possible to set up some automation to load and transcode them one at a time without any human intervention. You have to plan way ahead, anticipate future needs of your content, and maintain archive-quality copies reliably. If they are compressed at all, they should be stored using a lossless compression format so that a transcoder is able to down sample to any format that is required. You cannot transcode up to a higher-quality format and not see some artifacts as a result. At best you can blur them to hide the worst effects.

To be able to compare the various picture sizes and frame rates, the size of the data sets we are operating on must be taken into consideration. Then the capacity planning for a new compression workflow system can take place. The calculations are fairly simple, although finding the initial values to use is not easy.

Planning the capacity of a system to provide sufficient disk space for the projects being developed on it is vital. If the cost of storage is the limiting factor, then the equation is reversed. The decision then becomes how much video and at what picture size is going to be stored in the available disk space.

Moving-Image Formats

Now that we understand the basic requirements of a compression system, we can look at what kind of material is going to be processed. Film, TV, and computer data are all different.