Xilinx is found at the heart of all professional multimedia applications, with adaptable platforms capable of supporting the latest UHD and multichannel audio requirements.
Xilinx platforms are unrivalled in providing the digital signal processing (DSP) performance required in audio processing, interfacing, compression, embedding and conversion. The inherent parallelism of Xilinx architectures means that many channels of audio can be processed together using very efficient resources. Xilinx Vitis development tools support high level languages such as C++, so audio engineers can rapidly integrate audio functionality into their products, offering differentiation, channel density and fast time-to-market.
As imaging resolutions and frame rates increase to 4K and beyond, with higher frame rates and higher dynamic range, it is essential to perform real-time image and video processing within space-constrained designs of cameras and camcorders. In addition, new capabilities are required such as video analytics and metadata acquisition to enable efficient workflows and monetize content.
Xilinx platforms integrate high speed connectivity from the latest 4K and 8K sensors with a fully flexible image, ML and video processing pipeline. Add lossless or lossy encoding capabilities where needed and a variety of output connectivity standards, this is now all possible in a single device, offering significant space, cost and power savings.
Video streaming has become a ubiquitous part of life, with consumers demanding content anywhere and on any device, and workflows becoming decentralised with remote production driving the need for compressed content distribution. Streaming endpoints are essentially “codecs-in-a-box” and typically take one or two AV inputs from SDI or HDMI, compress using H.264/H.265 and stream over IP using Ethernet. In parallel, the emergence of similar functionality being integrated into equipment such as mini-switchers is turning streaming from a separate box into a feature.
Whether used in a standalone product, or integrated into another application, Xilinx Zynq UltraScale+ MPSoCs with on-board H.264/H.265 video codec supporting 4K60, and adaptable hardware to support AV connectivity and processing, plus machine learning for optimising bandwidth using region-of-interest (ROI) encoding, make it the ideal platform for any streaming design.
Whether a standalone rack unit or a dongle, or integrated as functionality in a larger AV product, keyboard, video, and mouse extension (KVM) with extremely low latency and high video quality is critical for access to video content regardless of where the client or server resides. Used in a variety of applications from broadcast, eSports, control rooms, ship command centers and airports, KVM is moving from simple point-to-point extenders and matrices to embrace IP and widen both its reach and functionality.
The emergence of new display technologies is pushing the boundaries of resolution, HDR and fast frame rates to support 4K120 and 8K video, meaning that the latest AV connectivity standards such as HDMI 2.1 and DisplayPort 1.4 need to be bridged to reliable and secure Ethernet networks. Xilinx platforms are the ideal platform for integrating connectivity with a choice of mezzanine codecs and H.264/H.265 for simultaneous LAN and WAN streaming.
With the wide variety of compressed and uncompressed video and audio interfaces and formats available, it’s inevitable that simple converter boxes will be needed around major installations to glue the network together. These simple but effective converters can take in a standard such as SDI and convert it to an HDMI output (or vice versa). Often capable of handling more than one channel, converters are typically low-cost and small form factor, so integration is key whilst not sacrificing any video or audio quality. Some converters offer video processing to scale from one format to another, or the latest versions will support the transition to IP networks such as ST 2110.
Supporting any-to-any connectivity, Xilinx platforms also provide integrated single-chip implementations with ARM® processors for system control and audio processing, video processing pipelines for format conversion and optional codecs for mezzanine, intraframe and long-GOP compression.
Whether building projection systems for cinema, enterprise or simulators, the need for high resolution, high brightness 4K and 8K images is key. In cinema and high-end projectors, the integrated media block (IMB) typically deals with video processing and compression. Scaling, color correction and advanced video warping (keystone correction, rotation, arbitrary warp, stitching and blending) can be integrated with 4K and 8K interfacing and video processing in our cost-optimized devices for office projectors, or high performance HBM platforms used in cinema, simulation and visualization.
Optical engines include the light source devices and are the heart of the projection system. Xilinx FPGAs accompany the world’s leading light source engines (DLP® and LCOS) to support Expanded Pixel Resolution (XPR), making it possible to use lower resolution light sources to create high resolution images.
Routers are the central connectivity component of any broadcast studio, live event, office building or campus, connecting multiple sources (cameras, media servers, microphones) to various displays (multiviewers, video walls, projectors) or archival systems. These have been predominantly point-to-point baseband audio and video connections supporting from 2 or 4 to hundreds of SDI, HDMI and DisplayPort channels in and out of a crosspoint switch. Add video processing, digital video effects (DVE) and compression with a control surface, and the router becomes a switcher. Recently routers and switchers have become hybrid devices, bridging Ethernet and IP networks as well as baseband, and using new AV-over-IP technologies such as ST 2110, Dante, NDI and IPMX.
Supporting a scalable number of 4K and 8K HDR channels with a variety of integrated and mezzanine codecs, audio and video effects, and minimal latency, Xilinx platforms continue to be the main system components in routers and switchers.
Xilinx offers a wide range of video IP to enable display vendors to deliver the highest quality video, the ability to differentiate through novel processing algorithms and feature enhancements, and provides a balance between cost and power requirements.
For large multiviewers, handling multiple uncompressed and compressed HD, 4K and 8K streams with video processing (particularly scaling and multiscaling) can be done in Xilinx adaptable platforms offering integrated high bandwidth memory (HBM) to enable more channels, or higher resolutions and frame rates.
Broadcast and Pro AV media servers combine storage, retrieval and playout in a variety of usecases from high-channel density OTT distribution to digital signage media players. Unlike traditional servers, media servers need to combine baseband AV, IP transport and file-based ingest with integrated playout features that can also incorporate logos and station branding graphics, local ad insertion, digital video effects and live switching. Incorporating Xilinx expertise in both multimedia and compute storage domains means that you can quickly build on-premise media servers that work with cloud playout services that can adapt to available bandwidth and scale usage requirements, whilst providing highest reliability for guaranteed service to your viewers.
Globally there are numerous standards for modulating, transmitting and receiving digital television signals. From ISDB to DVB, from ATSC to DOCSIS, each standard has been developed to support unique robustness requirements of satellite, cable and terrestrial transmisison, as well as tuning those technologies for the local geopolitical needs. However each standard has a relatively common signal processing chain, from mapping and channel coding (Forward Error Correction), to baseband shaping and up conversion to the transmission DAC. All of these functions are possible in a Xilinx FPGA, SoC or RFSoC, with the benefit of providing a dense, multichannel solution in a single chip (reducing rack space and cost) as well as offering adaptabilty to support emerging next generation standards. To accelerate time-to-market, Xilinx partners offer a range of modulation IP to support these applications, as well as design services to provide a tailored and optimized product.
For live events, news and sports coverage, having a wireless transmitter directly attached to the camera means a speedy deployment in any environment, but it has unique challenges. Latency needs to be as low as possible, particularly in multi-camera sports events. Video quality has to remain as high as possible even through noisy wireless transmission, and continuously adapt to available bandwidth. Portability means battery-powered and small form-factor, so power needs to be as low as possible and without the stress of thermal management.
Fortunately Xilinx has the ideal platform that balances all of these requirements. The Zynq UltraScale+ MPSoC has a hardened H.264/H.265 4:2:2 10-bit codec supporting 4K UHD and is already being used in many mobile wireless video systems. It has the industry’s lowest latency, can be finely controlled to balance VQ and bandwidth, and can integrate UHD-SDI channels and various video processing algorithms all in the same device, lowering system power and cost.
For contribution, there is typically a choice between mezzanine codecs (such as JPEG 2000 and JPEG-XS) or H.264/H.265. Xilinx Alliance partners offer a range of mezzanine codec IP cores with high VQ, small footprint and low latency. Xilinx also offers the Zynq UltraScale+ MPSoC with hardened H.264/H.265 4:2:2 10-bit codec supporting 4K UHD. Using Xilinx platforms can also enable the system integration of UHD-SDI, ST 2110 and ST 2022 for ingest and playout.
For distribution, whether for OTT (over-the-top Internet distribution) or OTA (over-the-air transmission), video service providers are faced with delivering high-quality user experiences, while managing their infrastructure and operating costs. This typically means choosing between broadcasting a small number of streams to large viewerships with the highest possible video quality, or optimized for a high volume of broadcasters with smaller number of viewers demanding high-density. In both cases, and whether on-premise or in the cloud, encoding, transcoding and decoding can be implemented on Xilinx platforms. The Zynq UltraScale+ MPSoC H.264/H.265 family is ideal for high density applications, and Virtex UltraScale+ FPGAs or Alveo accelerator cards a perfect choice for highest quality broadcast distribution using H.264, H.265, AV1 or the latest codec enhancement layers provided by LCEVC.
The broadcast industry is migrating to IP networks and has been using IP contribution using SMPTE ST 2022 for some time, while the deployment of all-IP facilities using ST 2110 is well underway. But there is still a huge amount of legacy equipment in use, and SDI still has advantages over IP networks, so it’s important to be able to move content around regardless of the connectivity standards in use.
Providing any media over any network, Xilinx devices are capable of integrating many channels of UHD-SDI and up to 400G Ethernet ports in high-performance broadcast networking platforms, or in more optimized modular networking card solutions for fewer channels that can be added when needed. Whether the media is compressed or uncompressed, Xilinx devices offer unrivalled transceiver performance, huge bandwidth capabilities and on-board high-bandiwdth memory (HBM) for packet and frame buffering. Add timing and synchronization using ST 2059 and open APIs for software control, Xilinx FPGAs and SoCs provide the ideal bridge between existing and new topologies, and an adaptable and scalable solution which can support emerging standards and technologies.
Digital signage applications range from information displays to advertising, and can be targeted to an individual viewer or large audiences. It could be a single flat panel display, perhaps with interactivity, a large video wall made of multiple displays synchronized together, or an LED wall made up of many individual tiles to create very large formats. Finer pitch LED walls are on the verge of shifting from large outdoor advertising displays to replacing projection in cinemas, meeting rooms and even the home. For LED walls, software-only systems running on CPUs are unable to meet the extremely high performance real-time AV processing required by larger formats and therefore rely on Xilinx platforms. For advertising, the introduction of cameras with machine learning means that viewer metrics can be analysed and relevant targeted ads displayed, offering monetization opportunities.
Video conferencing systems are morphing into much more capable collaboration systems that provide easy-to-use sharing of video, audio and content in real-time. In addition to traditional meeting rooms with high-end conferencing, there are estimated to be more than 30 million huddle rooms on the planet, of which less than 2% are video enabled. This untapped potential represents an enormous opportunity for collaboration equipment vendors to add intelligent video capabilities to meetings. Through edge-AI enabled cameras, backed up by real-time image warping, low-latency high quality compression and IP streaming, the user-experience is vastly improved, with seamless and lifelike interaction getting nearer and nearer.
Manufacturers can benefit from highly-integrated and cost-effective video processing, computer vision and streaming solutions built on Xilinx adaptable platforms, particularly the Zynq-7000 SoC for embedded processing, and the Zynq UltraScale+ MPSoC EV for integrated H.264/H.265 compression.
Xilinx offers a range of platforms for broadcast and professional multimedia systems. From FPGAs and ACAPs to MPSoCs with integrated H.264/H.265 video codec unit, and from Kria SOMs to Alveo accelerator cards, whether you want to implement chip-down or plug-in board, Xilinx provides the ideal real-time, low-latency AV processing platform.
Most multimedia systems use a combination of AV connectivity (both baseband and IP transport), audio and video processing, and compression. Xilinx and our Alliance Program Partners offer a huge portfolio of IP cores, reference designs and subsystems to get you faster to market.