There is a price to pay for the multiple benefits of an IT infrastructure that supports media over IP – not best measured in dollars, but in the effort to plan and deploy an effective way to manage nature. complex of a computer network and an endpoint according to Felix Poulin, director of the Media Transport Architecture Lab at CBC / Radio-Canada.
“Along with the greater flexibility that Media over IP technology will provide us with comes greater complexity compared to SDI,” he said on November 11 during the “Continuous Deployment of Media Data Center on CBC IP ”during the SMPTE2020 virtual event. .
Poulin, accompanied by Alexandre Dugas, Project Manager for Basic Systems Engineering at CBC / Radio Canada, discussed the Canadian national public broadcaster’s new IP production center in Montreal, which is expected to be completed in 2021, and d ‘other national deployments planned.
“…[W]We have each endpoint that requires configuration of thousands of settings, including IP and multicast address, PTP settings, and media specific configurations, ”Poulin said.
Recognizing the tedious and error-prone nature of manually configuring or reconfiguring every piece of equipment in its network, CBC researched and found in IT data centers the tools and strategies needed to support initial and subsequent configuration changes on the long term, he said. .
CBC’s approach is based on the application of “a lot of continuous integration and continuous deployment (CI / CD) concepts and Dev Ops practices that have [a] proven operating experience [at] data centers throughout their lifecycle, ”said Poulin.
These concepts include scripted provisioning and configuration, updating and changes, performing validity checks, and monitoring. “This is indeed a major change for broadcast operations,” he said.
CBC / Radio Canada’s new 420,000 square foot head office in Montreal is dedicated to the broadcaster’s French network. It operates a 24-hour news channel with 10 floors of studios under the control of four automated control rooms. There are also two television studios and two multi-function studios with two control rooms, several radio studios and post-production facilities, according to Poulin.
The facility’s centralized main control room, which controls all French-language stations across Canada, is designed to handle 20 TV channels and 80 radio stations broadcast via OTA and streaming, he added. .
CBC / Radio Canada is also preparing to relocate its Toronto headquarters engine room to a new data center and is at the start of a technology update for all of its facilities in Canada.
The Montreal facility is the broadcaster’s first major project to be designed around IP technology. The architecture is based on JT-NM TR-1001-1 and uses SMPTE ST 2110 (-20, -30 and -40) for transport and a double spine-leaf COTS network with SMPTE ST 2022-7 redundancy, said Poulin.
The IEEE 1588 precision time protocol working with the SMPTE ST 2059 profile maintains precise synchronization. The broadcaster also uses proprietary AMWA NMOS, Ember +, and API protocols for control, he said.
“With that… [size] project to be delivered in the years to come, we believe that studying and automating the deployment is the only way to achieve … [this] huge task, ”Poulin said.
AUTOMATION OF DEPLOYMENT
Poulin defined three steps to automate deployment, including proper network setup, which eliminates the need for separate prep stations and the risk of errors due to human manipulation; initial terminal staging, which focuses on programming the minimum set of device parameters to enable the device management interfaces to be reached; and device configuration, involving “software programming of all media-specific parameters to make the endpoint usable and ready for provisioning in the broadcast control room,” he said.
As Poulin tackled the high-level deployment, Dugas laid out the details of the solution.
Several components make up the architecture of the solution. They include the continuous delivery pipeline, an IP Address Manager (IPAM), a Dynamic Host Configuration Protocol (DHCP) server to automatically provide and assign IP addresses; a firmware repository running on a file server; a network infrastructure physical path generator (NIPP); and an integrity check pipeline that runs daily. “All these components… [revolve] around IT automation tools, ”he said.
“The role of the continuous delivery pipeline is to organize sets of tasks into what the broadcaster calls workflows – which are involved in the initial preparation process, may play a role in network setup, and ‘perform Function calls in an organized fashion with appropriate metadata that is pulled from the IT automation tool, ”Dugas said. “Continuous delivery pipeline tools provide a level of abstraction to the end user who is not necessarily familiar with the scripting language. “
The architecture has three “sources of truth” to ensure automated error-free configuration: a schematic database providing a clear view and understanding of the physical network, a source version control tool that regularly records the state of the network. ‘IT automation tool and IPAM, which provides “a source of truth for all logical components of the network,” said Dugas.
At the heart of the automated deployment system is an agentless IT automation tool that translates staging settings into a series of comments, synchronizes various system components, and provides current information on endpoints and infrastructure. network, he said.
Another basic principle of the automated deployment system is an “idempotence criterion”. “We can say that a task is idempotent if it provides exactly the same results regardless of the initial stage of the IT automation tool and also regardless of the number of times it has been executed,” said he explained.
To complete the deployment loop, the automatic health check pipeline, which runs after enrolling a new device, and then again once a day to report any difference in the current state of the devices as well. that the network and IPAM, NIPP and the source version control the sources of truth, he said.
LESSONS TO SHARE
Poulin closed the presentation by offering some of the lessons CBC / Radio Canada learned during the deployment.
“We realized that multimedia over IP devices take five to ten times longer to test, prepare, configure, troubleshoot… [as does the] equivalent equipment based on SDI, ”Poulin said. “It became clear that we needed automation tools like those found in data centers to… [make deployment and continuous operation] handy.”
Broadcasters looking to make the transition to IT must decide on a “single source of truth” for every aspect of the system, including IP and multicast addresses, VLAN assignments, device inventory list, and configuration. network, before starting a deployment, he said.
“Build your deployment tools before you actually build your system,” he said. “So you can recoup your investment by automating the initial deployment and avoid having to reconcile your system state with configuration scripts. [which would be necessary] if you expected to automate at a later stage of the project.
Poulin also advised broadcasters to involve all key stakeholders, including the architect, system designer and operations. “Using a deployment continuity methodology is a major change in practice, and you need to involve everyone so that they can effectively implement the new methodologies,” he said.
The Director of CBC / Radio Canada’s Media Transport Architecture Lab concluded by reminding presentation participants that this approach is not business as usual for broadcasters.
“This is a major change for a broadcast media organization,” he said. “But it is a change that is needed to embrace IT technology at the heart of our multimedia infrastructure and achieve the expected benefits of the flexibility and agility promised by the complex transition. “
SMPTE 2020 content is available on demand at 2020.smpte.org.