For the VFX and CG industries as a whole (and others relying on intense graphical artistry such as architecture and games), the search was on for a practical, flexible, and immediate solution.
It came as an enormous relief for many studios when they discovered that not only did such a solution exist, but that they already had it – sitting almost entirely unnoticed on their workstations. Its name is HP ZCentral Remote Boost, formerly known as RGS (Remote Graphics Software) and it comes bundled with every Z4, Z6 or Z8 sold.
The key to its success is that it moves pixels, not data. The Remote Boost sender uses 256-bit encryption to send a stream of pixels to the receiver via VPN, effectively turning the internet into a monitor cable. It’s a secure point-to-point stream, using the same security protocols an artist would use if working on-prem, but freeing them to harness the power of a workstation remotely using their own receiver machine. This receiver just needs enough processing power to decode the stream and deliver it to a local monitor, with mouse and keyboard inputs being sent back from the receiver. No data leaves the workstation at any time.
We spoke to two post-production/VFX studios to discover how this solution is helping them to stay operational and competitive while remote working is the rule, not the exception.
How quickly did the studios realise that they needed to implement a remote working studio?
“Early on there was the inkling that some people would have to start working from home and at that point I thought it would be sensible to look holistically at how we would do it for the entire studio,” explains Barry Zubel, Head of Systems for LIPSYNC Post, an established film and TV post-production house with recent projects including Fisherman’s Friends and Horrible Histories. “You can get away with some things when you’re dealing with low numbers, but a solution for some people is vastly different to a solution for everyone.”
“Our Tech Ops department could already work remotely, using Teradici to deal with DCP mastering,” continues Barry, “but it was the VFX department that was the problem. It’s traditionally a very secure environment and it was quite a frightening thought to allow them remote access.”
For Black Kite, a young, independent VFX and design company that works on advertising projects, music promos, fashion, and high-end long-form episodic work, the situation was very different. Founded by industry veterans who enjoyed successful careers at MPC, they felt it was simply time for a change.
“Scale isn’t everything; with that comes a lot of legacy technology infrastructure,” says Graham Bird, co-founder of Black Kite. “The technology architecture that the team devised was built for the future, to be agile and adaptive, and supporting and underpinning a talent-driven offer. But we didn't expect it to be tested so thoroughly by a global pandemic within 12 months of our launch.”
Because Black Kite had the benefit of building a technology pipeline to support this talent-first approach, with the aim of utilising artists from all over the world (we’re offered Norfolk and Kazakhstan as examples), its technology leaders were well aware of ZCentral Remote Boost and the role it would play.
“I had the privilege of talking to the developers at HP about RGS way back,” says Paula Wilkinson, Head of Technology. “And so I had it up my sleeve. We tried to play with it at Black Kite early on in its infancy and we were just testing it from one machine in one room to another machine in another room. But as soon as we got this other machine, Graham booked a job on it so we didn’t get too far with our testing.”
“But when lockdown happened we decided it was time to go back to our research, and we found it was quite easy to set up. Once we were happy we told the team that it was available, and the response was mainly ‘Sorry, what?’”
For Barry at LIPSYNC Post, Remote Boost was an unknown quantity. His first thought was to go with the Teradici solution he already knew, but delays in getting hold of the hardware – and bandwidth barriers with its software solution – meant it was a non-starter. “We had about two gigs of bandwidth and a couple of decent Firewalls, but they were only rated for VPN for about 350mbps each, so only 700mbps traffic.” With around 60 artists needing to access their workstations remotely, it was clear he needed another solution.
Although the entire VFX floor was HP workstations, he was unaware of Remote Boost. “My first question was ‘how much is it?’ I found out it was bundled with the price of the workstations and knew my Financial Director would be very happy! The cost of zero was very attractive, but how good was the quality?”
With LIPSYNC Post artists running Nuke for 2D and Maya for 3D work, the technology demands for working remotely were different for each group of users. “Maya over a remote connection can be… awkward,” Barry understates. “So we were looking for two different solutions. We’d already considered an NX server, which seemed great for 2D, but didn’t handle 3D. Being able to twist, rotate or transform objects is tough remotely. You get a lot of latency; it’s not pixel perfect so you may miss when trying to grab hold of a handle on something; and re-rendering the screen is very expensive because it takes time. All of those things together made it a non-optimal solution for something like Maya, or even Houdini.”
Barry then began testing HP ZCentral Remote Boost in the office, and was impressed with the quality immediately. “The quality being sent back was far in excess of anything else, and you could operate Nuke, Maya and Houdini very effectively.” The next step was to set up the security parameters (such as disabling clipboards and other ways to prevent the egress of data) and test it over VPN. ”Again we were pleasantly surprised. Even with the extra CPU overhead and latency of VPN, it worked almost as good as if you were in the office.”
Once Barry was convinced that Remote Boost was the solution, it was rolled out with Puppet – taking just 30 minutes across LIPSYNC’s entire sender estate. Only one small change to the existing infrastructure was required. “We had to pop a single port hole in the ACL, to allow access for RGS. Once we had a route from the VPN that would allow the RGS traffic to pass through, it was easy.”
On the receiver side of the equation, some artists were already pre-configured and had taken a workstation from the office. For the rest, who were mainly using their own machines, Barry sent a download link with instructions, with most people able to set up and access their workstations remotely very quickly.
Black Kite points to one particular factor when noting this ease of rollout: HP has developed the receiver for Windows, Mac OS and Linux.They predominantly run Z series Workstations at the East London Studio, but not many people have powerful Linux boxes in their homes. They might have an iMac, a MacBook Pro, or a Windows machine so the ability to install the receiver so easily and get someone on a VPN is tremendous. One example is an artist in Norfolk with an iMac who was given remote access to a workstation with Flame via VPN and Remote Boost.
With Remote Boost only requiring a 20mbps to 30mbps uplink to operate Flame, Maya, Nuke, Houdini etc smoothly and consistently over VPN on a 2560x1440 screen, will this lockdown-enforced way of working change the way the industry works in the future?
“Over the last ten years or so, technology has become progressively cheaper,” explains Adam Crocker, Head of VFX at Black Kite, “There’s Cloud technology etc, and you’re no longer reliant on huge infrastructures and machinery - we’re witnessing the complete democratisation of the industry. Now it’s become strictly back to talent, like in the early days of our industry.”
Remote Boost allows Black Kite, LIPSYNC post and any post-house or design studio to cherry pick from the best talent available based on skills, not location. “Corona has forced companies to bring forward their plans,” continues Adam. “When we come back, our industry will have been pushed further in the direction it was heading. Now you’ve had people tasting what it’s like to work like a freelancer - having a Flame suite in effect in your cupboard at home isn’t something most people are likely to have had before. We’re used to travelling into the office to use the Flame suite, then home again at the end of the day. Now after a day’s work, if you have an idea for something that you want to try out, for a pitch for example, you can just walk into the next room and do it immediately. Or at the weekend, you don’t have to travel in for an hour then out for another hour, you can just jump on and do something.”
“People are working completely different hours now,” confirms Barry at LIPSYNC post, “probably more than if they were in the office. But they’re fitting it around other things, not just working from nine until six.”
The way clients view remote working has also changed during the pandemic. Innovative studios have not only proved that this way of delivering projects can work, but that it is just as viable and secure as the on-premises model of old. Black Kite has seen its technology embraced by production companies and ad agencies, and has been engaged at the pre-production stage to explain how best to shoot and execute on a technology level. A strong example is the recent Rolling Stones video for Living In A Ghost Town, which captured the empty streets of London and other towns and cities around the world. Black Kite processed all the data remotely through its servers, whilst using HP ZCentral Remote Boost to build the timeline and edit, pulling shots as and when they needed them. Moving pixels not data, meant the project could be completed by people using the power of workstations remotely, from wherever they were under lockdown conditions.
Barry at LIPSYNC believes that any studio working remotely needs to be transparent with its clients. “Allowing remote access to your network and not telling your clients can cause problems. We told them what we needed to do, and how we were proposing to do it.”
“We use 2-factor on our credentials for logins, so there’s no easy way for a key logger to steal those credentials or get in any other way. We refer to it as a pane of glass into our network. A slight concern was that content is being displayed on a remote screen. We control the workstation driving it, but we aren’t in the room with that screen. In the interest of transparency, this was brought up with clients and in every single case the client was very understanding – largely I think because the client wanted their work completed very quickly! All that being said, we have not relaxed any of our other security measures that we have for our internal corporate network and our egresses, so we’re keeping a very close eye on traffic and transfers in and out. It’s probably higher than it usually is.”
Throughout this whole process, Escape Technology has been acting as a consultant, using our systems administration expertise to ensure that any potential issues are resolved before they become a real-world problem. Paula at Black Kite notes that although she has been working with Escape Technology for years now, this recent upheaval has nurtured a closer relationship than ever before, with Escape providing fantastic support. “They know the business, what’s not to love?” she concludes.
The sentiment is echoed by Graham, who recalls a day of technology company visits while they were piecing together the technology that would power Black Kite. “We shared our plan and they had ideas - they put forward different solutions and understood what it was like to work on the frontline dealing with clients. They understood the requirements. They stood out from everyone else we spoke to and were there to help.”
“What we set out to achieve 12 months ago has been proved by the lockdown,” finishes Graham. “We can have flexible, agile, technology infrastructure that allows us to work with talent anywhere in the world. We believed this was the right way to work. You don’t need 250 artists under one roof in expensive London real estate in order to compete and produce outstanding work for the world’s top agencies and directors.