Conversations With Cliff Plumer Part I: A Magical Move

3. Bringing The Farm To User Land

While Lucasfilm is most noted for its huge render farm to make their visuals shine, the move gave the company an opportunity to further expand the render farm out to the entire corporate Workstation population. One of Plumer's ideas was to pull plenty of fiber around the new campus, and make sure that all workstations could be remote controlled from a central location. This includes using them during off hours when all that CPU power can continue to work productively.

ILM's new campus is easy to get to.

The main render farm, lovingly called "the Death Star," is located in the 13,500 square foot data center. The farm includes about 1,000 machines. Three thousand user workstations are spread around the campus. All of the Lucas buildings at the Presidio have raised floors everywhere to facilitate installation of fiber. The company standardized on Opteron-based HP 9300 workstations, and is using the higher-end dual-core Athlons from AMD, too.

"Our machines are working 24x7. When the artists go home in the evening, those workstations become part of the render farm, thanks to our own job scheduling program," Plumer said. "The jobs go into a large queue and from there are assigned to a processor - it could be any AMD processor on our network, so the render farm can grow to 4,000 machines at night."

With all these PCs crunching away, you can imagine the cooling costs are enormous. "We are also clever at what we can do with power cycling the systems to keep the cooling costs down," Plumer said.

What Is The IT Department At Lucas Like?

The IT department at Lucasfilm isn't like the one at your average insurance company or bank, to be sure. Plumer supervises four different departments and about 150 people in total. There is his support staff, which at least now has to support a single consolidated campus and not run among several different locations. There is another department that develops tools and handles special projects for the film and game developers and does new product R&D when needed. There is your traditional information systems department that handles tracking the visual assets and production projects as well as the various databases that are the repository for this information. Finally, there is his system engineering team, a small group of guys that Plumer says he "can throw any problem at anytime."

As you might expect, the environment is pretty stressful but satisfying. Plumer said he is always looking for good people, but emphasizes that "we are a pretty intensive environment."

"ILM is known for delivering on time, and a lot of that is due to how we can scale up very quickly, and our throughput is second to no one. That puts a lot of pressure on our support team," Plumer said. "On the engineering side, with systems and computer graphics, we are always looking for people. We have many engineers with advanced PhDs in physics and imaging."

Plumer mentioned that he works with several Stanford University students as interns, who help with writing scripts in various languages such as perl and python. "We try to get people who have exposure to the bigger picture and understand the context of their work," he said. "We have a lot of perfectionists here."

Next week, we continue our conversation with Cliff Plumer about more of his computing challenges, including using mobile workstations to bring the special effects creation process right onto the set where a film is being shot.