Shaping The Future: SCAD’s XR Stage

The acquisition of an XR Stage has given SCAD a whole new set of tools for students to use. Real-time rendering in place of a green screen will continue to be in demand as the technology improves and studios begin to acquire XR stages of their own. Now that the stage is available for students and professors to use and teach with, we at Render Q thought it would be important to reach out to SCAD faculty and students to get their opinions on the stage and how they feel it will be used as a teaching tool.

Interview with Professor Galeano

What motivated SCAD towards investing in an XR stage?

It’s the latest technology, and it’s also the future of filmmaking. The XR stage is still in the early stages, so it’s a great idea that SCAD wanted to invest early on to train and prepare our students for what will be happening in the filmmaking industry.

What is the XR stage used for, and who can use it?

Anybody can use it. You can use it for music videos, fashion films, filmmaking, or any setup, even a podcast. So pretty much across the board, anybody can take advantage of the XR stage.

The most significant advantage is that the background, which is the LED wall, will have real-time rendering. So the reflections, the lighting, and everything that’s going to be cast on the talent will be the actual reflections and lighting from the background. In addition to this, the actor will be able to see what the environment they’re acting in looks like; they will be able to get into character more. Since we are using real-time rendering software like Unreal, you can use it as a real-time tool for camera movement. The real-life camera is cloned and attached to a CG camera inside Unreal, allowing for really nice integration right on set. So basically, what you shoot is what you get. There is minimal post-production which is one of the biggest things about the XR stage workflow. It solves the problem of post-production issues. And the ease of the camera movement gives you the freedom to do pretty much anything you want on stage.

How will the XR stage be used as a teaching tool?

It’s an essential tool for students to learn because it’s the future of filmmaking and other applications. So we want to go ahead and train our students as early as possible for what the future holds for them. Knowing Unreal software is fundamental because it’s a real-time rendering system that allows for faster renders. Unreal Engine, the software itself, is taking over the industry like a storm. Here at SCAD, we already teach the students how to use it for the visual production workflow. So it is essential for students need to learn early on what the future holds.

Do you see real-time rendering replacing the green screen in the near future? If so, how long do you think it will take?

We already live in it, and it’s already happening in terms of how long it takes. The issue or the challenge is that it’s not readily available because it’s expensive. Although for the more prominent production companies, they’re trying to get into it because they know it will be seen more in the future. It’s not one hundred percent going to replace any other technology because of the portability of green and blue screens to shoot on location unless we get a portable LED stage, which is not yet the case.

So it’s not going to replace it completely, and it’s not meant to do an entire project; it’s intended to be used for certain shots. This technology is designed to support and help the production workflow to make it more integrated and seamless. Still, it’s not meant to replace the green or blue screen or any other visual effects techniques. But it’s definitely something that’s going to be part of the workflow, and it’s going to make our students feel more prepared for what’s happening in the industry, and also, companies will want to hire them. Companies will see that not many people know how to work on virtual production, but our students are already learning it along with the industry. So this will make them more hireable and more desirable for companies, and they’re going to be able to get out there in a few years from now or less and understand the latest technology in filmmaking.

How did you feel knowing you would be teaching the class?

I was very humbled and excited. There was a little bit of the usual hesitation of, “Am I going to be able to do it” and all the more significant questions that you start asking yourself. It’s a whole new technology that we were reading about; we were seeing how the industry is incorporating it, but I was not trained on it, so it was both exciting and scary at the same time. I think it was more exciting than anything just because I knew that I would be able to figure it out with my skill set. It’s the latest technology in filmmaking, so I was excited to be involved with it and try to at least help figure out how we can best teach this new technology in harmony with other departments. So I think that challenge on its own was exciting, and it’s still very interesting and still a challenge that we today try to figure out the best way to make it work for all departments. We were the first class to ever record anything on that XR stage, so there is a lot of “never try never know,” so let’s go ahead and test it, then we fix it and improve it.

Briefly describe your experience as a professor using the XR stage teaching the class.

It’s an excellent opportunity for professors as well. It will be a good opportunity for students and professors like myself who’re teaching it to also learn. We have to learn in order to teach. Since it’s a technology that got started during the pandemic, we all have to catch up and learn on our own and do a lot of research about the best workflow with the technology.

This quarter is the first quarter ever to be teaching this class. It was an excellent troubleshooting experience for my students and me as a professor. That was the mentality from the beginning. We knew there would be many challenges and many things that we had to solve and a pipeline that we had to figure out, but that makes it even more exciting because we were all learning pretty much at the same time trying to troubleshoot. We’re not waiting until the technology is already in the industry for many years, and then we start teaching it. We’re teaching it and figuring it out as the industry is also figuring it out because even the industry hasn’t fully 100% about it yet. Everybody’s trying to figure it out, so we are right there with them. On another aspect, it invites cooperation from different departments. This quarter, we were able to work together with the film department and the production design department. Within my class, we had different skill sets from various departments, such as VR, Game Design, Visual Effects, and UX design. They all will benefit a lot from being involved with this workflow because it’s the future. It involves more communication than ever between different departments, and everyone has to be in agreement before you actually go in to shoot the project. Filmmakers, directors, producers, production designers, visual effects artists, whoever is involved in that production needs to be on the same page instead of having the mindset of “fix it all in post.”

Are there going to be more classes?

The idea is to have a continuation class. It won’t be quarter by quarter; it will be something that we continue working on each quarter so some students can continue working on what they started. So there’s a lot of conversations that are still happening as we speak. But our goal is to give the best opportunity to our students and train them in the best ways to prepare them for the future.

STUDENT’S POV

Interview with Daniel Rivera Aponte ( Production Design Senior)  and Nico Ramirez (Visual Effects Junior)

How would you define the XR stage class? How was it organized?

Both students emphasized the flexibility of this class’s approach. It was a little less structured than other SCAD classes since it was the first XR stage class. Visual Effects junior, Nico Ramirez, emphasized the dynamic of the curriculum as a very open one. “We had formed groups and came up with a concept of what we wanted to do on the XR Stage; this was all part of the heavy focus on previsualization. We then had to plan and make our immature model in collaboration with the film and production design class. After we created the model, filmed, and went through post-production, we presented it to various department heads to pitch our ideas. Then it had to be filmed with actors on the actual XR stage.”

What inspired you to take this first XR stage class? Why did you want to take this elective?

Nico Ramirez shared with us how all of the students in this class were not expecting to be the first students to work with SCAD’s XR stage. “During the Spring quarter of last year, I decided to take this class expecting to learn Unreal Engine, in preparation for virtual production classes. I didn’t expect to be working with the XR stage as soon as it was built. So to be the first-class working with the XR stage was a very welcomed surprise that came this fall quarter.”

Daniel Rivera mentioned the “560 electives” in the Production Design department, a group of electives where students get to experience how production works on a professional level. The XR stage class was part of one of his 560 electives. In the interview, he shared how learning about the XR stage for the first time in Disney’s The Mandalorian inspired him and ignited his curiosity. Daniel got very excited when he realized that SCAD was incorporating the XR stage within its facilities because it was an excellent opportunity for him and a wonderful learning experience.

What was your job during this collaborative production?

“I mostly worked in Post-production as a compositor,” said the Visual Effects junior. Compositing combines all the visual elements from separate sources into a final image, creating the illusion that the pieces are part of the same scene. Nico Ramirez mentioned that in addition to working with her main concentration, compositing, she also worked with her group in the ARVR class to help come up with the starting concepts. Her group’s idea consisted of a fantastical thematic with the design of a magical forest.

Daniel shared with us in the interview, “Production designers focus on how the set would look and the visual direction that the film is taking. We are responsible for designing sets that match the theme of the film.” Daniel was in charge of the look of the set to be accurate with the environment. The concept of his group was about a snowy, more realistic setting, mixing science and art. The team developed a miniature model of this scene so that during production, they had an idea of what elements were important in the midground, foreground, and background.

What challenges did you face? 
Nico shared with us some of the difficulties she faced during this process “Meeting outside of class with all the other students was definitely a challenge. Time as well, since at the beginning a lot of things were up in the air and not set in stone since it is a brand new class. Also, working in collaboration with the other classes, our end goal with the project was not the same at the beginning so getting on the same page was definitely a part of it.”

One of Daniel’s most significant challenges with the team was replicating the Areola Borealis lighting within the Unreal environment and accurately matching that with the miniature model. “The lighting needed to match the physical environment and the background to sell the idea.” The movement of the actor was another challenge. Figuring out what they wanted the actor to do in the scene was difficult for the team.

How did you feel about collaborating with others? Did you like this experience?

Nicole Ramirez shared with us, “I did enjoy the experience; it was frustrating at times but also incredibly rewarding. I’m very proud of all the hard work our group ended up putting into the project.”

Daniel mentions, “It was an insane opportunity to be the first student working on the XR stage. This was the first time that I academically collaborated with professors and students from different majors within a class. Collaborating was very intimidating and interesting at first. Nobody knew what we were doing. However, it is something that I had to deal with because the whole process is a collaboration” He shared how being in the whole process was amazing. Stress was part of it, but it was a worthwhile experience.

Would you recommend incoming students to take this class? If so, why? 

Nico Ramirez recommends that any students interested in virtual production should take this class. “It definitely was a one-of-a-kind experience. I am so fortunate to have been a part of this class and the first student to have ever worked with a stage like this at a collegiate level.”

Daniel says, “Absolutely. Visual effects, production design, and film, whichever major should definitely take this class. Even if it is just out of interest in learning this new technology despite your major, definitely take this class.” Daniel emphasizes how the XR stage is a new technology that is catching the world by storm already. Knowing how to work with the XR stage makes students more valuable and more interesting in the field since they would already have this experience of working with the stage. Students that take this class are at the forefront of the development of that technology. “You receive a good insight on how and why this technology works the way it works. The fact that you’re there experiencing for the first time is very valuable for personal and artistic growth.”

After taking this class, do you see yourself involved in virtual production projects in the future? 

Nico mentions that she would absolutely love to keep learning and taking classes involving virtual production. “As the program grows, I definitely see myself involved and pursuing it in the future.”

“Of course, 100%” Was Daniel’s enthusiastic answer. He is taking another XR stage elective next quarter. Daniel is interested in doing virtual concerts with the XR stage in the future. Eventually, he would like to learn how to create assets in an Unreal environment and work with them in a production design sense.

If you were to change something about the XR stage class, what would it be? 

Nico Ramirez mentions that as much as she loved being part of the first group to work with the XR stage, she would change the class structure. However, she emphasizes and recognizes that it was not something the students had control over. “We were guinea pigs, and our experience here will only make things better for future classes to come.”

Daniel shares that getting used to the collaborative process was intimidating and challenging. However, he says that overall, he wouldn’t change anything because it was a big learning opportunity to work with two other majors collaboratively.

Final thoughts? 

Nico closed the interview by mentioning how she was very happy to take the XR stage class with professor Galeano. “I couldn’t imagine someone more enthusiastic about the new technology and helping us through this quarter. I would love to take another class with him involving virtual production in the future.”

The VSFX Chair Words regarding the XR Stage

I spoke with SCAD VFX Chair Gray Marshall and asked about his perspective of the XR stage and how real-time rendering will be used in filmmaking as we advance. When I asked this question, Chair Marshall explained how an XR stage is another tool for filmmakers, but it can also be misused with any tool. For example, when I first learned about the volume and its use in the Mandalorian, I thought that this would be the end of a green screen, and we were going to be using this technology moving forward.

While it is likely that we will see more real-time rendering in the near future, Chair Marshall explained why that might not be the case. The XR stage can be misused when used as the shot’s focus rather than just a backdrop. He explained that the stage is practical when the subjects are the focus of the shot instead of the background being the focus. The reason is that if your eyes are drawn to the screen, the more likely the audience will tell that it’s fake. Chair Marshall believes that the technology isn’t quite ready for it to look as natural as possible.

Towards the end of our conversation, Chair Marshall pointed something out about real-time rendering that I didn’t think of, which was how the XR stage saves time in post-work, but then more time will have to be spent in pre-vis. When working on a green screen, everything is built-in post, so pre-production doesn’t take quite as long, but since you’re trying to pull off an in-camera effect, everything will need to be built in pre-production rather than in post. Doing all this work in pre-production does create some issues down the line. He pointed out that the movie doesn’t come together until it’s being edited, and this is when the filmmakers will most likely see what needs to be changed. If it’s shot on a green screen, then the process isn’t quite so hard since the artists can go back and change it, but if it’s done in real-time, there’s a lot more that you’re going to have to change.

I think the quote that summarizes his feelings the best is “virtual production is the use of real-time production… to help filmmakers ultimately; creatives, get more creative options.” I liked how he stressed that virtual production is just another tool for filmmakers rather than replacing the green screen altogether. He made excellent points about where the green screen can be more effective than an XR stage, especially when considering how much money it costs to build an XR stage. I appreciated Chair Marshall’s insight into virtual production as a whole, and the acquisition of an XR stage for SCAD students will undoubtedly help future filmmakers create significant visual effects.