The Tech Challenges to Photorealistic Games

Introduction to photorealism

What is photorealism?

Photorealism simply means that a simulated scene appears indistinguishable from a photograph, or by extension, from real life. The main obstacle to photorealism is processing power, said Jensen.

To make movies like "Avatar" and "Life of Pi" appear photorealistic, each frame of the movie is pre-rendered. That means that the digital models and scripted motion in a virtual scene are compiled into frames (usually up to 30 frames per second) and touched up, a process that can take hours per frame.

But video games don't have that kind of time. Because the action in a scene has to respond dynamically to player input, video games have to render their graphics in real time, usually at a rate of 60 frames per second.

That's why it's good to be skeptical of pre-rendered video game trailers. They inevitably look better than the final game, because they are essentially movies, not samples of how the game looks when you play it.

MORE: 20 Hottest Video Game Hunks

Currently, during actual play, games use a technique called rasterization to achieve somewhat realistic effects. Rasterization is an algorithm by which three-dimensional shapes are output for display on a flat screen.

Developers have had to find roundabout ways to create realistically lighted scenes, because rasterization is not modeled after the way visuals work in the real world.

For example, the best way to do shadows with rasterization in game engines like Unity is to texture an object with a darker filter, which recreates the appearance of shadow. The light sources in the scene aren't dynamically creating the shadow like a spotlight on a stage would; instead, you're drawing in the shadow yourself, like an artist doing a sketch.

The games industry is in the middle of a transition from rasterization to ray tracing, Jensen said. Unlike rasterization — which demands that designers hand-create effects like shadows, textures and shades of color — ray tracing works in a way that closely models the natural world.

Ray tracing means less work for artists and developers, not to mention a more realistic finished product. But it also means a lot more work for the computer: the amount of processing power needed to account for every light ray is enormous.

Jill Scharr is a creative writer and narrative designer in the videogame industry. She's currently Project Lead Writer at the games studio Harebrained Schemes, and has also worked at Bungie. Prior to that she worked as a Staff Writer for Tom's Guide, covering video games, online security, 3D printing and tech innovation among many subjects. 

  • KelvinTy
    Even for pre-rendered cut scenes, the quality totally depends on the developers. Take ME3 for example, they have the ability to render things in much higher resolution, yet, the 8bit background graphics is more like it's from '02 or '03.
    Having the tech and not using it is the biggest problem, not the tech itself.
  • dark_knight33
    Jill said:
    But even SVOGI, if it's true SVOGI, is too demanding for commercial gaming devices; the technique would require a machine of at least 1 teraFLOP capacity. For perspective, that's 40 times more powerful than the Xbox 360.

    The GeForce GTX TITAN:

    Delivers 4.5 Teraflops of single precision and 1.3 Teraflops of double precision processing power.

    Doesn't the Xbox 360 look like the poor cousin?

    Truth: Consoles are what's holding games back.
  • dark_knight33
    Also, TG's comment system sucks.
  • xomm
    Yeah, the statement claiming 1 TFLOP is too much for commercial gaming is kinda dated. The Xbox One, PS4, and most $150+ graphics card have that kind of power.
  • mirrormirror666
    this article is so full of horse shit. ray tracing is not the holy grail of graphics. ray tracing does not offer TRUE PHOTOREALISM. ray tracing is just much better than rasterization but you still have to do plenty of trickery in order to recreate some optical effects ( like caustics ) which only come naturally with path tracing.

    path tracing is a simpler (but yet more demanding algorithm) which also offers global illumination, while ray tracing does not. global illumination is very important in having photorealistic graphics.

    with path tracing you don't have to resort to tricks to implement various effects like you have to do with raytracing.

    and path tracing is not a SHORTCUT like the article mentions. path tracing is a full algorithm which solves the illumination equation in the full, it's also a so called "unbiased" algorithm. Last but not least, path tracing was not invented by that obscure company called "caustic", it's a well known algorithm.

    Don't let the marketing guys trick you, ray tracing is just a jump forward, the final solution is path tracing.