An introduction to path tracing was presented by me at university in a class on mathematical concepts of computer graphics. I decided to paste this presentation into the blog as well.
Introduction
The topic of the presentation was originally intended to be strictly ray tracing or reversed ray tracing, but since it has already been repeated several times even in today’s classes, in a word of introduction we will present some concepts that hook on ray tracing, but in the rest of the lecture we will focus on a technique called path tracing.
Here we see the words of David Kirk – a talented computer scientist and former chief scientist and vice president of architecture at a company that is one of the main players (if not the most important) in the graphics card market – NVIDIA.
He used to say that ray tracing is the technology of the future and will always remain so. We are at the beginning of 2023 and a few months ago there was the release of a new generation of NVIDIA graphics cards that support for hardware-accelerated Ray Tracing technology.
We will now start with a small introduction to terminology.
Rays
To begin with, we will define what a ray is and how we can define it. We will give it attributes such as origin which is the origin of the ray, which will be some point in the coordinate system. In our presentation we are assuming a three-dimensional space and we will define the point with scalars X Y Z. In addition, for the ray to make sense it must have some direction in which it goes. And this will simply be a three-dimensional vector.
Ray Casting
Next, we will define Ray Casting which is an attempt to project/shoot a ray from its origin in a defined direction. The purpose of Ray Casting is to see if any potential object will be hit by this ray. Here it can be mentioned that it is not actually a rendering algorithm but rather a tool that can be used in any way, depending on the project. For example, to check physical collisions in a computer game, a shooter, in which we check whether a fired bullet will hit some object.
Here, of course, we will use them for rendering. So, to summarize once again – ray casting is about “shooting” a ray and seeing if anything gets in its way.
Ray casting can also be used between two points and in this setting we can check if an object is between them. This is a good way to check if a point and also the space is in the shadow of an object. That is, we assume one point as our origin – the beginning of the ray set towards the other point and in this way we can determine if something is obscuring the other point. If so, it is in its shadow.
Backwards Ray Tracing
So much for the introduction, we have learned the fundamentals of rays and ray casting. Now let’s move on to generating some basic image, the actual and practical use of ray casting. Basically, every rendered image has a camera – an eye object – from which the user / player sees the image. So on the left we have the camera. Then we have the image – Image – or screen, a two-dimensional space that describes the three-dimensional world. Going further, each small square is a pixel, that’s how it should be correctly interpreted. This pixel contains RGB data, or Red Green Blue, in turn. So ray tracing technology tries to answer the question of what the ray passing through a given pixel hits. In other words, we iterate over all the pixels in the image and project a selected number of rays through that pixel collecting samples. The ray enters the environment of the scene (in our case, three-dimensional), meets objects on its way, or not, of course. And the object that is closest to it will be the first to be hit, and thus it is this object that will be displayed on the screen.
Then we can project the rays between the light source and the nearest seen object in the scene/environment. By doing so, we will check if anything is standing in the way of the light so that it reaches the selected spot and illuminates it with its full power. If anything is in the way of the light to the View Ray intersection point, it is a signal to us that this point is in shadow. Otherwise, it is illuminated. This is the Shadow Ray shown in the picture.
What is presented here is basically the use of the ray and ray casting ideas we mentioned earlier. We define for ourselves rays that enter the world and look for collisions. Then we check if there is any object between the two points (the View Ray intersection point and the light source).
Whitted’s Ray Tracing
What we have described is an algorithm whose genesis begins in 1980. This is when the first ray tracing algorithms were being developed. An example is the whitepaper written by Turner Whitted “An Improved Illumination Model for Shaded Display”. This whitepaper describes quite a few important algorithms that are basically still used today, such as anti-aliasing and bounding volume hierarchies (BVH).
In short, it was the beginning of the idea – how we are able to render reflections, refraction and shadows. And whether we are able to render it recursively at all.
Based on what was said earlier, we can deduce what ray tracing is. It is an algorithm. where on each pixel of the screen we project a ray into the environment. Then from the intersection point we project another ray in the direction of each existing light source in the scene so as to find out where the shadows are. And finally, we project rays from each material from the reflection/refraction values to extract the color for that material.
Kajiya’s Path Tracing
We, however, would like to focus more on path tracing technology. So moving on, in 1986 there was a big theoretical leap in terms of ray tracing algorithms. James T. Kajiya described it in his white paper The Rendering Equation. What if we say Sky is the Limit. We will re-project the rays from the camera/eye point and look for every ray that hits something. That is, the intersection point will be known. And what in case we do not know which way the reflection goes. Yes, in the case of a mirror it is clear, however, we are talking about, say, ceramics. In this case we totally have no idea from which point, from which side the light is coming. In fact, we know, as in real life, light comes from all possible directions. So we project rays from this point of intersection in many directions. What specifically am I talking about? Let’s see it.
Path Tracing
From the point of the eye / camera through one selected pixel, at one selected point of it, we let one ray into the environment. This ray hits this box. For the reason that we hit the box, we learned the intersection point of the ray with the game world.
And then from this point of intersection we project another ray, which can go in any direction. Here, for the sake of presentation, it goes back towards the sky and gets the color green. That is why it has been drawn here in green.
Later we project another ray just from this point of intersection, again in any direction, and this ray acquires color information. We see that it bounces twice and then flies somewhere further away, for the sake of presentation this level of recursion should allow you to understand the issue.
So we continue to project the rays, tracing their paths, thereby gaining more and more information about the color. It’s worth mentioning here that this doesn’t necessarily involve Shadow Ray, that is, those rays from the intersection point we are projecting directly into the light. No, Here we care about gaining information from where the light is most intense, because the light itself comes from all directions.
So we have paths, rays get information about the ground, information about the sky, about other objects nearby, etc. So quite a lot of extracted colors from the game environment. How does it look in terms of samples? Let’s move on.
Samples of our pixel are collected and in such a graphic / pictorial representation are as follows, seen here on the left.
With this solution, we get anti-aliasing basically completely for free which is a significant benefit. Such sampling allows us to get all the colors that go into the selected pixel, in one lecture they put it as “the whole pixel box.” In the standard approach, we would project rays only from the center of that pixel. Now all we have to do is sum these samples and we basically get the finished color for the selected pixel.
Here, of course, we decided to present only a few rays, so as to show the general outline of the algorithm. Taking the film industry under a magnifying glass, let’s say, we can learn that there, for each pixel, filmmakers project 1000 or even 3000 rays. That’s an overwhelming amount of rays and paths to calculate, with current hardware impossible to calculate in a 60-frame-per-second standard.
So path tracing is basically a simple invention, just project the rays and get the colors that describe the pixel.
The Rendering Equation
We have explained what path tracing is. Now let’s try to understand what equation describes this phenomenon. It represents exactly how light enters our eyes. It looks complicated however it is one of the greatest equations that computer graphics has seen. If we have any graphics fanatics here, this is an equation we can tattoo or print over our bed.
So we move on to understanding the equation. So let’s start with what the equation needs as input. This is the point in the scene X that the camera/eye is currently looking at. There is also an outgoing direction, or omega o. This is basically the direction to the camera / eye. In addition to that, we also have an incoming direction. This is located on the very right side as omega i. This variable describes the light coming from other directions to this point X. And next to omega i, we have the normal vector (that is, the vector perpendicular to the plane. That is, when we simply have a flat plane, the normal vector will be directly oriented upward, Well, and we also have the expression S squared. S squared tells us that we will be evaluating this integral along all directions from which the light comes.
As for the expressions in turn found in this equation:
Outgoing light on the very left, answers the question of what light we see / observe, with a given point and initial direction. In other words, we look in the selected direction at a point, what light we observe from that point.
Moving on, we have emitted light. This function basically relies on the same thing as outgoing light, except that this function is supposed to answer what light is coming from that exact point. So if there is a light source at that point that basically shines directly into our eye, then we won’t see any other color, it will just be light coming from that point only. Then we don’t have to solve the rest of this equation.
In the integral, on the other hand, we see three expressions, they are incoming light, material and lambert, respectively. So incoming light corresponds to what light a given point sees from the direction of omega i. This is already a different direction from that toward the eye, which is worth noting, it is omega i, not omega o.
A material function with data at the input of omega i, the direction of incoming light, and omega o, the direction of outgoing light, answers the question of which incoming light is reflected in the direction of outgoing light. So in the case of a mirror, the incoming and outgoing directions will be very much related. In contrast, other materials may already behave a bit differently.
Well, and finally we have the last expression, lambert. In short, it’s the product of the incoming light and the normal vector. Let’s move on to show it more easily.
Lambert’s Cosine Law
Lambert’s cosine law is already a pretty old idea, which even intuitively is true. If the light is directly over a surface, it will have the greatest effect on the color of that surface. But if the light source is moved in a different direction, that is, the light falling on the surface is coming from the side, it will have much less effect on the color of that surface. So looking at these images from left to right. we can see how light spreads across a surface when we change the direction of its origin.
Materials
In addition, a few words about the material. In the case of path tracing, when it comes to mirrors we have a direct reflection of our projected ray. So in the case of a mirror, all we need is a single ray that is perfectly reflected.
In the case of glossy material, we will need to trace the paths of a series of rays and we can choose any directions. Then adding them together should return a satisfactory result.
And finally, we have the aforementioned ceramics, a material that diffuses light, where we just mostly apply the previously mentioned Lambert’s law.
A quick example of each material taken from real life, namely Christmas baubles, a Styrofoam ball and a wooden ball.
Summary
So basically we have explained the most important equation behind the idea of path tracing. We add up all the light coming from all directions and this light is transmitted to the camera/eye as shown in the visualization.
Finally, we’d like to show a comparison of rendered scenes with different number of samples with just path tracing algorithms. On the left is an image rendered with 5 samples per pixel, while on the left 500 samples have already been used, thus the image is much clearer, with a trace amount of noise. Such minor noise can be seen precisely in games, where we are more interested in maintaining a stable rendering of 60 frames per second in order to preserve the fluidity of the game. In the film industry, on the other hand, they strive to generate photorealistic images, thus they can afford to use far more samples.
And the last comparison, where we have are images rendered with path tracing, ray tracing and standard rasterization algorithms. We can see that path tracing most closely represents the real world, fantastically representing reflections. Photorealism in its highest glory.
Bibliography
Turner Whitted, An Improved Illumination Model for Shaded Display, 1980
NVIDIA, OptiX: A General Purpose Ray Tracing Engine
James T. Kajiya, The Rendering Equation, 1986
NVIDIA, Ray Tracing Essentials, 2021
Rendering: Path Tracing Basics lecture by Bernhard Kerbl in TU Wien, Austria
Barbara Putz, GRAFIKA KOMPUTEROWA 3D, OKNO Politechnika Warszawska 2018
https://blogs.nvidia.com/blog/2022/03/23/what-is-path-tracing/
