VR Basics

Quality of a VR experience

• Immersion - result of emulating sensory input received by the body - visual, auditory, motor, etc. - the science of VR
• Presence - visceral feeling - a deep emotional or intuitive feeling - the art of VR
• Overall Goal: to increase presence and immersion to the point that you forget you are in a virtual world

Why is VR possible and affordable?

• 3D Viewing (i.e. displays)
• Head-pose tracking (i.e. sensors)
• The VR system knows where your head is at any time, so it can render the 3D scene for your viewpoint stereoscopically. Whenever you move, the scene changes to reflect your new perspective.

Stereoscopic 3D Viewing

• Separate views for the left and right eye.
• Views are slightly offset to create parallax which tricks the eyes into thinking that what they are seeing is actually a 3D view.
• Defn: the effect whereby the position or direction of an object appears to differ when viewed from different positions, e.g. through the viewfinder and the lens of a camera.
• The first thing that a VR camera has to do, then, is to render these side-by-side views

Lots of vocab

• Pincushion effect - image distortion that occurs when you look at a VR screen. The image is wide and tall b/c the headset lens is actually a very wide-angle lens = wide field of view
• Barrel distortion - the inverse of the pincushion effect - this image looks right to us when we view it through the headset lenses = occualr distortion correction - results in a field of view that includes a lot more than what you would normally see in your peripheral vision. Oculus Rift has about a 100 degree FOV
• Inter Pupillary Distance (IPD) - distance between your eyes - used to calculate the parallax and can differ from person to person

Folie 5

• Looking carefully at a VR screen, you might see color separations
• When a light passes through a lens, it gets refracted at different angles based on the light's wavelength
• Rendering software inverts the color separation so that it looks right to us - this is called chromatic abberration correction - makes the image look sharp

Display considerations

• Screen resolution is also important to avoid the screen-door effect - i.e. seeing pixels
• Need to consider pixels per inch (ppi) when considering a head mounted display versus worrying about pixel width and height of the display
• Pixel smearing and foveated rendering (i.e. showing higher-resolution details exactly where the eye is looking) help reduce screen door effect

Frames per second

• Frames per second (fps) is an important consideration when looking at a 3D scene in VR
• if the fps is too low, animation is choppy
• Things that affect fps:
• - GPU performance
• - complexity of the scene (# of polygons and lighting calculations)
• "This is compounded in VR b.c. you need to draw the scene once for each eye."
• New GPUs that are optimized for VR will improve frame rates.

Sound

• Biurnal audio gives each ear its own stereo view of a sound source so that the brain can imagine its location in 3D space. Regular headphones work.
• https:/www.youtube.com/watch?v=IUDT1vagjJA

Head Tracking

• Headset has a motion sensor (IMU - internal motion unit) that senses spatial acceleration and rotation rates on all 3 axes (x, y, z) providing six degrees of freedom
• Motion Detection - when you move your head, your current viewpoint is calculated and used when the next frame's image is drawn.

Oculus Rift Tracking

• Rift uses inside-out positional tracking which means that an array, or group, of invisible infrared LED's are read by an external optical sensor (infrared camera) to determine the viewer's position
• The camera has to be able to see you in order to work.

Vive Tracking

• outside-in positional tracking
• 2 or more dumb lasers placed in a room and an optical sensor on the headset determines the viewer's position

Windows MR headset tracking

• No external sensors or cameras
• integrated cameras do the spatial mapping of the environment around the viewer which locates and tracks viewer's position in space

Summary - what does it mean?

• Basically, the position, tilt and forward direction of the viewer's head (i.e. head pose) are used by the graphics software to redraw the scene from a particular perspective or vantage point.