Program 4 2019: texturing and transparency

Due: 11:59pm, Friday November 15

Goal: In this assignment you will learn about rendering textured and transparent models using the WebGL rasterization API.

Submission: Submit your assignment using this Google Form.


BASIC GRADING:
The main components of this programming assignment are:
  • 10% Part 0: properly turned in assignment
  • 40% Part 1: render the input triangles, textured but without lighting
  • 10% Part 2: render using lightmaps and texture
  • 10% Part 3: render using lighting and texture
  • 30% Part 4: render using lighting, texture and transparency
  • Participation: Receive participation credit (outside of this assignment) for posting images of your progress, good or bad, on the class forum!

General:
You will render triangles, described in the same sorts of JSON input files used in the second assignment. We will again test your program using several different input files, so it would be wise to test your program with several such files. The input files describe arrays of triangles using JSON. An example input files reside at https://ncsucgclass.github.io/prog4/triangles.jsonUse this URL in hardcode as the locations of the input triangle file — it will always be there. While testing, you should use a different URL referencing a file that you can manipulate, so that you can test multiple triangle files.

For this assignment, we have made additions to the file format supporting a transparency alpha, texture filenames and texture coordinates. All textures will reside at the same URL as the model input files. For example, if the triangles.json file makes a reference to texture1.jpg, then it will reside at https://ncsucgclass.github.io/prog4/texture1.jpg. When you load the texture from github, some browsers will deem it a cross-origin risk and invalidate ("taint") your canvas. To avoid this problem set the image's crossOrigin attribute to "Anonymous". For more details click hereTexture loads happen asynchronously (on a different thread from your javascript), so your texture may not appear immediately. We recommend that you load a one pixel "dummy" texture locally to avoid runtime errors in the meanwhile.

We will provide a shell in which you can build your code. You will run the shell here, and see its code here. This shell will be is a correct implementation of program 2, which also loads an image (which can ultimately become a texture) as the canvas background. Default viewing parameters are the same as in the first two assignments. We will also provide you with a solution for program 3, as soon as late turnin expires.

You should code the core of this assignment yourself. You may not use others' code to implement texturing, blend texture with lightmaps or lighting, or implement transparency. You may use math, matrix and modeling libraries you find, but you must credit them in comments. You may recommend libraries to one another, speak freely with one another about your code or theirs, but you may never directly provide any code to another student. If you are ever uncertain if what you are contemplating is permissible, simply ask me or the TA.

Part 0: Properly turned in assignment
Remember that 10% of your assignment grade is for correctly submitting your work! For more information about how to correctly submit, see this page on the class website.

Part 1: Render the input triangles, textured but without lighting
Use WebGL to render unlit triangles, giving each fragment its textured color. Do not perform lighting. You will have to use the fragment shader to find the appropriate color in the texture. UV coordinates are provided with triangle sets.

Part 2: Render with texture and lightmaps
Improve your renderer to shade fragments by mixing texture with a second special texture, a lightmap, which describes the light cast by a specific light source. Read in the second lightmap texture, and combine the two textures by multiplying them component by component in the fragment shader: C = CtCm, where Ct is the texture color, and Cm the lightmap color. Show the unlit texture when the u key is pressed, and the lightmapped texture when the m key is pressed. Example lightmaps can be found in here and here.

Part 3: Render with texture and lighting
Improve your renderer to shade fragments by mixing texture with Blinn-Phong lighting. A simple approach is modulation, which multiplies the lit fragment Cb with the the texture value Ct: C = CbCt. Show Blinn-Phong mixed with texture when the b key is pressed, the lightmapped texture when the m key is pressed, and the the unlit texture when the u key is pressed.

Part 4: Render with texture, lighting/lightmaps and transparency
Improve your renderer again by adding transparency (alpha) to its rendering. To avoid transparent objects occluding other objects, you will have to first render opaque objects with z-buffering on, then transparent objects with the z-write component of z-buffering off (gl.depthMask(false)). Include alpha you find in textures, and multiply it with alpha from the triangle to get the final alpha, as you did the RGB components.

EXTRA CREDIT GRADING: 
The extra credit components we suggest for this assignment are:
  • 461: 1% — support ellipses
  • 461: 2% — 561: 1% — improve transparency correctness with partial sorting
  • 461: 5% — 561: 2% — improve transparency correctness further with a BSP tree
Other extra credit is possible with instructor approval. You must note any extra credit in your readme.md file, otherwise you will likely not receive credit for it.

Extra credit: support ellipses
Render ellipses as well as triangles. The new complication here is that you will have to generate uv texture coordinates. We recommend using a latitude longitude mapping.

Extra credit: improve transparency correctness with a partial sort
Now sort your transparent triangles by depth to make transparency more correct. You must sort before rendering. You may have to issue a separate draw calls to ensure gpu parallelism does not undo your ordering.

Extra credit: improve transparency correctness further with a BSP tree
Sort your transparent triangles with a BSP tree to improve transparency further. This will split triangles when they intersect, improving correctness.

Target output for Program 3

Hey folks,

   Here is the output in the form of a GIF that I am getting so far for program 3. Following is the input that I have used:
 https://ncsucgclass.github.io/prog3/triangles2.json 
Note: Don't limit yourself with the sample input files that we have provided. Do test your program on different inputs, for example, input with more triangles, different orientation, or with an object made up of multiple triangles.


Cheers,
Ajinkya

Thursday in 561: NVIDIA's David Luebke

David Luebke, VP Graphics Research at NVIDIA
Thursday's guest speaker in 561 will be David Luebke, VP Graphics Research at NVIDIA, who will speak on the use of AI in computer graphics.

David co-founded NVIDIA Research in 2006, after eight rewarding years on the faculty at the University of Virginia. He runs graphics research at NVIDIA, with his research interests including VR/AR, ray tracing, real-time rendering, GPU computing, and the intersection of machine learning with computer graphics. He was recently elevated to IEEE Fellow "for contributions to GPU computing and computer graphics."

Tuesday: NVIDIA's Josef Spjut

Josef Spjut, research scientist at NVIDIA    @josefspjut
Tuesday's guest speaker will be Josef Spjut of NVIDIA Research, who will speak on ray tracing in high framerate displays.

Josef works to improve human experience through the development of hardware and software to enable new human computer interactions, such as virtual and augmented reality. Lately, his research has begun to focus on esports. Previously, Josef worked on the “RT Core” ray tracing hardware in the Turing GPU architecture. Prior to his work at NVIDIA, Josef was a visiting professor at Harvey Mudd. He earned his PhD in Computing at the University of Utah, and his BS at UC Riverside.

Come to our next NexUX meetup Friday afternoon, and earn extra credit!

Come to the nexUX Meetup!
 
Leah Kaufman, Senior UX Research Manager, Lenovo Global UX eCommerce Team   Leah on linkedin
Leah Kaufman, Senior UX Research Manager, Lenovo Global UX eCommerce Team

Your Digital Wardrobe: Studying Tech Use Over Time— Friday, October 18 3PM @ Design’s Burns Auditorium

Your laptop, Alexa, phone, Nest, Roomba, baby monitor, Smart Doorbell — our devices weave in and out of our daily activities. But what happened to the ones we had five years ago? Ten years ago? Our devices mediate relationships, memories, events, and more; and every time we add, drop or change devices, the way we view ourselves shifts just a bit. During this session we’ll look at the chronology of our digital wardrobes and how to study our relationship with technology.
Please bring your questions to the meetup!
Leah Kaufman has been senior UX research manager at Lenovo since 2015, leading a team of 5 researchers who study and test the usability of Lenovo's websites world-wide. She has been UX since before it was "UX", studying cognitive psychology at U. Washington in the late 80s, working on Microsoft’s usability team through the 90s, and doing UX research in Seattle’s start-up scene and at Getty Images.
As always with our meetups, this is not a "sit back and listen" meeting, but a chance for us to bring together a cross disciplinary group of people to think about the relationship between UX, technology, and design. Mostly though, we'll get to know one another, and have some fun. If you'll come, please RSVP!
NOTE: This year’s meetups will take place at NC State’s College of Design, in its Burns Auditorium. Please see our meetups page for details.
RSVP to this Meetup
Tweet
Share
Share
+1
Forward
Location
This year, our meetups happen at Burns Auditorium in Kamphoefner Hall (gmap) at the College of Design on NCSU’s Main Campus, with a few exceptions.
To get to the meetup by:
  • foot: From Pullen Road, go down the steps on either side of the columns of Brooks Hall to enter the inner courtyard, Kamphoefner Hall is across the courtyard. From Stinson Drive, enter the corner of Kamphoefner Hall. Burns Auditorium is on the first and second floors.
  • bus: Free NCSU Wolfline buses service the Hunt Library on several routes. See the full system map (PDF) or the real-time bus tracking map. Get off on Pullen Road next to Brooks Hall.
  • car: Park in an hourly lot, and pay with credit card ($2/hr). The closest lots, in order, are by the clock tower, the Ferndell lot across Hillsborough, and the across the track under the Coliseum Deck. There is also paid city parking on Hillsborough and Pullen.
Sponsors

Program 3 2019: rasterization, projection, transforms and lighting

Due: 11:59pm, Friday Oct 25

Goal: In this assignment you will practice basic modeling and implement transforms and lighting on 3D objects using the WebGL rasterization API.

Submission: Submit your assignment using this Google Form.


BASIC GRADING:
The main components of this programming assignment are:
  • 10% Part 0: properly turned in assignment
  • 10% Part 1: render the input triangles, without lighting
  • 20% Part 2: light the triangles
  • 20% Part 3: interactively change view
  • 10% Part 4: interactively select a model
  • 10% Part 5: interactively change projection
  • 20% Part 6: interactively transform the triangles
  • Participation: Receive participation credit (outside of this assignment) for posting images of your progress, good or bad, on the class forum!

General:
You will only render triangles in this assignment, described in the same sorts of JSON input files used before. We will test your program using several different input files, so it would be wise to test your program with several such files. The input files describe arrays of triangles and using JSON. Example input files reside at https://ncsucgclass.github.io/prog3/triangles.json. When you turn in your program, you should use these URLs in hardcode as the locations of the input triangles files — they will always be there. While testing, you should use a different URL referencing a file that you can manipulate, so that you can test multiple triangles files. Note that browser security makes loading local files difficult, so we encourage you to access any input files with HTTP GET requests.

We provide a small shell in which you can build your code. You can run the shell here, and see its code and assets here. The shell shows how to draw triangles using WebGL without any model or view transform, and how to parse the input triangles.json file. It is the same as the program 2 shell; we will release the program 2 solution as soon as late deadlines pass.

The default view and light are as in the first assignment. The eye is at (0.5,0.5,-0.5), view up of [0 1 0], look at vector [0 0 1]. Locate the window a distance of 0.5 from the eye, and make it a 1x1 square normal to the look at vector and centered at (0.5,0.5,0), and parallel to the view up vector. With this scheme, you can assume that everything in the world is in view if it is located in a 1x1x1 box with one corner at the origin, and another at (1,1,1).

You should code the core of this assignment yourself. You may not use others' code to transform or project models or perform Blinn-Phong lighting. You may use math, matrix and modeling libraries you find, but you must credit them in comments. You may recommend libraries to one another, speak freely with one another about your code or theirs, but you may never directly provide any code to another student. If you are ever uncertain if what you are contemplating is permissible, simply ask me or the TA.

Part 0: Properly turned in assignment
Remember that 10% of your assignment grade is for correctly submitting your work! For more information about how to correctly submit, see this page on the class website.

Part 1: Render the input triangles, without lighting
Use rasterization to render unlit triangles, giving each triangle its unmodified diffuse color (e.g, if the diffuse color of the triangle is (1,0,0), every pixel in it should be red). You will have to use vertex shaders to perform viewing and perspective transforms, and fragment shaders to select the diffuse color. We recommend the use of the glMatrix library for creating these transforms.

Part 2: Light the triangles
Shade the triangles using per-fragment shading and the Blinn-Phong illumination model, using the reflectivity coefficients you find in the input files. Use triangle normals during lighting. Your fragment shaders will perform the lighting calculation.

Part 3: interactively change view
Use the following key to action table to enable the user to change the view:
  • a and d — translate view left and right along view X
  • w and — translate view forward and backward along view Z
  • and e — translate view up and down along view Y
  • A and D — rotate view left and right around view Y (yaw)
  • W and — rotate view forward and backward around view X (pitch)
To implement these changes you will need to change the eye, lookAt and lookUp vectors used to form your viewing transform.

Part 4: Interactively select a model
Use the following key to action table to interactively select a certain model:
    • left and right — select and highlight the next/previous triangle set (previous off)
    • space — deselect and turn off highlight
    A triangle set is one entry in the input triangle array. To highlight, uniformly scale the selection by 20% (multiply x y and z by 1.2). To turn highlighting off, remove this scaling. You will have to associate a transform matrix with each triangle to maintain state, and apply this transform in your vertex shaders. glMatrix will also be helpful here.

    Part 5: Interactively change projection of a model
    Use the following key to action table to interactively change projection on the selected model:
    • — use perspective projection (the default)
    • — use parallel projection
    When toggling between perspective and parallel projection, apply this change to globally to all models. All other changes should apply only to the selected model.

    Part 6: Interactively transform models
    Use the following key to action table to interactively transform the selected model:
    • k and ; — translate selection left and right along view X
    • o and — translate selection forward and backward along view Z
    • and p — translate selection up and down along view Y
    • K and : — rotate selection left and right around view Y (yaw)
    • O and — rotate selection forward and backward around view X (pitch)
    • and P — rotate selection clockwise and counterclockwise around view Z (roll)
    Translate the model after you rotate it (so the model rotates around itself), and after the highlighting scale (see above, so the model doesn't translate as it scales).


    EXTRA CREDIT GRADING: 
    The extra credit components we suggest for this assignment are below:
    • 461: 1% — arbitrarily sized viewports
    • 461: 1% — off-axis and rectangular projections
    • 461: 1% — multiple lights at arbitrary locations
    • 461: 3% — 561: 1% — smooth shading with vertex normals
    • 461: 4% — 561: 2% — render ellipsoids
    Students in 561 should not perform components that will not earn extra credit. Other components are possible with instructor approval. You must note any extra credit in your readme.md file, otherwise you will likely not receive credit for it.

    Extra credit: Arbitrarily sized viewports 
    Accept a new square canvas (viewport) width/height through your UI. Size your canvas to match.

    Extra credit: Support off-axis and rectangular projections 
    Accept new window parameters in viewing coordinates through your UI (left, right, top, bottom). Adjust your projection matrix to this new window, and render the scene.

    Extra credit: Multiple and arbitrarily located lights
    Read in an additional lights.json file that contains an array of objects describing light location and color. Note that these lights will have distinct ambient, diffuse and specular colors. Render the scene with all of these lights. You can find an example lights.json file here. Assume that the input lights file will always reside at this URL when you turn in your code.

    Extra credit: Smooth shading with vertex normals 
    Using only triangle normals, your shapes will look disappointingly faceted. To represent curvature more accurately, you need vertex normals. When you read in triangles, check for vertex normals in the input file. As you apply the composited modeling, viewing and projection matrices to your vertices, apply the inverse transpose of the modeling transform to your vertex normals. During lighting, use these normals rather than the face normal. The rasterizer will interpolate them for you.

    Extra credit: Render ellipsoids
    Render ellipsoids as described in input. You can find an example ellipsoids.json file here. There are no ellipsoid primitives available in WebGL, so you will have to build an ellipsoid out of triangles, then transform it to the right location and size. You can do this statically with a hardcoded sphere model, or procedurally with a latitude/longitude parameterization. Again you will have to use vertex shaders to perform viewing and perspective transforms, fragment shaders to select color. The ellipsoids should be shaded like triangles, and should use vertex normals if you are claiming that extra credit.

    Program 2 2019: intro to webgl

    Due: 11:59pm, Friday Oct 4

    Goal: In this assignment you will focus on gaining basic understanding of the WebGL rasterization API, by learning how to render and manage a few triangles.

    Submission: Submit your assignment using this Google Form.


    GRADING:
    This assignment is a forgiving introduction to WebGL, with correspondingly forgiving grading. The main components of this programming assignment are:
    • 25% Part 1: attempt improvement of vertex and index arrays
    • 25% Part 2: render input triangles correctly
    • 25% Part 3: attempt improvement of shader to handle parameters
    • 25% Part 4: triangles in input color
    • Participation: Receive participation credit (outside of this assignment) for posting images of your progress, good or bad, on the class forum!

    General:
    You will only render triangles in this assignment, described in the same sorts of JSON input files used in the first. We will test your program using several different input files, so it would be wise to test your program with several such files. The input files describe arrays of triangles using JSON. An example input file resides at https://ncsucgclass.github.io/prog2/triangles.json. When you turn in your program, you should use this URL in hardcode as the location of the input triangles file — it will always be there. While testing, you should use a different URL referencing a file that you can manipulate, so that you can test multiple triangles files. Note that browser security makes loading local files difficult, so we encourage you to access any input files with HTTP GET requests.

    We provide a small shell in which you can build your code. You can run the shell here, and see its code and assets here. The shell shows how to draw triangles using WebGL, treating them all the same way. It also shows how to parse the input triangles.json file.

    We are using webGL's default view setup, with the eye at the origin, and the window from -1 to 1 horizontally and vertically.

    We encourage you to help one another with this assignment, and will reserve a class for you to do so. Feel free to suggest how other students might solve problems, and to help them debug their code — just don't write their code for them. The code you turn in should still be your own (except for the shell). This is a simple assignment, and should not need other third party libraries. As always, if you are ever uncertain if what you are contemplating is permissible, simply ask me or the TA. For information about how to correctly submit, see this page on the class website.

    Parts 1 & 2: Attempt to improve vertex and index array, and render triangles correctly
    For these parts of the assignment, your goal is to display the input triangles in the correct position. The shell only renders the first set of triangles, in solid white. You must change the code to display all the triangles, improving two arrays: one that describes the vertices and their attributes, and another that describes the triangles by listing their vertex indices. If you make a solid attempt at improving these arrays, you will receive full credit for Part 1. If you correctly render all the triangles in white, you will earn full credit for Part 2.

    Parts 3 & 4: Attempt to improve the shader, and render the input colors correctly
    For these parts of the assignment, your goal is to color the input triangles with their diffuse color. The shell includes basic shader code that renders everything in white. You must change the shader to accept and use a parameter, and the code to send the shader a different value for the parameter for each triangle set. If you make a solid attempt at this, you will receive full credit for Part 3. If you correctly render each triangle set with its unique color, you will earn full credit for Part 4.