Vertex_shader_rounding_tests

Draws a light grey 400x200 background, then a 4x2 grid of adjacent 100x100 green quads, each offset by the bias in x and y, through a passthrough vertex shader. Grey seams between neighboring quads show inconsistent rounding of shared edges. This case uses no offset.

As AdjacentGeometry_0.0000 with a bias of 0.0010 (a tiny offset).

As AdjacentGeometry_0.0000 with a bias of 0.4999 (just under half a pixel).

As AdjacentGeometry_0.0000 with a bias of 0.5000 (exactly half a pixel).

As AdjacentGeometry_0.0000 with a bias of 0.5624 (just under the 9/16 subpixel boundary).

As AdjacentGeometry_0.0000 with a bias of 0.5625 (exactly 9/16 of a pixel, a 1/16 subpixel boundary).

As AdjacentGeometry_0.0000 with a bias of 0.5626 (just over the 9/16 subpixel boundary).

As AdjacentGeometry_0.0000 with a bias of 0.9990 (just under one pixel).

As AdjacentGeometry_0.0000 with a bias of 1.0000 (exactly one pixel).

As Compositing_-4, but the final full screen quad is drawn at z = -2.

Fills a framebuffer sized texture with a centered 256x256 checkerboard, points the color surface at that texture and draws the left half of the texture onto itself four times through a passthrough vertex shader, with the combiner outputting the texture's color and half its alpha. The texture is then drawn to the screen with a fixed function full screen quad at z = -4. Any offset or blurring in the left half shows texel misalignment that accumulates over the repeated self compositing.

As Compositing_-4, but the final full screen quad is drawn at z = 2.

Draws a 240x240 magenta quad centered on the screen, then a green quad over it whose top and left edges are offset by the bias (the right and bottom edges are unchanged), both through a passthrough vertex shader. Any magenta left visible along the top or left edge shows how the offset edge was snapped to the 1/16 subpixel grid and rasterized. This case uses no offset.

As Geometry_0.0000 with a bias of 0.0010 (a tiny offset).

As Geometry_0.0000 with a bias of 0.4999 (just under half a pixel).

As Geometry_0.0000 with a bias of 0.5000 (exactly half a pixel).

As Geometry_0.0000 with a bias of 0.5624 (just under the 9/16 subpixel boundary).

As Geometry_0.0000 with a bias of 0.5625 (exactly 9/16 of a pixel, a 1/16 subpixel boundary).

As Geometry_0.0000 with a bias of 0.5626 (just over the 9/16 subpixel boundary).

As Geometry_0.0000 with a bias of 0.9990 (just under one pixel).

As Geometry_0.0000 with a bias of 1.0000 (exactly one pixel).

For each of four sizes (128x128, 128x32, 32x128 and 16x64), points the color surface at texture memory sized exactly to the quad, fills it with magenta on the CPU and draws a green quad covering the whole surface offset by the bias in x and y through a passthrough vertex shader. Each result is then drawn to the screen as a texture in a 2x2 grid. Magenta shows texels the offset quad did not cover. This case uses no offset.

As GeometrySubscreen_0.0000 with a bias of 0.0010 (a tiny offset).

As GeometrySubscreen_0.0000 with a bias of 0.4999 (just under half a pixel).

As GeometrySubscreen_0.0000 with a bias of 0.5000 (exactly half a pixel).

As GeometrySubscreen_0.0000 with a bias of 0.5624 (just under the 9/16 subpixel boundary).

As GeometrySubscreen_0.0000 with a bias of 0.5625 (exactly 9/16 of a pixel, a 1/16 subpixel boundary).

As GeometrySubscreen_0.0000 with a bias of 0.5626 (just over the 9/16 subpixel boundary).

As GeometrySubscreen_0.0000 with a bias of 0.9990 (just under one pixel).

As GeometrySubscreen_0.0000 with a bias of 1.0000 (exactly one pixel).

As GeometrySubscreen, but the color surface is always 1024x1024 and only the first width * height texels of it (contiguously, not as a rectangle) are filled with magenta before drawing the offset green quad of each size. The top left width x height region of the surface is drawn to the screen for each size. This case uses no offset.

As GeometrySuperscreen_0.0000 with a bias of 0.0010 (a tiny offset).

As GeometrySuperscreen_0.0000 with a bias of 0.4999 (just under half a pixel).

As GeometrySuperscreen_0.0000 with a bias of 0.5000 (exactly half a pixel).

As GeometrySuperscreen_0.0000 with a bias of 0.5624 (just under the 9/16 subpixel boundary).

As GeometrySuperscreen_0.0000 with a bias of 0.5625 (exactly 9/16 of a pixel, a 1/16 subpixel boundary).

As GeometrySuperscreen_0.0000 with a bias of 0.5626 (just over the 9/16 subpixel boundary).

As GeometrySuperscreen_0.0000 with a bias of 0.9990 (just under one pixel).

As GeometrySuperscreen_0.0000 with a bias of 1.0000 (exactly one pixel).

As OpRounding_MUL, but measures the rounding of ADD. Each case uses operands a and b whose exact sum lies 0.25 or 0.75 ulp beyond a representable float (for positive and negative sums), or exactly 0.5 ulp above an odd one so that round-to-nearest-even rounds up ("Tie"), with c the IEEE round-to-nearest-even sum. The shader computes d = ADD(a, b) - c. IEEE round to nearest would give d = 0 for every case. Hardware truncates the sum toward zero, giving d = 0, -1, 0, 1 and -1 ulp.

As OpRounding_ADD, but with operands that probe the alignment window of the nv2a adder. Hardware aligns the smaller operand to the larger one and truncates it to a multiple of 2^-28 relative to the larger operand's leading bit (24 bits of mantissa plus 5 more, with no sticky bit) before summing exactly and truncating the sum toward zero. The cases compute 1 - 2^-28 ("Kept", inside the window), 1 - 2^-29 ("Lost", outside it) and 1 - (2^-24 + 2^-29) ("Strad", whose 2^-29 bit is outside it). Hardware gives depth - offset = -16, 0 and 0 (d = -0.5, 0 and 0 ulp). Round to nearest gives 0, 0 and 0; truncating the exact sum gives -16, -16 and -16.

As OpRounding_MUL, but d is computed by a single DP4 of (a, c, 0, 0) and (b, -1, 0, 2^23). On hardware the product is truncated before it is summed, giving the same results as OpRounding_MUL.

Computes d = DP4((a, b, t2, 0), (1, 1, 1, 0)) - c with c, the IEEE round-to-nearest-even value of a + b + t2, subtracted by a separate ADD so that it does not take part in the DP4's accumulation. Hardware truncates each product, aligns all of them to the largest one in the same 2^-28 window as ADD (see OpRounding_ADDWindow), sums the aligned terms exactly and truncates the result toward zero. "+0.75" and "-0.75" sum to 0.75 ulp beyond +1 and -1, "Kept" is 1 + 2^-28 - 1 (scaled by 2^32), "Strad" is 1 - (2^-24 + 2^-29) and "Order" is 2^32 - 2^32 + 2^-32 (scaled by 2^36), whose small term is outside the window anchored to the largest term even though the large terms cancel first. Hardware gives depth - offset = -32, 32, 0, 0 and -16 (d = -1, 1, 0 and 0 ulp, and d = -c for "Order"). Round to nearest gives 0, 0, 0, 0 and 0; truncating the exact sum gives -32, 32, 0, -16 and 0; truncating every partial sum in order gives -32, 32, -16, -16 and 0.

As OpRounding_ADD, but d is computed by a single DP4 of (a, b, c, 0) and (1, 1, -1, 0), so all products are exact and only the accumulation can round. Summing left to right or pairwise adds a + b first, which would give the OpRounding_ADD results if intermediate sums were rounded. Hardware gives the exact distance between a + b and c, d = 0.25, -0.25, -0.25, 0.25 and -0.5 ulp, so DP4 does not round intermediate sums. The accumulation is only exact within the adder's alignment window (see OpRounding_DP4Sum3).

As OpRounding_DP4SumAB, but with (a, c, b, 0) and (1, -1, 1, 0). Left to right or pairwise summation computes a - c first, which is exact, and then adds b exactly, so d is 0.25, -0.25, -0.25, 0.25 and -0.5 ulp regardless of how additions round; any other result would mean the accumulation adds b before subtracting c. Hardware gives these values.

As OpRounding_MUL, but d is computed by a single MAD (a * b - c). On hardware the product is truncated before the addition (the operation is not fused), giving the same results as OpRounding_MUL.

As OpRounding_ADD, but d is computed by a single MAD of a * 1 + b, so the product is exact and only MAD's addition can round. Hardware gives the same results as OpRounding_ADD: MAD's addition truncates toward zero like ADD.

As OpRounding_MADExactProduct, with the OpRounding_ADDWindow operands. MAD's addition uses the same alignment window as ADD, so hardware gives the OpRounding_ADDWindow results: depth - offset = -16, 0 and 0.

As OpRounding_MADExactProduct, but with a multiplier m so that both the product and the sum of MAD(a, m, b) are inexact: 0x3F800800 * 0x3F800C00 + 2^-24 ("+both") and its negation ("-both"), with c the IEEE round-to-nearest-even value of the exact a * m + b. Hardware truncates the product and then the sum, giving depth - offset = -32 and 32 (d = -1 and 1 ulp). A fused MAD rounded to nearest gives 0 and 0, and rounding both steps to nearest gives 32 and -32.

Measures the rounding behavior of the MUL vertex shader instruction. Each case uses operands a and b whose exact product lies 0.25 or 0.75 ulp above a representable float (for positive and negative products), or exactly 0.5 ulp above one (the -1.0014837 * -40 product from Rugby 2005, xemu-project/xemu#2388), along with c, the IEEE round-to-nearest-even value of the product, all passed as shader constants. The shader computes d = MUL(a, b) - c (the subtraction is exact) and writes z = d * 2^28 + 2^23 to a Z24S8 surface, which is read back on the CPU. The screen lists, per case, the depth minus 2^23 and d in ulps of c. IEEE round to nearest would give d = 0 for every case. Hardware truncates the product toward zero, giving d = 0, -1, 0, 1 and -1 ulp. Rows whose depth differs from the hardware result are drawn over a red rectangle.

As AdjacentGeometry, but positions are unprojected through a D3D style perspective projection with z varying from the top to the bottom of each quad. Quads alternate in a checkerboard between the programmable pipeline (dark green, 0xFF009900) and the fixed function pipeline (light green, 0xFF00BB00), so seams between them show differences in how the two pipelines round projected vertices. This case uses no offset.

As ProjAdjacentGeometry_0.0000 with a bias of 0.0010 (a tiny offset).

As ProjAdjacentGeometry_0.0000 with a bias of 0.4999 (just under half a pixel).

As ProjAdjacentGeometry_0.0000 with a bias of 0.5000 (exactly half a pixel).

As ProjAdjacentGeometry_0.0000 with a bias of 0.5624 (just under the 9/16 subpixel boundary).

As ProjAdjacentGeometry_0.0000 with a bias of 0.5625 (exactly 9/16 of a pixel, a 1/16 subpixel boundary).

As ProjAdjacentGeometry_0.0000 with a bias of 0.5626 (just over the 9/16 subpixel boundary).

As ProjAdjacentGeometry_0.0000 with a bias of 0.9990 (just under one pixel).

As ProjAdjacentGeometry_0.0000 with a bias of 1.0000 (exactly one pixel).

Points the color surface at a 128x128 render target that is prefilled with translucent magenta, sets the viewport offset to (320.53125, 240.53125) and the viewport scale to (320, -240, 16777215), and draws a quad from (-1, 1) to (1, -1) with a manually loaded vertex program ending in the XDK screen space epilogue (MUL by c[58], RCC of w, MAD with c[59]). The render target is then drawn to the screen with the fixed function pipeline. Magenta shows texels the quad did not cover.

Tests the rasterizer's top-left fill rule and 1/16 subpixel snapping through a passthrough vertex shader. Draws thin vertical quads and triangles whose left edges sit at x + 0.5 (or the next float below it) and step by 1/16 pixel, a 0.2 pixel wide quad that should be invisible without multisampling, quads stacked to share edges at x + 0.5625, black to white horizontal gradient quads whose expected interpolated colors are 0x808080, 0x808080 and 0x000000 from top to bottom, and the same edge patterns rotated to horizontal on the right side of the screen.

As TopLeftRaster, but the geometry is drawn through the fixed function pipeline with identity model view and projection matrices and a zero viewport scale and offset.