REX3: make IRIX OpenGL blending work on Newport (BLENDALPHA, host alpha, A_LINE alpha)

[atomchild411]

IRIX's OpenGL and IRIS GL never blended on the emulated Newport: every GL_SRC_ALPHA/GL_ONE_MINUS_SRC_ALPHA draw came out opaque. This fixes that and two bugs it had been hiding.

1. BLENDALPHA only changes how alpha itself is blended (6ed5999)

DRAWMODE1 bit 27 was applied to all four channels, so with it clear BF_SA was 1.0 for red, green and blue too and nothing was ever attenuated. rex3.pdf §3.8 says the bit decides how the alpha component is blended ("alpha component can be blended in two different ways…"), and the pin table names it "Blend source alpha with alpha". IRIX's GL draws SA/MSA with BLENDALPHA=0, so half-transparent quads, antialiased text and transparent texels all wrote at full strength.

This is also what rules/rex3/blendalpha-and-alpha-blending.md was looking at in blast: the brown/pink haze filling the nebula billboard's whole quad. With the fix the surround is transparent and only the nebula shows. The note gets a correction header; the rest is left as written.

2. ALPHAHOST with 8-bit host fields: the field is the alpha (f6dc11c)

IRIX draws antialiased points (pntsmooth, GL_POINT_SMOOTH) as one ILINE GO per pixel with ALPHAHOST, HOSTDEPTH 0 and the CPU-computed coverage in the top byte of a HOSTRW0 word (0xdd000000…), colour from the DDA (§3.9: "the HOSTRW1,0 alpha fields are to be used to blend the DDA R,G,B"). The field was unpacked as a 4-bit colour, so the alpha was always 0. Before fix 1 that only showed as hard white blobs; after it, blast's stars vanished. Now they draw as soft specks, as on the emulated XZ. 32-bit fields are unchanged; 16-bit fields (host depth 12) are left alone since nothing says where the alpha sits there.

3. A_LINE alpha is coverage, not COLORALPHA (f6dc11c)

IRIX's OpenGL draws GL_LINE_SMOOTH as A_LINE with blending on and never loads COLORALPHA (it holds whatever the previous primitive left, ≈16 in my trace). The hardware uses the line's coverage as alpha (§3.6). Coverage isn't modelled, so A_LINE pixels now count as fully covered: solid lines, as before fix 1, rather than lines that fade to nothing.

Both the interpreter and the REX JIT are changed. Findings are written up in rules/rex3/alphahost-fields-and-aline-alpha.md, with a CHANGELOG entry.

Testing

• cargo test --features rex-jit --lib: 971 passed. New tests cover the alpha channel's factor under both BLENDALPHA settings, host-field alpha (interpreter vs JIT), and A_LINE alpha (both). The old BLENDALPHA tests that encoded the previous reading are rewritten.
• Indy R5000 + Newport, IRIX 6.5.22, compared with the emulated XZ as a second opinion:
  • a GL probe (textured and untextured quads, 12-bit double-buffered and 24-bit single-buffered visuals) now blends as it does on the XZ;
  • smooth lines draw at full strength (255; was 17);
  • blast -T -p (run from its data directory, as blast_audio does) shows the nebula without the haze and with its stars.
• Before/after screenshots of blast are available if useful.

Not addressed

• Real A_LINE coverage (AWEIGHT, S1/S2).
• The interpreter and the JIT walk an integer-coordinate A_LINE one row apart. The new A_LINE test checks the pixels' strength, not their position.
• blast's white triangle and smeared rows along the billboard's upper-left edge. They show identically before and after, so they're unrelated to blending.
• Unrelated and already on main: after warm-up, a 12-bit double-buffered GL window under the REX JIT reads back garbage with glReadPixels. It's correct with rex jit off, and the same on db16e3c without these changes.

🤖 Generated with Claude Code

[atomchild411]

This one fixes part of the rendering artifacts in 'blast', but when textures are turned on in the game there is still a funky artifacts (unrelated to this fix/pr). claude is going to try to capture that one tonight while I'm sleeping and send another follow up pr.