Valve BSP, WAD, and external texture support

[Toodles2You]

Half-Life map support!

This is actually two features in one: Valve format support and external wad texture support. The former relies on the latter; Half-Life maps don't usually embed textures since the wads are shipped with the game.

Test maps

External WAD textures

Textures are now loaded from wads if they aren't embedded within the map itself.

Upon loading a map, all wads listed in the "wad" field of the world are "mounted" (if present) for searching during texture loading. Wads are looked for in the game dir and gfx subdir.

This actually isn't tied to the Valve formats. It's been implemented to work with Quake bsps and wads, too. To showcase this, I provided e1m1 compiled using the ericw tools -notex option. The game loads gfx/base.wad for all of the textures.

The external_textures cvar disables this feature entirely.

WAD3 textures

Each texture has its own color palette, allowing for full color textures!

Textures prefixed with ! are treated as liquids.

Bright textures

These didn't work in Half-Life; Presumably for the same reason they didn't work in GLQuake. They do here, though!
Textures prefixed with ~ or +[0-9/A-Z]~ work under the same rules as Quake: Palette indices >=224 are fullbright.

BSP version 30

Lighting is stored as 24-bit RGB.

Embedded textures are assumed to be WAD3 textures.

Note: By default, Half-Life maps use different hull sizes than Quake; This implementation is for modders more-so than using existing maps.

[vsonnier]

Thanks @Toodles2You for your PR ! I'll look into it in more detail later.

For now, I don't think we really need #ifdef USE_VALVE_FORMATS anywhere:

• your changes are not that invasive,
• All additions are either VALVE qualified one way or another or in specific functions,
• Git is there to track the changes in any case

That way you could get rid of that conditinal compilation you used in meson and probably also of the nasty goto _load_regular in gl_model_c (could be something cleaner and simpler ?)

ELIM5 questions and remarks :

• Does these external Wad still work from within regular .pak ?
• Apparently BSP30 formats have the same limits as BSP from Quake ? Nowadays BSP2 is erywhere, I don't think creators would like to limit themselves so, just for prettier lighmaps and such ?
• What modern tools (ericw-tools, Trenchbroom, others ? ) would needed to create such maps ?

In order to preserve Git history please just add commits to this PR instead of amending it, that would be better.

[Toodles2You]

Thanks for responding @vsonnier !

I'll make those changes you requested.

Responses:

• Wad loading does still work from .pak files! I actually did test that but, forgot to mention it. It's why I used the FS_* file io replacements.
• Yes, BSP30 limits are the same as BSP. Modern compilers can bypass the original hard-coded limits but, the actual data structures are identical.
• Trenchbroom or J.A.C.K. for mapping. SDHLT (modern VHLT fork) for compiling. I recall ericw-tools being able to compile BSP30, as well. WadMaker for wads.

Really, the BSP30 support was the original starting point for this experiment, which ended up being more focused on WAD3 support and wad loading. It let me implement the full color textures before the wad mounting system. Should I just cut BSP30 loading? Or is it worth keeping, even if just as a fun novelty?

Also, I have the original separate commits in my local branch I used for development. Would it be preferable if I pushed those?

[vsonnier]

Hello @Toodles2You, Thanks for your answers.

• BSP version 30
  Lighting is stored as 24-bit RGB.
  Embedded textures are assumed to be WAD3 textures.
• Yes, BSP30 limits are the same as BSP.

I don't think we need to trouble ourselves with this BSP30 format, because RGB lighting is already possible using *.lit as a standard method, so the killer feature is really the external and WAD3 support.

For now, keep BSP29 support present and enabled like for instance: (As a Half-life map "viewer"...what for ?)

#ifndef BSP29_VALVE_SUPPORT
    #define BSP29_VALVE_SUPPORT 1 
#endif 

put into quakedef.h ?

And for the WAD support enable it unconditionaly.

I've toyed a little bit with Trenchbroom, and noticed that with your sample wads, a Half-life profile would only see the WAD3 format (sample.wad) while with the Quake profile, only the base.wad was seen....

I think that would be complicated to make a Half-life profile supporting Quake entities (Perhaps not, but this would disrupt the Quake creators workflows I imagine) so maybe the only think we need to do is make the Quake profile recognize the WAD3 textures.

After comparing the both GameConfig.cfg I came up with the following change in Quake GameConfig.cfg:

 "materials": {
        "root": "textures",
        "extensions": [".C", ".D"],  #note the .C addition, from the Half-life profile. Don't ask me what it means, Trenchbroom src is not clearer in that regard.
        "palette": "gfx/palette.lmp",
        "attribute": "wad"
    },

Also, I have the original separate commits in my local branch I used for development. Would it be preferable if I pushed those?

No no don't trouble yourself, I was speaking of commits on top of that current one within that PR.

Thank you for this evolution for vk !. Now to be honest, these additions are only going to be used by artists if the whole QS familly supports this, not only vkQuake.

So I think adding that PR for Quakespasm as well is somewhat inevitable... (QSS and Ironwail will get that eventually by rebasing on top of QS as they do regularly)

I could try to do it myself for @sezero in order to free you of that if you prefer.

[Toodles2You]

@vsonnier Alright, I've made the changes you requested! The BSP support is enabled via BSP29_VALVE in quakedef.h.

It's great to see that both WAD types can be loaded from a single TrenchBroom profile! I didn't try that, myself.

note the .C addition, from the Half-life profile. Don't ask me what it means, Trenchbroom src is not clearer in that regard.

I'm guessing the character extensions correlate with WAD 64 + x lump types. 64 + ('D' - 'A') + 1 == TYP_MIPTEX

these additions are only going to be used by artists if the whole QS familly supports this

Yes, having this in the entire QS family would be awesome! I'll attempt to do that myself; I appreciate you offering help, though. I imagine it's mostly a matter of adjusting the code to use Hunk_* allocations.

[vsonnier]

Thanks ! I think Con_DWarning should be changed to Con_Warning because the former only prints if developper 1 is activated. In your case, you want explicit warnings all the time if something is ammiss with the external WADs

I'll attempt to do that myself; I appreciate you offering help, though. I imagine it's mostly a matter of adjusting the code to use Hunk_* allocations.

Thank you :) I think Hunk_* are for speeding up allocations "at runtime" (while playing) but for loading level phase with 'immutable' allocation during the level play, scoped allocations like LoadWadList / FreeWadList are okay to use normal malloc/free.
The other condition is to never mix malloc pointers with HunkXXX alloc ones. (obviously)

[vsonnier]

Annnd I would like the same fixes you got from the other side :) (if applicable) For lack of time I didn't have time to review precisly the code, but I wouldn't have the insight @sezero have anyway.

[vsonnier]

Thanks. This is good for me, so let's wait if @sezero and others has anything more to add, and I'll merge the PR.

[vsonnier]

Done ! Thanks for your contribution !

[Novum]

Cool change

[Toodles2You]

Awesome! Happy to contribute.

@vsonnier Thank you for your time.

[pbdot]

Nice job! Looks fantastic in vkQuake. A couple notes in case anyone stumbles upon this PR:

1. ericwa tools technically process the sample map, though without colored lighting, broken spotlight, and the player start ends up outside the map with a warning about no entities in empty space

2. SDHLT looks cool though has a very broken license

• This code is protected by the GPL, a link to the GPL can be found at the end of this page.
• In addition to the GPL, the Valve SDK 2.3 EULA overrides any rights you may have obtained in the GPL, when needed.
• The iD Quake 2 Licence overrides portions of both the Valve EULA, and the GPL where needed, please contact iD for information on this subject.

That's not how licenses work :)

[vsonnier]

Nice job! Looks fantastic in vkQuake. A couple notes in case anyone stumbles upon this PR:
ericwa tools technically process the sample map, though without colored lighting, broken spotlight, and the player start ends up
outside the map with a warning about no entities in empty space
SDHLT looks cool though has a very broken license
This code is protected by the GPL, a link to the GPL can be found at the end of this page.
In addition to the GPL, the Valve SDK 2.3 EULA overrides any rights you may have obtained in the GPL, when needed.
The iD Quake 2 Licence overrides portions of both the Valve EULA, and the GPL where needed, please contact iD for information on this subject.
That's not how licenses work :)

Thanks ! IMHO this is all the more a reason not to do BSP30 for Quake. To be honest, the BSP30 has been left activated more as a gadget than anything else, because there is no point in doing HL format levels for Quake:

• BSP2 has increased limits compared to Valve BSP30
• RGB colored lighting is supported already in Quake by external .lit files
• WAD3 + external WAD support do not depends on the BSP30 format, and can be supported by either BSP or BSP2 already used by Quake thanks to this PR.

What is left is a way to make WAD3 textures recognized by the Quake level design tools, like Trenchbroom.

[pbdot]

Just wanted to share that ericwa tools with -notex and trenchroom with the .C texture format is really great... WAD3 with independent palettes per texture and easy bright pixel palettes is very nice.

Note these shots use a bit extra bounce over the originals in the PR, though that is all adjustable. Great work :)