Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New filesystem #76

Open
wants to merge 63 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
f497335
new filesystem: Copy in initial filesystem files
Chomenor Apr 12, 2018
c462edc
new filesystem: Initial tremulous porting work
Chomenor Apr 12, 2018
85ba05a
new filesystem: Refactor source dir definitions to avoid hardcoded so…
Chomenor Apr 15, 2018
cfaf833
new filesystem: Support reading from mac app bundle path
Chomenor Apr 15, 2018
2334088
new filesystem: Fix FS_FOpenFile null handle ptr return values to be …
Chomenor Apr 15, 2018
83f9701
new filesystem: Build with new filesystem ifdefs by default
Chomenor Apr 15, 2018
e352456
new filesystem: Run clang-format on everything in filesystem directory
Chomenor Apr 16, 2018
61c7449
new filesystem: Initial base directory and hash precedence support
Chomenor May 17, 2018
9cd31b3
new filesystem: Try to fix compile errors
Chomenor May 17, 2018
843e850
new filesystem: Try again to fix the compile error
Chomenor May 17, 2018
5100805
new filesystem: Mod dir handling work
Chomenor May 18, 2018
7e34608
new filesystem: Improved download/pure list handling
Chomenor May 24, 2018
2a2b09a
new filesystem: Rename 'system paks' to 'default paks'
Chomenor May 26, 2018
57d4706
new filesystem: Prioritize pk3dirs over pk3s with same name
Chomenor May 26, 2018
4edf2bb
Merge branch 'master' of https://github.com/GrangerHub/tremulous
Chomenor Jan 3, 2020
2b02f2d
new filesystem: Add folder browse functions
Chomenor Jan 3, 2020
3058a7b
new filesystem: Merge filesystem updates
Chomenor Jan 5, 2020
5504900
new filesystem: FS_FOpenFile cleanup and improvements
Chomenor Jan 6, 2020
35e8dab
new filesystem: Allow vms to read any files on disk when pure
Chomenor Jan 12, 2020
50780ae
new filesystem: Improve tremulous base directory handling
Chomenor Jan 13, 2020
5e0d768
new filesystem: Update tremulous core pk3 precedence
Chomenor Jan 13, 2020
0fed5aa
new filesystem: Update default.cfg handling
Chomenor Jan 14, 2020
9423942
new filesystem: Remove arch string from game dll search filename
Chomenor Jan 16, 2020
4281c6d
new filesystem: Use original error handling behavior for non-pk3 curl…
Chomenor Jan 16, 2020
55c6592
new filesystem: Fix ordering of duplicate pure list entries
Chomenor Jan 17, 2020
8cef060
new filesystem: Don't redownload core pk3s to different base directories
Chomenor Jan 17, 2020
813d60f
new filesystem: Simplify pk3_list_lookup function
Chomenor Jan 18, 2020
024325e
new filesystem: Update precedence handling and move some functions to…
Chomenor Jan 18, 2020
4f89e73
new filesystem: Rearrange some Tremulous defines
Chomenor Jan 18, 2020
82821ff
new filesystem: Provisional core game dll support
Chomenor Jan 18, 2020
d0004db
new filesystem: Path generation tweaks
Chomenor Jan 23, 2020
f8a6bec
new filesystem: Fix browse folder function for directory names contai…
Chomenor Jan 23, 2020
f4f95da
Build system: Adjust the build system for the new file system.
dGr8LookinSparky Jan 21, 2020
224e98e
Merge pull request #1 from dGr8LookinSparky/filesystem
Chomenor Jan 23, 2020
091abc4
new filesystem: Use fs_basegame for engine writes regardless of curre…
Chomenor Jan 23, 2020
65fc4a8
new filesystem: Fix old UI remaining loaded after demo playback finishes
Chomenor Jan 25, 2020
272c80e
- change the naming of the core-cgame and core-ui dynamic libraries b…
dGr8LookinSparky Jan 25, 2020
27fa4c3
Merge pull request #2 from dGr8LookinSparky/filesystem
Chomenor Jan 25, 2020
81ee257
new filesystem: Allow browse folder commands to create missing direct…
Chomenor Jan 25, 2020
0a21f7b
build system: fix zip errors.
dGr8LookinSparky Jan 25, 2020
1d63b5f
Merge pull request #3 from dGr8LookinSparky/filesystem
Chomenor Jan 25, 2020
b2a0b56
build system:remove unnecessary files in the travis-ci release builds.
dGr8LookinSparky Jan 26, 2020
1d7a696
build system: fix zip error for travis release build of macos.
dGr8LookinSparky Jan 26, 2020
cbeddac
build system: rename core-game to core_game.
dGr8LookinSparky Jan 26, 2020
280a9aa
game module: set the default of vm_game to 0.
dGr8LookinSparky Jan 26, 2020
87761ff
main menu: Add basemod/ as an option in the Browse Folders menu.
dGr8LookinSparky Jan 28, 2020
cc3ddbf
new filesystem: Allow parse functions to read files on disk when conn…
Chomenor Feb 4, 2020
9da72c9
Revert "new filesystem: Allow parse functions to read files on disk w…
Chomenor Feb 5, 2020
54917f8
new filesystem: Allow FS_FOpenFile to read files with specific extens…
Chomenor Feb 5, 2020
9cd1b2d
Correct the directory paths in the vms pk3 files.
dGr8LookinSparky Feb 16, 2020
db111bf
Merge branch 'master' of https://github.com/GrangerHub/tremulous
Chomenor Feb 28, 2020
539b722
git: change .gitattributes
dGr8LookinSparky Feb 28, 2020
cda5508
Merge branch 'master' into filesystem
dGr8LookinSparky Feb 28, 2020
4a18039
assets: update the assets to v1.3.0-alpha.0.14.10
dGr8LookinSparky Feb 28, 2020
81668c2
new filesystem: Support UI pk3 download progress stats
Chomenor Feb 28, 2020
6e107c0
new filesystem: Fix UI pk3 download progress stats
Chomenor Feb 29, 2020
706ee83
new filesystem: Merge pk3 manifest processing updates
Chomenor Mar 6, 2020
9ae3c6d
Merge branch 'master' of https://github.com/GrangerHub/tremulous
Chomenor Mar 26, 2020
62ba61a
new filesystem: Initial support for multi-version pk3 manifests
Chomenor Mar 26, 2020
7b39045
renderer: make lightmap not cheat protected.
dGr8LookinSparky Mar 30, 2020
eeeb57f
default assets: Update the Trem 1.3 default assets to v1.3.0-alpha.0.…
dGr8LookinSparky Mar 30, 2020
89ea9ea
Merge branch 'master' of https://github.com/GrangerHub/tremulous
Chomenor Mar 30, 2020
798ad46
tremulous: fix the bucket system for QVMs.
dGr8LookinSparky Mar 31, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 60 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,8 @@ RCOMMONDIR=$(MOUNT_DIR)/renderercommon
RGL1DIR=$(MOUNT_DIR)/renderergl1
RGL2DIR=$(MOUNT_DIR)/renderergl2
CMDIR=$(MOUNT_DIR)/qcommon
FSDIR=$(MOUNT_DIR)/filesystem
FSDIR_FSCORE=$(MOUNT_DIR)/filesystem/fscore
SDLDIR=$(MOUNT_DIR)/sdl
ASMDIR=$(MOUNT_DIR)/asm
SYSDIR=$(MOUNT_DIR)/sys
Expand Down Expand Up @@ -828,6 +830,10 @@ ifneq ($(BUILD_DATA_PK3),0)
$(B)/$(BASEGAME)/data-$(VERSION).pk3
endif

ifneq ($(NEW_FILESYSTEM),0)
BASE_CFLAGS += -DNEW_FILESYSTEM
endif

ifeq ($(USE_OPENAL),1)
CLIENT_CFLAGS += -DUSE_OPENAL
ifeq ($(USE_OPENAL_DLOPEN),1)
Expand Down Expand Up @@ -1833,7 +1839,28 @@ Q3OBJ = \
$(B)/client/sdl_snd.o \
\
$(B)/client/con_log.o \
$(B)/client/sys_main.o
$(B)/client/sys_main.o \
\
$(B)/client/fsc_cache.o \
$(B)/client/fsc_crosshair.o \
$(B)/client/fsc_gameparse.o \
$(B)/client/fsc_iteration.o \
$(B)/client/fsc_main.o \
$(B)/client/fsc_md4.o \
$(B)/client/fsc_misc.o \
$(B)/client/fsc_os.o \
$(B)/client/fsc_pk3.o \
$(B)/client/fsc_sha256.o \
$(B)/client/fsc_shader.o \
$(B)/client/fs_commands.o \
$(B)/client/fs_download.o \
$(B)/client/fs_fileio.o \
$(B)/client/fs_filelist.o \
$(B)/client/fs_lookup.o \
$(B)/client/fs_main.o \
$(B)/client/fs_misc.o \
$(B)/client/fs_reference.o \
$(B)/client/fs_trusted_vms.o

ifdef MINGW
Q3OBJ += \
Expand Down Expand Up @@ -2351,7 +2378,28 @@ Q3DOBJ = \
$(B)/ded/null_snddma.o \
\
$(B)/ded/con_log.o \
$(B)/ded/sys_main.o
$(B)/ded/sys_main.o \
\
$(B)/ded/fsc_cache.o \
$(B)/ded/fsc_crosshair.o \
$(B)/ded/fsc_gameparse.o \
$(B)/ded/fsc_iteration.o \
$(B)/ded/fsc_main.o \
$(B)/ded/fsc_md4.o \
$(B)/ded/fsc_misc.o \
$(B)/ded/fsc_os.o \
$(B)/ded/fsc_pk3.o \
$(B)/ded/fsc_sha256.o \
$(B)/ded/fsc_shader.o \
$(B)/ded/fs_commands.o \
$(B)/ded/fs_download.o \
$(B)/ded/fs_fileio.o \
$(B)/ded/fs_filelist.o \
$(B)/ded/fs_lookup.o \
$(B)/ded/fs_main.o \
$(B)/ded/fs_misc.o \
$(B)/ded/fs_reference.o \
$(B)/ded/fs_trusted_vms.o

ifeq ($(ARCH),x86)
Q3DOBJ += \
Expand Down Expand Up @@ -2646,6 +2694,11 @@ $(B)/client/%.o: $(SDIR)/%.c
$(B)/client/%.o: $(CMDIR)/%.c
$(DO_CC)

$(B)/client/%.o: $(FSDIR)/%.cpp
$(DO_CXX)
$(B)/client/%.o: $(FSDIR_FSCORE)/%.cpp
$(DO_CXX)

$(B)/client/%.o: $(CDIR)/%.cpp
$(DO_CXX)

Expand Down Expand Up @@ -2754,6 +2807,11 @@ $(B)/ded/%.o: $(SDIR)/%.cpp
$(B)/ded/%.o: $(CMDIR)/%.c
$(DO_DED_CC)

$(B)/ded/%.o: $(FSDIR)/%.cpp
$(DO_DED_CXX)
$(B)/ded/%.o: $(FSDIR_FSCORE)/%.cpp
$(DO_DED_CXX)

$(B)/ded/%.o: $(CMDIR)/%.cpp
$(DO_DED_CXX)

Expand Down
9 changes: 9 additions & 0 deletions src/client/cl_avi.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -608,7 +608,16 @@ bool CL_CloseAVI( void )
// Write index

// Open the temp index file
#ifdef NEW_FILESYSTEM
indexSize = 0;
{ char path[FS_MAX_PATH];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regrettably, It's not documented anywhere and some files (such as this one) have not yet been updated, but please try to follow a general style of

  indexSize = 0;
  {
      char path[FS_MAX_PATH];
      if(fs_generate_path_writedir(FS_GetCurrentGameDir(), idxFileName, 0, FS_ALLOW_SLASH, path, sizeof(path)))
          afd.idxF = fs_direct_read_handle_open(0, path, (unsigned int *)&indexSize);
  }

Notably

  1. Curly braces on a separate line (exceptions for small one line conditionals and functions)
  2. 4 space indent (no tabs)

if(fs_generate_path_writedir(FS_GetCurrentGameDir(), idxFileName,
0, FS_ALLOW_SLASH, path, sizeof(path)))
afd.idxF = fs_direct_read_handle_open(0, path, (unsigned int *)&indexSize); }
if(indexSize <= 0)
#else
if( ( indexSize = FS_FOpenFileRead( idxFileName, &afd.idxF, true ) ) <= 0 )
#endif
{
FS_FCloseFile( afd.f );
return false;
Expand Down
27 changes: 27 additions & 0 deletions src/client/cl_cgame.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,9 @@ void CL_ShutdownCGame( void ) {
return;
}
VM_Call( cls.cgame, CG_SHUTDOWN );
#ifdef NEW_FILESYSTEM
fs_close_owner_handles(FS_HANDLEOWNER_CGAME);
#endif
VM_Free( cls.cgame );
cls.cgame = NULL;
}
Expand Down Expand Up @@ -459,17 +462,33 @@ intptr_t CL_CgameSystemCalls( intptr_t *args )
Cmd_LiteralArgsBuffer( (char*)VMA(1), args[2] );
return 0;
case CG_FS_FOPENFILE:
#ifdef NEW_FILESYSTEM
return FS_FOpenFileByModeOwner( (const char*)VMA(1), (fileHandle_t*)VMA(2), (FS_Mode)args[3], FS_HANDLEOWNER_CGAME );
#else
return FS_FOpenFileByMode( (const char*)VMA(1), (fileHandle_t*)VMA(2), (FS_Mode)args[3] );
#endif
case CG_FS_READ:
#ifdef NEW_FILESYSTEM
if(fs_handle_get_owner(args[3]) != FS_HANDLEOWNER_CGAME) return 0;
#endif
FS_Read( VMA(1), args[2], args[3] );
return 0;
case CG_FS_WRITE:
#ifdef NEW_FILESYSTEM
if(fs_handle_get_owner(args[3]) != FS_HANDLEOWNER_CGAME) return 0;
#endif
FS_Write( VMA(1), args[2], args[3] );
return 0;
case CG_FS_FCLOSEFILE:
#ifdef NEW_FILESYSTEM
if(fs_handle_get_owner(args[1]) != FS_HANDLEOWNER_CGAME) return 0;
#endif
FS_FCloseFile( args[1] );
return 0;
case CG_FS_SEEK:
#ifdef NEW_FILESYSTEM
if(fs_handle_get_owner(args[1]) != FS_HANDLEOWNER_CGAME) return 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to fix the indenting throughout here

#endif
return FS_Seek( (fileHandle_t)args[1], args[2], (FS_Origin)args[3] );
case CG_FS_GETFILELIST:
return FS_GetFileList( (const char*)VMA(1), (const char*)VMA(2), (char*)VMA(3), args[4] );
Expand Down Expand Up @@ -786,14 +805,22 @@ void CL_InitCGame( void ) {
mapname = Info_ValueForKey( info, "mapname" );
Com_sprintf( cl.mapname, sizeof( cl.mapname ), "maps/%s.bsp", mapname );

#ifdef NEW_FILESYSTEM
// Give the map pk3 slightly higher precedence in the filesystem to help
// load the correct textures and such
fs_register_current_map(cl.mapname);
#endif

// load the dll or bytecode
interpret = (vmInterpret_t)Cvar_VariableValue("vm_cgame");
#ifndef NEW_FILESYSTEM
if(cl_connectedToPureServer)
{
// if sv_pure is set we only allow qvms to be loaded
if(interpret != VMI_COMPILED && interpret != VMI_BYTECODE)
interpret = VMI_COMPILED;
}
#endif

cls.cgame = VM_Create( "cgame", CL_CgameSystemCalls, interpret );
if ( !cls.cgame ) {
Expand Down
20 changes: 20 additions & 0 deletions src/client/cl_curl.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,10 @@ void CL_cURL_BeginDownload( const char *localName, const char *remoteURL )
CL_cURL_Cleanup();
Q_strncpyz(clc.downloadURL, remoteURL, sizeof(clc.downloadURL));
Q_strncpyz(clc.downloadName, localName, sizeof(clc.downloadName));
#ifdef NEW_FILESYSTEM
if(!clc.activeCURLNotGameRelated) Com_sprintf(clc.downloadTempName, sizeof(clc.downloadTempName), "download.temp");
else
#endif
Com_sprintf(clc.downloadTempName, sizeof(clc.downloadTempName),
"%s.tmp", localName);

Expand Down Expand Up @@ -347,18 +351,34 @@ void CL_cURL_PerformDownload(void)
return;
}
FS_FCloseFile(clc.download);
#ifdef NEW_FILESYSTEM
clc.download = 0;
#endif
if(msg->msg == CURLMSG_DONE && msg->data.result == CURLE_OK) {
#ifdef NEW_FILESYSTEM
if(!clc.activeCURLNotGameRelated) fs_finalize_download();
else
#endif
FS_SV_Rename(clc.downloadTempName, clc.downloadName, false);
clc.downloadRestart = true;
}
else {
long code;

qcurl_easy_getinfo(msg->easy_handle, CURLINFO_RESPONSE_CODE, &code);
#ifdef NEW_FILESYSTEM
Com_Printf("Download Error: %s Code: %ld URL: %s\n",
qcurl_easy_strerror(msg->data.result), code, clc.downloadURL);
#else
Com_Error(ERR_DROP, "Download Error: %s Code: %ld URL: %s",
qcurl_easy_strerror(msg->data.result),
code, clc.downloadURL);
#endif
}

#ifdef NEW_FILESYSTEM
if(clc.activeCURLNotGameRelated) CL_cURL_Shutdown();
else
#endif
CL_NextDownload();
}
Loading