Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re-add actions to build xmake with cosmocc #6010

Open
A2va opened this issue Dec 26, 2024 · 8 comments
Open

Re-add actions to build xmake with cosmocc #6010

A2va opened this issue Dec 26, 2024 · 8 comments
Labels
feature request

Comments

@A2va
Copy link
Contributor

A2va commented Dec 26, 2024

Is your feature request related to a problem? Please describe.

Previously, a workflow was created to build xmake with cosmocc
https://github.com/xmake-io/xmake/blob/b657911e2b0c34674570368c73c5025eff9d7649/.github/workflows/cosmocc.yml
However, this was not successful as it didn't run, but I noticed that this build was made using macos, so I tried to build it on linux.

First, I just tested that a normal xmake build (without embed) would work

a2va:~/xmake/core$ xmake f -p linux --cosmocc=y

The executable runs fine, then I enabled embed

a2va:~/xmake/core$ xmake f -p linux --cosmocc=y

Again, no problem with the executable, then I copied the file to Windows (the build was done in WSL Ubuntu). And it starts, but with this error:

PS D:\\xm-tt> .\\xmake-tt.exe --version
Error: @programdir/core/base/task.lua:398: @programdir/core/base/winos.lua:238: Field '_registry_query' is not accessible (a null value)
Stack traceback:
    [@programdir/core/base/winos.lua:238]: in function 'registry_query'.
    [@programdir/core/base/cpu.lua:222]: in function '_info'.
    [@programdir/core/base/cpu.lua:286]: in function 'Vendor'
    [@programdir/core/base/cpu.lua:352]:
    [@programdir/core/base/os.lua:1443]: in function 'default_njob'.
    [@programdir/plugins/pack/xmake.lua:31]: in main chunk

I suspect this is because Cosmopolitan doesn't implement the Windows API for editing the registry, but those can added if needed (eg: jart/cosmopolitan#1318), or the other solution would be to completely disable this functionality.

Describe the solution you'd like

With this feature the xmake bundle executable which is for now windows only could be extended to other platforms.

Describe alternatives you've considered

No response

Additional context

No response
@waruqi
Copy link
Member

waruqi commented Dec 27, 2024

I tried it before, but it still has many problems, such as it will stuck on macos ci and some windows api problems.

But I don't have time to solve them recently, so I can only delete it temporarily. If you can solve these problems, you can submit a PR.

@A2va
Copy link
Contributor Author

A2va commented Dec 27, 2024

I managed to built xmake in the CI but one of the test fails, I don't know if this is due to cmake or not.

> /usr/bin/gcc -c -m64 -DAL_LIBTYPE_STATIC -isystem /home/runner/.xmake/packages/o/openal-soft/1.23.1/ac313711f22a45f2a1a2f9b24f64d1d1/include -isystem /home/runner/.xmake/packages/l/libsndio/1.9.0/aa973ae5d04a4f53a02cf28d02c69457/include -isystem /home/runner/.xmake/packages/a/alsa-lib/1.2.10/60f829908b344b8193f7d458e8a93b5a/include -o /tmp/.xmake1001/241227/_B5599DC13D654B00859AC66A1E9E1B10.o /tmp/.xmake1001/241227/_76360DFFA3454E3488F03C9FE8D87B65.c
> /usr/bin/g++ -o /tmp/.xmake1001/241227/_B5599DC13D654B00859AC66A1E9E1B10.b /tmp/.xmake1001/241227/_B5599DC13D654B00859AC66A1E9E1B10.o -m64 -L/home/runner/.xmake/packages/o/openal-soft/1.23.1/ac313711f22a45f2a1a2f9b24f64d1d1/lib -L/home/runner/.xmake/packages/l/libsndio/1.9.0/aa973ae5d04a4f53a02cf28d02c69457/lib -L/home/runner/.xmake/packages/a/alsa-lib/1.2.10/60f829908b344b8193f7d458e8a93b5a/lib -lopenal -lsndio -latopology -lasound -lpthread -ldl
> checking for c includes(AL/al.h)
> checking for c funcs(alGetProcAddress)
> checking for c links(openal, sndio, atopology, asound, pthread, dl)
> checking for c snippet(has_cfuncs)
  => install openal-soft 1.23.1 .. ok
  => download https://www.sfml-dev.org/files/SFML-2.5.1-sources.zip .. ok
error: @programdir/core/sandbox/modules/os.lua:273: -- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - donCMake Error at /usr/local/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find OpenGL (missing: OPENGL_opengl_LIBRARY OPENGL_glx_LIBRARY
  OPENGL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.31/Modules/FindOpenGL.cmake:579 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  cmake/Macros.cmake:279 (find_package)
  src/SFML/Window/CMakeLists.txt:257 (sfml_find_package)


r remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Configuring incomplete, errors occurred!

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [@programdir/core/sandbox/modules/os.lua:273]:
    [@programdir/core/sandbox/modules/os.lua:291]: in function 'vrunv'
    [@programdir/modules/package/tools/cmake.lua:1317]: in function 'configure'
    [@programdir/modules/package/tools/cmake.lua:1359]: in function 'install'
    [./xmake.lua:192]: in function 'script'
    [...dir/modules/private/action/require/impl/utils/filter.lua:114]: in function 'call'
    [.../modules/private/action/require/impl/actions/install.lua:452]:

  => install sfml 2.5.1 .. failed
error: @programdir/core/main.lua:329: @programdir/modules/async/runjobs.lua:325: .../modules/private/action/require/impl/actions/install.lua:561: install failed!
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [.../modules/private/action/require/impl/actions/install.lua:561]: in function 'catch'
    [@programdir/core/sandbox/modules/try.lua:123]: in function 'try'
    [.../modules/private/action/require/impl/actions/install.lua:419]:
    [...modules/private/action/require/impl/install_packages.lua:510]: in function 'jobfunc'
    [@programdir/modules/async/runjobs.lua:241]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:1075: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:329: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
>>     test failed: @programdir/core/sandbox/modules/os.lua:337: exec(xmake f -c -D -y) failed(255)
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [@programdir/core/sandbox/modules/os.lua:337]: in function 'exec'
    [./tests/test_utils/test_build.lua:4]: in function 'build'
    [./tests/projects/package/components/test.lua:10]:

>>       function test_main tests/projects/package/components/test.lua:1
error: @programdir/core/main.lua:329: ./tests/test_utils/print_error.lua:46: aborting because of unhandled error ...
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [./tests/test_utils/print_error.lua:46]: in function 'print_error'
    [./tests/runner.lua:58]: in function 'catch'
    [@programdir/core/sandbox/modules/try.lua:123]: in function 'try'
    [./tests/runner.lua:45]:
    [./tests/run.lua:7]: in function '_run_test'
    [./tests/run.lua:42]:
    [@programdir/plugins/lua/main.lua:124]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:244]:
    [@programdir/core/base/task.lua:491]: in function 'run'
    [@programdir/core/main.lua:327]: in function 'cotask'
    [@programdir/core/base/scheduler.lua:406]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:1075: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:329: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
Error: Process completed with exit code 255.

https://github.com/A2va/xmake/blob/cosmocc-ci/.github/workflows/cosmocc.yml
https://github.com/A2va/xmake/actions/runs/12518925059/job/34922147269#step:7:2163

@waruqi
Copy link
Member

waruqi commented Jan 5, 2025

I don't know why. I haven't had time to look at it, but if you can get it to pass on all ci tests, you can open a PR to rebuild xmake with cosmocc binary

@A2va
Copy link
Contributor Author

A2va commented Jan 8, 2025

I just updated my fork to the latest commit of the master branch and ran to both linux and cosmocc ci. Both fail on the same test, which seems to indicate that xmake cosmocc is acting the same as the regular one.

What is weird is that the same test on the xmake main repo with the linux ci passes.

error: @programdir/core/main.lua:329: @programdir/actions/build/main.lua:148: @programdir/modules/async/runjobs.lua:325: @programdir/actions/build/kinds/binary.lua:53: @programdir/core/sandbox/modules/os.lua:378: execv(/usr/bin/clang++ -o build/linux/x86_64/release/dependence2 build/.objs/dependence2/linux/x86_64/release/src/main.cpp.o build/.objs/dependence2/linux/x86_64/release/src/foo.mpp.o build/.objs/dependence2/linux/x86_64/release/src/bar.mpp.o build/.objs/dependence2/linux/x86_64/release/src/zoo.mpp.o build/.objs/dependence2/linux/x86_64/release/src/cat.mpp.o -m64 -s -stdlib=libc++) failed(1)
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [@programdir/core/sandbox/modules/os.lua:378]: in function 'execv'
    [@programdir/modules/core/tools/gcc.lua:599]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:244]:
    [@programdir/core/tool/linker.lua:232]: in function 'link'
    [@programdir/actions/build/kinds/binary.lua:53]: in function 'callback'
    [@programdir/modules/core/project/depend.lua:217]: in function 'on_changed'
    [@programdir/actions/build/kinds/binary.lua:41]: in function '_do_link_target'
    [@programdir/actions/build/kinds/binary.lua:83]:
    [@programdir/actions/build/kinds/binary.lua:110]: in function '_link_target'
    [@programdir/actions/build/kinds/binary.lua:138]: in function 'jobfunc'
    [@programdir/modules/async/runjobs.lua:241]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:244]: in function 'trycall'
    [@programdir/core/sandbox/modules/try.lua:117]: in function 'try'
    [@programdir/modules/async/runjobs.lua:224]: in function 'cotask'
    [@programdir/core/base/scheduler.lua:406]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:1075: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:329: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
warning: std and std.compat modules not found! maybe try to add --sdk=<PATH/TO/LLVM> or install libc++
warning: No clang-scan-deps found ! using fallback scanner

Linux: https://github.com/A2va/xmake/actions/runs/12675795887/job/35327578205#step:8:944
Cosmocc: https://github.com/A2va/xmake/actions/runs/12675715771/job/35327336647#step:7:958

@waruqi
Copy link
Member

waruqi commented Jan 9, 2025

I don't know why. you can diff ci yml.

@A2va A2va mentioned this issue Jan 15, 2025
Merged
@A2va
Copy link
Contributor Author

A2va commented Jan 15, 2025

The PR #6065 seems to have the test problem I was getting on my fork, so I made a PR to add a cosmocc action.

There is still this windows crash that I need to resolve. At first I tought that it was a missing stub in the win32 api of cosmocc, but that does not seems the case.
What I suspect is that the winos functions is not compiled because TB_CONFIG_OS_WINDOWS is not defined.

xmake/core/src/xmake/engine.c

Lines 383 to 398 in 58c00fc

#ifdef TB_CONFIG_OS_WINDOWS
static luaL_Reg const g_winos_functions[] =
{
{ "cp_info", xm_winos_cp_info }
, { "console_cp", xm_winos_console_cp }
, { "console_output_cp", xm_winos_console_output_cp }
, { "oem_cp", xm_winos_oem_cp }
, { "ansi_cp", xm_winos_ansi_cp }
, { "logical_drives", xm_winos_logical_drives }
, { "registry_query", xm_winos_registry_query }
, { "registry_keys", xm_winos_registry_keys }
, { "registry_values", xm_winos_registry_values }
, { "short_path", xm_winos_short_path }
, { tb_null, tb_null }
};
#endif

And so the winos function doesn't get registered with ``xm_lua_register'', and it returns nil.

xmake/core/src/xmake/engine.c

Line 1356 in 58c00fc

#ifdef TB_CONFIG_OS_WINDOWS

I tried to build in debug, to see if the stacktrace tells a bit more

[xmake]: init: ..
[xmake]: version: xmake_v2_9_7_20250114_x86_64_sse2_elf by gnu c/c++ 14.1
[tbox]: init: ..
[tbox]: [backtrace]: [warning]: noimpl at tb_backtrace_frames(): 136, src/tbox/tbox/src/tbox/platform/libc/backtrace.c
[tbox]: version: tbox_v1_7_6_20250114_x86_64_sse2_elf by gnu c/c++ 14.1
[tbox]: init: ok
[xmake]: init: ok
[xmake]: [engine]: project: /D/Telechargements/xmake-cosmocc-x86_64
[xmake]: [engine]: programfile: /D/Telechargements/xmake-cosmocc-x86_64/xmakec.exe
[xmake]: [engine]: programdir: /tmp/.xmake13793/2.9.7+test-cosmocc.34a7088
[xmake]: [engine]: main: /tmp/.xmake13793/2.9.7+test-cosmocc.34a7088/core/_xmake_main.lua
error: @programdir/core/base/task.lua:400: @programdir/core/base/winos.lua:242: attempt to call a nil value (field '_registry_query')
stack traceback:
    [@programdir/core/base/winos.lua:242]: in function 'registry_query'
    [@programdir/core/base/cpu.lua:222]: in function '_info'
    [@programdir/core/base/cpu.lua:286]: in function 'vendor'
    [@programdir/core/base/cpu.lua:352]:
    [@programdir/core/base/os.lua:1514]: in function 'default_njob'
    [@programdir/plugins/pack/xmake.lua:31]: in main chunk

[tbox]: [lock_profiler]: 
[tbox]: [lock_profiler]: lock: 0x6ffffffb07a4, name: native_large_allocator, occupied: 0
[tbox]: [lock_profiler]: lock: 0x6ffffffb214c, name: allocator, occupied: 0
[tbox]: [small_allocator]: 
[tbox]: [static_fixed_pool]: [16]: peak_size: 48, wast_rate: 4074/10000, pred_failed: 0, item_maxn: 245, free_count: 5, malloc_count: 5
[tbox]: [static_fixed_pool]: [32]: peak_size: 32, wast_rate: 2558/10000, pred_failed: 0, item_maxn: 246, free_count: 2, malloc_count: 2
[tbox]: [static_fixed_pool]: [64]: peak_size: 64, wast_rate: 1466/10000, pred_failed: 0, item_maxn: 247, free_count: 19, malloc_count: 19
[tbox]: [static_fixed_pool]: [128]: peak_size: 384, wast_rate: 791/10000, pred_failed: 0, item_maxn: 249, free_count: 206, malloc_count: 206
[tbox]: [static_fixed_pool]: [192]: peak_size: 384, wast_rate: 541/10000, pred_failed: 0, item_maxn: 250, free_count: 166, malloc_count: 166
[tbox]: [static_fixed_pool]: [256]: peak_size: 256, wast_rate: 411/10000, pred_failed: 0, item_maxn: 251, free_count: 1, malloc_count: 1
[tbox]: [static_fixed_pool]: [384]: peak_size: 1152, wast_rate: 278/10000, pred_failed: 0, item_maxn: 252, free_count: 3, malloc_count: 3
[tbox]: [static_fixed_pool]: [512]: peak_size: 512, wast_rate: 210/10000, pred_failed: 0, item_maxn: 253, free_count: 1, malloc_count: 1
[tbox]: [static_fixed_pool]: [1024]: peak_size: 3072, wast_rate: 106/10000, pred_failed: 0, item_maxn: 254, free_count: 233, malloc_count: 233
[tbox]: [static_fixed_pool]: [2048]: peak_size: 2048, wast_rate: 53/10000, pred_failed: 0, item_maxn: 255, free_count: 33, malloc_count: 33
[tbox]: [static_fixed_pool]: [3072]: peak_size: 3072, wast_rate: 35/10000, pred_failed: 0, item_maxn: 255, free_count: 9, malloc_count: 9
[tbox]: [native_large_allocator]: 
[tbox]: [native_large_allocator]: peak_size: 2467336
[tbox]: [native_large_allocator]: wast_rate: 12/10000
[tbox]: [native_large_allocator]: free_count: 285
[tbox]: [native_large_allocator]: malloc_count: 285
[tbox]: [native_large_allocator]: ralloc_count: 96
[tbox]: exit: ok

How can I fix this ?

@waruqi
Copy link
Member

waruqi commented Jan 15, 2025 •
edited
Loading

I don't know how cosmocc calls the windows api。

@A2va
Copy link
Contributor Author

A2va commented Jan 15, 2025

They have a windows.h as well as some stub functions for win32
https://github.com/jart/cosmopolitan/blob/master/libc/nt/windows.h
https://github.com/jart/cosmopolitan/blob/master/libc/nt/advapi32/RegCloseKey.S

I will try to compile the winos module with cosmocc, and adding some check at runtime in the lua script to verify that is runs on windows.

@waruqi waruqi mentioned this issue Jan 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@waruqi @A2va