Can I auto-tunning SD models by myself?

[felixslu]

with tvm.transform.PassContext(opt_level=3): ex = relax.build(mod_deploy, args.target)
args.target： “cuda”
pip install -I mlc_ai_nightly_cu121 -f https://mlc.ai/wheels

but,get errors as below!

Traceback (most recent call last):
File "web-stable-diffusion/build.py", line 184, in
build(mod, ARGS)
File "web-stable-diffusion/build.py", line 151, in build
ex = relax.build(mod_deploy, args.target)
File "/usr/local/lib/python3.8/dist-packages/tvm/relax/vm_build.py", line 338, in build
return _vmlink(builder, target, tir_mod, ext_libs, params, system_lib=system_lib)
File "/usr/local/lib/python3.8/dist-packages/tvm/relax/vm_build.py", line 242, in _vmlink
lib = tvm.build(
File "/usr/local/lib/python3.8/dist-packages/tvm/driver/build_module.py", line 281, in build
rt_mod_host = _driver_ffi.tir_to_runtime(annotated_mods, target_host)
File "tvm/_ffi/_cython/./packed_func.pxi", line 331, in tvm._ffi._cy3.core.PackedFuncBase.call
File "tvm/_ffi/_cython/./packed_func.pxi", line 262, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 251, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 181, in tvm._ffi._cy3.core.CHECK_CALL
tvm._ffi.base.TVMError: Traceback (most recent call last):
10: TVMFuncCall
9: ZN3tvm7runtime13PackedFuncObj9ExtractorINS0_16PackedFuncSubObjIZNS0_15TypedPackedFuncIFNS0_6ModuleERKNS0_3MapINS_6TargetENS_8IRModuleEvvEES7_EE17AssignTypedLambdaINS_UlSB_S7_E4_EEEvT_SsEUlRKNS0_7TVMArgsEPNS0_11TVMRetValueEE_EEE4CallEPKS1_SH_SL
8: tvm::TIRToRuntime(tvm::runtime::Map<tvm::Target, tvm::IRModule, void, void> const&, tvm::Target const&)
7: tvm::SplitMixedModule(tvm::IRModule, tvm::Target const&, tvm::Target const&)
6: tvm::ApplyPasses(tvm::IRModule, tvm::transform::Sequential)
5: tvm::transform::Pass::operator()(tvm::IRModule) const
4: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
3: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
2: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
1: tvm::transform::ModulePassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
0: ZN3tvm7runtime13PackedFuncObj9ExtractorINS0_16PackedFuncSubObjIZNS0_15TypedPackedFuncIFNS_8IRModuleES5_NS_9transform11PassContextEEE17AssignTypedLambdaIZNS_3tir9transform12VerifyMemoryEvEUlS5_S7_E_EEvT_EUlRKNS0_7TVMArgsEPNS0_11TVMRetValueEE_EEE4CallEPKS1_SF_SJ
Did you forget to bind?
Variable B is directly accessed by host memory (it is not contained in a thread environment or in the function arguments.
Variable A is directly accessed by host memory (it is not contained in a thread environment or in the function arguments.
Variable matmul is directly accessed by host memory (it is not contained in a thread environment or in the function arguments.
Variable matmul is directly accessed by host memory (it is not contained in a thread environment or in the function arguments.
Variable matmul is directly accessed by host memory (it is not contained in a thread environment or in the function arguments.
File "/workspace/tvm/src/tir/analysis/verify_memory.cc", line 205
RuntimeError: Memory verification failed with the following errors:
from tvm.script import tir as T

@T.prim_func
def matmul20(A: T.Buffer((T.int64(2), T.int64(256), T.int64(1280)), "float32"), B: T.Buffer((T.int64(1280), T.int64(1280)), "float32"), matmul: T.Buffer((T.int64(2), T.int64(256), T.int64(1280)), "float32")):
T.func_attr({"global_symbol": "matmul20", "op_pattern": 4, "target": T.target({"arch": "sm_86", "host": {"keys": ["cpu"], "kind": "llvm", "tag": ""}, "keys": ["cuda", "gpu"], "kind": "cuda", "max_num_threads": 1024, "tag": "", "thread_warp_size": 32}), "tir.noalias": T.bool(True)})
for i0, i1, i2, k in T.grid(2, 256, 1280, 1280):
cse_var_2: T.int32 = i0 * 327680 + i1 * 1280
cse_var_1: T.int32 = cse_var_2 + i2
matmul_1 = T.Buffer((T.int64(655360),), data=matmul.data)
if k == 0:
matmul_1[cse_var_1] = T.float32(0)
A_1 = T.Buffer((T.int64(655360),), data=A.data)
B_1 = T.Buffer((T.int64(1638400),), data=B.data)
matmul_1[cse_var_1] = matmul_1[cse_var_1] + A_1[cse_var_2 + k] * B_1[k * 1280 + i2]

[felixslu]

I wonder whether we could get tunning script from your team to tunning sd models by myself.

such as links below:
mlc-ai/mlc-llm@8aeb3df#diff-643d01e2455cf9344c3c81c40c42c8d6aad9cd7ad207aa72712c0b1556c2d014
mlc_llm/tuning.py

[TigerVersusT]

@felixslu any solution to this problem? I get the same error.