From 563b3e73c126a56dcabc8d2b17bf6a27347e37ff Mon Sep 17 00:00:00 2001 From: Scott Todd Date: Wed, 23 Oct 2024 08:02:15 -0700 Subject: [PATCH] Document new external ONNX model and linalg operator test suites. (#18819) Progress on https://github.com/iree-org/iree-test-suites/issues/2 and https://github.com/iree-org/iree-test-suites/issues/6 . --- .../developers/debugging/model-development.md | 2 +- .../docs/developers/general/testing-guide.md | 21 +++++++++++++++++++ .../website/docs/guides/ml-frameworks/onnx.md | 3 ++- 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/docs/website/docs/developers/debugging/model-development.md b/docs/website/docs/developers/debugging/model-development.md index ea7bbbae0116..c7ba61a71ae3 100644 --- a/docs/website/docs/developers/debugging/model-development.md +++ b/docs/website/docs/developers/debugging/model-development.md @@ -147,7 +147,7 @@ if crashes, bugs, numerical issues, etc. can be reproduced at that scale. Some existing test suites can be found at these locations: * -* +* * * * diff --git a/docs/website/docs/developers/general/testing-guide.md b/docs/website/docs/developers/general/testing-guide.md index 070085dae253..50957e24e38f 100644 --- a/docs/website/docs/developers/general/testing-guide.md +++ b/docs/website/docs/developers/general/testing-guide.md @@ -412,6 +412,18 @@ repository also contains tests for many machine learning models. Some of these tests are planned to be migrated into [iree-org/iree-test-suites](https://github.com/iree-org/iree-test-suites). +### linalg operator tests + +Tests for operators in the MLIR linalg dialect like `matmul`, and `convolution` +are being migrated from folders like +[`tests/e2e/matmul/`](https://github.com/iree-org/iree/tree/main/tests/e2e/matmul) +in the +[iree-org/iree](https://github.com/iree-org/iree) repository to +[`linalg_ops/`](https://github.com/iree-org/iree-test-suites/tree/main/linalg_ops) +in the +[iree-org/iree-test-suites](https://github.com/iree-org/iree-test-suites) +repository. + ### ONNX operator tests Tests for individual ONNX operators are included at @@ -483,3 +495,12 @@ The workflow job that failed should then upload a new config file as an committed: ![image](https://github.com/user-attachments/assets/b5dbdcb4-4c0a-4ff2-adc6-9021614179b2) + +### ONNX model tests + +Tests for ONNX models are included at +[`onnx_models/`](https://github.com/iree-org/iree-test-suites/tree/main/onnx_models) +in the +[iree-org/iree-test-suites](https://github.com/iree-org/iree-test-suites) +repository. These tests use models from the upstream +[onnx/models](https://github.com/onnx/models) repository. diff --git a/docs/website/docs/guides/ml-frameworks/onnx.md b/docs/website/docs/guides/ml-frameworks/onnx.md index 5abe0112f862..d2dda65fe9f7 100644 --- a/docs/website/docs/guides/ml-frameworks/onnx.md +++ b/docs/website/docs/guides/ml-frameworks/onnx.md @@ -118,7 +118,8 @@ graph LR | Code samples | | | -- | -- | Generated op tests | [iree-test-suites `onnx_ops`](https://github.com/iree-org/iree-test-suites/tree/main/onnx_ops) -Curated op and model tests | [SHARK-TestSuite `e2eshark/onnx`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/e2eshark/onnx) +Public model tests | [iree-test-suites `onnx_models`](https://github.com/iree-org/iree-test-suites/tree/main/onnx_models) +Curated op and model tests | SHARK-TestSuite [`e2eshark/onnx`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/e2eshark/onnx) and [`alt_e2eshark/onnx_tests`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/alt_e2eshark/onnx_tests) Importer tests | [torch-mlir `test/python/onnx_importer`](https://github.com/llvm/torch-mlir/tree/main/test/python/onnx_importer) ## :octicons-question-16: Troubleshooting