This repository provides guidance on how to:
Convert DICOM files into padded 1024*1024 PNG images. Deploy image segmentation and diagnostic report generation models (medImageParse and CXRReportGen) from the Azure Machine Learning (AML) model catalog. Create a real-time endpoint for generating diagnostic reports for chest X-rays (CXR) and identifying abnormal pathological images in chest CT scans with segmentation.
此repo指导如何将DICOM文件转为padded 1024*1024 png图片, 在Azure Machine Learning (AML)的model catalog部署图像分割模型和诊断报告生成模型medImageParse and CXRReportGen, 并创建realtime endpoint,生成CXR诊断报告和胸部等CT的异常病理图像识别和分割.
上述创建的endpoint可以在本repo的notebook里代码方式使用, AML没有playground.
如果直接在Azure Foundry Portal(AI Studio) model catalog里创建endpoint, 则可以直接在AI Foundry Healthcare Playground里UI方式使用. AML和AI Foundry不能互相引用.
Key Features:
1.DICOM to PNG Conversion: Transforms DICOM files into padded 1024*1024 PNG format required by Microsoft's medImageParse healthcare AI image segmentation model.
Endpoint Usage:
Via AML Notebooks: The endpoint created in AML can be utilized programmatically using code provided in this repository's notebooks. Note that AML does not offer a playground for direct UI interactions.
Via Azure Foundry Portal (AI Studio): Endpoints created in the AI Foundry Portal (model catalog) can be directly accessed through the UI in the AI Foundry Healthcare Playground.
AML and AI Foundry endpoints are independent and cannot reference each other.
三个主要功能:
1.将DICOM文件转为符合微软healthcareai图片分割模型medimagaparse所必须的padded 1024*1024 png图片格式.
2.Use the medImageParse Model for Chest CT Abnormal Pathological Image Identification and Segmentation
2.使用medimageparse模型对胸部CT进行异常病理图像识别分割, 用户输入病理位置以及异常定位的text Prompt, 结果以mask方式展示.
Process:
Users provide the pathological location and abnormality description through a text prompt.
Output:
The model processes the input and displays the results as a mask overlay on the chest CT images, highlighting the identified regions.
3.Further Visualize the Contour of Abnormal Regions
3.可以进一步对异常区域的边缘可视化
Code Execution:
0.Install the az command for logging into your Azure account. Download the Azure CLI for Windows/Ubuntu/Mac from the following link: Azure CLI Installation Guide For Windows, select the MSI installer that includes PowerShell. Log in to your Azure account using the Azure CLI. 代码执行:
- 安装az命令, 用于登录azure账号, 下载windows/ubuntu/mac版本的Azure CLI, 选择带有Powershell的windows安装程序MSI
https://learn.microsoft.com/zh-cn/cli/azure/install-azure-cli
通过Azure CLI登陆到Azure账号
az login
1.Package Installation Create a new Conda environment with Python version >=3.9, <3.12 (Python 3.10 is recommended). Execute the package installation using the following command:
- 代码包安装
创建一个新的conda环境, python>=3.9,<3.12, 推荐3.10 执行package包的安装: pip install -e package
2.Environment Configuration
Deploy medImageParse and CXRReportGen models in the AML model catalog.
Note: The deployment process takes approximately 40 minutes.
2. 配置环境
在AML model catelog里deploy medImageParse, CXRReportGen.部署所需时间40分钟左右.
3.Set Up Your Environment
Create a .env file with the following format for the MIP_MODEL_ENDPOINT:
3. 设置自己的env
.env: MIP_MODEL_ENDPOINT格式如下:
Example format: "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.MachineLearningServices/workspaces/{workspace-name}/onlineEndpoints/{endpoint-name}"
MIP_MODEL_ENDPOINT = "/subscriptions/0d3f39ba-7349-4bd7-8122-649ff18f0a4a/resourceGroups/wanmeng-healthcare/providers/Microsoft.MachineLearningServices/workspaces/wanmeng-7491/onlineEndpoints/wanmeng-7491-yyfba"
resource-group和workspace-name可以在打开模型的URL里找到.
You can find the URL in the opened model details page in the AML model catalog.
- 下载config.json, 从AML workspace首页或AI Studio的project overview首页都可以找到config的下载
config放在项目根目录下面, 会在建立ml_client的时候从本地读取, 否则会与.env认证不符. Download config.json
You can download the config.json file from either the AML Workspace homepage or the AI Studio Project Overview homepage.
Place the config.json file in the root directory of your project.
Note: The config.json file will be read locally when creating the ml_client. Ensure it is correctly placed, as mismatched authentication with .env will cause issues.
5.Execute the Image Segmentation Notebook 5. 执行图像分割notebook: /Users/wanmeng/repository/healthcareai-examples/azureml/medimageparse/medimageparse_segmentation_demo.ipynb
6.Execute the CXR Report Generation
6. 执行CXR报告稿生成:
/Users/wanmeng/repository/healthcareai-examples/azureml/cxrreportgen/cxr-deploy-aml.ipynb
