Refactor app_tests/integration_tests/llm for easy addition of more models for testing

[renxida]

Bundled up loose parameter passing in the testing infra to support support easily adding models from HF, Azure, and local like:

TEST_MODELS = {
    "open_llama_3b": ModelConfig(
        source=ModelSource.HUGGINGFACE,
        repo_id="SlyEcho/open_llama_3b_v2_gguf",
        model_file="open-llama-3b-v2-f16.gguf",
        tokenizer_id="openlm-research/open_llama_3b_v2",
        batch_sizes=(1, 4),
        device_settings=device_settings.CPU,
    ),
    "llama3.1_8b": ModelConfig(
        source=ModelSource.LOCAL,
        local_path=Path("/data/llama3.1/8b/llama8b_f16.irpa"),
        model_file="llama8b_f16.irpa",
        tokenizer_id="NousResearch/Meta-Llama-3.1-8B",
        batch_sizes=(1, 4),
        device_settings=device_settings.CPU,
    ),
    "azure_llama": ModelConfig(
        source=ModelSource.AZURE,
        azure_config=AzureConfig(
            account_name="sharkblobs",
            container_name="halo-models",
            blob_path="llm-dev/llama3_8b/8b_f16.irpa",
        ),
        model_file="azure-llama.irpa",
        tokenizer_id="openlm-research/open_llama_3b_v2",
        batch_sizes=(1, 4),
        device_settings=device_settings.CPU,
    ),
}

Sharktank has a similar list of models in hf_datasets.py that only supports huggingface.

Might be of interest to draw from that at some point.

[renxida]

BTW CI is actually passing, I just forgot to re-xfail the concur tests we previously xfailed.