v1.2.5

What's Changed

• docs: Add client code examples without context manager by @Sherlock113 in #4512
• docs: Update docs by @Sherlock113 in #4515
• docs: Add authorization docs by @Sherlock113 in #4517
• docs: Change sample input to one line by @Sherlock113 in #4518
• docs: Update ControlNet use case docs by @Sherlock113 in #4519
• docs: Update the distributed services and get started docs by @Sherlock113 in #4521
• refactor(cli): make CLI commands available as modules by @frostming in #4487
• docs: Refactor BentoCloud docs by @Sherlock113 in #4525
• docs: Update token docs by @Sherlock113 in #4526
• fix: fix list deployment exception by @FogDong in #4522
• feat: add get endpoint url for deployment by @FogDong in #4516
• docs: Add BentoCloud CLI and API ref docs by @Sherlock113 in #4527
• fix: add missing to dict for deployment config by @FogDong in #4529
• feat(client): disallow media type other than json for HTTP clients by @frostming in #4520
• chore: clean unneeded dependencies by @frostming in #4528
• docs: Add call deployment endpoints doc by @Sherlock113 in #4531
• docs: Update all use cases by @Sherlock113 in #4532
• fix: correctly encode ValueError in response by @frostming in #4530
• fix: improve CLI message by @frostming in #4533
• chore(io): allow implicit pydantic type conversion by @aarnphm in #4534
• fix: type convertion of io descriptor by @frostming in #4537
• feat: add back service hooks by @frostming in #4524
• docs: fix typo by @sauyon in #4540
• docs: Add BentoSVD Use case doc by @Sherlock113 in #4535
• doc: Add adaptive batching doc by @Sherlock113 in #4536
• doc: allow reuse of existing directory by @jianshen92 in #4543
• feat: cloud context from env vars by @frostming in #4542
• docs: Update the clients doc by @Sherlock113 in #4539
• docs: Update adaptive batching docs by @Sherlock113 in #4547
• docs: Update vLLM use case doc by @Sherlock113 in #4548
• feat: export component context at top namespace by @frostming in #4551
• feat: use starlette request directly in service context by @frostming in #4552
• feat: mount asgi app and fastapi support by @frostming in #4538
• fix(sdk): build envs into dockerfile by @bojiang in #4546
• fix: rename server context attributes by @frostming in #4553
• fix: crash when pulling bento from organization with proxy mode by @frostming in #4554
• fix: add str in enum for deployment schema by @FogDong in #4549

Full Changelog: v1.2.4...v1.2.5

v1.2.4

What's Changed

• docs: Update the get started docs by @Sherlock113 in #4513
• fix(sdk): incorrect bento_path if not provided by @bojiang in #4514

Full Changelog: v1.2.3...v1.2.4

v1.2.3

What's Changed

• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #4488
• infra: update to use Ruff formatter by @aarnphm in #4269
• feat: tpu_type support by @frostming in #4493
• chore(cloud cli): add cluster and expose access_point by @bojiang in #4490
• docs: Update the whisperx use case by @Sherlock113 in #4495
• docs: Update the readme by @Sherlock113 in #4496
• fix(sdk): current directory for built bentos by @bojiang in #4505
• chore(cloud cli): rename cluster to region by @bojiang in #4508
• doc: Add the lcm lora use case doc by @Sherlock113 in #4510
• fix(sdk): clean bentoml version by @bojiang in #4511
• fix: bug: Dataframes not serializing correctly in the new API by @frostming in #4491

Full Changelog: v1.2.2...v1.2.3

v1.2.2

What's Changed

• docs: Remove direct execution lines by @Sherlock113 in #4480
• docs: Refactor deployment docs by @Sherlock113 in #4482
• docs: Add build options doc by @Sherlock113 in #4483
• feat: lock dependencies by default considering the current bentoml version by @frostming in #4484
• docs: Add distributed services doc by @Sherlock113 in #4485
• fix(server): clean the request resources after the response is consumed by @frostming in #4481

Full Changelog: v1.2.1...v1.2.2

v1.2.1

Full Changelog: v1.2.1a1...v1.2.1

What's Changed

• docs: Add more featured use cases by @Sherlock113 in #4476
• docs: Add API reference doc by @Sherlock113 in #4475
• docs: Clean API ref docs by @Sherlock113 in #4477
• fix: fix the cluster from display name to name and url by @FogDong in #4478
• fix(cli): add start runner server back by @bojiang in #4479

Full Changelog: v1.2.0...v1.2.1

v1.2.1a1

What's Changed

• docs: Add more featured use cases by @Sherlock113 in #4476
• docs: Add API reference doc by @Sherlock113 in #4475
• docs: Clean API ref docs by @Sherlock113 in #4477
• fix: fix the cluster from display name to name and url by @FogDong in #4478
• fix(cli): add start runner server back by @bojiang in #4479

Full Changelog: v1.2.0...v1.2.1a1

v1.2.0

🍱 We are excited to share with you that we have released BentoML v1.2, the biggest release since the launch of v1.0. This release includes improvements from all the learning and feedback from our community over the past year. We invite you to read our release blog post for a comprehensive overview of the new features and the motivations behind their development.

Here are a few key points to note before we delve into the new features:

• v1.2 ensures complete backward compatibility, meaning that Bentos built with v1.1 will continue to function seamlessly with this release.
• We remain committed to supporting v1.1. Critical bug fixes and security updates will be backported to the v1.1 branch.
• BentoML documentation has been updated with examples and guides for v1.2. More guides are being added every week.
• BentoCloud is fully equipped to handle deployments from both v1.1 and v1.2 releases of BentoML.

⛏️ Introduced a simplified service SDK to empower developers with greater control and flexibility.

• Simplified the service and API interfaces as Python classes, allowing developers to add custom logic and use third party libraries flexibly with ease.
• Introduced @bentoml.service and @bentoml.api decorators to customize the behaviors of services and APIs.
• Moved configuration from YAML files to the service decorator @bentoml.service next to the class definition.
• See this example demonstrating the flexibility of the service API by initializing a vLLM AsyncEngine in the service constructor and run inference with continuous batching in the service API.

🔭 Revamped IO descriptors with more familiar input and output types.

• Enable use of Pythonic types directly, without the need for additional IO descriptor definitions or decorations.
• Integrated with Pydantic to leverage its robust validation capabilities and wide array of supported types.
• Expanded support to ML and Generative AI specific IO types.

📦 Updated model saving and loading API to be more generic to enable integration with more ML frameworks.

• Allow flexible saving and loading models using the bentoml.models.create API instead of framework specific APIs, e.g. bentoml.pytorch.save_model, bentoml.tensorflow.save_model.

🚚 Streamlined the deployment workflow to allow more rapid development iterations and a faster time to production.

• Enabled direct deployment to production through CLI and Python API from Git projects.

🎨 Improved API development experience with generated web UI and rich Python client.

• All bentos are now accompanied by a custom-generated UI in the BentoCloud Playground, tailored to their API definitions.
• BentoClient offers a Pythonic way to invoke the service endpoint, allowing parameters to be supplied in native Python format, letting the client efficiently handles the necessary serialization while ensuring compatibility and performance.

🎭 We've learned that the best way to showcase what BentoML can do is not through dry, conceptual documentation but through real-world examples. Check out our current list of examples, and we'll continue to publish new ones to the gallery as exciting new models are released.

• BentoVLLM
• BentoControlNet
• BentoSDXLTurbo
• BentoWhisperX
• BentoXTTS
• BentoCLIP

🙏 Thank you for your continued support!

What's Changed

• chore(deps): bump h2 from 0.3.20 to 0.3.24 in /grpc-client/rust by @dependabot in #4434
• fix: Remove trailing character when building bento with API on Windows by @holzweber in #4455
• fix: Replace backslahes by normal slashes, making bentoml pull possible on windows by @holzweber in #4456
• fix(monitoring): Missing f string by @jianshen92 in #4463
• feat: 1.2 staging by @bojiang in #4366
• chore(deps): bump pdm-project/setup-pdm from 3 to 4 by @dependabot in #4457
• fix(client): Convert string to Path if it isn't like a URL by @frostming in #4469
• docs: Add Model Store doc by @Sherlock113 in #4471
• docs: Clean up BentoCloud doc and add get started doc by @Sherlock113 in #4472
• docs: Update the quickstart by @Sherlock113 in #4474

New Contributors

• @holzweber made their first contribution in #4455

Full Changelog: v1.2.0rc1...v1.2.0

v1.2.0rc1

What's Changed

• fix: correct schema for optional type by @frostming in #4459
• fix: client sending request fail if the input is a list of files by @frostming in #4461
• fix: worker number of start-http-server by @bojiang in #4462

Full Changelog: v1.2.0a7...v1.2.0rc1

v1.2.0a7

What's Changed

• feat: allow disabling GPU allocation via env by @frostming in #4453

Full Changelog: v1.2.0a6...v1.2.0a7

v1.2.0a6

What's Changed

• feat: add adaptive batch size histogram by @frostming in #4438
• ci: Use nox as the task runner by @frostming in #4432
• feat(cli): remove or hide unneeded commands by @frostming in #4441
• docs: Add whisperx use case doc by @Sherlock113 in #4436
• docs: Add Docker deployment doc by @Sherlock113 in #4443
• docs: Add io types doc by @Sherlock113 in #4444
• fix: only inline display images by @frostming in #4442
• docs: Update homepage by @Sherlock113 in #4445
• feat: skip access log for infra endpoints by @frostming in #4447
• feat: add schema in bentoInfo by @xianml in #4439
• fix: fix deployment cli log by @FogDong in #4450
• fix: convert generator to async gen before response by @frostming in #4448

Full Changelog: v1.2.0a5...v1.2.0a6