Skip to content

Commit

Permalink
Merge pull request #1416 from qdrant/docs-gpu-tweaks
Browse files Browse the repository at this point in the history
Minor GPU documentation tweaks
  • Loading branch information
timvisee authored Jan 23, 2025
2 parents 0b0ac76 + 60d72a7 commit cd26e40
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions qdrant-landing/content/documentation/guides/running-with-GPU.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ weight: 10

Starting from version v1.13.0, Qdrant offers support for GPU acceleration.

However, GPU support is not included in the default Qdrant binary due to additional dependencies and libraries. Instead, you will need to use dedicated Docker images with GPU support.
However, GPU support is not included in the default Qdrant binary due to additional dependencies and libraries. Instead, you will need to use dedicated Docker ([NVIDIA](#nvidia-gpus), [AMD](#amd-gpus)) images with GPU support.


## Configuration
Expand Down Expand Up @@ -64,7 +64,7 @@ For standalone usage, you can build Qdrant with GPU support by running the follo
cargo build --release --features gpu
```

Ensure your device supports Vulkan API v1.3. This enables compatibility with Apple Silicon, Intel GPUs, and CPU emulators.
Ensure your device supports Vulkan API v1.3. This includes compatibility with Apple Silicon, Intel GPUs, and CPU emulators. Note that `gpu.indexing: true` must be set in your configuration to use GPUs at runtime.

## NVIDIA GPUs

Expand All @@ -76,21 +76,21 @@ To use Docker with NVIDIA GPU support, ensure the following are installed on you

Most AI or CUDA images on Amazon/GCP come pre-configured with the NVIDIA container toolkit.

### Docker images with NVidia GPU support
### Docker images with NVIDIA GPU support

Docker images with NVIDIA GPU support use the tag suffix `gpu-nvidia`, e.g., `qdrant/qdrant:v1.13.0-gpu-nvidia`. These images include all necessary dependencies.

To enable GPU support, use the `--gpus=all` flag with Docker settings. Example:

```bash
# `--gpus=all` flag says to Docker that we want to use GPUs.
# `-e QDRANT__GPU__indexing=1` flag says to QDrant that we want to use GPUs for indexing.
# `-e QDRANT__GPU__INDEXING=1` flag says to Qdrant that we want to use GPUs for indexing.
docker run \
--rm \
--gpus=all \
-p 6333:6333 \
-p 6334:6334 \
-e QDRANT__GPU__indexing=1 \
-e QDRANT__GPU__INDEXING=1 \
qdrant/qdrant:gpu-nvidia-latest
```

Expand Down Expand Up @@ -151,14 +151,14 @@ To enable GPU for Docker, you need additional `--device /dev/kfd --device /dev/d

```bash
# `--device /dev/kfd --device /dev/dri` flags say to Docker that we want to use GPUs.
# `-e QDRANT__GPU__indexing=1` flag says to QDrant that we want to use GPUs for indexing.
# `-e QDRANT__GPU__INDEXING=1` flag says to Qdrant that we want to use GPUs for indexing.
docker run \
--rm \
--device /dev/kfd --device /dev/dri \
-e QDRANT__log_level=debug \
-p 6333:6333 \
-p 6334:6334 \
-e QDRANT__GPU__indexing=1 \
-e QDRANT__LOG_LEVEL=debug \
-e QDRANT__GPU__INDEXING=1 \
qdrant/qdrant:gpu-amd-latest
```

Expand All @@ -181,19 +181,20 @@ This concludes the setup. In a basic scenario, you won't need to configure anyth

Due to this limitation, you should not create segments where either original vectors OR quantized vectors are larger than 16GB.

For example, the collection with 1536d vectors and scalar quatization can have at most
For example, a collection with 1536d vectors and scalar quantization can have at most:

```text
16Gb / 1536 ~= 11 million vectors per segment
```

And without quantization
And without quantization:

```text
16Gb / 1536 * 4 ~= 2.7 million vectors per segment
```

Maximal size of the segment can be configured in collection settings.
The maximum size of each segment can be configured in the collection settings.
Use the following operation to [change](/documentation/concepts/collections/#update-collection-parameters) on your existing collection:

```http
PATCH collections/{collection_name}
Expand All @@ -204,5 +205,4 @@ PATCH collections/{collection_name}
}
```

Note, that `max_segment_size` is specified in KiloBytes.

Note that `max_segment_size` is specified in KiloBytes.

0 comments on commit cd26e40

Please sign in to comment.