Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump citylens to 1.16 #559

Merged
merged 6 commits into from
Dec 26, 2024
Merged

Bump citylens to 1.16 #559

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
825ef64
Bump citylens to 1.16
Dec 24, 2024
48a165a
Fix naming style & .Values.pro.baseUrl param description
Dec 24, 2024
70a4407
Bump to 1.16.2 & fix readme
Dec 24, 2024
858244b
Add required to map matching host & key if map matching enabled
Dec 24, 2024
40d5125
Citylens 1.16.3: map matcher: process only new tracks if map matching…
Dec 24, 2024
b1a7f4d
Allow to overide default timeout for kafka message producer
Dec 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/citylens/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: application
description: A Helm chart for Kubernetes to deploy Citylens service

version: 1.32.0
appVersion: 1.15.0
appVersion: 1.16.1

maintainers:
- name: 2gis
Expand Down
39 changes: 30 additions & 9 deletions charts/citylens/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ See the [documentation]() to learn about:
| Name | Description | Value |
| ---------------------- | ------------ | ------------------------------ |
| `api.image.repository` | Repository. | `2gis-on-premise/citylens-api` |
| `api.image.tag` | Tag. | `1.15.0` |
| `api.image.tag` | Tag. | `1.16.1` |
| `api.image.pullPolicy` | Pull Policy. | `IfNotPresent` |

### Resources settings
Expand Down Expand Up @@ -133,7 +133,7 @@ See the [documentation]() to learn about:
| Name | Description | Value |
| ---------------------- | ------------ | ------------------------------ |
| `web.image.repository` | Repository. | `2gis-on-premise/citylens-web` |
| `web.image.tag` | Tag. | `1.15.0` |
| `web.image.tag` | Tag. | `1.16.1` |
| `web.image.pullPolicy` | Pull Policy. | `IfNotPresent` |

### Resources settings
Expand Down Expand Up @@ -185,11 +185,12 @@ See the [documentation]() to learn about:

### Custom settings

| Name | Description | Value |
| -------------------- | ------------------------------------------------------------------------ | -------------- |
| `web.logLevel` | Log level. | `WARNING` |
| `web.metricsAppName` | Value for service prometheus metrics label "app_name". | `citylens-web` |
| `web.pgPoolSize` | Postgres: maximum number of connections in connections pool to maintain. | `5` |
| Name | Description | Value |
| ----------------------- | ------------------------------------------------------------------------------------------- | -------------- |
| `web.logLevel` | Log level. | `WARNING` |
| `web.metricsAppName` | Value for service prometheus metrics label "app_name". | `citylens-web` |
| `web.pgPoolSize` | Postgres: maximum number of connections in connections pool to maintain. | `5` |
| `web.pgPoolMaxOverflow` | Postgres: maximum number of extra connections in connections pool (relative of pgPoolSize). | `10` |

### Metadata settings

Expand Down Expand Up @@ -341,6 +342,26 @@ See the [documentation]() to learn about:
| `worker.tracksUploader.tolerations` | Kubernetes pod [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |
| `worker.tracksUploader.affinity` | Kubernetes pod [affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity) settings. | `{}` |

### Citylens Map Matcher worker's settings

| Name | Description | Value |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `worker.mapMatcher.enabled` | If Map Matcher worker is enabled for the service. | `false` |
| `worker.mapMatcher.replicas` | A replica count for the pod. | `1` |
| `worker.mapMatcher.revisionHistoryLimit` | Revision history limit (used for [rolling back](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) a deployment). | `3` |
| `worker.mapMatcher.host` | Map Matching API address. Ex.: http://navi-front.svc | `""` |
| `worker.mapMatcher.key` | Map Matching API key. | `""` |
| `worker.mapMatcher.interpolation` | Set to `true` for compatibility with Map Matching API which requires gps points timestamps in seconds as integers (w/o millisecond precision support). | `true` |
| `worker.mapMatcher.retries` | Total number of retries in case of Map Matching API unavailability/errors. | `3` |
| `worker.mapMatcher.timeoutSeconds` | Map Matching API timeout, seconds. | `30` |
| `worker.mapMatcher.annotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `worker.mapMatcher.labels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `worker.mapMatcher.podAnnotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `worker.mapMatcher.podLabels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `worker.mapMatcher.nodeSelector` | Kubernetes pod [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
| `worker.mapMatcher.tolerations` | Kubernetes pod [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |
| `worker.mapMatcher.affinity` | Kubernetes pod [affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity) settings. | `{}` |

### Citylens Dashboard batch events worker's settings

| Name | Description | Value |
Expand All @@ -352,7 +373,7 @@ See the [documentation]() to learn about:
| Name | Description | Value |
| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
| `worker.dashboardBatchEvents.image.repository` | Repository. | `2gis-on-premise/citylens-workers` |
| `worker.dashboardBatchEvents.image.tag` | Tag. | `1.15.0` |
| `worker.dashboardBatchEvents.image.tag` | Tag. | `1.16.1` |
| `worker.dashboardBatchEvents.image.pullPolicy` | Pull Policy. | `IfNotPresent` |
| `worker.dashboardBatchEvents.logLevel` | Worker's log level. | `INFO` |
| `worker.dashboardBatchEvents.revisionHistoryLimit` | Revision history limit (used for [rolling back](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) a deployment). | `3` |
Expand All @@ -370,7 +391,7 @@ See the [documentation]() to learn about:
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
| `migrations.enabled` | If migrations needed. | `true` |
| `migrations.image.repository` | Repository. | `2gis-on-premise/citylens-database` |
| `migrations.image.tag` | Tag. | `1.15.0` |
| `migrations.image.tag` | Tag. | `1.16.1` |
| `migrations.image.pullPolicy` | Pull Policy | `IfNotPresent` |
| `migrations.resources.requests.cpu` | A CPU request. | `100m` |
| `migrations.resources.requests.memory` | A memory request. | `1Gi` |
Expand Down
14 changes: 14 additions & 0 deletions charts/citylens/templates/helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ Expand the name of the chart.
{{ include "citylens.name" . }}-logs-saver
{{- end }}

{{- define "citylens.map-matcher.name" -}}
{{ include "citylens.name" . }}-map-matcher
{{- end }}

{{- define "citylens.predictions-saver.name" -}}
{{ include "citylens.name" . }}-predictions-saver
{{- end }}
Expand Down Expand Up @@ -112,6 +116,16 @@ app.kubernetes.io/instance: {{ include "citylens.logs-saver.name" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}

{{- define "citylens.map-matcher.selectorLabels" -}}
app.kubernetes.io/name: {{ .Release.Name }}
app.kubernetes.io/instance: {{ include "citylens.map-matcher.name" . }}
{{- end }}

{{- define "citylens.map-matcher.labels" -}}
{{ include "citylens.map-matcher.selectorLabels" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}

{{- define "citylens.predictions-saver.selectorLabels" -}}
app.kubernetes.io/name: {{ .Release.Name }}
app.kubernetes.io/instance: {{ include "citylens.predictions-saver.name" . }}
Expand Down
11 changes: 11 additions & 0 deletions charts/citylens/templates/web/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ data:
db_connections:
postgres: {{ include "citylens.pgDSN" . | squote }}
postgres_pool_size: {{ .Values.web.pgPoolSize }}
postgres_pool_max_overflow: {{ .Values.web.pgPoolMaxOverflow }}
kafka:
{{- with .Values.kafka }}
bootstrap_servers: {{ required "A valid .Values.kafka.bootstrap.servers entry required" .bootstrapServer | squote }}
Expand Down Expand Up @@ -107,6 +108,16 @@ data:
coords: [{{ join "," .coords }}]
{{- end }}
initial_project: {{ .Values.map.initialProject | squote }}
{{- with .Values.worker.mapMatcher }}
matching:
enabled: {{ .enabled }}
host: {{ .host }}
key: {{ .key }}
interpolation: {{ .interpolation }}
retries_total: {{ .retries }}
retries_backoff_factor: 0.8
timeout: {{ .timeoutSeconds }}
{{- end }}
register_predictor_ttl_seconds: 600
enabled_modules:
header_links:
Expand Down
81 changes: 81 additions & 0 deletions charts/citylens/templates/web/workers/deployment-map-matcher.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
{{- if .Values.worker.mapMatcher.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "citylens.map-matcher.name" . }}
{{- if .Values.worker.mapMatcher.annotations }}
annotations:
{{- toYaml .Values.worker.mapMatcher.annotations | nindent 4 }}
{{- end }}
labels:
{{- include "citylens.map-matcher.labels" . | nindent 4 }}
{{- if .Values.worker.mapMatcher.labels }}
{{- toYaml .Values.worker.mapMatcher.labels | nindent 4 }}
{{- end }}
spec:
replicas: {{ .Values.worker.mapMatcher.replicas }}
revisionHistoryLimit: {{ .Values.worker.mapMatcher.revisionHistoryLimit }}
selector:
matchLabels:
{{- include "citylens.map-matcher.selectorLabels" . | nindent 6 }}
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
annotations:
checksum/config: {{ include (print $.Template.BasePath "/web/configmap.yaml") . | sha256sum }}
checksum/custom-ca: {{ include "citylens.checksum" (merge (dict "path" "/custom-ca.configmap.yaml") $) }}
{{- with .Values.worker.mapMatcher.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "citylens.map-matcher.labels" . | nindent 8 }}
spec:
containers:
- name: {{ include "citylens.map-matcher.name" . }}
image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.web.image.repository }}:{{ .Values.web.image.tag }}
imagePullPolicy: {{ .Values.web.image.pullPolicy }}
command: [ "/bin/sh", "-c" ]
args: [ "flask map_matching match" ]
env:
- name: CONFIG_PATH
value: /opt/worker/config/dashboard_config.yaml
{{- if $.Values.customCAs.bundle }}
- name: SSL_CERT_DIR
value: {{ include "citylens.customCA.mountPath" $ }}
{{- end }}
resources:
{{- toYaml .Values.worker.resources | nindent 12 }}
volumeMounts:
- name: config-volume
mountPath: /opt/worker/config
{{- if $.Values.customCAs.bundle }}
- mountPath: {{ include "citylens.customCA.mountPath" $ }}
name: custom-ca
readOnly: true
{{- end }}
volumes:
- name: config-volume
configMap:
name: {{ include "citylens.web.name" . }}-configmap
{{- if $.Values.customCAs.bundle }}
- name: custom-ca
configMap:
name: {{ include "citylens.name" $ }}-custom-ca-configmap
{{- end }}
{{- with .Values.worker.mapMatcher.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.worker.mapMatcher.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.worker.mapMatcher.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
1 change: 1 addition & 0 deletions charts/citylens/templates/workers/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ data:
s3_endpoint: {{ required "A valid .Values.s3.endpoint entry required" .endpoint | squote }}
bucket_prefix: {{ required "A valid .Values.s3.bucketPrefix entry required" .bucketPrefix | squote }}
frame_key_template: {{ include "citylens.s3_constants.frame_key_template" . | squote }}
crop_frame_key_template: {{ include "citylens.s3_constants.crop_frame_key_template" . | squote }}
{{- end }}
topics:
{{- with .Values.kafka.topics }}
Expand Down
56 changes: 52 additions & 4 deletions charts/citylens/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ api:
image:
repository: 2gis-on-premise/citylens-api
pullPolicy: IfNotPresent
tag: 1.15.0
tag: 1.16.1

replicas: 4

Expand Down Expand Up @@ -215,6 +215,7 @@ api:
# @param web.logLevel Log level.
# @param web.metricsAppName Value for service prometheus metrics label "app_name".
# @param web.pgPoolSize Postgres: maximum number of connections in connections pool to maintain.
# @param web.pgPoolMaxOverflow Postgres: maximum number of extra connections in connections pool (relative of pgPoolSize).

# @section Metadata settings

Expand All @@ -231,7 +232,7 @@ web:
image:
repository: 2gis-on-premise/citylens-web
pullPolicy: IfNotPresent
tag: 1.15.0
tag: 1.16.1

replicas: 1

Expand Down Expand Up @@ -279,6 +280,7 @@ web:
pkce: false

pgPoolSize: 5
pgPoolMaxOverflow: 10

logLevel: WARNING
metricsAppName: citylens-web
Expand Down Expand Up @@ -584,6 +586,52 @@ worker:
tolerations: {}


# @section Citylens Map Matcher worker's settings

# @param worker.mapMatcher.enabled If Map Matcher worker is enabled for the service.

# @param worker.mapMatcher.replicas A replica count for the pod.

# @param worker.mapMatcher.revisionHistoryLimit Revision history limit (used for [rolling back](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) a deployment).

# @param worker.mapMatcher.host Map Matching API address. Ex.: http://navi-front.svc
# @param worker.mapMatcher.key Map Matching API key.
# @param worker.mapMatcher.interpolation Set to `true` for compatibility with Map Matching API which requires gps points timestamps in seconds as integers (w/o millisecond precision support).
# @param worker.mapMatcher.retries Total number of retries in case of Map Matching API unavailability/errors.
# @param worker.mapMatcher.timeoutSeconds Map Matching API timeout, seconds.

# @param worker.mapMatcher.annotations Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/).
# @param worker.mapMatcher.labels Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).
# @param worker.mapMatcher.podAnnotations Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/).
# @param worker.mapMatcher.podLabels Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).

# @param worker.mapMatcher.nodeSelector Kubernetes pod [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector).
# @param worker.mapMatcher.tolerations Kubernetes pod [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings.
# @param worker.mapMatcher.affinity Kubernetes pod [affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity) settings.

mapMatcher:

enabled: false

replicas: 1

host: ''
dbelyaev-nsk marked this conversation as resolved.
Show resolved Hide resolved
key: ''
interpolation: true
retries: 3
timeoutSeconds: 30

revisionHistoryLimit: 3

annotations: {}
labels: {}
podAnnotations: {}
podLabels: {}
nodeSelector: {}
affinity: {}
tolerations: {}


# @section Citylens Dashboard batch events worker's settings

# @param worker.dashboardBatchEvents.replicas A replica count for the pod.
Expand Down Expand Up @@ -614,7 +662,7 @@ worker:
image:
repository: 2gis-on-premise/citylens-workers
pullPolicy: IfNotPresent
tag: 1.15.0
tag: 1.16.1

revisionHistoryLimit: 3

Expand Down Expand Up @@ -650,7 +698,7 @@ migrations:
image:
repository: 2gis-on-premise/citylens-database
pullPolicy: IfNotPresent
tag: 1.15.0
tag: 1.16.1

resources:
requests:
Expand Down
Loading