Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump citylens to 1.16 #559

Merged
merged 6 commits into from
Dec 26, 2024
Merged

Bump citylens to 1.16 #559

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
825ef64
Bump citylens to 1.16
Dec 24, 2024
48a165a
Fix naming style & .Values.pro.baseUrl param description
Dec 24, 2024
70a4407
Bump to 1.16.2 & fix readme
Dec 24, 2024
858244b
Add required to map matching host & key if map matching enabled
Dec 24, 2024
40d5125
Citylens 1.16.3: map matcher: process only new tracks if map matching…
Dec 24, 2024
b1a7f4d
Allow to overide default timeout for kafka message producer
Dec 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/citylens/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: application
description: A Helm chart for Kubernetes to deploy Citylens service

version: 1.32.0
appVersion: 1.15.0
appVersion: 1.16.4

maintainers:
- name: 2gis
Expand Down
42 changes: 32 additions & 10 deletions charts/citylens/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ See the [documentation]() to learn about:
| Name | Description | Value |
| ---------------------- | ------------ | ------------------------------ |
| `api.image.repository` | Repository. | `2gis-on-premise/citylens-api` |
| `api.image.tag` | Tag. | `1.15.0` |
| `api.image.tag` | Tag. | `1.16.4` |
| `api.image.pullPolicy` | Pull Policy. | `IfNotPresent` |

### Resources settings
Expand Down Expand Up @@ -133,7 +133,7 @@ See the [documentation]() to learn about:
| Name | Description | Value |
| ---------------------- | ------------ | ------------------------------ |
| `web.image.repository` | Repository. | `2gis-on-premise/citylens-web` |
| `web.image.tag` | Tag. | `1.15.0` |
| `web.image.tag` | Tag. | `1.16.4` |
| `web.image.pullPolicy` | Pull Policy. | `IfNotPresent` |

### Resources settings
Expand Down Expand Up @@ -185,11 +185,12 @@ See the [documentation]() to learn about:

### Custom settings

| Name | Description | Value |
| -------------------- | ------------------------------------------------------------------------ | -------------- |
| `web.logLevel` | Log level. | `WARNING` |
| `web.metricsAppName` | Value for service prometheus metrics label "app_name". | `citylens-web` |
| `web.pgPoolSize` | Postgres: maximum number of connections in connections pool to maintain. | `5` |
| Name | Description | Value |
| ----------------------- | ------------------------------------------------------------------------------------------- | -------------- |
| `web.logLevel` | Log level. | `WARNING` |
| `web.metricsAppName` | Value for service prometheus metrics label "app_name". | `citylens-web` |
| `web.pgPoolSize` | Postgres: maximum number of connections in connections pool to maintain. | `5` |
| `web.pgPoolMaxOverflow` | Postgres: maximum number of extra connections in connections pool (relative of pgPoolSize). | `10` |

### Metadata settings

Expand Down Expand Up @@ -341,6 +342,26 @@ See the [documentation]() to learn about:
| `worker.tracksUploader.tolerations` | Kubernetes pod [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |
| `worker.tracksUploader.affinity` | Kubernetes pod [affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity) settings. | `{}` |

### Citylens Map Matcher worker's settings

| Name | Description | Value |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `worker.mapMatcher.enabled` | If Map Matcher worker is enabled for the service. | `false` |
| `worker.mapMatcher.replicas` | A replica count for the pod. | `1` |
| `worker.mapMatcher.revisionHistoryLimit` | Revision history limit (used for [rolling back](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) a deployment). | `3` |
| `worker.mapMatcher.baseUrl` | Map Matching API address. Ex.: http://navi-front.svc **Required** | `""` |
| `worker.mapMatcher.key` | Map Matching API key. **Required** | `""` |
| `worker.mapMatcher.interpolation` | Set to `true` for compatibility with Map Matching API which requires gps points timestamps in seconds as integers (w/o millisecond precision support). | `true` |
| `worker.mapMatcher.retries` | Total number of retries in case of Map Matching API unavailability/errors. | `3` |
| `worker.mapMatcher.timeoutSeconds` | Map Matching API timeout, seconds. | `30` |
| `worker.mapMatcher.annotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `worker.mapMatcher.labels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `worker.mapMatcher.podAnnotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `worker.mapMatcher.podLabels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `worker.mapMatcher.nodeSelector` | Kubernetes pod [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
| `worker.mapMatcher.tolerations` | Kubernetes pod [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |
| `worker.mapMatcher.affinity` | Kubernetes pod [affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity) settings. | `{}` |

### Citylens Dashboard batch events worker's settings

| Name | Description | Value |
Expand All @@ -352,7 +373,7 @@ See the [documentation]() to learn about:
| Name | Description | Value |
| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
| `worker.dashboardBatchEvents.image.repository` | Repository. | `2gis-on-premise/citylens-workers` |
| `worker.dashboardBatchEvents.image.tag` | Tag. | `1.15.0` |
| `worker.dashboardBatchEvents.image.tag` | Tag. | `1.16.4` |
| `worker.dashboardBatchEvents.image.pullPolicy` | Pull Policy. | `IfNotPresent` |
| `worker.dashboardBatchEvents.logLevel` | Worker's log level. | `INFO` |
| `worker.dashboardBatchEvents.revisionHistoryLimit` | Revision history limit (used for [rolling back](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) a deployment). | `3` |
Expand All @@ -370,7 +391,7 @@ See the [documentation]() to learn about:
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
| `migrations.enabled` | If migrations needed. | `true` |
| `migrations.image.repository` | Repository. | `2gis-on-premise/citylens-database` |
| `migrations.image.tag` | Tag. | `1.15.0` |
| `migrations.image.tag` | Tag. | `1.16.1` |
| `migrations.image.pullPolicy` | Pull Policy | `IfNotPresent` |
| `migrations.resources.requests.cpu` | A CPU request. | `100m` |
| `migrations.resources.requests.memory` | A memory request. | `1Gi` |
Expand All @@ -385,6 +406,7 @@ See the [documentation]() to learn about:
| `kafka.bootstrapServer` | A Kafka broker endpoint. **Required** | `""` |
| `kafka.username` | A Kafka username for connection. **Required** | `""` |
| `kafka.password` | A Kafka password for connection. **Required** | `""` |
| `kafka.produceTimeoutSeconds` | Timeout for producer. | `5` |
| `kafka.topics.frames` | List of topics for Frames saver worker. **Required** | `""` |
| `kafka.topics.tracks` | List of topics for Tracks metadata worker. **Required** | `""` |
| `kafka.topics.pro` | Topic for frames synchronization with Pro (used by Reporter pro worker). **Required** | `""` |
Expand Down Expand Up @@ -445,7 +467,7 @@ See the [documentation]() to learn about:

| Name | Description | Value |
| ------------------- | ----------------------------------------------------------------------------------------------------- | ------ |
| `pro.baseUrl` | PRO API URL (used for filters actualization). Ex: http(s)://pro-api.svc/your_asset_name/filters | `""` |
| `pro.baseUrl` | PRO API URL. Ex: http(s)://pro-api.svc | `""` |
| `pro.key` | PRO API auth token | `""` |
| `pro.verifySsl` | Set to `false` if pro.baseUrl must be accessed via https without certificate validation. **Required** | `true` |
| `pro.framesAssetId` | PRO frames asset id (used for filters actualization). Ex: your_asset_name | `""` |
Expand Down
14 changes: 14 additions & 0 deletions charts/citylens/templates/helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ Expand the name of the chart.
{{ include "citylens.name" . }}-logs-saver
{{- end }}

{{- define "citylens.map-matcher.name" -}}
{{ include "citylens.name" . }}-map-matcher
{{- end }}

{{- define "citylens.predictions-saver.name" -}}
{{ include "citylens.name" . }}-predictions-saver
{{- end }}
Expand Down Expand Up @@ -112,6 +116,16 @@ app.kubernetes.io/instance: {{ include "citylens.logs-saver.name" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}

{{- define "citylens.map-matcher.selectorLabels" -}}
app.kubernetes.io/name: {{ .Release.Name }}
app.kubernetes.io/instance: {{ include "citylens.map-matcher.name" . }}
{{- end }}

{{- define "citylens.map-matcher.labels" -}}
{{ include "citylens.map-matcher.selectorLabels" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}

{{- define "citylens.predictions-saver.selectorLabels" -}}
app.kubernetes.io/name: {{ .Release.Name }}
app.kubernetes.io/instance: {{ include "citylens.predictions-saver.name" . }}
Expand Down
16 changes: 15 additions & 1 deletion charts/citylens/templates/web/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,11 @@ data:
db_connections:
postgres: {{ include "citylens.pgDSN" . | squote }}
postgres_pool_size: {{ .Values.web.pgPoolSize }}
postgres_pool_max_overflow: {{ .Values.web.pgPoolMaxOverflow }}
kafka:
{{- with .Values.kafka }}
bootstrap_servers: {{ required "A valid .Values.kafka.bootstrap.servers entry required" .bootstrapServer | squote }}
bootstrap_servers: {{ required "A valid .Values.kafka.bootstrap.servers entry required" .bootstrapServer | squote }}
producer_timeout_seconds: {{ .produceTimeoutSeconds }}
security:
security_protocol: 'SASL_PLAINTEXT'
sasl_mechanism: 'SCRAM-SHA-512'
Expand Down Expand Up @@ -107,6 +109,18 @@ data:
coords: [{{ join "," .coords }}]
{{- end }}
initial_project: {{ .Values.map.initialProject | squote }}
{{- with .Values.worker.mapMatcher }}
{{- if .enabled }}
matching:
enabled: {{ .enabled }}
host: {{ required "A valid .Values.worker.mapMatcher.baseUrl entry required" .baseUrl }}
key: {{ required "A valid .Values.worker.mapMatcher.key entry required" .key }}
interpolation: {{ .interpolation }}
retries_total: {{ .retries }}
retries_backoff_factor: 0.8
timeout: {{ .timeoutSeconds }}
{{- end }}
{{- end }}
register_predictor_ttl_seconds: 600
enabled_modules:
header_links:
Expand Down
81 changes: 81 additions & 0 deletions charts/citylens/templates/web/workers/deployment-map-matcher.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
{{- if .Values.worker.mapMatcher.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "citylens.map-matcher.name" . }}
{{- if .Values.worker.mapMatcher.annotations }}
annotations:
{{- toYaml .Values.worker.mapMatcher.annotations | nindent 4 }}
{{- end }}
labels:
{{- include "citylens.map-matcher.labels" . | nindent 4 }}
{{- if .Values.worker.mapMatcher.labels }}
{{- toYaml .Values.worker.mapMatcher.labels | nindent 4 }}
{{- end }}
spec:
replicas: {{ .Values.worker.mapMatcher.replicas }}
revisionHistoryLimit: {{ .Values.worker.mapMatcher.revisionHistoryLimit }}
selector:
matchLabels:
{{- include "citylens.map-matcher.selectorLabels" . | nindent 6 }}
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
annotations:
checksum/config: {{ include (print $.Template.BasePath "/web/configmap.yaml") . | sha256sum }}
checksum/custom-ca: {{ include "citylens.checksum" (merge (dict "path" "/custom-ca.configmap.yaml") $) }}
{{- with .Values.worker.mapMatcher.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "citylens.map-matcher.labels" . | nindent 8 }}
spec:
containers:
- name: {{ include "citylens.map-matcher.name" . }}
image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.web.image.repository }}:{{ .Values.web.image.tag }}
imagePullPolicy: {{ .Values.web.image.pullPolicy }}
command: [ "/bin/sh", "-c" ]
args: [ "flask map_matching match" ]
env:
- name: CONFIG_PATH
value: /opt/worker/config/dashboard_config.yaml
{{- if $.Values.customCAs.bundle }}
- name: SSL_CERT_DIR
value: {{ include "citylens.customCA.mountPath" $ }}
{{- end }}
resources:
{{- toYaml .Values.worker.resources | nindent 12 }}
volumeMounts:
- name: config-volume
mountPath: /opt/worker/config
{{- if $.Values.customCAs.bundle }}
- mountPath: {{ include "citylens.customCA.mountPath" $ }}
name: custom-ca
readOnly: true
{{- end }}
volumes:
- name: config-volume
configMap:
name: {{ include "citylens.web.name" . }}-configmap
{{- if $.Values.customCAs.bundle }}
- name: custom-ca
configMap:
name: {{ include "citylens.name" $ }}-custom-ca-configmap
{{- end }}
{{- with .Values.worker.mapMatcher.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.worker.mapMatcher.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.worker.mapMatcher.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
1 change: 1 addition & 0 deletions charts/citylens/templates/workers/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ data:
s3_endpoint: {{ required "A valid .Values.s3.endpoint entry required" .endpoint | squote }}
bucket_prefix: {{ required "A valid .Values.s3.bucketPrefix entry required" .bucketPrefix | squote }}
frame_key_template: {{ include "citylens.s3_constants.frame_key_template" . | squote }}
crop_frame_key_template: {{ include "citylens.s3_constants.crop_frame_key_template" . | squote }}
{{- end }}
topics:
{{- with .Values.kafka.topics }}
Expand Down
Loading
Loading