Skip to content

Commit

Permalink
Add more details in guideline for Auth Istio&Apisix
Browse files Browse the repository at this point in the history 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
  • Loading branch information
@XinyaoWa
XinyaoWa committed Jan 20, 2025
1 parent 781345b commit 0e89d91
Show file tree
Hide file tree
Showing 5 changed files with 192 additions and 48 deletions.
112 changes: 78 additions & 34 deletions authN-authZ/auth-apisix/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,19 @@

Follow the steps to enable authentication and authorization of OPEA services using APISIX api gateway and Identity provider like keycloak

## Prerequisites
## Start ChatQnA service

1. Make sure ChatQnA service is running and accessible locally
Please refer to [GenAIExamples ChatQnA](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/kubernetes) to start `chatqna` megaservice.

2. Run keycloak, setup a realm with OIDC based authentication and add users with passwords for each user.
## Starting and Configuring Keycloak

Steps to start keycloak from official keycloak helm chart
In this stage, we run keycloak, setup a realm with OIDC based authentication. For demonstration, we will add a realm called `apisix` and add a user called `mary` with password. In the authentication step, only the user from `apisix` realm can access the chatQnA pipeline.

### Step 1: Prerequisite.

Create a PersistentVolume of 9Gi for keycloak-postgress with RWO access (to persist updated keycloak configuration)

```sh
# Prerequisite: Create a PersistentVolume of 9Gi for keycloak-postgress with RWO access (to persist updated keycloak configuration)
# Below is a reference to create PersistentVolume which will be used by keycloak-postgress
kubectl apply -f - <<EOF
apiVersion: v1
Expand All @@ -26,90 +29,131 @@ spec:
accessModes:
- ReadWriteOnce
EOF
```

# install keycloak with helm by setting
### Step 2: Install keycloak

```bash
# (Option 1) install keycloak with helm by setting, you can change the user and password to your customized setting
helm install keycloak oci://registry-1.docker.io/bitnamicharts/keycloak --version 22.1.0 --set auth.adminUser=admin --set auth.adminPassword=admin

# Access keycloak UI through service/keycloak to do the necessary configurations
#(Option 2) install keycloak with kubectl and configuration file
cd ..
kubectl apply -f ./keycloak_install.yaml
```

### Step 3: Determine keycloak service ip and port.

```bash
export HOST_IP=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}' | cut -d '/' -f3 | cut -d ':' -f1)
export KEYCLOAK_PORT=$(kubectl get svc keycloak -o jsonpath='{.spec.ports[0].nodePort}')
export KEYCLOAK_ADDR=${HOST_IP}:${KEYCLOAK_PORT}
```

**Note:** Double check if the host ip captured is the correct ip.

### Step 4. Configuration.

Access the Keycloak admin console through the `KEYCLOAK_ADDR` and use the username and password specified in step 2 to login. Then we configure the users, here we create a user named "mary" and assign "user" role to her.

The user management is done via Keycloak and the configuration steps look like this:

1. Create a new realm named `apisix` (change if needed) within Keycloak.

2. Create a new client called `apisix` (change if needed) , set `Client authentication` to `On`, `Save` setting and go to the `Credentials` page, copy `Client Secret` for later use.

3. From the left pane select the Realm roles and create a new role name as `user`.

4. Create a new user name as `mary`, set password for her (set 'Temporary' to 'Off'). Select Role mapping on the top, assign the `user` role to `mary`.

5. Turn off all 'Required actions' buttons under the 'Authentication' section in Keycloak

Then set some environment variables.

```bash
export USER='mary'
export PASSWORD=<password>
export KEYCLOAK_REALM='apisix' # change if needed
export KEYCLOAK_CLIENT_ID='apisix' # change if needed
export KEYCLOAK_CLIENT_SECRET=<keycloak client secret>
```

Once the keycloak pod is up and running, access the UI through the Keycloak's NodePort service and set the necessary configuration
**Trouble Shooting: https required**

If you meet "https required" issue when you open the console, you can fix with the following steps:

```bash
kubectl exec -it ${keycloak_pods_id} -- /bin/bash
cd /opt/keycloak/bin/
./kcadm.sh config credentials --server ${KEYCLOAK_ADDR} --realm master --user admin ## need to type in password set before
./kcadm.sh update realms/master -s sslRequired=NONE --server ${KEYCLOAK_ADDR}
```

Then goto the Keycloak console and find the "Realm setting" for `apisix` realm, set "Require SSL" to "None".

## Update values

Update the following values in values.yaml
Update the following values in values.yaml, you can take values_megaservice.yaml as a reference.

1. Update all the entries in oidc config pertaining to your identity provider

2. Update all the entries in API specific configs

## Install
## Install APISIX

```sh
# Install apisix api gateway and ingress controller [reference: https://apisix.apache.org/docs/apisix/installation-guide/]
# Install APISIX api gateway and ingress controller [reference: https://apisix.apache.org/docs/apisix/installation-guide/]
helm repo add apisix https://charts.apiseven.com
helm repo update
helm install auth-apisix apisix/apisix -f values_apisix_gw.yaml --create-namespace --namespace auth-apisix

# WAIT UNTIL apisix-ingress-controller POD IS READY by checking status with 'kubectl get -n auth-apisix pods'
# WAIT UNTIL the apisix-ingress-controller POD IS READY by checking it's status with 'kubectl get -n auth-apisix pods'
# The pod is ready when READY status shows 1/1

# Publish authenticated APIs in APISIX gateway
helm install auth-apisix-crds . --namespace auth-apisix
```

</br>
APISIX helm chart provides configs to change the service type to other options like LoadBalancer (apisix.service.type) and externalTrafficPolicy to 'local'(apisix.service.externalTrafficPolicy). These can be added in values_apisix_gw.yaml </br></br>

## Usage

The published APIs in apisix gateway are accessible through auth-apisix-gateway kubernetes service. By default, it is a NodePort service and can be accessed as:
The published APIs in APISIX gateway are accessible through auth-apisix-gateway kubernetes service. By default, it is a NodePort service and can be accessed as:

```sh
export NODE_PORT=$(kubectl get --namespace auth-apisix -o jsonpath="{.spec.ports[0].nodePort}" services auth-apisix-gateway)
export NODE_IP=$(kubectl get nodes --namespace auth-apisix -o jsonpath="{.items[0].status.addresses[0].address}")

# the authenticated endpoint published in APISIX gateway can be accessed as: http://$NODE_IP:$NODE_PORT/<published endpoint uri>
export accessUrl=http://$NODE_IP:$NODE_PORT/<your published endpoint uri>


export accessUrl="http://$NODE_IP:$NODE_PORT/chatqna-oidc"
```

</br>
Apisix helm chart provides configs to change the service type to other options like LoadBalancer (apisix.service.type) and externalTrafficPolicy to 'local'(apisix.service.externalTrafficPolicy). These can be added in values_apisix_gw.yaml </br></br>
While accessing the published APIs, the HTTP Authorization header of the request should contain the Access token provided by Identity provider as 'Bearer &ltAccess Token>'. </br></br>
The access token, refresh token, userinfo, user roles and OIDC scopes assigned to user can be obtained by invoking OIDC auth endpoint through UI or token endpoint through curl and providing user credentials. </br></br>

Below steps can be followed to get access token from keycloak and access the APISIX published ChatQnA API through curl

```sh
# Get access token for specified user from keycloak
export USER=<username>
export PASSWORD=<password>
export KEYCLOAK_ADDR=<keycloak url>
export KEYCLOAK_REALM=<keycloak realm>
export KEYCLOAK_CLIENT_ID=<keycloak client id>
export KEYCLOAK_CLIENT_SECRET=<keycloak client secret>

#Invoke Keycloak's OIDC token endpoint to get access token, refresh token and expirt times. (Only Access token is used in the example below)
#Invoke Keycloak's OIDC token endpoint to get access token, refresh token and expiry times. (Only Access token is used in the example below)
export TOKEN=$(curl -X POST http://${KEYCLOAK_ADDR}/realms/${KEYCLOAK_REALM}/protocol/openid-connect/token -H 'Content-Type: application/x-www-form-urlencoded' -d "grant_type=password&client_id=${KEYCLOAK_CLIENT_ID}&client_secret=${KEYCLOAK_CLIENT_SECRET}&username=${USER}&password=${PASSWORD}" | jq -r .access_token)

# follow instructions above to fetch the NODE_IP and NODE_PORT
export accessUrl="http://$NODE_IP:$NODE_PORT/chatqna-oidc"

# try without token. Shall get response: "Authorization required 401 error"
curl -X POST $accessUrl -d '{"text":"What is the revenue of Nike in 2023?","parameters":{"max_new_tokens":17, "do_sample": true}}' -sS -H 'Content-Type: application/json' -w " %{http_code}\n"
curl -X POST $accessUrl -d '{"messages": "What is the revenue of Nike in 2023?", "max_new_tokens":17, "do_sample": true}' -sS -H 'Content-Type: application/json' -w " %{http_code}\n"

# try with token. Shall get the correct response from ChatQnA with http code 200
curl -X POST $accessUrl -d '{"text":"What is the revenue of Nike in 2023?","parameters":{"max_new_tokens":17, "do_sample": true}}' -sS -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -w " %{http_code}\n"
curl -X POST $accessUrl -d '{"messages": "What is the revenue of Nike in 2023?", "max_new_tokens":17, "do_sample": true}' -sS -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -w " %{http_code}\n"

```

## Uninstall

```sh
# Uninstall apisix
# Uninstall APISIX
helm uninstall auth-apisix-crds --namespace auth-apisix
helm uninstall auth-apisix --namespace auth-apisix
```

The crds installed by apisix won't be deleted by helm uninstall. Need to manually delete those crds </br>
The crds installed by APISIX won't be deleted by helm uninstall. Need to manually delete those crds </br>
All APISIX specific crds can be obtained by 'kubectl get crds | grep apisix' </br>
Each crd can be manually deleted by 'kubectl delete crd/\<crd name\>' </br>
2 changes: 1 addition & 1 deletion authN-authZ/auth-apisix/templates/apisix-chatqna-route.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ spec:
introspection_endpoint_auth_method: client_secret_basic
scope: openid profile email
bearer_only: true
realm: myrealm
realm: {{ .Values.oidc.realm }}
- name: proxy-rewrite
enable: true
config:
Expand Down
20 changes: 20 additions & 0 deletions authN-authZ/auth-apisix/values_megaservice.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# Identity provider OIDC config
oidc:
realm: ${KEYCLOAK_REALM} # replace with your realm name for OPEA apps
client_id: ${KEYCLOAK_CLIENT_ID} # replace with your oidc client id
client_secret: ${KEYCLOAK_CLIENT_SECRET} # your oidc client secret
discovery: http://${KEYCLOAK_ADDR}/realms/${KEYCLOAK_CLIENT_ID}/.well-known/openid-configuration
introspection_endpoint: http://${KEYCLOAK_ADDR}/realms/${KEYCLOAK_CLIENT_ID}/protocol/openid-connect/token/introspect

# APISIX chatqna api config
chatqna:
namespace: default # namespace in which your chatqna service is running
hostname: your-hostname # 'Host' HTTP header from incoming request should match this. Wildcards like '*' allowed too
query_api:
path: /chatqna-oidc # This is the path that will be published in apisix and this should be used by UI to access the chatqna service
backend_service: chatqna # your kubernetes service name to access chatqna megaservice or gmc without .<namespace>.svc.cluster.local
service_port: 8888 # port on which chatqna mega service or gmc is running
service_path: "/v1/chatqna" # path to access chatqna mega service or gmc backend
Loading

0 comments on commit 0e89d91

Please sign in to comment.