用户可以使用 Karmada 提供的 MultiClusterIngress API 将外部流量导入成员集群中的服务。

前提

多集群间服务可以相互访问，参考karmada多集群服务发现
成员集群的 Kubernetes 版本必须为 v1.21 或更高版本。

部署测试

在主机集群上部署ingress-nginx

我这里没有使用官方提供的 multi-cluster-ingress-nginx ，而是使用了nginx-ingress官方的yaml部署的。

使用MCS

MCS即多集群服务发现。安装 ServiceExport and ServiceImport CRDs，步骤见上篇文章。

在member1集群中部署web服务

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: hello-app
        #image: gcr.io/google-samples/hello-app:1.0 拉不到docker search替换。
        image: hansd/hello-app:latest
        ports:
        - containerPort: 8080
          protocol: TCP
---      
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  ports:
  - port: 81
    targetPort: 8080
  selector:
    app: web
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: mci-workload
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: web
    - apiVersion: v1
      kind: Service
      name: web
  placement:
    clusterAffinity:
      clusterNames:
        - member1

从 member1 集群导出 web 服务

apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceExport
metadata:
  name: web
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: web-export-policy
spec:
  resourceSelectors:
    - apiVersion: multicluster.x-k8s.io/v1alpha1
      kind: ServiceExport
      name: web
  placement:
    clusterAffinity:
      clusterNames:
        - member1

将 web 服务导入 member2 集群

apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceImport
metadata:
  name: web
spec:
  type: ClusterSetIP
  ports:
  - port: 81
    protocol: TCP
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: web-import-policy
spec:
  resourceSelectors:
    - apiVersion: multicluster.x-k8s.io/v1alpha1
      kind: ServiceImport
      name: web
  placement:
    clusterAffinity:
      clusterNames:
        - member2

在 karmada-controlplane 上部署 multiclusteringress

apiVersion: networking.karmada.io/v1alpha1
kind: MultiClusterIngress
metadata:
  name: demo-localhost
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: demo.web.com
    http:
      paths:
      - backend:
          service:
            name: web
            port:
              number: 81
        path: /web
        pathType: Prefix

测试访问

本地添加hosts后，访问 http://demo.web.com:32267/web 无法访问。原因是 nginx-ingress-controller 无法访问 karmada-apiserver 来监听资源的变化（例如 multiclusteringress、endpointslices 和服务），没有相应的rbac权限。

下载multi-cluster-ingress-nginx yaml

git clone https://github.com/karmada-io/multi-cluster-ingress-nginx.git

修改镜像

# k8s.gcr.io/ingress-nginx/controller:v1.1.1@sha256:0bc88eb15f9e7f84e8e56c14fa5735aaa488b840983f87bd79b1054190e660de
# k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
# 镜像已经推送到我的dockerhub中了，可以直接使用。
docker pull wgh9626/ingress-nginx-controller:v1.1.1
docker pull wgh9626/kube-webhook-certgen:v1.1.1
cd multi-cluster-ingress-nginx-main/charts/ingress-nginx
vim values.yaml
helm install ingress-nginx . -n ingress-nginx

可以看到ingress-nginx的clusterrole添加了multiclusteringresses的rbac权限。

创建secret

# get the 'karmada-apiserver' kubeconfig information and direct it to file /tmp/kubeconfig.yaml
kubectl -n karmada-system get secret kubeconfig --template={{.data.kubeconfig}} | base64 -d > /tmp/kubeconfig.yaml
# create secret with name 'kubeconfig' from file /tmp/kubeconfig.yaml
kubectl -n ingress-nginx create secret generic kubeconfig --from-file=kubeconfig=/tmp/kubeconfig.yaml

修改ingress-nginx yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  ...
spec:
  #...
  template:
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --karmada-kubeconfig=/etc/kubeconfig  # new line
        #...
        volumeMounts:
        #...
        - mountPath: /etc/kubeconfig            # new line
          name: kubeconfig                      # new line
          subPath: kubeconfig                   # new line
      volumes:
      #...
      - name: kubeconfig                        # new line
        secret:                                 # new line
          secretName: kubeconfig                # new line