[release-3.3] fix the issue that the upload app template did not display icons (#5493)

fix the issue that the upload app template did not display icons

Co-authored-by: xiaoliu <978911210@qq.com>

.dockerignore

@@ -1,3 +0,0 @@
-# exclude all files and folders except bin folder
-**
-!bin

.env

@@ -1,2 +0,0 @@
-DATA_PATH=./tmp
-KUBESPHERE_LOG_LEVEL=debug

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,5 +1,6 @@
 ---
 name: Bug report
+labels: ["kind/bug"]
 about: Create a report to help us improve
 ---
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,30 +1,54 @@
-**What type of PR is this?**
-> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
->
-> /kind api-change
-> /kind bug
-> /kind cleanup
-> /kind design
-> /kind documentation
-> /kind failing-test
-> /kind feature
-> /kind flake
+<!-- Thanks for sending a pull request! Here are some tips for you:
 
-**What this PR does / why we need it**:
+1. If you want **faster** PR reviews, read how: https://github.com/kubesphere/community/blob/master/developer-guide/development/the-pr-author-guide-to-getting-through-code-review.md
+2. In case you want to know how your PR got reviewed, read: https://github.com/kubesphere/community/blob/master/developer-guide/development/code-review-guide.md
+3. Here are some coding convetions followed by KubeSphere community: https://github.com/kubesphere/community/blob/master/developer-guide/development/coding-conventions.md
+-->
 
-**Which issue(s) this PR fixes**:
+### What type of PR is this?
+<!-- 
+Add one of the following kinds:
+/kind bug
+/kind cleanup
+/kind documentation
+/kind feature
+/kind design
+
+Optionally add one or more of the following kinds if applicable:
+/kind api-change
+/kind deprecation
+/kind failing-test
+/kind flake
+/kind regression
+-->
+
+
+### What this PR does / why we need it:
+
+### Which issue(s) this PR fixes:
 <!--
 Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
 _If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
 -->
 Fixes #
 
-**Special notes for reviewers**:
+### Special notes for reviewers:
 ```
 ```
 
-**Additional documentation, usage docs, etc.**:
+### Does this PR introduced a user-facing change?
+<!--
+If no, just write "None" in the release-note block below.
+If yes, a release note is required:
+Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".
 
+For more information on release notes see: https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md
+-->
+```release-note
+
+```
+
+### Additional documentation, usage docs, etc.:
 <!--
 This section can be blank if this pull request does not require a release note.
 Please use the following format for linking documentation or pass the

.github/workflows/build-multiarch.yaml

@@ -0,0 +1,42 @@
+name: BuildContainerImage
+
+on:
+  push:
+    branches:
+      - 'master'
+      - 'release-*'
+    tags:
+      - 'v*'
+  pull_request:
+    branches:
+      - 'master'
+      - 'release-*'
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    env:
+      GO111MODULE: on
+    steps:
+      - name: Check out code into the Go module directory
+        uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+        with:
+          platforms: all
+
+      - name: Set up Docker buildx
+        uses: docker/setup-buildx-action@v1
+
+      - name: Build and push docker images
+        env:
+          DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
+          DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
+        if: github.event_name == 'push'
+        run: |
+          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
+          REPO=kubespheredev TAG="${GITHUB_REF#refs/*/}" make container-cross-push

.github/workflows/build.yml

@@ -1,72 +0,0 @@
-name: Go
-
-on:
-  push:
-    branches:
-      - 'master'
-      - 'release*'
-    tags:
-      - 'v*'
-  pull_request:
-    branches:
-      - 'master'
-      - 'release*'
-
-jobs:
-  build:
-    name: Build
-    runs-on: ubuntu-latest
-    env:
-      GO111MODULE: on
-    steps:
-      - name: Set up Go 1.13
-        uses: actions/setup-go@v2
-        with:
-          go-version: 1.13
-        id: go
-
-      - name: Check out code into the Go module directory
-        uses: actions/checkout@v2
-        with:
-          fetch-depth: 0
-
-      - name: Check pr is properly formatted
-        run: diff -u <(echo -n) <(gofmt -d ./pkg ./cmd ./tools ./test)
-
-      - name: Verify goimports
-        run: go get -u golang.org/x/tools/cmd/goimports && bash hack/verify-goimports.sh
-
-      - name: Downloading go dependencies
-        run: go mod vendor
-
-      - name: Install kubebuilder
-        run: bash hack/install_kubebuilder.sh
-
-      - name: Build
-        run: make all
-
-      - name: Make OpenAPI Spec
-        run: make openapi
-
-      - name: Uploading code coverage
-        uses: codecov/codecov-action@v1
-        with:
-          token: ${{ secrets.CODECOV_TOKEN }}
-          file: ./coverage.txt
-          flags: unittests
-          name: codecov-umbrella
-          fail_ci_if_error: false
-
-      - name: Get branch name
-        id: extract_branch
-        shell: bash
-        run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
-
-      - name: Build and push docker images
-        env:
-          DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
-          DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
-        if: github.event_name == 'push'
-        run: |
-          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
-          bash hack/docker_build.sh ${{ steps.extract_branch.outputs.branch }}

.github/workflows/e2e-test.yml

@@ -13,20 +13,17 @@ jobs:
       GO111MODULE: on
     steps:
 
-      - name: Set up Go 1.13
-        uses: actions/setup-go@v1
+      - name: Set up Go 1.16
+        uses: actions/setup-go@v2
         with:
-          go-version: 1.13
+          go-version: 1.16
         id: go
 
       - name: Check out code into the Go module directory
         uses: actions/checkout@v2
 
-      - name: Downloading go dependencies
-        run: go mod vendor
-
       - name: Create kind cluster
-        uses: helm/kind-action@v1.0.0-rc.1
+        uses: helm/kind-action@v1.2.0
         with:
           config: .github/workflows/kind/kind.yaml
 
@@ -34,4 +31,14 @@ jobs:
         run: KIND_CLUSTER_NAME=chart-testing hack/deploy-kubesphere.sh
 
       - name: Run e2e testing
-        run: go test ./test/e2e
+        run: go test ./test/e2e
+
+      - name: slack
+        uses: 8398a7/action-slack@v3
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+        with:
+          status: ${{ job.status }}
+          fields: repo,message,commit,author,action,eventName,ref,workflow,job,took
+        if: failure()
+

.github/workflows/kind/kind.yaml

@@ -2,7 +2,7 @@ kind: Cluster
 apiVersion: kind.x-k8s.io/v1alpha4
 nodes:
 - role: control-plane
-  image: kindest/node:v1.19.7
+  image: kindest/node:v1.21.1
   extraMounts:
   - hostPath: /etc/localtime
     containerPath: /etc/localtime

.github/workflows/nightly-builds.yml

@@ -15,10 +15,10 @@ jobs:
       GO111MODULE: on
     steps:
 
-      - name: Set up Go 1.13
-        uses: actions/setup-go@v1
+      - name: Set up Go 1.16
+        uses: actions/setup-go@v2
         with:
-          go-version: 1.13
+          go-version: 1.16
         id: go
 
       - name: Check out code into the Go module directory
@@ -26,17 +26,13 @@ jobs:
         with:
           fetch-depth: 0
 
-      - name: Downloading go dependencies
-        run: go mod vendor
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+        with:
+          platforms: all
 
-      - name: Install kubebuilder
-        run: bash hack/install_kubebuilder.sh
-
-      - name: Build
-        run: make all
-
-      - name: Make OpenAPI Spec
-        run: make openapi
+      - name: Set up Docker buildx
+        uses: docker/setup-buildx-action@v1
 
       - name: Build and push docker images
         env:
@@ -44,19 +40,8 @@ jobs:
           DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
         run: |
           echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
-          bash hack/docker_build.sh master
-
-          if [[ $? == 0 ]]; then
-            tag=nightly-$(date '+%Y%m%d')
-
-            docker tag kubespheredev/ks-apiserver kubespheredev/ks-apiserver:${tag}
-            docker tag kubespheredev/ks-controller-manager kubespheredev/ks-controller-manager:${tag}
-
-            docker push kubespheredev/ks-apiserver:${tag}
-            docker push kubespheredev/ks-controller-manager:${tag}
-          else
-            exit -1
-          fi
+          tag=nightly-$(date '+%Y%m%d')
+          REPO=kubespheredev TAG=${tag} make container-cross-push
 
       - name: slack
         uses: 8398a7/action-slack@v3
@@ -65,4 +50,4 @@ jobs:
         with:
           status: ${{ job.status }}
           fields: repo,message,commit,author,action,eventName,ref,workflow,job,took
-        if: always()
+        if: failure()

.gitignore

@@ -32,3 +32,4 @@ kustomize/network/etcd
 apiserver.local.config
 tmp/
 kubesphere.yaml
+testbin/

.licenserc.yaml

@@ -0,0 +1,62 @@
+#
+#  Copyright 2022 The KubeSphere Authors.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+#
+
+header:
+  license:
+    spdx-id: Apache-2.0
+    copyright-owner: KubeSphere Authors
+
+    content: |
+      Copyright 2022 The KubeSphere Authors.
+  
+      Licensed under the Apache License, Version 2.0 (the "License");
+      you may not use this file except in compliance with the License.
+      You may obtain a copy of the License at
+      
+          http://www.apache.org/licenses/LICENSE-2.0
+      
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+      See the License for the specific language governing permissions and
+      limitations under the License.
+
+
+  paths-ignore:
+    - 'api'
+    - 'build'
+    - 'docs'
+    - 'config'
+    - 'test'
+    - 'install'
+    - 'hack'
+    - 'vendor'
+    - 'staging'
+    - 'LICENSE'
+    - 'OWNERS'
+    - 'Makefile'
+    - 'pkg/client/**'
+    - 'pkg/simple/client/**'
+    - '**/*.md'
+    - '**/*.json'
+    - '**/go.mod'
+    - '**/go.sum'
+    - '.github/**'
+    - '.gitignore'
+    - '.gitattributes'
+    - 'pkg/controller/application/status.go'
+
+  comment: on-failure

Makefile

@@ -2,9 +2,16 @@
 # Use of this source code is governed by a Apache license
 # that can be found in the LICENSE file.
 
+
 # Produce CRDs that work back to Kubernetes 1.11 (no version conversion)
 CRD_OPTIONS ?= "crd:trivialVersions=true"
 
+GV="network:v1alpha1 servicemesh:v1alpha2 tenant:v1alpha1 tenant:v1alpha2 devops:v1alpha1 iam:v1alpha2 devops:v1alpha3 cluster:v1alpha1 storage:v1alpha1 auditing:v1alpha1 types:v1beta1 quota:v1alpha2 application:v1alpha1 notification:v2beta1 gateway:v1alpha1"
+MANIFESTS="application/* cluster/* iam/* network/v1alpha1 quota/* storage/* tenant/* gateway/*"
+
+# App Version
+APP_VERSION = v3.2.0
+
 # Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set)
 ifeq (,$(shell go env GOBIN))
 GOBIN=$(shell go env GOPATH)/bin
@@ -13,7 +20,10 @@ GOBIN=$(shell go env GOBIN)
 endif
 
 OUTPUT_DIR=bin
-GOFLAGS=-mod=vendor
+ifeq (${GOFLAGS},)
+	# go build with vendor by default.
+	export GOFLAGS=-mod=vendor
+endif
 define ALL_HELP_INFO
 # Build code.
 #
@@ -34,94 +44,118 @@ define ALL_HELP_INFO
 #           debugging tools like delve.
 endef
 .PHONY: all
-all: test ks-apiserver ks-controller-manager
+all: test ks-apiserver ks-controller-manager;$(info $(M)...Begin to test and build all of binary.) @ ## Test and build all of binary.
+
+help:
+	@grep -hE '^[ a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \
+		awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-17s\033[0m %s\n", $$1, $$2}'
+
+.PHONY: binary
+# Build all of binary
+binary: | ks-apiserver ks-controller-manager; $(info $(M)...Build all of binary.) @ ## Build all of binary.
 
 # Build ks-apiserver binary
-ks-apiserver: fmt vet
-	hack/gobuild.sh cmd/ks-apiserver
+ks-apiserver: ; $(info $(M)...Begin to build ks-apiserver binary.)  @ ## Build ks-apiserver.
+	 hack/gobuild.sh cmd/ks-apiserver;
 
 # Build ks-controller-manager binary
-ks-controller-manager: fmt vet
+ks-controller-manager: ; $(info $(M)...Begin to build ks-controller-manager binary.)  @ ## Build ks-controller-manager.
 	hack/gobuild.sh cmd/controller-manager
 
+# Run all verify scripts hack/verify-*.sh
+verify-all: ; $(info $(M)...Begin to run all verify scripts hack/verify-*.sh.)  @ ## Run all verify scripts hack/verify-*.sh.
+	hack/verify-all.sh
+
 # Build e2e binary
-e2e: fmt vet
+e2e: ;$(info $(M)...Begin to build e2e binary.)  @ ## Build e2e binary.
 	hack/build_e2e.sh test/e2e
 
-# Run go fmt against code 
-fmt:
+kind-e2e: ;$(info $(M)...Run e2e test.) @ ## Run e2e test in kind.
+	hack/kind_e2e.sh
+
+# Run go fmt against code
+fmt: ;$(info $(M)...Begin to run go fmt against code.)  @ ## Run go fmt against code.
 	gofmt -w ./pkg ./cmd ./tools ./api
 
-goimports:
+# Format all import, `goimports` is required.
+goimports: ;$(info $(M)...Begin to Format all import.)  @ ## Format all import, `goimports` is required.
 	@hack/update-goimports.sh
 
 # Run go vet against code
-vet:
+vet: ;$(info $(M)...Begin to run go vet against code.)  @ ## Run go vet against code.
 	go vet ./pkg/... ./cmd/...
 
 # Generate manifests e.g. CRD, RBAC etc.
-manifests:
-	go run ./vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go object:headerFile=./hack/boilerplate.go.txt paths=./pkg/apis/... rbac:roleName=controller-perms ${CRD_OPTIONS} output:crd:artifacts:config=config/crds
+manifests: ;$(info $(M)...Begin to generate manifests e.g. CRD, RBAC etc..)  @ ## Generate manifests e.g. CRD, RBAC etc.
+	hack/generate_manifests.sh ${CRD_OPTIONS} ${MANIFESTS}
 
-deploy: manifests
+deploy: manifests ;$(info $(M)...Begin to deploy.)  @ ## Deploy.
 	kubectl apply -f config/crds
 	kustomize build config/default | kubectl apply -f -
 
-mockgen:
+mockgen: ;$(info $(M)...Begin to mockgen.)  @ ## Mockgen.
 	mockgen -package=openpitrix -source=pkg/simple/client/openpitrix/openpitrix.go -destination=pkg/simple/client/openpitrix/mock.go
 
-deepcopy:
-	GO111MODULE=on go install -mod=vendor k8s.io/code-generator/cmd/deepcopy-gen
-	${GOPATH}/bin/deepcopy-gen -i kubesphere.io/kubesphere/pkg/apis/... -h ./hack/boilerplate.go.txt -O zz_generated.deepcopy
+deepcopy: ;$(info $(M)...Begin to deepcopy.)  @ ## Deepcopy.
+	hack/generate_group.sh "deepcopy" kubesphere.io/api kubesphere.io/api ${GV} --output-base=staging/src/  -h "hack/boilerplate.go.txt"
 
-openapi:
-	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./pkg/apis/tenant/v1alpha1 -p kubesphere.io/kubesphere/pkg/apis/tenant/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list
-	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./pkg/apis/servicemesh/v1alpha2 -p kubesphere.io/kubesphere/pkg/apis/servicemesh/v1alpha2 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list
-	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/api/networking/v1,./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/k8s.io/apimachinery/pkg/util/intstr,./pkg/apis/network/v1alpha1 -p kubesphere.io/kubesphere/pkg/apis/network/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list
-	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./pkg/apis/devops/v1alpha1,./vendor/k8s.io/apimachinery/pkg/runtime,./vendor/k8s.io/api/core/v1 -p kubesphere.io/kubesphere/pkg/apis/devops/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list
-	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./pkg/apis/cluster/v1alpha1,./vendor/k8s.io/apimachinery/pkg/runtime,./vendor/k8s.io/api/core/v1 -p kubesphere.io/kubesphere/pkg/apis/cluster/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list
-	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./pkg/apis/devops/v1alpha3,./vendor/k8s.io/apimachinery/pkg/runtime -p kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list
+openapi: ;$(info $(M)...Begin to openapi.)  @ ## Openapi.
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/kubesphere.io/api/tenant/v1alpha1 -p kubesphere.io/api/tenant/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/kubesphere.io/api/network/v1alpha1 -p kubesphere.io/api/network/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/kubesphere.io/api/servicemesh/v1alpha2 -p kubesphere.io/api/servicemesh/v1alpha2 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/api/networking/v1,./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/k8s.io/apimachinery/pkg/util/intstr,./vendor/kubesphere.io/api/network/v1alpha1 -p kubesphere.io/api/network/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/kubesphere.io/api/devops/v1alpha1,./vendor/k8s.io/apimachinery/pkg/runtime,./vendor/k8s.io/api/core/v1 -p kubesphere.io/api/devops/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/kubesphere.io/api/cluster/v1alpha1,./vendor/k8s.io/apimachinery/pkg/runtime,./vendor/k8s.io/api/core/v1 -p kubesphere.io/api/cluster/v1alpha1 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
+	go run ./vendor/k8s.io/kube-openapi/cmd/openapi-gen/openapi-gen.go -O openapi_generated -i ./vendor/k8s.io/apimachinery/pkg/apis/meta/v1,./vendor/kubesphere.io/api/devops/v1alpha3,./vendor/k8s.io/apimachinery/pkg/runtime -p kubesphere.io/api/devops/v1alpha3 -h ./hack/boilerplate.go.txt --report-filename ./api/api-rules/violation_exceptions.list  --output-base=staging/src/
 	go run ./tools/cmd/crd-doc-gen/main.go
 	go run ./tools/cmd/doc-gen/main.go
-# Build the docker image
-docker-build: all
-	hack/docker_build.sh
-docker-build-no-test: ks-apiserver ks-controller-manager
+
+container: ;$(info $(M)...Begin to build the docker image.)  @ ## Build the docker image.
+	DRY_RUN=true hack/docker_build.sh
+
+container-push: ;$(info $(M)...Begin to build and push.)  @ ## Build and Push.
 	hack/docker_build.sh
 
+container-cross: ; $(info $(M)...Begin to build container images for multiple platforms.)  @ ## Build container images for multiple platforms. Currently, only linux/amd64,linux/arm64 are supported.
+	DRY_RUN=true hack/docker_build_multiarch.sh
+
+container-cross-push: ; $(info $(M)...Begin to build and push.)  @ ## Build and Push.
+	hack/docker_build_multiarch.sh
+
+helm-package: ; $(info $(M)...Begin to helm-package.)  @ ## Helm-package.
+	ls config/crds/ | xargs -i cp -r config/crds/{} config/ks-core/crds/
+	helm package config/ks-core --app-version=${APP_VERSION} --version=0.1.0 -d ./bin
+
+helm-deploy: ; $(info $(M)...Begin to helm-deploy.)  @ ## Helm-deploy.
+	ls config/crds/ | xargs -i cp -r config/crds/{} config/ks-core/crds/
+	- kubectl create ns kubesphere-controls-system
+	helm upgrade --install ks-core ./config/ks-core -n kubesphere-system --create-namespace
+	kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/roles/ks-core/prepare/files/ks-init/role-templates.yaml
+
+helm-uninstall: ; $(info $(M)...Begin to helm-uninstall.)  @ ## Helm-uninstall.
+	- kubectl delete ns kubesphere-controls-system
+	helm uninstall ks-core -n kubesphere-system
+	kubectl delete -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/roles/ks-core/prepare/files/ks-init/role-templates.yaml
+
 # Run tests
-test: fmt vet
-	export KUBEBUILDER_CONTROLPLANE_START_TIMEOUT=2m; go test ./pkg/... ./cmd/... -covermode=atomic -coverprofile=coverage.txt
+ENVTEST_ASSETS_DIR=$(shell pwd)/testbin
+test: vet test-env ;$(info $(M)...Begin to run tests.)  @ ## Run tests.
+	export KUBEBUILDER_ASSETS=$(shell pwd)/testbin/bin; go test ./pkg/... ./cmd/... -covermode=atomic -coverprofile=coverage.txt
+	cd staging/src/kubesphere.io/api ; GOFLAGS="" go test ./...
+	cd staging/src/kubesphere.io/client-go ; GOFLAGS="" go test ./...
+
+.PHONY: test-env
+test-env: ;$(info $(M)...Begin to setup test env) @ ## Download unit test libraries e.g. kube-apiserver etcd.
+	@hack/setup-kubebuilder-env.sh
 
 .PHONY: clean
-clean:
+clean: ;$(info $(M)...Begin to clean.)  @ ## Clean.
 	-make -C ./pkg/version clean
 	@echo "ok"
 
-# find or download controller-gen
-# download controller-gen if necessary
-clientset: 
-	./hack/generate_client.sh
+clientset:  ;$(info $(M)...Begin to find or download controller-gen.)  @ ## Find or download controller-gen,download controller-gen if necessary.
+	./hack/generate_client.sh ${GV}
 
-
-# Currently in the upgrade phase of controller tools.
-# But the new controller tools are not compatible with the old version.
-# With these commands you may need to manually modify the generated code
-# So don't use it unless you know it very deeply
-internal-crds:
-	$(CONTROLLER_GEN) $(CRD_OPTIONS) rbac:roleName=manager-role webhook paths="./pkg/apis/network/..." output:crd:artifacts:config=config/crd/bases
-
-internal-generate-apis: internal-controller-gen
-	$(CONTROLLER_GEN) object:headerFile=./hack/boilerplate.go.txt paths=./pkg/apis/network/...
-
-internal-controller-gen:
-ifeq (, $(shell which controller-gen))
-	go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.2.0-beta.4
-CONTROLLER_GEN=$(GOBIN)/controller-gen
-else
-CONTROLLER_GEN=$(shell which controller-gen)
-endif
-
-network-rbac:
-	$(CONTROLLER_GEN) paths=./pkg/controller/network/provider/ paths=./pkg/controller/network/ rbac:roleName=network-manager output:rbac:artifacts:config=kustomize/network/calico-k8s
-	$(CONTROLLER_GEN) paths=./pkg/controller/network/ rbac:roleName=network-manager output:rbac:artifacts:config=kustomize/network/calico-etcd
+# Fix invalid file's license.
+update-licenses: ;$(info $(M)...Begin to update licenses.)
+	@hack/update-licenses.sh

OWNERS

@@ -1,6 +1,9 @@
 approvers:
     - zryfish #oncall
     - rayzhou2017
+    - wansir
+    - zheng1
+    - benjaminhuo
 
 reviewers:
     - rayzhou2017
@@ -8,16 +11,16 @@ reviewers:
     - benjaminhuo
     - calvinyv
     - FeynmanZhou
-    - huanggze
-    - huojiao2006
-    - Ma-Dan
-    - magicsong
     - pixiake
-    - runzexia
     - wansir
-    - wnxn
     - zheng1
-    - soulseen
-    - shaowenchen
     - stoneshi-yunify
     - linuxsuren
+    - RolandMa1986
+    - wanjunlei
+    - xyz-li
+    - junotx
+    - yuswift
+    - zhu733756
+    - JohnNiang
+    - dkeven

PROJECT

@@ -1,3 +0,0 @@
-version: "1"
-domain: kubesphere.io
-repo: kubesphere.io/kubesphere

README.md

@@ -1,11 +1,20 @@
-# KubeSphere Container Platform
+<p align="center">
+<a href="https://kubesphere.io/"><img src="docs/images/kubesphere-icon.gif" alt="banner" width="200px"></a>
+</p>
 
-[![License](http://img.shields.io/badge/license-apache%20v2-blue.svg)](https://github.com/KubeSphere/KubeSphere/blob/master/LICENSE)
-[![Build Status](https://travis-ci.org/kubesphere/kubesphere.svg?branch=master)](https://travis-ci.org/kubesphere/kubesphere)
-[![Go Report Card](https://goreportcard.com/badge/github.com/kubesphere/kubesphere)](https://goreportcard.com/report/github.com/kubesphere/kubesphere)
-[![KubeSphere release](https://img.shields.io/github/release/kubesphere/kubesphere.svg?color=release&label=release&logo=release&logoColor=release)](https://github.com/kubesphere/kubesphere/releases/tag/v3.0.0)
+<p align="center">
+<b>The container platform tailored for <i>Kubernetes multi-cloud, datacenter, and edge</i> management</b>
+</p>
+
+<p align=center>
+<a href="https://goreportcard.com/report/github.com/kubesphere/kubesphere"><img src="https://goreportcard.com/badge/github.com/kubesphere/kubesphere" alt="A+"></a>
+<a href="https://hub.docker.com/r/kubesphere/ks-installer"><img src="https://img.shields.io/docker/pulls/kubesphere/ks-installer"></a>
+<a href="https://github.com/search?q=user%3Akubesphere+user%3Akubesphere-sigs+label%3A%22good+first+issue%22+state%3Aopen&type=Issues&ref=advsearch&l=&l="><img src="https://img.shields.io/github/issues/badges/shields/good%20first%20issue" alt="good first"></a>
+<a href="https://twitter.com/intent/follow?screen_name=KubeSphere"><img src="https://img.shields.io/twitter/follow/KubeSphere?style=social" alt="follow on Twitter"></a>
+<a href="https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE"><img src="https://img.shields.io/badge/Slack-600%2B-blueviolet?logo=slack&amp;logoColor=white"></a>
+<a href="https://www.youtube.com/channel/UCyTdUQUYjf7XLjxECx63Hpw"><img src="https://img.shields.io/youtube/channel/subscribers/UCyTdUQUYjf7XLjxECx63Hpw?style=social"></a>
+</p>
 
-![logo](docs/images/kubesphere-logo.png)
 
 ----
 
@@ -13,7 +22,7 @@
 
 > English | [中文](README_zh.md)
 
-[KubeSphere](https://kubesphere.io/) is a **distributed operating system providing cloud native stack** with [Kubernetes](https://kubernetes.io) as its kernel, and aims to be plug-and-play architecture for third-party applications seamless integration to boost its ecosystem. KubeSphere is also a multi-tenant enterprise-grade container platform with full-stack automated IT operation and streamlined DevOps workflows. It provides developer-friendly wizard web UI, helping enterprises to build out a more robust and feature-rich platform, which includes most common functionalities needed for enterprise Kubernetes strategy, see [Feature List](#features) for details.
+[KubeSphere](https://kubesphere.io/) is a **distributed operating system for cloud-native application management**, using [Kubernetes](https://kubernetes.io) as its kernel. It provides a plug-and-play architecture, allowing third-party applications to be seamlessly integrated into its ecosystem. KubeSphere is also a multi-tenant container platform with full-stack automated IT operation and streamlined DevOps workflows. It provides developer-friendly wizard web UI, helping enterprises to build out a more robust and feature-rich platform, which includes most common functionalities needed for enterprise Kubernetes strategy, see [Feature List](#features) for details.
 
 The following screenshots give a close insight into KubeSphere. Please check [What is KubeSphere](https://kubesphere.io/docs/introduction/what-is-kubesphere/) for further information.
 
@@ -36,68 +45,128 @@ The following screenshots give a close insight into KubeSphere. Please check [Wh
   </tr>
 </table>
 
-## Demo Environment
+## Demo environment
 
-Using the account `demo1 / Demo123` to log in the [demo environment](https://demo.kubesphere.io/). Please note the account is granted view access. You can also have a quick view of [KubeSphere Demo Video](https://youtu.be/u5lQvhi_Xlc).
+🎮 Using the account `demo1 / Demo123` to log in the [demo environment](https://demo.kubesphere.io/). Please note the account is granted view access. 
 
-## Architecture
-
-KubeSphere uses a loosely-coupled architecture that separates the [frontend](https://github.com/kubesphere/console) from the [backend](https://github.com/kubesphere/kubesphere). External systems can access the components of the backend which are delivered as Docker containers through the REST APIs. See [Architecture](https://kubesphere.io/docs/introduction/architecture/) for details.
-
-![Architecture](docs/images/architecture.png)
+🖥 You can also have a quick view of [Demo video](https://youtu.be/YxZ1YUv0CYs).
 
 ## Features
 
-|Feature|Description|
-|---|---|
-| Provisioning Kubernetes Cluster|Support deploy Kubernetes on your infrastructure out of box, including online and air gapped installation|
-| Multi-cluster Management | Provide a centralized control plane to manage multiple Kubernetes Clusters, support application distribution across multiple clusters and cloud providers|
-| Kubernetes Resource Management | Provide web console for creating and managing Kubernetes resources, with powerful observability including monitoring, logging, events, alerting and notification |
-| DevOps System | Provide out-of-box CI/CD based on Jenkins, and offers automated workflow tools including binary-to-image (B2I) and source-to-image (S2I) |
-| Application Store | Provide application store for Helm-based applications, and offers application lifecycle management |
-| Service Mesh (Istio-based) | Provide fine-grained traffic management, observability and tracing for distributed microservice applications, provides visualization for traffic topology |
-| Rich Observability | Provide multi-dimensional monitoring metrics, and provides multi-tenant logging, events and [auditing](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) management, support alerting and notification for both application and infrastructure |
-| Multi-tenant Management | Provide unified authentication with fine-grained roles and three-tier authorization system, supports AD/LDAP authentication |
-| Infrastructure Management | Support node management and monitoring, and supports adding new nodes for Kubernetes cluster |
-| Storage Support | Support GlusterFS, CephRBD, NFS, LocalPV (default), etc. open source storage solutions, provides CSI plugins to consume storage from cloud providers |
-| Network Support | Support Calico, Flannel, etc., provides [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) management, and load balancer plug-in [Porter](https://github.com/kubesphere/porter) for bare metal.|
-| GPU Support | Support add GPU node, support vGPU, enables running ML applications on Kubernetes, e.g. TensorFlow |
+<details>
+  <summary><b>🕸 Provisioning Kubernetes Cluster</b></summary>
+  Support deploy Kubernetes on any infrastructure, support online and air-gapped installation, <a href="https://kubesphere.io/docs/installing-on-linux/introduction/intro/">learn more</a>.
+  </details>
 
-Please see the [Feature and Benefits](https://kubesphere.io/docs/introduction/features/) for further information.
+<details>
+  <summary><b>🔗 Kubernetes Multi-cluster Management</b></summary>
+  Provide a centralized control plane to manage multiple Kubernetes clusters, support propagate an app to multiple K8s clusters across different cloud providers.
+  </details>
+
+<details>
+  <summary><b>🤖 Kubernetes DevOps</b></summary>
+  Provide out-of-box CI/CD based on Jenkins, and offers automated workflow tools including binary-to-image (B2I) and source-to-image (S2I), <a href="https://kubesphere.io/devops/">learn more</a>.
+  </details>
+
+<details>
+  <summary><b>🔎 Cloud Native Observability</b></summary>
+  Multi-dimensional monitoring, events and auditing logs are supported; multi-tenant log query and collection, alerting and notification are built-in, <a href="https://kubesphere.io/observability/">learn more</a>.
+  </details>
+
+<details>
+  <summary><b>🧩 Service Mesh (Istio-based)</b></summary>
+  Provide fine-grained traffic management, observability and tracing for distributed microservice applications, provides visualization for traffic topology, <a href="https://kubesphere.io/service-mesh/">learn more</a>.
+  </details>
+
+<details>
+  <summary><b>💻 App Store</b></summary>
+  Provide an App Store for Helm-based applications, and offer application lifecycle management on Kubernetes platform, <a href="https://kubesphere.io/docs/pluggable-components/app-store/">learn more</a>.
+  </details>
+
+<details>
+  <summary><b>💡 Edge Computing Platform</b></summary>
+  KubeSphere integrates <a href="https://kubeedge.io/en/">KubeEdge</a> to enable users to deploy applications on the edge devices and view logs and monitoring metrics of them on the console, <a href="https://kubesphere.io/docs/pluggable-components/kubeedge/">learn more</a>.
+  </details>
+
+<details>
+  <summary><b>📊 Metering and Billing</b></summary>
+  Track resource consumption at different levels on a unified dashboard, which helps you make better-informed decisions on planning and reduce the cost, <a href="https://kubesphere.io/docs/toolbox/metering-and-billing/view-resource-consumption/">learn more</a>.
+  </details>
+
+<details>
+  <summary><b>🗃 Support Multiple Storage and Networking Solutions</b></summary>
+  <li>Support GlusterFS, CephRBD, NFS, LocalPV solutions, and provide CSI plugins to consume storage from multiple cloud providers.</li><li>Provide Load Balancer Implementation <a href="https://github.com/kubesphere/openelb">OpenELB</a> for Kubernetes in bare-metal, edge, and virtualization.</li><li> Provides network policy and Pod IP pools management, support Calico, Flannel, Kube-OVN</li>.</li>.
+  </details>
+
+<details>
+  <summary><b>🏘 Multi-tenancy</b></summary>
+  Provide unified authentication with fine-grained roles and three-tier authorization system, and support AD/LDAP authentication.
+  </details>
+
+<details>
+  <summary><b>🧠 GPU Workloads Scheduling and Monitoring</b></summary>
+  Create GPU workloads on the GUI, schedule GPU resources, and manage GPU resource quotas by tenant.
+  </details>
+
+
+## Architecture
+
+KubeSphere uses a loosely-coupled architecture that separates the [frontend](https://github.com/kubesphere/console) from the [backend](https://github.com/kubesphere/kubesphere). External systems can access the components of the backend through the REST APIs. 
+
+![Architecture](docs/images/architecture.png)
 
 ----
 
-## Latest Release
-
-KubeSphere 3.0.0 is now generally available! See the [Release Notes For 3.0.0](https://kubesphere.io/docs/release/release-v300/) for the updates.
+## Latest release
 
+🎉 KubeSphere 3.2.1 was released on Dec 20! It brought enhancements and better user experience, see the [Release Notes For 3.2.1](https://kubesphere.io/docs/release/release-v321/) for the updates.
 ## Installation
 
-KubeSphere can run anywhere from on-premise datacenter to any cloud to edge. In addition, it can be deployed on any version-compatible running Kubernetes cluster.
+KubeSphere can run anywhere from on-premise datacenter to any cloud to edge. In addition, it can be deployed on any version-compatible Kubernetes cluster. The installer will start a minimal installation by default, you can [enable other pluggable components before or after installation](https://kubesphere.io/docs/quick-start/enable-pluggable-components/).
+### Quick start
+#### Installing on K8s/K3s
 
-### QuickStarts
+If your cluster meets the [prerequisites](https://kubesphere.io/docs/quick-start/minimal-kubesphere-on-k8s/#prerequisites), then run the following commands to install KubeSphere on an exiting Kubernetes cluster:
 
-[Quickstarts](https://kubesphere.io/docs/quick-start/) include six hands-on lab exercises that help you quickly get started with KubeSphere.
+```yaml
+kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/kubesphere-installer.yaml
+   
+kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/cluster-configuration.yaml
+```
+#### All-in-one
 
-### Installing on Existing Kubernetes Cluster
+👨‍💻 No Kubernetes? You can use [KubeKey](https://github.com/kubesphere/kubekey) to install both KubeSphere and Kubernetes/K3s in single-node mode on your Linux machine. Let's take K3s as an example:
 
-- [Installing KubeSphere on Amazon EKS](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-eks/)
-- [Installing KubeSphere on Azure AKS](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-aks/)
-- [Installing KubeSphere on Google GKE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-gke/)
-- [Installing KubeSphere on DigitalOcean Kubernetes](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-do/)
-- [Installing KubeSphere on Oracle OKE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-oke/)
-- [Installing KubeSphere on Tencent TKE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-tencent-tke/)
-- [Installing KubeSphere on Huaweicloud CCE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-huawei-cce/)
+```yaml
+# Download KubeKey
+curl -sfL https://get-kk.kubesphere.io | VERSION=v2.0.0 sh -
+# Make kk executable
+chmod +x kk
+# Create a cluster
+./kk create cluster --with-kubernetes v1.21.4-k3s --with-kubesphere v3.2.1
+```
 
-### Installing on Linux
+You can run the following command to view the installation logs. After KubeSphere is successfully installed, you can access the KubeSphere web console at `http://IP:30880` and log in using the default administrator account (admin/P@88w0rd).
 
-- [Installing KubeSphere on Azure VM](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/install-ks-on-azure-vms/)
-- [Installing KubeSphere on VMware vSphere](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/on-premises/install-kubesphere-on-vmware-vsphere/)
-- [Installing KubeSphere on QingCloud Instance](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/kubesphere-on-qingcloud-instance/)
-- [Installing on Alibaba Cloud ECS](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/install-kubesphere-on-ali-ecs/)
-- [Installing on Huaweicloud VM](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/install-ks-on-huaweicloud-ecs/)
+```yaml
+kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
+``` 
+### 🐯 Katacoda for quick learning
 
-## Contributing, Support, Discussion, and Community
+[Katacoda](https://www.katacoda.com/) allows you to explore how to install KubeSphere on an existing Kubernetes cluster in a browser. You can start the [Katacoda scenario with KubeSphere](https://www.katacoda.com/kubesphere/scenarios/install-kubesphere-on-kubernetes) in minutes.
+### KubeSphere for hosted Kubernetes services
+
+KubeSphere is hosted on the following cloud providers, you can try KubeSphere by one-click installation on their hosted Kubernetes services. 
+
+- [KubeSphere for Amazon EKS](https://aws.amazon.com/quickstart/architecture/qingcloud-kubesphere/)
+- [KubeSphere for Azure AKS](https://market.azure.cn/marketplace/apps/qingcloud.kubesphere)
+- [KubeSphere for DigitalOcean Kubernetes](https://marketplace.digitalocean.com/apps/kubesphere)
+- [KubeSphere on QingCloud AppCenter(QKE)](https://www.qingcloud.com/products/kubesphereqke)
+
+You can also install KubeSphere on other hosted Kubernetes services within minutes, see the [step-by-step guides](https://kubesphere.io/docs/installing-on-kubernetes/) to get started.
+
+> 👨‍💻 No internet access? Refer to the [Air-gapped Installation on Kubernetes](https://kubesphere.io/docs/installing-on-kubernetes/on-prem-kubernetes/install-ks-on-linux-airgapped/) or [Air-gapped Installation on Linux](https://kubesphere.io/docs/installing-on-linux/introduction/air-gapped-installation/) for instructions on how to use private registry to install KubeSphere.
+## Contributing, support, discussion, and community
 
 We :heart: your contribution. The [community](https://github.com/kubesphere/community) walks you through how to get started contributing KubeSphere. The [development guide](https://github.com/kubesphere/community/tree/master/developer-guide/development) explains how to set up development environment.
 
@@ -106,18 +175,16 @@ We :heart: your contribution. The [community](https://github.com/kubesphere/comm
 - [Follow us on Twitter](https://twitter.com/KubeSphere)
 
 Please submit any KubeSphere bugs, issues, and feature requests to [KubeSphere GitHub Issue](https://github.com/kubesphere/kubesphere/issues).
-
 ## Who are using KubeSphere
 
-The [user case studies](https://kubesphere.io/case/) page includes the user list of the project. You can [submit a PR](https://github.com/kubesphere/kubesphere/blob/master/docs/powered-by-kubesphere.md) to add your institution name and homepage if you are using KubeSphere.
-
+The [user case studies](https://kubesphere.io/case/) page includes the user list of the project. You can [leave a comment](https://github.com/kubesphere/kubesphere/issues/4123) to let us know your use case.
 ## Landscapes
 
 <p align="center">
 <br/><br/>
-<img src="https://landscape.cncf.io/images/left-logo.svg" width="150"/>&nbsp;&nbsp;<img src="https://landscape.cncf.io/images/right-logo.svg" width="200"/>&nbsp;&nbsp;<img src="https://www.cncf.io/wp-content/uploads/2017/11/certified_kubernetes_color.png" height="40" width="30"/>
+<img src="https://landscape.cncf.io/images/left-logo.svg" width="150"/>&nbsp;&nbsp;<img src="https://landscape.cncf.io/images/right-logo.svg" width="200"/>&nbsp;&nbsp;
 <br/><br/>
 KubeSphere is a member of CNCF and a <a href="https://www.cncf.io/certification/software-conformance/#logos">Kubernetes Conformance Certified platform
-</a>, which enriches the <a href="https://landscape.cncf.io/landscape=observability-and-analysis&license=apache-license-2-0">CNCF CLOUD NATIVE Landscape.
+</a>, which enriches the <a href="https://landscape.cncf.io/?landscape=observability-and-analysis&license=apache-license-2-0">CNCF CLOUD NATIVE Landscape.
 </a>
 </p>

README_zh.md

@@ -1,10 +1,20 @@
-# KubeSphere 容器平台
+<p align="center">
+<a href="https://kubesphere.com.cn/"><img src="docs/images/kubesphere-icon.gif" alt="banner" width="200px"></a>
+</p>
 
-[![License](http://img.shields.io/badge/license-apache%20v2-blue.svg)](https://github.com/KubeSphere/KubeSphere/blob/master/LICENSE)
-[![Build Status](https://travis-ci.org/kubesphere/kubesphere.svg?branch=master)](https://travis-ci.org/kubesphere/kubesphere)
-[![KubeSphere release](https://img.shields.io/github/release/kubesphere/kubesphere.svg?color=release&label=release&logo=release&logoColor=release)](https://github.com/kubesphere/kubesphere/releases/tag/v3.0.0)
+<p align="center">
+<b>适用于<i> Kubernetes 多云、数据中心和边缘 </i>管理的容器平台</b>
+</p>
+
+<p align=center>
+<a href="https://goreportcard.com/report/github.com/kubesphere/kubesphere"><img src="https://goreportcard.com/badge/github.com/kubesphere/kubesphere" alt="A+"></a>
+<a href="https://hub.docker.com/r/kubesphere/ks-installer"><img src="https://img.shields.io/docker/pulls/kubesphere/ks-installer"></a>
+<a href="https://github.com/search?q=user%3Akubesphere+user%3Akubesphere-sigs+label%3A%22good+first+issue%22+state%3Aopen&type=Issues&ref=advsearch&l=&l="><img src="https://img.shields.io/github/issues/badges/shields/good%20first%20issue" alt="good first"></a>
+<a href="https://twitter.com/intent/follow?screen_name=KubeSphere"><img src="https://img.shields.io/twitter/follow/KubeSphere?style=social" alt="follow on Twitter"></a>
+<a href="https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE"><img src="https://img.shields.io/badge/Slack-600%2B-blueviolet?logo=slack&amp;logoColor=white"></a>
+<a href="https://www.youtube.com/channel/UCyTdUQUYjf7XLjxECx63Hpw"><img src="https://img.shields.io/youtube/channel/subscribers/UCyTdUQUYjf7XLjxECx63Hpw?style=social"></a>
+</p>
 
-![logo](docs/images/kubesphere-logo.png)
 
 ----
 
@@ -12,9 +22,9 @@
 
 > [English](README.md) | 中文
 
-[KubeSphere](https://kubesphere.com.cn) 是在 [Kubernetes](https://kubernetes.io) 之上构建的面向云原生应用的 **容器混合云**，支持多云与多集群管理，提供全栈的 IT 自动化运维的能力，简化企业的 DevOps 工作流。KubeSphere 提供了运维友好的向导式操作界面，帮助企业快速构建一个强大和功能丰富的容器云平台。KubeSphere 愿景是打造一个基于 Kubernetes 的云原生分布式操作系统，它的架构可以很方便地与云原生生态进行即插即用（plug-and-play）的集成。
+[KubeSphere](https://kubesphere.io/zh/) 愿景是打造一个以 [Kubernetes](https://kubernetes.io/zh/) 为内核的 **云原生分布式操作系统**，它的架构可以非常方便地使第三方应用与云原生生态组件进行即插即用（plug-and-play）的集成，支持云原生应用在多云与多集群的统一分发和运维管理。 KubeSphere 也是一个多租户容器平台，提供全栈的 IT 自动化运维的能力，简化企业的 DevOps 工作流。KubeSphere 提供了运维友好的向导式操作界面，帮助企业快速构建一个强大和功能丰富的容器云平台，详情请参阅 [平台功能](#平台功能) 。
 
-KubeSphere 目前最新的版本为 3.0.0，所有版本 100% 开源，关于 KubeSphere 更详细的介绍与说明请参阅 [什么是 KubeSphere](https://kubesphere.com.cn/docs/introduction/what-is-kubesphere/)。
+下面的屏幕截图让我们进一步了解 KubeSphere，关于 KubeSphere 更详细的介绍与说明请参阅 [什么是 KubeSphere](https://kubesphere.io/zh/docs/introduction/what-is-kubesphere/) 。
 
 <table>
   <tr>
@@ -35,89 +45,150 @@ KubeSphere 目前最新的版本为 3.0.0，所有版本 100% 开源，关于 Ku
   </tr>
 </table>
 
-## 快速体验
+## Demo 环境
 
-使用体验账号 `demo1 / Demo123` 登录 [Demo 环境](https://demo.kubesphere.io/)，该账号仅授予了 view 权限，建议自行安装体验完整的管理功能。您还可以访问 Youtube 查看 [KubeSphere Demo 视频](https://youtu.be/u5lQvhi_Xlc)。
+🎮 使用账号 `demo1 / Demo123` 登录 [Demo 环境](https://demo.kubesphere.io/) 。请注意，该帐户仅授予了 view 权限。
 
-## 架构
+🖥 您还可以快速查看[Demo 视频](https://youtu.be/YxZ1YUv0CYs) 。
 
-KubeSphere 采用了前后端分离的架构设计，后端的各个功能组件可通过 REST API 对接外部系统，详见 [架构说明](https://kubesphere.com.cn/docs/zh-CN/introduction/architecture/)。本仓库仅包含后端代码，前端代码参考 [Console 项目](https://github.com/kubesphere/console)。
+## 平台功能
+
+<details>
+  <summary><b>🕸 部署 Kubernetes 集群</b></summary>
+  支持在任何基础设施上部署 Kubernetes，支持在线安装和离线安装，<a href="https://kubesphere.io/zh/docs/installing-on-linux/introduction/intro/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>🔗 Kubernetes 多集群管理</b></summary>
+  提供集中控制平台来管理多个 Kubernetes 集群，支持将应用程序发布到跨不同云供应商的多个 k8s 集群上。
+  </details>
+
+<details>
+  <summary><b>🤖 Kubernetes DevOps</b></summary>
+  提供开箱即用的基于 Jenkins 的 CI/CD，并内置自动化流水线插件，包括 Binary-to-Image (B2I) 和 Source-to-Image (S2I)，<a href="https://kubesphere.io/zh/devops/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>🔎 云原生可观测性</b></summary>
+  支持多维度监控、事件和审计日志；内置多租户日志查询和收集，告警和通知，<a href="https://kubesphere.io/zh/observability/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>🧩 基于 Istio 的微服务治理</b></summary>
+  为分布式微服务应用程序提供细粒度的流量管理、可观测性和服务跟踪，支持可视化的流量拓扑，<a href="https://kubesphere.io/zh/service-mesh/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>💻 应用商店</b></summary>
+  为基于 Helm 的应用程序提供应用商店，并在 Kubernetes 平台上提供应用程序生命周期管理功能，<a href="https://kubesphere.io/zh/docs/pluggable-components/app-store/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>💡 Kubernetes 边缘节点管理</b></summary>
+  基于 <a href="https://kubeedge.io/zh/">KubeEdge</a> 实现应用与工作负载在云端与边缘节点的统一分发与管理，解决在海量边、端设备上完成应用交付、运维、管控的需求，<a href= "https://kubesphere.io/zh/docs/pluggable-components/kubeedge/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>📊 多维度计量与计费</b></summary>
+  提供基于集群与租户的多维度资源计量与计费的监控报表，让 Kubernetes 运营成本更透明，<a href="https://kubesphere.io/zh/docs/toolbox/metering-and-billing/view-resource-consumption/">了解更多</a>。
+  </details>
+
+<details>
+  <summary><b>🗃 支持多种存储和网络解决方案</b></summary>
+  <li>支持 GlusterFS、CephRBD、NFS、LocalPV ，并提供多个 CSI 插件对接公有云与企业级存储。</li><li>提供 Kubernetes 在裸机、边缘和虚拟化中的负载均衡器实现 <a href="https://github.com/kubesphere/openelb">OpenELB</a> 。</li><li>提供网络策略和容器组 IP 池管理，支持 Calico、Flannel、Kube-OVN。</li>
+  </details>
+
+<details>
+  <summary><b>🏘 多租户与统一鉴权认证</b></summary>
+  提供统一的认证鉴权与细粒度的基于角色的授权系统，支持对接 AD/LDAP 。
+  </details>
+
+<details>
+  <summary><b>🧠 GPU 工作负载调度与监控</b></summary>
+  支持可视化创建 GPU 工作负载，支持 GPU 监控，同时还支持对 GPU 资源进行租户级配额管理。
+  </details>
+
+## 架构说明
+
+KubeSphere 使用前后端分离的架构，将 [前端](https://github.com/kubesphere/console) 与 [后端](https://github.com/kubesphere/kubesphere) 分开。后端的各个功能组件可通过 REST API 对接外部系统。
 
 ![Architecture](docs/images/architecture.png)
 
-## 核心功能
-
-|功能 |介绍 |
-| --- | ---|
-|多云与多集群管理|提供多云与多集群的中央管理面板，支持集群导入，支持应用在多云与多集群一键分发|
-| Kubernetes 集群搭建与运维 | 支持在线 & 离线安装、升级与扩容 K8s 集群，支持安装 “云原生全家桶” |
-| Kubernetes 资源可视化管理 | 可视化纳管原生 Kubernetes 资源，支持向导式创建与管理 K8s 资源 |
-| 基于 Jenkins 的 DevOps 系统 | 支持图形化与脚本两种方式构建 CI/CD 流水线，内置 Source to Image（S2I）和 Binary to Image（B2I）等 CD 工具 |
-| 应用商店与应用生命周期管理 | 提供应用商店，内置 Redis、MySQL 等 15 个常用应用，支持应用的生命周期管理 |
-| 基于 Istio 的微服务治理 (Service Mesh) | 提供可视化无代码侵入的 **灰度发布、熔断、流量治理与流量拓扑、分布式 Tracing** |
-| 多租户管理 | 提供基于角色的细粒度多租户统一认证，支持 **对接企业 LDAP/AD**，提供多层级的权限管理 |
-| 丰富的可观察性功能 | 提供集群/工作负载/Pod/容器等多维度的监控，提供基于多租户的日志查询与日志收集，支持节点与应用层级的告警与通知 |
-|基础设施管理|支持 Kubernetes 节点管理，支持节点扩容与集群升级，提供基于节点的多项监控指标与告警规则 |
-| 存储管理 | 支持对接 Ceph、GlusterFS、NFS、Local PV，支持可视化运维管理 PVC、StorageClass，提供 CSI 插件对接云平台存储 |
-| 网络管理 | 提供租户网络隔离与 K8s [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) 管理，支持 Calico、Flannel，提供 [Porter LB](https://github.com/kubesphere/porter) 用于暴露物理环境 K8s 集群的 LoadBalancer 服务 |
-| GPU support | 集群支持添加 GPU 与 vGPU，可运行 TensorFlow 等 ML 框架 |
-
-以上功能说明详见 [产品功能](https://kubesphere.com.cn/docs/zh-CN/introduction/features/)。
-
 ----
 
-## 最新发布
+## 最新版本
 
-KubeSphere 3.0.0 已于 2020 年 8 月 31 日正式 GA！点击 [Release Notes For 3.0.0](https://kubesphere.com.cn/docs/release/release-v300/) 查看 3.0.0 版本的更新详情。
-
-## 安装 3.0.0
+🎉 KubeSphere 3.2.1 全新发布！！多项功能优化，带来更好的用户体验，详见 [v3.2.1 发行记录](https://kubesphere.com.cn/docs/release/release-v321/) 。
+## 安装
 
+KubeSphere 支持在任意平台运行，从本地数据中心到混合多云再走向边缘。此外，KubeSphere 可以部署在任何版本兼容的 Kubernetes 集群上。Installer 默认将执行最小化安装，您可以在安装前或安装后自定义[安装可插拔功能组件](https://kubesphere.com.cn/docs/quick-start/enable-pluggable-components/)。
 ### 快速入门
+#### 在 K8s/K3s 上安装
 
-[快速入门系列](https://kubesphere.com.cn/docs/quick-start/) 提供了快速安装与入门示例，供初次安装体验参考。
+请确保您的集群满足安装的[前提条件](https://kubesphere.io/zh/docs/quick-start/minimal-kubesphere-on-k8s/)，运行以下命令以在现有 Kubernetes 集群上安装 KubeSphere：
 
-### 在已有 Kubernetes 之上安装 KubeSphere
+```yaml
+kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/kubesphere-installer.yaml
+   
+kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/cluster-configuration.yaml
+```
+#### All-in-one（Linux 单节点安装）
 
-- [基于 Kubernetes 的安装介绍](https://kubesphere.com.cn/docs/installing-on-kubernetes/introduction/overview/)
-- [在阿里云 ACK 安装 KubeSphere](https://kubesphere.com.cn/forum/d/1745-kubesphere-v3-0-0-dev-on-ack)
-- [在腾讯云 TKE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-tencent-tke/)
-- [在华为云 CCE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-huawei-cce/)
-- [在 AWS EKS 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-eks/)
-- [在 Google GKE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-gke/)
-- [在 Azure AKS 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-aks/)
-- [在 DigitalOcean 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-do/)
-- [在 Oracle OKE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-oke/)
+👨‍💻 没有 Kubernetes 集群? 可以用 [KubeKey](https://github.com/kubesphere/kubekey) 在 Linux 环境以 All-in-one 快速安装单节点 K8s/K3s 和 KubeSphere，下面以 K3s 为例：
 
-### 基于 Linux 安装 KubeSphere
+```yaml
+# 下载 KubeKey
+curl -sfL https://get-kk.kubesphere.io | VERSION=v1.2.0 sh -
+# 为 kk 赋予可执行权限
+chmod +x kk
+# 创建集群
+./kk create cluster --with-kubernetes v1.21.4-k3s --with-kubesphere v3.2.1
+```
 
-- [多节点安装介绍（以三节点为例）](https://kubesphere.com.cn/docs/installing-on-linux/introduction/multioverview/)
-- [在 VMware vSphere 安装高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/on-premises/install-kubesphere-on-vmware-vsphere/)
-- [在青云QingCloud 安装高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/kubesphere-on-qingcloud-instance/)
-- [在阿里云 ECS 部署高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/install-kubesphere-on-ali-ecs/)
+可使用以下命令查看安装日志。如果安装成功，可使用 `http://IP:30880` 访问 KubeSphere Console，管理员登录帐密为 `admin/P@88w0rd`。
 
-- [在华为云 VM 部署高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/install-kubesphere-on-huaweicloud-ecs/)
-- [在 Azure VM 安装高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/install-kubesphere-on-azure-vms/)
+```yaml
+kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
+``` 
+### 🐯 使用 Katacoda 在线安装体验 KubeSphere
 
-## 技术社区
+[Katacoda](https://www.katacoda.com/) 是一个在线的云原生技术学习实验平台，你可以使用它在浏览器中快速 [安装体验 KubeSphere](https://www.katacoda.com/kubesphere/scenarios/install-kubesphere-on-kubernetes) 。
+### 在托管 Kubernetes 上部署 KubeSphere
 
-[KubeSphere 社区](https://github.com/kubesphere/community) 包含所有社区的信息，包括如何开发，兴趣小组(SIG)等。比如[开发指南](https://github.com/kubesphere/community/tree/master/developer-guide/development) 详细说明了如何从源码编译、KubeSphere 的 GitHub 工作流、如何贡献代码以及如何测试等。
+KubeSphere 托管在以下云供应商上，您可以通过在其托管的 Kubernetes 服务上一键安装来部署 KubeSphere。
+
+- [在 Amazon EKS 上部署 KubeSphere](https://aws.amazon.com/quickstart/architecture/qingcloud-kubesphere/)
+- [在 Azure AKS 上部署 KubeSphere](https://market.azure.cn/marketplace/apps/qingcloud.kubesphere)
+- [在 DigitalOcean 上部署 KubeSphere](https://marketplace.digitalocean.com/apps/kubesphere)
+- [在青云QingCloud QKE 上部署 KubeSphere](https://www.qingcloud.com/products/kubesphereqke)
+
+您还可以在几分钟内在其他托管的 Kubernetes 服务上安装 KubeSphere，请参阅 [官方文档](https://kubesphere.io/zh/docs/installing-on-kubernetes/) 以开始使用。
+
+> 👨‍💻 不能访问网络？参考 [在Kubernetes上离线安装](https://kubesphere.io/zh/docs/installing-on-kubernetes/on-prem-kubernetes/install-ks-on-linux-airgapped/) 或者 [在 Linux 上离线安装](https://kubesphere.io/zh/docs/installing-on-linux/introduction/air-gapped-installation/) 了解如何使用私有仓库来安装 KubeSphere。
+
+## 贡献、支持、讨论和社区
+
+我们 :heart: 您的贡献。[社区](https://github.com/kubesphere/community) 将引导您了解如何开始贡献 KubeSphere。[开发指南](https://github.com/kubesphere/community/tree/master/developer-guide/development) 说明了如何安装开发环境。
 
 - [中文论坛](https://kubesphere.com.cn/forum/)
-- [Slack Channel](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
 - [社区微信群（见官网底部）](https://kubesphere.com.cn/)
-- [Bug 与建议反馈（GitHub Issue）](https://github.com/kubesphere/kubesphere/issues)
+- [Slack Channel](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
+- [Bilibili](https://space.bilibili.com/438908638)
+- [在推特上关注我们](https://twitter.com/KubeSphere)
+
+请将任何 KubeSphere 的 Bug、问题和需求提交到 [KubeSphere GitHub Issue](https://github.com/kubesphere/kubesphere/issues) 。
 
 ## 谁在使用 KubeSphere
 
-[Powered by KubeSphere](https://kubesphere.com.cn/case/) 列出了哪些企业在使用 KubeSphere，如果您所在的企业已安装使用了 KubeSphere，欢迎[提交 PR](https://github.com/kubesphere/kubesphere/blob/master/docs/powered-by-kubesphere.md)。
+[用户案例学习](https://kubesphere.com.cn/case/) 列出了哪些企业在使用 KubeSphere。欢迎 [发表评论](https://github.com/kubesphere/kubesphere/issues/4123) 来分享您的使用案例。
 
 ## Landscapes
 
 <p align="center">
 <br/><br/>
-<img src="https://landscape.cncf.io/images/left-logo.svg" width="150"/>&nbsp;&nbsp;<img src="https://landscape.cncf.io/images/right-logo.svg" width="200"/>&nbsp;&nbsp;<img src="https://www.cncf.io/wp-content/uploads/2017/11/certified_kubernetes_color.png" height="40" width="30"/>
+<img src="https://landscape.cncf.io/images/left-logo.svg" width="150"/>&nbsp;&nbsp;<img src="https://landscape.cncf.io/images/right-logo.svg" width="200"/>&nbsp;&nbsp;
 <br/><br/>
 KubeSphere 是 CNCF 基金会成员并且通过了 <a href="https://www.cncf.io/certification/software-conformance/#logos">Kubernetes 一致性认证
-</a>，进一步丰富了 <a href="https://landscape.cncf.io/landscape=observability-and-analysis&license=apache-license-2-0">CNCF 云原生的生态。
+</a>，进一步丰富了 <a href="https://landscape.cncf.io/?landscape=observability-and-analysis&license=apache-license-2-0">CNCF 云原生的生态。
 </a>
 </p>

SECURITY.md

@@ -0,0 +1,50 @@
+# Security Policy
+
+## Supported Versions
+
+Use this section to tell people about which versions of your project are
+currently being supported with security updates.
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 3.2.x   | :white_check_mark: |
+| 3.1.x   | :white_check_mark: |
+| 3.0.x   | :white_check_mark: |
+| 2.1.x   | :white_check_mark: |
+| < 2.1.x   | :x:                |
+
+## Reporting a Vulnerability
+
+# Security Vulnerability Disclosure and Response Process
+
+To ensure KubeSphere security, a security vulnerability disclosure and response process is adopted. And the security team is set up in KubeSphere community, also any issue and PR is welcome for every contributors.
+
+The primary goal of this process is to reduce the total exposure time of users to publicly known vulnerabilities. To quickly fix vulnerabilities of KubeSphere, the security team is responsible for the entire vulnerability management process, including internal communication and external disclosure.
+
+If you find a vulnerability or encounter a security incident involving vulnerabilities of KubeSphere, please report it as soon as possible to the KubeSphere security team (security@kubesphere.io).
+
+Please kindly help provide as much vulnerability information as possible in the following format:
+
+- Issue title(Please add 'Security' lable)*:
+
+- Overview*:
+
+- Affected components and version number*:
+
+- CVE number (if any):
+
+- Vulnerability verification process*:
+
+- Contact information*:
+
+The asterisk (*) indicates the required field.
+
+# Response Time
+
+The KubeSphere security team will confirm the vulnerabilities and contact you within 2 working days after your submission.
+
+We will publicly thank you after fixing the security vulnerability. To avoid negative impact, please keep the vulnerability confidential until we fix it. We would appreciate it if you could obey the following code of conduct:
+
+The vulnerability will not be disclosed until KubeSphere releases a patch for it.
+
+The details of the vulnerability, for example, exploits code, will not be disclosed.

api/api-rules/violation_exceptions.list

@@ -1,6 +1,3 @@
-API rule violation: list_type_missing,./pkg/apis/devops/v1alpha3,DevOpsProjectList,Items
-API rule violation: list_type_missing,./pkg/apis/devops/v1alpha3,NoScmPipeline,Parameters
-API rule violation: list_type_missing,./pkg/apis/devops/v1alpha3,PipelineList,Items
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIGroup,ServerAddressByClientCIDRs
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIGroup,Versions
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIGroupList,Groups
@@ -9,16 +6,15 @@ API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIRe
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIResourceList,APIResources
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIVersions,ServerAddressByClientCIDRs
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIVersions,Versions
+API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,ApplyOptions,DryRun
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,CreateOptions,DryRun
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,DeleteOptions,DryRun
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,FieldsV1,Raw
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,LabelSelector,MatchExpressions
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,LabelSelectorRequirement,Values
-API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,List,Items
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,ObjectMeta,Finalizers
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,ObjectMeta,ManagedFields
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,ObjectMeta,OwnerReferences
-API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,PartialObjectMetadataList,Items
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,PatchOptions,DryRun
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,RootPaths,Paths
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,StatusDetails,Causes
@@ -29,63 +25,7 @@ API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,Table
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,UpdateOptions,DryRun
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/runtime,RawExtension,Raw
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/runtime,Unknown,Raw
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,ApiUri
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromForks
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromOrigin
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,DiscarderProperty,DaysToKeep
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,DiscarderProperty,NumToKeep
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,ApiUri
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverPRFromForks
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverPRFromOrigin
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,ApiUri
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverPRFromForks
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverPRFromOrigin
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,ServerName
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchJobTrigger,CreateActionJobsToTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchJobTrigger,DeleteActionJobsToTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,BitbucketServerSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitHubSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitlabSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,MultiBranchJobTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,ScriptPath
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,SingleSvnSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,SourceType
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,SvnSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,TimerTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,NoScmPipeline,DisableConcurrent
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,NoScmPipeline,RemoteTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,NoScmPipeline,TimerTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,Parameter,DefaultValue
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,PipelineSpec,MultiBranchPipeline
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SingleSvnSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SingleSvnSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SvnSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SvnSource,ScmId
+API rule violation: list_type_missing,kubesphere.io/api/devops/v1alpha3,NoScmPipeline,Parameters
 API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,APIResourceList,APIResources
 API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,Duration,Duration
 API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,InternalEvent,Object
@@ -96,3 +36,60 @@ API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,Time,Time
 API rule violation: names_match,k8s.io/apimachinery/pkg/runtime,Unknown,ContentEncoding
 API rule violation: names_match,k8s.io/apimachinery/pkg/runtime,Unknown,ContentType
 API rule violation: names_match,k8s.io/apimachinery/pkg/runtime,Unknown,Raw
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,ApiUri
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,CloneOption
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,CredentialId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromForks
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromOrigin
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,DiscoverTags
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,RegexFilter
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,BitbucketServerSource,ScmId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,DiscarderProperty,DaysToKeep
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,DiscarderProperty,NumToKeep
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitSource,CloneOption
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitSource,CredentialId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitSource,DiscoverTags
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitSource,RegexFilter
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitSource,ScmId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,ApiUri
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,CloneOption
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,CredentialId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,DiscoverPRFromForks
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,DiscoverPRFromOrigin
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,DiscoverTags
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,RegexFilter
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GithubSource,ScmId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,ApiUri
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,CloneOption
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,CredentialId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,DiscoverPRFromForks
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,DiscoverPRFromOrigin
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,DiscoverTags
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,RegexFilter
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,ScmId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,GitlabSource,ServerName
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchJobTrigger,CreateActionJobsToTrigger
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchJobTrigger,DeleteActionJobsToTrigger
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,BitbucketServerSource
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,GitHubSource
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,GitSource
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,GitlabSource
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,MultiBranchJobTrigger
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,ScriptPath
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,SingleSvnSource
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,SourceType
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,SvnSource
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,MultiBranchPipeline,TimerTrigger
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,NoScmPipeline,DisableConcurrent
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,NoScmPipeline,RemoteTrigger
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,NoScmPipeline,TimerTrigger
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,Parameter,DefaultValue
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,PipelineSpec,MultiBranchPipeline
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,SingleSvnSource,CredentialId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,SingleSvnSource,ScmId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,SvnSource,CredentialId
+API rule violation: names_match,kubesphere.io/api/devops/v1alpha3,SvnSource,ScmId

build/ks-apiserver/Dockerfile

@@ -1,23 +1,48 @@
 # Copyright 2020 The KubeSphere Authors. All rights reserved.
 # Use of this source code is governed by an Apache license
 # that can be found in the LICENSE file.
-FROM alpine:3.11
 
+# Download dependencies
+FROM alpine:3.11 as base_os_context
+
+ARG TARGETARCH
+ARG TARGETOS
 ARG HELM_VERSION=v3.5.2
 
+ENV OUTDIR=/out
+RUN mkdir -p ${OUTDIR}/usr/local/bin/
+
+WORKDIR /tmp
+
 RUN apk add --no-cache ca-certificates
+
 # install helm
-RUN wget https://get.helm.sh/helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    tar xvf helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    rm helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    mv linux-amd64/helm /usr/bin/ && \
-    rm -rf linux-amd64
-# To speed up building process, we copy binary directly from make
-# result instead of building it again, so make sure you run the 
-# following command first before building docker image
-#   make ks-apiserver
-#
-COPY  /bin/cmd/ks-apiserver /usr/local/bin/
+ADD https://get.helm.sh/helm-${HELM_VERSION}-${TARGETOS}-${TARGETARCH}.tar.gz /tmp
+RUN tar xvzf /tmp/helm-${HELM_VERSION}-${TARGETOS}-${TARGETARCH}.tar.gz -C /tmp
+RUN mv /tmp/${TARGETOS}-${TARGETARCH}/helm ${OUTDIR}/usr/local/bin/
+
+# Build 
+FROM golang:1.16.3 as build_context
+
+ENV OUTDIR=/out
+RUN mkdir -p ${OUTDIR}/usr/local/bin/
+
+WORKDIR /workspace
+ADD . /workspace/
+
+RUN make ks-apiserver
+RUN mv /workspace/bin/cmd/ks-apiserver ${OUTDIR}/usr/local/bin/
+
+##############
+# Final image
+#############
+
+FROM alpine:3.11 
+
+COPY --from=base_os_context /out/ /
+COPY --from=build_context /out/ /
+
+WORKDIR /
 
 EXPOSE 9090
 CMD ["sh"]

build/ks-controller-manager/Dockerfile

@@ -1,25 +1,66 @@
 # Copyright 2020 The KubeSphere Authors. All rights reserved.
 # Use of this source code is governed by an Apache license
 # that can be found in the LICENSE file.
-FROM alpine:3.11
 
+# Download dependencies
+FROM alpine:3.11 as base_os_context
+
+ARG TARGETARCH
+ARG TARGETOS
 ARG HELM_VERSION=v3.5.2
-ARG KUSTOMIZE_VERSION=v4.1.0
+ARG KUSTOMIZE_VERSION=v4.2.0
+ARG INGRESS_NGINX_VERSION=4.0.13
+
+ENV OUTDIR=/out
+RUN mkdir -p ${OUTDIR}/usr/local/bin
+RUN mkdir -p ${OUTDIR}/var/helm-charts
+
+WORKDIR /tmp
 
 RUN apk add --no-cache ca-certificates
-# install helm
-RUN wget https://get.helm.sh/helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    tar xvf helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    rm helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    mv linux-amd64/helm /usr/bin/ && \
-    rm -rf linux-amd64
+
+# Install helm
+ADD https://get.helm.sh/helm-${HELM_VERSION}-${TARGETOS}-${TARGETARCH}.tar.gz /tmp
+RUN tar xvzf /tmp/helm-${HELM_VERSION}-${TARGETOS}-${TARGETARCH}.tar.gz -C /tmp
+RUN mv /tmp/${TARGETOS}-${TARGETARCH}/helm ${OUTDIR}/usr/local/bin/
+
 # install kustomize
-RUN wget https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F${KUSTOMIZE_VERSION}/kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
-    tar xvf kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
-    rm kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
-    mv kustomize /usr/bin
-    
-COPY  /bin/cmd/controller-manager /usr/local/bin/
+ADD https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F${KUSTOMIZE_VERSION}/kustomize_${KUSTOMIZE_VERSION}_${TARGETOS}_${TARGETARCH}.tar.gz /tmp
+RUN tar xvzf /tmp/kustomize_${KUSTOMIZE_VERSION}_${TARGETOS}_${TARGETARCH}.tar.gz -C /tmp
+RUN mv /tmp/kustomize ${OUTDIR}/usr/local/bin/
+
+
+# Install Nginx Ingress Helm Chart
+ADD https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-${INGRESS_NGINX_VERSION}/ingress-nginx-${INGRESS_NGINX_VERSION}.tgz /tmp
+RUN tar xvzf /tmp/ingress-nginx-${INGRESS_NGINX_VERSION}.tgz -C /tmp
+RUN mv /tmp/ingress-nginx ${OUTDIR}/var/helm-charts/
+
+# Build
+
+FROM golang:1.16.3 as build_context
+
+ENV OUTDIR=/out
+RUN mkdir -p ${OUTDIR}/usr/local/bin/
+RUN mkdir -p ${OUTDIR}/var/helm-charts
+
+WORKDIR /workspace
+ADD . /workspace/
+
+RUN make ks-controller-manager
+RUN mv /workspace/bin/cmd/controller-manager ${OUTDIR}/usr/local/bin/
+
+# Copy gateway config and helm chart
+RUN mv /workspace/config/gateway ${OUTDIR}/var/helm-charts/
+RUN mv /workspace/config/watches.yaml ${OUTDIR}/var/helm-charts/
+
+# Final Image
+
+FROM alpine:3.11 
+
+COPY --from=base_os_context /out/ /
+COPY --from=build_context /out/ /
+
+WORKDIR /
 
 EXPOSE 8443 8080
 

cmd/controller-manager/app/controllers.go

@@ -17,20 +17,48 @@ limitations under the License.
 package app
 
 import (
-	"k8s.io/apimachinery/pkg/runtime"
+	"fmt"
+	"time"
+
+	"github.com/kubesphere/pvc-autoresizer/runners"
+	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apimachinery/pkg/util/sets"
 	"k8s.io/client-go/tools/cache"
 	"k8s.io/klog"
+	ctrl "sigs.k8s.io/controller-runtime"
+	runtimeclient "sigs.k8s.io/controller-runtime/pkg/client"
 	"sigs.k8s.io/controller-runtime/pkg/manager"
 	"sigs.k8s.io/kubefed/pkg/controller/util"
 
-	iamv1alpha2 "kubesphere.io/kubesphere/pkg/apis/iam/v1alpha2"
-	authoptions "kubesphere.io/kubesphere/pkg/apiserver/authentication/options"
+	"kubesphere.io/kubesphere/cmd/controller-manager/app/options"
+	"kubesphere.io/kubesphere/pkg/controller/application"
+	"kubesphere.io/kubesphere/pkg/controller/helm"
+	"kubesphere.io/kubesphere/pkg/controller/namespace"
+	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmapplication"
+	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmcategory"
+	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmrelease"
+	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmrepo"
+	"kubesphere.io/kubesphere/pkg/controller/quota"
+	"kubesphere.io/kubesphere/pkg/controller/serviceaccount"
+	"kubesphere.io/kubesphere/pkg/controller/user"
+	"kubesphere.io/kubesphere/pkg/controller/workspace"
+	"kubesphere.io/kubesphere/pkg/controller/workspacerole"
+	"kubesphere.io/kubesphere/pkg/controller/workspacerolebinding"
+	"kubesphere.io/kubesphere/pkg/controller/workspacetemplate"
+	"kubesphere.io/kubesphere/pkg/models/kubeconfig"
+	"kubesphere.io/kubesphere/pkg/simple/client/devops"
+	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
+	ldapclient "kubesphere.io/kubesphere/pkg/simple/client/ldap"
+	"kubesphere.io/kubesphere/pkg/simple/client/s3"
+
+	"kubesphere.io/kubesphere/pkg/controller/storage/snapshotclass"
+
+	iamv1alpha2 "kubesphere.io/api/iam/v1alpha2"
+
 	"kubesphere.io/kubesphere/pkg/controller/certificatesigningrequest"
 	"kubesphere.io/kubesphere/pkg/controller/cluster"
 	"kubesphere.io/kubesphere/pkg/controller/clusterrolebinding"
 	"kubesphere.io/kubesphere/pkg/controller/destinationrule"
-	"kubesphere.io/kubesphere/pkg/controller/devopscredential"
-	"kubesphere.io/kubesphere/pkg/controller/devopsproject"
 	"kubesphere.io/kubesphere/pkg/controller/globalrole"
 	"kubesphere.io/kubesphere/pkg/controller/globalrolebinding"
 	"kubesphere.io/kubesphere/pkg/controller/group"
@@ -41,269 +69,509 @@ import (
 	"kubesphere.io/kubesphere/pkg/controller/network/nsnetworkpolicy"
 	"kubesphere.io/kubesphere/pkg/controller/network/nsnetworkpolicy/provider"
 	"kubesphere.io/kubesphere/pkg/controller/notification"
-	"kubesphere.io/kubesphere/pkg/controller/pipeline"
-	"kubesphere.io/kubesphere/pkg/controller/s2ibinary"
-	"kubesphere.io/kubesphere/pkg/controller/s2irun"
 	"kubesphere.io/kubesphere/pkg/controller/storage/capability"
-	"kubesphere.io/kubesphere/pkg/controller/storage/expansion"
-	"kubesphere.io/kubesphere/pkg/controller/user"
 	"kubesphere.io/kubesphere/pkg/controller/virtualservice"
 	"kubesphere.io/kubesphere/pkg/informers"
-	"kubesphere.io/kubesphere/pkg/simple/client/devops"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
-	ldapclient "kubesphere.io/kubesphere/pkg/simple/client/ldap"
-	"kubesphere.io/kubesphere/pkg/simple/client/multicluster"
-	"kubesphere.io/kubesphere/pkg/simple/client/network"
 	ippoolclient "kubesphere.io/kubesphere/pkg/simple/client/network/ippool"
-	"kubesphere.io/kubesphere/pkg/simple/client/s3"
 )
 
-func addControllers(
-	mgr manager.Manager,
-	client k8s.Client,
-	informerFactory informers.InformerFactory,
-	devopsClient devops.Interface,
-	s3Client s3.Interface,
-	ldapClient ldapclient.Interface,
-	options *k8s.KubernetesOptions,
-	authenticationOptions *authoptions.AuthenticationOptions,
-	multiClusterOptions *multicluster.Options,
-	networkOptions *network.Options,
-	serviceMeshEnabled bool,
-	kubectlImage string,
-	stopCh <-chan struct{}) error {
+var allControllers = []string{
+	"user",
+	"workspacetemplate",
+	"workspace",
+	"workspacerole",
+	"workspacerolebinding",
+	"namespace",
 
+	"helmrepo",
+	"helmcategory",
+	"helmapplication",
+	"helmapplicationversion",
+	"helmrelease",
+	"helm",
+
+	"application",
+	"serviceaccount",
+	"resourcequota",
+
+	"virtualservice",
+	"destinationrule",
+	"job",
+	"storagecapability",
+	"volumesnapshot",
+	"pvcautoresizer",
+	"workloadrestart",
+	"loginrecord",
+	"cluster",
+	"nsnp",
+	"ippool",
+	"csr",
+
+	"clusterrolebinding",
+
+	"fedglobalrolecache",
+	"globalrole",
+	"fedglobalrolebindingcache",
+	"globalrolebinding",
+
+	"groupbinding",
+	"group",
+
+	"notification",
+}
+
+// setup all available controllers one by one
+func addAllControllers(mgr manager.Manager, client k8s.Client, informerFactory informers.InformerFactory,
+	cmOptions *options.KubeSphereControllerManagerOptions,
+	stopCh <-chan struct{}) error {
+	var err error
+
+	////////////////////////////////////
+	// begin init necessary informers
+	////////////////////////////////////
 	kubernetesInformer := informerFactory.KubernetesSharedInformerFactory()
 	istioInformer := informerFactory.IstioSharedInformerFactory()
 	kubesphereInformer := informerFactory.KubeSphereSharedInformerFactory()
+	////////////////////////////////////
+	// end informers
+	////////////////////////////////////
 
-	multiClusterEnabled := multiClusterOptions.Enable
+	////////////////////////////////////
+	// begin init necessary clients
+	////////////////////////////////////
+	kubeconfigClient := kubeconfig.NewOperator(client.Kubernetes(),
+		informerFactory.KubernetesSharedInformerFactory().Core().V1().ConfigMaps().Lister(),
+		client.Config())
 
-	var vsController, drController manager.Runnable
+	var devopsClient devops.Interface
+	if cmOptions.DevopsOptions != nil && len(cmOptions.DevopsOptions.Host) != 0 {
+		devopsClient, err = jenkins.NewDevopsClient(cmOptions.DevopsOptions)
+		if err != nil {
+			return fmt.Errorf("failed to connect jenkins, please check jenkins status, error: %v", err)
+		}
+	}
+
+	var ldapClient ldapclient.Interface
+	// when there is no ldapOption, we set ldapClient as nil, which means we don't need to sync user info into ldap.
+	if cmOptions.LdapOptions != nil && len(cmOptions.LdapOptions.Host) != 0 {
+		if cmOptions.LdapOptions.Host == ldapclient.FAKE_HOST { // for debug only
+			ldapClient = ldapclient.NewSimpleLdap()
+		} else {
+			ldapClient, err = ldapclient.NewLdapClient(cmOptions.LdapOptions, stopCh)
+			if err != nil {
+				return fmt.Errorf("failed to connect to ldap service, please check ldap status, error: %v", err)
+			}
+		}
+	} else {
+		klog.Warning("ks-controller-manager starts without ldap provided, it will not sync user into ldap")
+	}
+	////////////////////////////////////
+	// end init clients
+	////////////////////////////////////
+
+	////////////////////////////////////////////////////////
+	// begin init controller and add to manager one by one
+	////////////////////////////////////////////////////////
+
+	// "user" controller
+	if cmOptions.IsControllerEnabled("user") {
+		userController := &user.Reconciler{
+			MultiClusterEnabled:     cmOptions.MultiClusterOptions.Enable,
+			MaxConcurrentReconciles: 4,
+			LdapClient:              ldapClient,
+			DevopsClient:            devopsClient,
+			KubeconfigClient:        kubeconfigClient,
+			AuthenticationOptions:   cmOptions.AuthenticationOptions,
+		}
+		addControllerWithSetup(mgr, "user", userController)
+	}
+
+	// "workspacetemplate" controller
+	if cmOptions.IsControllerEnabled("workspacetemplate") {
+		workspaceTemplateReconciler := &workspacetemplate.Reconciler{MultiClusterEnabled: cmOptions.MultiClusterOptions.Enable}
+		addControllerWithSetup(mgr, "workspacetemplate", workspaceTemplateReconciler)
+	}
+
+	// "workspace" controller
+	if cmOptions.IsControllerEnabled("workspace") {
+		workspaceReconciler := &workspace.Reconciler{}
+		addControllerWithSetup(mgr, "workspace", workspaceReconciler)
+	}
+
+	// "workspacerole" controller
+	if cmOptions.IsControllerEnabled("workspacerole") {
+		workspaceRoleReconciler := &workspacerole.Reconciler{MultiClusterEnabled: cmOptions.MultiClusterOptions.Enable}
+		addControllerWithSetup(mgr, "workspacerole", workspaceRoleReconciler)
+	}
+
+	// "workspacerolebinding" controller
+	if cmOptions.IsControllerEnabled("workspacerolebinding") {
+		workspaceRoleBindingReconciler := &workspacerolebinding.Reconciler{MultiClusterEnabled: cmOptions.MultiClusterOptions.Enable}
+		addControllerWithSetup(mgr, "workspacerolebinding", workspaceRoleBindingReconciler)
+	}
+
+	// "namespace" controller
+	if cmOptions.IsControllerEnabled("namespace") {
+		namespaceReconciler := &namespace.Reconciler{GatewayOptions: cmOptions.GatewayOptions}
+		addControllerWithSetup(mgr, "namespace", namespaceReconciler)
+	}
+
+	// "helmrepo" controller
+	if cmOptions.IsControllerEnabled("helmrepo") {
+		helmRepoReconciler := &helmrepo.ReconcileHelmRepo{}
+		addControllerWithSetup(mgr, "helmrepo", helmRepoReconciler)
+	}
+
+	// "helmcategory" controller
+	if cmOptions.IsControllerEnabled("helmcategory") {
+		helmCategoryReconciler := &helmcategory.ReconcileHelmCategory{}
+		addControllerWithSetup(mgr, "helmcategory", helmCategoryReconciler)
+	}
+
+	var opS3Client s3.Interface
+	if !cmOptions.OpenPitrixOptions.AppStoreConfIsEmpty() {
+		opS3Client, err = s3.NewS3Client(cmOptions.OpenPitrixOptions.S3Options)
+		if err != nil {
+			klog.Fatalf("failed to connect to s3, please check openpitrix s3 service status, error: %v", err)
+		}
+
+		// "helmapplication" controller
+		if cmOptions.IsControllerEnabled("helmapplication") {
+			reconcileHelmApp := (&helmapplication.ReconcileHelmApplication{})
+			addControllerWithSetup(mgr, "helmapplication", reconcileHelmApp)
+		}
+
+		// "helmapplicationversion" controller
+		if cmOptions.IsControllerEnabled("helmapplicationversion") {
+			reconcileHelmAppVersion := (&helmapplication.ReconcileHelmApplicationVersion{})
+			addControllerWithSetup(mgr, "helmapplicationversion", reconcileHelmAppVersion)
+		}
+	}
+
+	// "helmrelease" controller
+	if cmOptions.IsControllerEnabled("helmrelease") {
+		reconcileHelmRelease := &helmrelease.ReconcileHelmRelease{
+			// nil interface is valid value.
+			StorageClient:      opS3Client,
+			KsFactory:          informerFactory.KubeSphereSharedInformerFactory(),
+			MultiClusterEnable: cmOptions.MultiClusterOptions.Enable,
+			WaitTime:           cmOptions.OpenPitrixOptions.ReleaseControllerOptions.WaitTime,
+			MaxConcurrent:      cmOptions.OpenPitrixOptions.ReleaseControllerOptions.MaxConcurrent,
+			StopChan:           stopCh,
+		}
+		addControllerWithSetup(mgr, "helmrelease", reconcileHelmRelease)
+	}
+
+	// "helm" controller
+	if cmOptions.IsControllerEnabled("helm") {
+		if !cmOptions.GatewayOptions.IsEmpty() {
+			helmReconciler := &helm.Reconciler{GatewayOptions: cmOptions.GatewayOptions}
+			addControllerWithSetup(mgr, "helm", helmReconciler)
+		}
+	}
+
+	// "application" controller
+	if cmOptions.IsControllerEnabled("application") {
+		selector, _ := labels.Parse(cmOptions.ApplicationSelector)
+		applicationReconciler := &application.ApplicationReconciler{
+			Scheme:              mgr.GetScheme(),
+			Client:              mgr.GetClient(),
+			Mapper:              mgr.GetRESTMapper(),
+			ApplicationSelector: selector,
+		}
+		addControllerWithSetup(mgr, "application", applicationReconciler)
+	}
+
+	// "serviceaccount" controller
+	if cmOptions.IsControllerEnabled("serviceaccount") {
+		saReconciler := &serviceaccount.Reconciler{}
+		addControllerWithSetup(mgr, "serviceaccount", saReconciler)
+	}
+
+	// "resourcequota" controller
+	if cmOptions.IsControllerEnabled("resourcequota") {
+		resourceQuotaReconciler := &quota.Reconciler{
+			MaxConcurrentReconciles: quota.DefaultMaxConcurrentReconciles,
+			ResyncPeriod:            quota.DefaultResyncPeriod,
+			InformerFactory:         informerFactory.KubernetesSharedInformerFactory(),
+		}
+		addControllerWithSetup(mgr, "resourcequota", resourceQuotaReconciler)
+	}
+
+	serviceMeshEnabled := cmOptions.ServiceMeshOptions != nil && len(cmOptions.ServiceMeshOptions.IstioPilotHost) != 0
 	if serviceMeshEnabled {
-		vsController = virtualservice.NewVirtualServiceController(kubernetesInformer.Core().V1().Services(),
-			istioInformer.Networking().V1alpha3().VirtualServices(),
-			istioInformer.Networking().V1alpha3().DestinationRules(),
-			kubesphereInformer.Servicemesh().V1alpha2().Strategies(),
-			client.Kubernetes(),
-			client.Istio(),
-			client.KubeSphere())
+		// "virtualservice" controller
+		if cmOptions.IsControllerEnabled("virtualservice") {
+			vsController := virtualservice.NewVirtualServiceController(kubernetesInformer.Core().V1().Services(),
+				istioInformer.Networking().V1alpha3().VirtualServices(),
+				istioInformer.Networking().V1alpha3().DestinationRules(),
+				kubesphereInformer.Servicemesh().V1alpha2().Strategies(),
+				client.Kubernetes(),
+				client.Istio(),
+				client.KubeSphere())
+			addController(mgr, "virtualservice", vsController)
+		}
 
-		drController = destinationrule.NewDestinationRuleController(kubernetesInformer.Apps().V1().Deployments(),
-			istioInformer.Networking().V1alpha3().DestinationRules(),
-			kubernetesInformer.Core().V1().Services(),
-			kubesphereInformer.Servicemesh().V1alpha2().ServicePolicies(),
-			client.Kubernetes(),
-			client.Istio(),
-			client.KubeSphere())
+		// "destinationrule" controller
+		if cmOptions.IsControllerEnabled("destinationrule") {
+			drController := destinationrule.NewDestinationRuleController(kubernetesInformer.Apps().V1().Deployments(),
+				istioInformer.Networking().V1alpha3().DestinationRules(),
+				kubernetesInformer.Core().V1().Services(),
+				kubesphereInformer.Servicemesh().V1alpha2().ServicePolicies(),
+				client.Kubernetes(),
+				client.Istio(),
+				client.KubeSphere())
+			addController(mgr, "destinationrule", drController)
+		}
 	}
 
-	jobController := job.NewJobController(kubernetesInformer.Batch().V1().Jobs(), client.Kubernetes())
+	// "job" controller
+	if cmOptions.IsControllerEnabled("job") {
+		jobController := job.NewJobController(kubernetesInformer.Batch().V1().Jobs(), client.Kubernetes())
+		addController(mgr, "job", jobController)
+	}
 
-	var s2iBinaryController, s2iRunController, devopsProjectController, devopsPipelineController, devopsCredentialController manager.Runnable
-	if devopsClient != nil {
-		s2iBinaryController = s2ibinary.NewController(client.Kubernetes(),
-			client.KubeSphere(),
-			kubesphereInformer.Devops().V1alpha1().S2iBinaries(),
-			s3Client,
+	// "storagecapability" controller
+	if cmOptions.IsControllerEnabled("storagecapability") {
+		storageCapabilityController := capability.NewController(
+			client.Kubernetes().StorageV1().StorageClasses(),
+			kubernetesInformer.Storage().V1().StorageClasses(),
+			kubernetesInformer.Storage().V1().CSIDrivers(),
 		)
-
-		s2iRunController = s2irun.NewS2iRunController(client.Kubernetes(),
-			client.KubeSphere(),
-			kubesphereInformer.Devops().V1alpha1().S2iBinaries(),
-			kubesphereInformer.Devops().V1alpha1().S2iRuns())
-
-		devopsProjectController = devopsproject.NewController(client.Kubernetes(),
-			client.KubeSphere(), devopsClient,
-			informerFactory.KubernetesSharedInformerFactory().Core().V1().Namespaces(),
-			informerFactory.KubeSphereSharedInformerFactory().Devops().V1alpha3().DevOpsProjects(),
-			informerFactory.KubeSphereSharedInformerFactory().Tenant().V1alpha1().Workspaces())
-
-		devopsPipelineController = pipeline.NewController(client.Kubernetes(),
-			client.KubeSphere(),
-			devopsClient,
-			informerFactory.KubernetesSharedInformerFactory().Core().V1().Namespaces(),
-			informerFactory.KubeSphereSharedInformerFactory().Devops().V1alpha3().Pipelines())
-
-		devopsCredentialController = devopscredential.NewController(client.Kubernetes(),
-			devopsClient,
-			informerFactory.KubernetesSharedInformerFactory().Core().V1().Namespaces(),
-			informerFactory.KubernetesSharedInformerFactory().Core().V1().Secrets())
-
+		addController(mgr, "storagecapability", storageCapabilityController)
 	}
 
-	storageCapabilityController := capability.NewController(
-		client.KubeSphere().StorageV1alpha1().StorageClassCapabilities(),
-		kubesphereInformer.Storage().V1alpha1(),
-		client.Kubernetes().StorageV1().StorageClasses(),
-		kubernetesInformer.Storage().V1().StorageClasses(),
-		capability.SnapshotSupported(client.Kubernetes().Discovery()),
-		client.Snapshot().SnapshotV1beta1().VolumeSnapshotClasses(),
-		informerFactory.SnapshotSharedInformerFactory().Snapshot().V1beta1().VolumeSnapshotClasses(),
-		kubernetesInformer.Storage().V1beta1().CSIDrivers(),
-	)
-
-	volumeExpansionController := expansion.NewVolumeExpansionController(
-		client.Kubernetes(),
-		kubernetesInformer.Core().V1().PersistentVolumeClaims(),
-		kubernetesInformer.Storage().V1().StorageClasses(),
-		kubernetesInformer.Core().V1().Pods(),
-		kubernetesInformer.Apps().V1().Deployments(),
-		kubernetesInformer.Apps().V1().ReplicaSets(),
-		kubernetesInformer.Apps().V1().StatefulSets())
-
-	var fedUserCache, fedGlobalRoleBindingCache, fedGlobalRoleCache cache.Store
-	var fedUserCacheController, fedGlobalRoleBindingCacheController, fedGlobalRoleCacheController cache.Controller
-
-	if multiClusterEnabled {
-		fedUserClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedUserResource)
-		if err != nil {
-			klog.Error(err)
-			return err
-		}
-		fedGlobalRoleClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedGlobalRoleResource)
-		if err != nil {
-			klog.Error(err)
-			return err
-		}
-		fedGlobalRoleBindingClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedGlobalRoleBindingResource)
-		if err != nil {
-			klog.Error(err)
-			return err
-		}
-
-		fedUserCache, fedUserCacheController = util.NewResourceInformer(fedUserClient, "", &iamv1alpha2.FedUserResource, func(object runtime.Object) {})
-		fedGlobalRoleCache, fedGlobalRoleCacheController = util.NewResourceInformer(fedGlobalRoleClient, "", &iamv1alpha2.FedGlobalRoleResource, func(object runtime.Object) {})
-		fedGlobalRoleBindingCache, fedGlobalRoleBindingCacheController = util.NewResourceInformer(fedGlobalRoleBindingClient, "", &iamv1alpha2.FedGlobalRoleBindingResource, func(object runtime.Object) {})
-
-		go fedUserCacheController.Run(stopCh)
-		go fedGlobalRoleCacheController.Run(stopCh)
-		go fedGlobalRoleBindingCacheController.Run(stopCh)
+	// "volumesnapshot" controller
+	if cmOptions.IsControllerEnabled("volumesnapshot") {
+		volumeSnapshotController := snapshotclass.NewController(
+			kubernetesInformer.Storage().V1().StorageClasses(),
+			client.Snapshot().SnapshotV1().VolumeSnapshotClasses(),
+			informerFactory.SnapshotSharedInformerFactory().Snapshot().V1().VolumeSnapshotClasses(),
+		)
+		addController(mgr, "volumesnapshot", volumeSnapshotController)
 	}
 
-	userController := user.NewUserController(client.Kubernetes(), client.KubeSphere(), client.Config(),
-		kubesphereInformer.Iam().V1alpha2().Users(),
-		kubesphereInformer.Iam().V1alpha2().LoginRecords(),
-		fedUserCache, fedUserCacheController,
-		kubernetesInformer.Core().V1().ConfigMaps(),
-		ldapClient, devopsClient,
-		authenticationOptions, multiClusterEnabled)
+	// "pvc-autoresizer"
+	monitoringOptionsEnable := cmOptions.MonitoringOptions != nil && len(cmOptions.MonitoringOptions.Endpoint) != 0
+	if monitoringOptionsEnable {
+		if cmOptions.IsControllerEnabled("pvc-autoresizer") {
+			if err := runners.SetupIndexer(mgr, false); err != nil {
+				return err
+			}
+			promClient, err := runners.NewPrometheusClient(cmOptions.MonitoringOptions.Endpoint)
+			if err != nil {
+				return err
+			}
+			pvcAutoResizerController := runners.NewPVCAutoresizer(
+				promClient,
+				mgr.GetClient(),
+				ctrl.Log.WithName("pvc-autoresizer"),
+				1*time.Minute,
+				mgr.GetEventRecorderFor("pvc-autoresizer"),
+			)
+			addController(mgr, "pvcautoresizer", pvcAutoResizerController)
+		}
+	}
 
-	loginRecordController := loginrecord.NewLoginRecordController(
-		client.Kubernetes(),
-		client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().LoginRecords(),
-		kubesphereInformer.Iam().V1alpha2().Users(),
-		authenticationOptions.LoginHistoryRetentionPeriod,
-		authenticationOptions.LoginHistoryMaximumEntries)
+	if cmOptions.IsControllerEnabled("pvc-workload-restarter") {
+		restarter := runners.NewRestarter(
+			mgr.GetClient(),
+			ctrl.Log.WithName("pvc-workload-restarter"),
+			1*time.Minute,
+			mgr.GetEventRecorderFor("pvc-workload-restarter"),
+		)
+		addController(mgr, "pvcworkloadrestarter", restarter)
+	}
 
-	csrController := certificatesigningrequest.NewController(client.Kubernetes(),
-		kubernetesInformer.Certificates().V1beta1().CertificateSigningRequests(),
-		kubernetesInformer.Core().V1().ConfigMaps(), client.Config())
-
-	clusterRoleBindingController := clusterrolebinding.NewController(client.Kubernetes(),
-		kubernetesInformer.Rbac().V1().ClusterRoleBindings(),
-		kubernetesInformer.Apps().V1().Deployments(),
-		kubernetesInformer.Core().V1().Pods(),
-		kubesphereInformer.Iam().V1alpha2().Users(),
-		kubectlImage)
-
-	globalRoleController := globalrole.NewController(client.Kubernetes(), client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().GlobalRoles(), fedGlobalRoleCache, fedGlobalRoleCacheController)
-
-	globalRoleBindingController := globalrolebinding.NewController(client.Kubernetes(), client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().GlobalRoleBindings(),
-		fedGlobalRoleBindingCache, fedGlobalRoleBindingCacheController,
-		multiClusterEnabled)
-
-	groupBindingController := groupbinding.NewController(client.Kubernetes(), client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().GroupBindings(),
-		kubesphereInformer.Types().V1beta1().FederatedGroupBindings(),
-		multiClusterEnabled)
-
-	groupController := group.NewController(client.Kubernetes(), client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().Groups(),
-		kubesphereInformer.Types().V1beta1().FederatedGroups(),
-		multiClusterEnabled)
-
-	var clusterController manager.Runnable
-	if multiClusterEnabled {
-		clusterController = cluster.NewClusterController(
+	// "loginrecord" controller
+	if cmOptions.IsControllerEnabled("loginrecord") {
+		loginRecordController := loginrecord.NewLoginRecordController(
 			client.Kubernetes(),
-			client.Config(),
-			kubesphereInformer.Cluster().V1alpha1().Clusters(),
-			client.KubeSphere().ClusterV1alpha1().Clusters(),
-			multiClusterOptions.ClusterControllerResyncSecond)
+			client.KubeSphere(),
+			kubesphereInformer.Iam().V1alpha2().LoginRecords(),
+			kubesphereInformer.Iam().V1alpha2().Users(),
+			cmOptions.AuthenticationOptions.LoginHistoryRetentionPeriod,
+			cmOptions.AuthenticationOptions.LoginHistoryMaximumEntries)
+		addController(mgr, "loginrecord", loginRecordController)
 	}
 
-	var nsnpController manager.Runnable
-	if networkOptions.EnableNetworkPolicy {
-		nsnpProvider, err := provider.NewNsNetworkPolicyProvider(client.Kubernetes(), kubernetesInformer.Networking().V1().NetworkPolicies())
-		if err != nil {
-			return err
+	// "csr" controller
+	if cmOptions.IsControllerEnabled("csr") {
+		csrController := certificatesigningrequest.NewController(client.Kubernetes(),
+			kubernetesInformer.Certificates().V1().CertificateSigningRequests(),
+			kubernetesInformer.Core().V1().ConfigMaps(), client.Config())
+		addController(mgr, "csr", csrController)
+	}
+
+	// "clusterrolebinding" controller
+	if cmOptions.IsControllerEnabled("clusterrolebinding") {
+		clusterRoleBindingController := clusterrolebinding.NewController(client.Kubernetes(),
+			kubernetesInformer.Rbac().V1().ClusterRoleBindings(),
+			kubernetesInformer.Apps().V1().Deployments(),
+			kubernetesInformer.Core().V1().Pods(),
+			kubesphereInformer.Iam().V1alpha2().Users(),
+			cmOptions.AuthenticationOptions.KubectlImage)
+		addController(mgr, "clusterrolebinding", clusterRoleBindingController)
+	}
+
+	// "fedglobalrolecache" controller
+	var fedGlobalRoleCache cache.Store
+	var fedGlobalRoleCacheController cache.Controller
+	if cmOptions.IsControllerEnabled("fedglobalrolecache") {
+		if cmOptions.MultiClusterOptions.Enable {
+			fedGlobalRoleClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedGlobalRoleResource)
+			if err != nil {
+				klog.Fatalf("Unable to create FedGlobalRole controller: %v", err)
+			}
+			fedGlobalRoleCache, fedGlobalRoleCacheController = util.NewResourceInformer(fedGlobalRoleClient, "",
+				&iamv1alpha2.FedGlobalRoleResource, func(object runtimeclient.Object) {})
+			go fedGlobalRoleCacheController.Run(stopCh)
+			addSuccessfullyControllers.Insert("fedglobalrolecache")
 		}
-
-		nsnpController = nsnetworkpolicy.NewNSNetworkPolicyController(client.Kubernetes(),
-			client.KubeSphere().NetworkV1alpha1(),
-			kubesphereInformer.Network().V1alpha1().NamespaceNetworkPolicies(),
-			kubernetesInformer.Core().V1().Services(),
-			kubernetesInformer.Core().V1().Nodes(),
-			kubesphereInformer.Tenant().V1alpha1().Workspaces(),
-			kubernetesInformer.Core().V1().Namespaces(), nsnpProvider, networkOptions.NSNPOptions)
 	}
 
-	var ippoolController manager.Runnable
-	ippoolProvider := ippoolclient.NewProvider(kubernetesInformer, client.KubeSphere(), client.Kubernetes(), networkOptions.IPPoolType, options)
-	if ippoolProvider != nil {
-		ippoolController = ippool.NewIPPoolController(kubesphereInformer, kubernetesInformer, client.Kubernetes(), client.KubeSphere(), ippoolProvider)
-	}
-
-	controllers := map[string]manager.Runnable{
-		"virtualservice-controller":     vsController,
-		"destinationrule-controller":    drController,
-		"job-controller":                jobController,
-		"s2ibinary-controller":          s2iBinaryController,
-		"s2irun-controller":             s2iRunController,
-		"storagecapability-controller":  storageCapabilityController,
-		"volumeexpansion-controller":    volumeExpansionController,
-		"user-controller":               userController,
-		"loginrecord-controller":        loginRecordController,
-		"cluster-controller":            clusterController,
-		"nsnp-controller":               nsnpController,
-		"csr-controller":                csrController,
-		"clusterrolebinding-controller": clusterRoleBindingController,
-		"globalrolebinding-controller":  globalRoleBindingController,
-		"ippool-controller":             ippoolController,
-		"groupbinding-controller":       groupBindingController,
-		"group-controller":              groupController,
-	}
-
-	if devopsClient != nil {
-		controllers["pipeline-controller"] = devopsPipelineController
-		controllers["devopsprojects-controller"] = devopsProjectController
-		controllers["devopscredential-controller"] = devopsCredentialController
-	}
-
-	if multiClusterEnabled {
-		controllers["globalrole-controller"] = globalRoleController
-		notificationController, err := notification.NewController(client.Kubernetes(), mgr.GetClient(), mgr.GetCache())
-		if err != nil {
-			return err
+	// "globalrole" controller
+	if cmOptions.IsControllerEnabled("globalrole") {
+		if cmOptions.MultiClusterOptions.Enable {
+			globalRoleController := globalrole.NewController(client.Kubernetes(), client.KubeSphere(),
+				kubesphereInformer.Iam().V1alpha2().GlobalRoles(), fedGlobalRoleCache, fedGlobalRoleCacheController)
+			addController(mgr, "globalrole", globalRoleController)
 		}
-		controllers["notification-controller"] = notificationController
 	}
 
-	for name, ctrl := range controllers {
-		if ctrl == nil {
-			klog.V(4).Infof("%s is not going to run due to dependent component disabled.", name)
-			continue
+	// "fedglobalrolebindingcache" controller
+	var fedGlobalRoleBindingCache cache.Store
+	var fedGlobalRoleBindingCacheController cache.Controller
+	if cmOptions.IsControllerEnabled("fedglobalrolebindingcache") {
+		if cmOptions.MultiClusterOptions.Enable {
+			fedGlobalRoleBindingClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedGlobalRoleBindingResource)
+			if err != nil {
+				klog.Fatalf("Unable to create FedGlobalRoleBinding controller: %v", err)
+			}
+			fedGlobalRoleBindingCache, fedGlobalRoleBindingCacheController = util.NewResourceInformer(fedGlobalRoleBindingClient, "",
+				&iamv1alpha2.FedGlobalRoleBindingResource, func(object runtimeclient.Object) {})
+			go fedGlobalRoleBindingCacheController.Run(stopCh)
+			addSuccessfullyControllers.Insert("fedglobalrolebindingcache")
 		}
+	}
 
-		if err := mgr.Add(ctrl); err != nil {
-			klog.Error(err, "add controller to manager failed", "name", name)
-			return err
+	// "globalrolebinding" controller
+	if cmOptions.IsControllerEnabled("globalrolebinding") {
+		globalRoleBindingController := globalrolebinding.NewController(client.Kubernetes(), client.KubeSphere(),
+			kubesphereInformer.Iam().V1alpha2().GlobalRoleBindings(),
+			fedGlobalRoleBindingCache, fedGlobalRoleBindingCacheController,
+			cmOptions.MultiClusterOptions.Enable)
+		addController(mgr, "globalrolebinding", globalRoleBindingController)
+	}
+
+	// "groupbinding" controller
+	if cmOptions.IsControllerEnabled("groupbinding") {
+		groupBindingController := groupbinding.NewController(client.Kubernetes(), client.KubeSphere(),
+			kubesphereInformer.Iam().V1alpha2().GroupBindings(),
+			kubesphereInformer.Types().V1beta1().FederatedGroupBindings(),
+			cmOptions.MultiClusterOptions.Enable)
+		addController(mgr, "groupbinding", groupBindingController)
+	}
+
+	// "group" controller
+	if cmOptions.IsControllerEnabled("group") {
+		groupController := group.NewController(client.Kubernetes(), client.KubeSphere(),
+			kubesphereInformer.Iam().V1alpha2().Groups(),
+			kubesphereInformer.Types().V1beta1().FederatedGroups(),
+			cmOptions.MultiClusterOptions.Enable)
+		addController(mgr, "group", groupController)
+	}
+
+	// "cluster" controller
+	if cmOptions.IsControllerEnabled("cluster") {
+		if cmOptions.MultiClusterOptions.Enable {
+			clusterController := cluster.NewClusterController(
+				client.Kubernetes(),
+				client.KubeSphere(),
+				client.Config(),
+				kubesphereInformer.Cluster().V1alpha1().Clusters(),
+				kubesphereInformer.Iam().V1alpha2().Users().Lister(),
+				cmOptions.MultiClusterOptions.ClusterControllerResyncPeriod,
+				cmOptions.MultiClusterOptions.HostClusterName,
+			)
+			addController(mgr, "cluster", clusterController)
+		}
+	}
+
+	// "nsnp" controller
+	if cmOptions.IsControllerEnabled("nsnp") {
+		if cmOptions.NetworkOptions.EnableNetworkPolicy {
+			nsnpProvider, err := provider.NewNsNetworkPolicyProvider(client.Kubernetes(), kubernetesInformer.Networking().V1().NetworkPolicies())
+			if err != nil {
+				klog.Fatalf("Unable to create NSNetworkPolicy controller: %v", err)
+			}
+
+			nsnpController := nsnetworkpolicy.NewNSNetworkPolicyController(client.Kubernetes(),
+				client.KubeSphere().NetworkV1alpha1(),
+				kubesphereInformer.Network().V1alpha1().NamespaceNetworkPolicies(),
+				kubernetesInformer.Core().V1().Services(),
+				kubernetesInformer.Core().V1().Nodes(),
+				kubesphereInformer.Tenant().V1alpha1().Workspaces(),
+				kubernetesInformer.Core().V1().Namespaces(), nsnpProvider, cmOptions.NetworkOptions.NSNPOptions)
+			addController(mgr, "nsnp", nsnpController)
+		}
+	}
+
+	// "ippool" controller
+	if cmOptions.IsControllerEnabled("ippool") {
+		ippoolProvider := ippoolclient.NewProvider(kubernetesInformer, client.KubeSphere(), client.Kubernetes(),
+			cmOptions.NetworkOptions.IPPoolType, cmOptions.KubernetesOptions)
+		if ippoolProvider != nil {
+			ippoolController := ippool.NewIPPoolController(kubesphereInformer, kubernetesInformer, client.Kubernetes(),
+				client.KubeSphere(), ippoolProvider)
+			addController(mgr, "ippool", ippoolController)
+		}
+	}
+
+	// "notification" controller
+	if cmOptions.IsControllerEnabled("notification") {
+		if cmOptions.MultiClusterOptions.Enable {
+			notificationController, err := notification.NewController(client.Kubernetes(), mgr.GetClient(), mgr.GetCache())
+			if err != nil {
+				klog.Fatalf("Unable to create Notification controller: %v", err)
+			}
+			addController(mgr, "notification", notificationController)
+		}
+	}
+
+	// log all controllers process result
+	for _, name := range allControllers {
+		if cmOptions.IsControllerEnabled(name) {
+			if addSuccessfullyControllers.Has(name) {
+				klog.Infof("%s controller is enabled and added successfully.", name)
+			} else {
+				klog.Infof("%s controller is enabled but is not going to run due to its dependent component being disabled.", name)
+			}
+		} else {
+			klog.Infof("%s controller is disabled by controller selectors.", name)
 		}
 	}
 
 	return nil
 }
+
+var addSuccessfullyControllers = sets.NewString()
+
+type setupableController interface {
+	SetupWithManager(mgr ctrl.Manager) error
+}
+
+func addControllerWithSetup(mgr manager.Manager, name string, controller setupableController) {
+	if err := controller.SetupWithManager(mgr); err != nil {
+		klog.Fatalf("Unable to create %v controller: %v", name, err)
+	}
+	addSuccessfullyControllers.Insert(name)
+}
+
+func addController(mgr manager.Manager, name string, controller manager.Runnable) {
+	if err := mgr.Add(controller); err != nil {
+		klog.Fatalf("Unable to create %v controller: %v", name, err)
+	}
+	addSuccessfullyControllers.Insert(name)
+}

cmd/controller-manager/app/options/options.go

@@ -18,9 +18,18 @@ package options
 
 import (
 	"flag"
+	"fmt"
 	"strings"
 	"time"
 
+	"kubesphere.io/kubesphere/pkg/simple/client/monitoring/prometheus"
+
+	controllerconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
+
+	"k8s.io/apimachinery/pkg/util/sets"
+
+	"kubesphere.io/kubesphere/pkg/apiserver/authentication"
+
 	"k8s.io/apimachinery/pkg/labels"
 
 	"github.com/spf13/pflag"
@@ -28,8 +37,8 @@ import (
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
 
-	authoptions "kubesphere.io/kubesphere/pkg/apiserver/authentication/options"
 	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
+	"kubesphere.io/kubesphere/pkg/simple/client/gateway"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
 	ldapclient "kubesphere.io/kubesphere/pkg/simple/client/ldap"
 	"kubesphere.io/kubesphere/pkg/simple/client/multicluster"
@@ -43,12 +52,14 @@ type KubeSphereControllerManagerOptions struct {
 	KubernetesOptions     *k8s.KubernetesOptions
 	DevopsOptions         *jenkins.Options
 	S3Options             *s3.Options
-	AuthenticationOptions *authoptions.AuthenticationOptions
+	AuthenticationOptions *authentication.Options
 	LdapOptions           *ldapclient.Options
 	OpenPitrixOptions     *openpitrix.Options
 	NetworkOptions        *network.Options
 	MultiClusterOptions   *multicluster.Options
 	ServiceMeshOptions    *servicemesh.Options
+	GatewayOptions        *gateway.Options
+	MonitoringOptions     *prometheus.Options
 	LeaderElect           bool
 	LeaderElection        *leaderelection.LeaderElectionConfig
 	WebhookCertDir        string
@@ -61,6 +72,19 @@ type KubeSphereControllerManagerOptions struct {
 	//      "kubesphere.io/creator=" means reconcile applications with this label key
 	//      "!kubesphere.io/creator" means exclude applications with this key
 	ApplicationSelector string
+
+	// ControllerGates is the list of controller gates to enable or disable controller.
+	// '*' means "all enabled by default controllers"
+	// 'foo' means "enable 'foo'"
+	// '-foo' means "disable 'foo'"
+	// first item for a particular name wins.
+	//     e.g. '-foo,foo' means "disable foo", 'foo,-foo' means "enable foo"
+	// * has the lowest priority.
+	//     e.g. *,-foo, means "disable 'foo'"
+	ControllerGates []string
+
+	// Enable gops or not.
+	GOPSEnabled bool
 }
 
 func NewKubeSphereControllerManagerOptions() *KubeSphereControllerManagerOptions {
@@ -73,7 +97,8 @@ func NewKubeSphereControllerManagerOptions() *KubeSphereControllerManagerOptions
 		NetworkOptions:        network.NewNetworkOptions(),
 		MultiClusterOptions:   multicluster.NewOptions(),
 		ServiceMeshOptions:    servicemesh.NewServiceMeshOptions(),
-		AuthenticationOptions: authoptions.NewAuthenticateOptions(),
+		AuthenticationOptions: authentication.NewOptions(),
+		GatewayOptions:        gateway.NewGatewayOptions(),
 		LeaderElection: &leaderelection.LeaderElectionConfig{
 			LeaseDuration: 30 * time.Second,
 			RenewDeadline: 15 * time.Second,
@@ -82,12 +107,13 @@ func NewKubeSphereControllerManagerOptions() *KubeSphereControllerManagerOptions
 		LeaderElect:         false,
 		WebhookCertDir:      "",
 		ApplicationSelector: "",
+		ControllerGates:     []string{"*"},
 	}
 
 	return s
 }
 
-func (s *KubeSphereControllerManagerOptions) Flags() cliflag.NamedFlagSets {
+func (s *KubeSphereControllerManagerOptions) Flags(allControllerNameSelectors []string) cliflag.NamedFlagSets {
 	fss := cliflag.NamedFlagSets{}
 
 	s.KubernetesOptions.AddFlags(fss.FlagSet("kubernetes"), s.KubernetesOptions)
@@ -99,7 +125,7 @@ func (s *KubeSphereControllerManagerOptions) Flags() cliflag.NamedFlagSets {
 	s.NetworkOptions.AddFlags(fss.FlagSet("network"), s.NetworkOptions)
 	s.MultiClusterOptions.AddFlags(fss.FlagSet("multicluster"), s.MultiClusterOptions)
 	s.ServiceMeshOptions.AddFlags(fss.FlagSet("servicemesh"), s.ServiceMeshOptions)
-
+	s.GatewayOptions.AddFlags(fss.FlagSet("gateway"), s.GatewayOptions)
 	fs := fss.FlagSet("leaderelection")
 	s.bindLeaderElectionFlags(s.LeaderElection, fs)
 
@@ -116,6 +142,13 @@ func (s *KubeSphereControllerManagerOptions) Flags() cliflag.NamedFlagSets {
 	gfs.StringVar(&s.ApplicationSelector, "application-selector", s.ApplicationSelector, ""+
 		"Only reconcile application(sigs.k8s.io/application) objects match given selector, this could avoid conflicts with "+
 		"other projects built on top of sig-application. Default behavior is to reconcile all of application objects.")
+	gfs.StringSliceVar(&s.ControllerGates, "controllers", []string{"*"}, fmt.Sprintf(""+
+		"A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller "+
+		"named 'foo', '-foo' disables the controller named 'foo'.\nAll controllers: %s",
+		strings.Join(allControllerNameSelectors, ", ")))
+
+	gfs.BoolVar(&s.GOPSEnabled, "gops", s.GOPSEnabled, "Whether to enable gops or not.  When enabled this option, "+
+		"controller-manager will listen on a random port on 127.0.0.1, then you can use the gops tool to list and diagnose the controller-manager currently running.")
 
 	kfs := fss.FlagSet("klog")
 	local := flag.NewFlagSet("klog", flag.ExitOnError)
@@ -128,25 +161,58 @@ func (s *KubeSphereControllerManagerOptions) Flags() cliflag.NamedFlagSets {
 	return fss
 }
 
-func (s *KubeSphereControllerManagerOptions) Validate() []error {
+// Validate Options and Genetic Options
+func (o *KubeSphereControllerManagerOptions) Validate(allControllerNameSelectors []string) []error {
 	var errs []error
-	errs = append(errs, s.DevopsOptions.Validate()...)
-	errs = append(errs, s.KubernetesOptions.Validate()...)
-	errs = append(errs, s.S3Options.Validate()...)
-	errs = append(errs, s.OpenPitrixOptions.Validate()...)
-	errs = append(errs, s.NetworkOptions.Validate()...)
-	errs = append(errs, s.LdapOptions.Validate()...)
+	errs = append(errs, o.DevopsOptions.Validate()...)
+	errs = append(errs, o.KubernetesOptions.Validate()...)
+	errs = append(errs, o.S3Options.Validate()...)
+	errs = append(errs, o.OpenPitrixOptions.Validate()...)
+	errs = append(errs, o.NetworkOptions.Validate()...)
+	errs = append(errs, o.LdapOptions.Validate()...)
+	errs = append(errs, o.MultiClusterOptions.Validate()...)
 
-	if len(s.ApplicationSelector) != 0 {
-		_, err := labels.Parse(s.ApplicationSelector)
+	// genetic option: application-selector
+	if len(o.ApplicationSelector) != 0 {
+		_, err := labels.Parse(o.ApplicationSelector)
 		if err != nil {
 			errs = append(errs, err)
 		}
 	}
 
+	// genetic option: controllers, check all selectors are valid
+	allControllersNameSet := sets.NewString(allControllerNameSelectors...)
+	for _, selector := range o.ControllerGates {
+		if selector == "*" {
+			continue
+		}
+		selector = strings.TrimPrefix(selector, "-")
+		if !allControllersNameSet.Has(selector) {
+			errs = append(errs, fmt.Errorf("%q is not in the list of known controllers", selector))
+		}
+	}
+
 	return errs
 }
 
+// IsControllerEnabled check if a specified controller enabled or not.
+func (o *KubeSphereControllerManagerOptions) IsControllerEnabled(name string) bool {
+	hasStar := false
+	for _, ctrl := range o.ControllerGates {
+		if ctrl == name {
+			return true
+		}
+		if ctrl == "-"+name {
+			return false
+		}
+		if ctrl == "*" {
+			hasStar = true
+		}
+	}
+
+	return hasStar
+}
+
 func (s *KubeSphereControllerManagerOptions) bindLeaderElectionFlags(l *leaderelection.LeaderElectionConfig, fs *pflag.FlagSet) {
 	fs.DurationVar(&l.LeaseDuration, "leader-elect-lease-duration", l.LeaseDuration, ""+
 		"The duration that non-leader candidates will wait after observing a leadership "+
@@ -162,3 +228,19 @@ func (s *KubeSphereControllerManagerOptions) bindLeaderElectionFlags(l *leaderel
 		"The duration the clients should wait between attempting acquisition and renewal "+
 		"of a leadership. This is only applicable if leader election is enabled.")
 }
+
+// MergeConfig merge new config without validation
+// When misconfigured, the app should just crash directly
+func (s *KubeSphereControllerManagerOptions) MergeConfig(cfg *controllerconfig.Config) {
+	s.KubernetesOptions = cfg.KubernetesOptions
+	s.DevopsOptions = cfg.DevopsOptions
+	s.S3Options = cfg.S3Options
+	s.AuthenticationOptions = cfg.AuthenticationOptions
+	s.LdapOptions = cfg.LdapOptions
+	s.OpenPitrixOptions = cfg.OpenPitrixOptions
+	s.NetworkOptions = cfg.NetworkOptions
+	s.MultiClusterOptions = cfg.MultiClusterOptions
+	s.ServiceMeshOptions = cfg.ServiceMeshOptions
+	s.GatewayOptions = cfg.GatewayOptions
+	s.MonitoringOptions = cfg.MonitoringOptions
+}

cmd/controller-manager/app/options/options_test.go

@@ -0,0 +1,81 @@
+// Copyright 2022 The KubeSphere Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+package options
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+// ref: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/controller-manager/app/helper_test.go
+func TestIsControllerEnabled(t *testing.T) {
+	testcases := []struct {
+		name            string
+		controllerName  string
+		controllerFlags []string
+		expected        bool
+	}{
+		{
+			name:            "on by name",
+			controllerName:  "bravo",
+			controllerFlags: []string{"alpha", "bravo", "-charlie"},
+			expected:        true,
+		},
+		{
+			name:            "off by name",
+			controllerName:  "charlie",
+			controllerFlags: []string{"alpha", "bravo", "-charlie"},
+			expected:        false,
+		},
+		{
+			name:            "on by default",
+			controllerName:  "alpha",
+			controllerFlags: []string{"*"},
+			expected:        true,
+		},
+		{
+			name:            "on by star, not off by name",
+			controllerName:  "alpha",
+			controllerFlags: []string{"*", "-charlie"},
+			expected:        true,
+		},
+		{
+			name:            "off by name with star",
+			controllerName:  "charlie",
+			controllerFlags: []string{"*", "-charlie"},
+			expected:        false,
+		},
+		{
+			name:            "off then on",
+			controllerName:  "alpha",
+			controllerFlags: []string{"-alpha", "alpha"},
+			expected:        false,
+		},
+		{
+			name:            "on then off",
+			controllerName:  "alpha",
+			controllerFlags: []string{"alpha", "-alpha"},
+			expected:        true,
+		},
+	}
+
+	for _, tc := range testcases {
+		option := NewKubeSphereControllerManagerOptions()
+		option.ControllerGates = tc.controllerFlags
+		actual := option.IsControllerEnabled(tc.controllerName)
+		assert.Equal(t, tc.expected, actual, "%v: expected %v, got %v", tc.name, tc.expected, actual)
+	}
+}

cmd/controller-manager/app/server.go

@@ -17,43 +17,30 @@ limitations under the License.
 package app
 
 import (
+	"context"
 	"fmt"
 	"os"
 
+	"github.com/google/gops/agent"
 	"github.com/spf13/cobra"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/labels"
 	utilerrors "k8s.io/apimachinery/pkg/util/errors"
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
 	"k8s.io/klog/klogr"
 	ctrl "sigs.k8s.io/controller-runtime"
 	"sigs.k8s.io/controller-runtime/pkg/manager"
-	"sigs.k8s.io/controller-runtime/pkg/runtime/signals"
+	"sigs.k8s.io/controller-runtime/pkg/manager/signals"
 	"sigs.k8s.io/controller-runtime/pkg/webhook"
 
 	"kubesphere.io/kubesphere/cmd/controller-manager/app/options"
 	"kubesphere.io/kubesphere/pkg/apis"
 	controllerconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
-	"kubesphere.io/kubesphere/pkg/controller/application"
-	"kubesphere.io/kubesphere/pkg/controller/namespace"
 	"kubesphere.io/kubesphere/pkg/controller/network/webhooks"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmapplication"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmcategory"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmrelease"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmrepo"
 	"kubesphere.io/kubesphere/pkg/controller/quota"
-	"kubesphere.io/kubesphere/pkg/controller/serviceaccount"
 	"kubesphere.io/kubesphere/pkg/controller/user"
-	"kubesphere.io/kubesphere/pkg/controller/workspace"
-	"kubesphere.io/kubesphere/pkg/controller/workspacerole"
-	"kubesphere.io/kubesphere/pkg/controller/workspacerolebinding"
-	"kubesphere.io/kubesphere/pkg/controller/workspacetemplate"
 	"kubesphere.io/kubesphere/pkg/informers"
-	"kubesphere.io/kubesphere/pkg/simple/client/devops"
-	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
-	ldapclient "kubesphere.io/kubesphere/pkg/simple/client/ldap"
 	"kubesphere.io/kubesphere/pkg/simple/client/s3"
 	"kubesphere.io/kubesphere/pkg/utils/metrics"
 	"kubesphere.io/kubesphere/pkg/utils/term"
@@ -75,6 +62,8 @@ func NewControllerManagerCommand() *cobra.Command {
 			NetworkOptions:        conf.NetworkOptions,
 			MultiClusterOptions:   conf.MultiClusterOptions,
 			ServiceMeshOptions:    conf.ServiceMeshOptions,
+			GatewayOptions:        conf.GatewayOptions,
+			MonitoringOptions:     conf.MonitoringOptions,
 			LeaderElection:        s.LeaderElection,
 			LeaderElect:           s.LeaderElect,
 			WebhookCertDir:        s.WebhookCertDir,
@@ -85,14 +74,22 @@ func NewControllerManagerCommand() *cobra.Command {
 
 	cmd := &cobra.Command{
 		Use:  "controller-manager",
-		Long: `KubeSphere controller manager is a daemon that`,
+		Long: `KubeSphere controller manager is a daemon that embeds the control loops shipped with KubeSphere.`,
 		Run: func(cmd *cobra.Command, args []string) {
-			if errs := s.Validate(); len(errs) != 0 {
+			if errs := s.Validate(allControllers); len(errs) != 0 {
 				klog.Error(utilerrors.NewAggregate(errs))
 				os.Exit(1)
 			}
 
-			if err = run(s, signals.SetupSignalHandler()); err != nil {
+			if s.GOPSEnabled {
+				// Add agent to report additional information such as the current stack trace, Go version, memory stats, etc.
+				// Bind to a random port on address 127.0.0.1
+				if err := agent.Listen(agent.Options{}); err != nil {
+					klog.Fatal(err)
+				}
+			}
+
+			if err = Run(s, controllerconfig.WatchConfigChange(), signals.SetupSignalHandler()); err != nil {
 				klog.Error(err)
 				os.Exit(1)
 			}
@@ -101,7 +98,7 @@ func NewControllerManagerCommand() *cobra.Command {
 	}
 
 	fs := cmd.Flags()
-	namedFlagSets := s.Flags()
+	namedFlagSets := s.Flags(allControllers)
 
 	for _, f := range namedFlagSets.FlagSets {
 		fs.AddFlagSet(f)
@@ -116,7 +113,7 @@ func NewControllerManagerCommand() *cobra.Command {
 
 	versionCmd := &cobra.Command{
 		Use:   "version",
-		Short: "Print the version of KubeSphere ks-apiserver",
+		Short: "Print the version of KubeSphere controller-manager",
 		Run: func(cmd *cobra.Command, args []string) {
 			cmd.Println(version.Get())
 		},
@@ -127,7 +124,41 @@ func NewControllerManagerCommand() *cobra.Command {
 	return cmd
 }
 
-func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{}) error {
+func Run(s *options.KubeSphereControllerManagerOptions, configCh <-chan controllerconfig.Config, ctx context.Context) error {
+	ictx, cancelFunc := context.WithCancel(context.TODO())
+	errCh := make(chan error)
+	defer close(errCh)
+	go func() {
+		if err := run(s, ictx); err != nil {
+			errCh <- err
+		}
+	}()
+
+	// The ctx (signals.SetupSignalHandler()) is to control the entire program life cycle,
+	// The ictx(internal context)  is created here to control the life cycle of the controller-manager(all controllers, sharedInformer, webhook etc.)
+	// when config changed, stop server and renew context, start new server
+	for {
+		select {
+		case <-ctx.Done():
+			cancelFunc()
+			return nil
+		case cfg := <-configCh:
+			cancelFunc()
+			s.MergeConfig(&cfg)
+			ictx, cancelFunc = context.WithCancel(context.TODO())
+			go func() {
+				if err := run(s, ictx); err != nil {
+					errCh <- err
+				}
+			}()
+		case err := <-errCh:
+			cancelFunc()
+			return err
+		}
+	}
+}
+
+func run(s *options.KubeSphereControllerManagerOptions, ctx context.Context) error {
 
 	kubernetesClient, err := k8s.NewKubernetesClient(s.KubernetesOptions)
 	if err != nil {
@@ -135,32 +166,8 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 		return err
 	}
 
-	var devopsClient devops.Interface
-	if s.DevopsOptions != nil && len(s.DevopsOptions.Host) != 0 {
-		devopsClient, err = jenkins.NewDevopsClient(s.DevopsOptions)
-		if err != nil {
-			return fmt.Errorf("failed to connect jenkins, please check jenkins status, error: %v", err)
-		}
-	}
-
-	var ldapClient ldapclient.Interface
-	// when there is no ldapOption, we set ldapClient as nil, which means we don't need to sync user info into ldap.
-	if s.LdapOptions != nil && len(s.LdapOptions.Host) != 0 {
-		if s.LdapOptions.Host == ldapclient.FAKE_HOST { // for debug only
-			ldapClient = ldapclient.NewSimpleLdap()
-		} else {
-			ldapClient, err = ldapclient.NewLdapClient(s.LdapOptions, stopCh)
-			if err != nil {
-				return fmt.Errorf("failed to connect to ldap service, please check ldap status, error: %v", err)
-			}
-		}
-	} else {
-		klog.Warning("ks-controller-manager starts without ldap provided, it will not sync user into ldap")
-	}
-
-	var s3Client s3.Interface
 	if s.S3Options != nil && len(s.S3Options.Endpoint) != 0 {
-		s3Client, err = s3.NewS3Client(s.S3Options)
+		_, err = s3.NewS3Client(s.S3Options)
 		if err != nil {
 			return fmt.Errorf("failed to connect to s3, please check s3 service status, error: %v", err)
 		}
@@ -207,113 +214,19 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 	// register common meta types into schemas.
 	metav1.AddToGroupVersion(mgr.GetScheme(), metav1.SchemeGroupVersion)
 
-	workspaceTemplateReconciler := &workspacetemplate.Reconciler{MultiClusterEnabled: s.MultiClusterOptions.Enable}
-	if err = workspaceTemplateReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace template controller: %v", err)
-	}
-
-	workspaceReconciler := &workspace.Reconciler{}
-	if err = workspaceReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace controller: %v", err)
-	}
-
-	workspaceRoleReconciler := &workspacerole.Reconciler{MultiClusterEnabled: s.MultiClusterOptions.Enable}
-	if err = workspaceRoleReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace role controller: %v", err)
-	}
-
-	workspaceRoleBindingReconciler := &workspacerolebinding.Reconciler{MultiClusterEnabled: s.MultiClusterOptions.Enable}
-	if err = workspaceRoleBindingReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace role binding controller: %v", err)
-	}
-
-	namespaceReconciler := &namespace.Reconciler{}
-	if err = namespaceReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create namespace controller: %v", err)
-	}
-
-	err = helmrepo.Add(mgr)
-	if err != nil {
-		klog.Fatal("Unable to create helm repo controller")
-	}
-
-	err = helmcategory.Add(mgr)
-	if err != nil {
-		klog.Fatal("Unable to create helm category controller")
-	}
-
-	var opS3Client s3.Interface
-	if !s.OpenPitrixOptions.AppStoreConfIsEmpty() {
-		opS3Client, err = s3.NewS3Client(s.OpenPitrixOptions.S3Options)
-		if err != nil {
-			klog.Fatalf("failed to connect to s3, please check openpitrix s3 service status, error: %v", err)
-		}
-		err = (&helmapplication.ReconcileHelmApplication{}).SetupWithManager(mgr)
-		if err != nil {
-			klog.Fatalf("Unable to create helm application controller, error: %s", err)
-		}
-
-		err = (&helmapplication.ReconcileHelmApplicationVersion{}).SetupWithManager(mgr)
-		if err != nil {
-			klog.Fatalf("Unable to create helm application version controller, error: %s ", err)
-		}
-	}
-
-	err = (&helmrelease.ReconcileHelmRelease{
-		// nil interface is valid value.
-		StorageClient:      opS3Client,
-		KsFactory:          informerFactory.KubeSphereSharedInformerFactory(),
-		MultiClusterEnable: s.MultiClusterOptions.Enable,
-		WaitTime:           s.OpenPitrixOptions.ReleaseControllerOptions.WaitTime,
-		MaxConcurrent:      s.OpenPitrixOptions.ReleaseControllerOptions.MaxConcurrent,
-		StopChan:           stopCh,
-	}).SetupWithManager(mgr)
-
-	if err != nil {
-		klog.Fatalf("Unable to create helm release controller, error: %s", err)
-	}
-
-	selector, _ := labels.Parse(s.ApplicationSelector)
-	applicationReconciler := &application.ApplicationReconciler{
-		Scheme:              mgr.GetScheme(),
-		Client:              mgr.GetClient(),
-		Mapper:              mgr.GetRESTMapper(),
-		ApplicationSelector: selector,
-	}
-	if err = applicationReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create application controller: %v", err)
-	}
-
-	saReconciler := &serviceaccount.Reconciler{}
-	if err = saReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create ServiceAccount controller: %v", err)
-	}
-
-	resourceQuotaReconciler := quota.Reconciler{}
-	if err := resourceQuotaReconciler.SetupWithManager(mgr, quota.DefaultMaxConcurrentReconciles, quota.DefaultResyncPeriod, informerFactory.KubernetesSharedInformerFactory()); err != nil {
-		klog.Fatalf("Unable to create ResourceQuota controller: %v", err)
-	}
-
 	// TODO(jeff): refactor config with CRD
-	servicemeshEnabled := s.ServiceMeshOptions != nil && len(s.ServiceMeshOptions.IstioPilotHost) != 0
-	if err = addControllers(mgr,
+	// install all controllers
+	if err = addAllControllers(mgr,
 		kubernetesClient,
 		informerFactory,
-		devopsClient,
-		s3Client,
-		ldapClient,
-		s.KubernetesOptions,
-		s.AuthenticationOptions,
-		s.MultiClusterOptions,
-		s.NetworkOptions,
-		servicemeshEnabled,
-		s.AuthenticationOptions.KubectlImage, stopCh); err != nil {
+		s,
+		ctx.Done()); err != nil {
 		klog.Fatalf("unable to register controllers to the manager: %v", err)
 	}
 
 	// Start cache data after all informer is registered
 	klog.V(0).Info("Starting cache resource from apiserver...")
-	informerFactory.Start(stopCh)
+	informerFactory.Start(ctx.Done())
 
 	// Setup webhooks
 	klog.V(2).Info("setting up webhook server")
@@ -323,6 +236,7 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 	hookServer.Register("/validate-email-iam-kubesphere-io-v1alpha2", &webhook.Admission{Handler: &user.EmailValidator{Client: mgr.GetClient()}})
 	hookServer.Register("/validate-network-kubesphere-io-v1alpha1", &webhook.Admission{Handler: &webhooks.ValidatingHandler{C: mgr.GetClient()}})
 	hookServer.Register("/mutate-network-kubesphere-io-v1alpha1", &webhook.Admission{Handler: &webhooks.MutatingHandler{C: mgr.GetClient()}})
+	hookServer.Register("/persistentvolumeclaims", &webhook.Admission{Handler: &webhooks.AccessorHandler{C: mgr.GetClient()}})
 
 	resourceQuotaAdmission, err := quota.NewResourceQuotaAdmission(mgr.GetClient(), mgr.GetScheme())
 	if err != nil {
@@ -336,7 +250,7 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 	mgr.AddMetricsExtraHandler("/kapis/metrics", metrics.Handler())
 
 	klog.V(0).Info("Starting the controllers.")
-	if err = mgr.Start(stopCh); err != nil {
+	if err = mgr.Start(ctx); err != nil {
 		klog.Fatalf("unable to run the manager: %v", err)
 	}
 

cmd/ks-apiserver/app/options/options.go

@@ -20,24 +20,30 @@ import (
 	"crypto/tls"
 	"flag"
 	"fmt"
+	"net/http"
+	"strings"
+	"sync"
 
+	openpitrixv1 "kubesphere.io/kubesphere/pkg/kapis/openpitrix/v1"
+	"kubesphere.io/kubesphere/pkg/utils/clusterclient"
+
+	"kubesphere.io/kubesphere/pkg/apiserver/authentication/token"
+
+	"k8s.io/client-go/kubernetes/scheme"
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
 	runtimecache "sigs.k8s.io/controller-runtime/pkg/cache"
+	runtimeclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	"kubesphere.io/kubesphere/pkg/apis"
 	"kubesphere.io/kubesphere/pkg/apiserver"
 	apiserverconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
-	"kubesphere.io/kubesphere/pkg/client/clientset/versioned/scheme"
 	"kubesphere.io/kubesphere/pkg/informers"
 	genericoptions "kubesphere.io/kubesphere/pkg/server/options"
 	"kubesphere.io/kubesphere/pkg/simple/client/alerting"
 	auditingclient "kubesphere.io/kubesphere/pkg/simple/client/auditing/elasticsearch"
 	"kubesphere.io/kubesphere/pkg/simple/client/cache"
 
-	"net/http"
-	"strings"
-
 	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
 	eventsclient "kubesphere.io/kubesphere/pkg/simple/client/events/elasticsearch"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
@@ -53,15 +59,18 @@ type ServerRunOptions struct {
 	ConfigFile              string
 	GenericServerRunOptions *genericoptions.ServerRunOptions
 	*apiserverconfig.Config
+	schemeOnce sync.Once
+	DebugMode  bool
 
-	//
-	DebugMode bool
+	// Enable gops or not.
+	GOPSEnabled bool
 }
 
 func NewServerRunOptions() *ServerRunOptions {
 	s := &ServerRunOptions{
 		GenericServerRunOptions: genericoptions.NewServerRunOptions(),
 		Config:                  apiserverconfig.New(),
+		schemeOnce:              sync.Once{},
 	}
 
 	return s
@@ -70,13 +79,14 @@ func NewServerRunOptions() *ServerRunOptions {
 func (s *ServerRunOptions) Flags() (fss cliflag.NamedFlagSets) {
 	fs := fss.FlagSet("generic")
 	fs.BoolVar(&s.DebugMode, "debug", false, "Don't enable this if you don't know what it means.")
+	fs.BoolVar(&s.GOPSEnabled, "gops", false, "Whether to enable gops or not. When enabled this option, "+
+		"ks-apiserver will listen on a random port on 127.0.0.1, then you can use the gops tool to list and diagnose the ks-apiserver currently running.")
 	s.GenericServerRunOptions.AddFlags(fs, s.GenericServerRunOptions)
 	s.KubernetesOptions.AddFlags(fss.FlagSet("kubernetes"), s.KubernetesOptions)
 	s.AuthenticationOptions.AddFlags(fss.FlagSet("authentication"), s.AuthenticationOptions)
 	s.AuthorizationOptions.AddFlags(fss.FlagSet("authorization"), s.AuthorizationOptions)
 	s.DevopsOptions.AddFlags(fss.FlagSet("devops"), s.DevopsOptions)
 	s.SonarQubeOptions.AddFlags(fss.FlagSet("sonarqube"), s.SonarQubeOptions)
-	s.RedisOptions.AddFlags(fss.FlagSet("redis"), s.RedisOptions)
 	s.S3Options.AddFlags(fss.FlagSet("s3"), s.S3Options)
 	s.OpenPitrixOptions.AddFlags(fss.FlagSet("openpitrix"), s.OpenPitrixOptions)
 	s.NetworkOptions.AddFlags(fss.FlagSet("network"), s.NetworkOptions)
@@ -165,21 +175,23 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 		apiServer.SonarClient = sonarqube.NewSonar(sonarClient.SonarQube())
 	}
 
-	var cacheClient cache.Interface
-	if s.RedisOptions != nil && len(s.RedisOptions.Host) != 0 {
-		if s.RedisOptions.Host == fakeInterface && s.DebugMode {
-			apiServer.CacheClient = cache.NewSimpleCache()
-		} else {
-			cacheClient, err = cache.NewRedisClient(s.RedisOptions, stopCh)
-			if err != nil {
-				return nil, fmt.Errorf("failed to connect to redis service, please check redis status, error: %v", err)
-			}
-			apiServer.CacheClient = cacheClient
+	// If debug mode is on or CacheOptions is nil, will create a fake cache.
+	if s.CacheOptions.Type != "" {
+		if s.DebugMode {
+			s.CacheOptions.Type = cache.DefaultCacheType
 		}
+		cacheClient, err := cache.New(s.CacheOptions, stopCh)
+		if err != nil {
+			return nil, fmt.Errorf("failed to create cache, error: %v", err)
+		}
+		apiServer.CacheClient = cacheClient
 	} else {
-		klog.Warning("ks-apiserver starts without redis provided, it will use in memory cache. " +
-			"This may cause inconsistencies when running ks-apiserver with multiple replicas.")
-		apiServer.CacheClient = cache.NewSimpleCache()
+		s.CacheOptions = &cache.Options{Type: cache.DefaultCacheType}
+		// fake cache has no error to return
+		cacheClient, _ := cache.New(s.CacheOptions, stopCh)
+		apiServer.CacheClient = cacheClient
+		klog.Warning("ks-apiserver starts without cache provided, it will use in memory cache. " +
+			"This may cause inconsistencies when running ks-apiserver with multiple replicas, and memory leak risk")
 	}
 
 	if s.EventsOptions.Host != "" {
@@ -206,6 +218,13 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 		apiServer.AlertingClient = alertingClient
 	}
 
+	if s.Config.MultiClusterOptions.Enable {
+		cc := clusterclient.NewClusterClient(informerFactory.KubeSphereSharedInformerFactory().Cluster().V1alpha1().Clusters())
+		apiServer.ClusterClient = cc
+	}
+
+	apiServer.OpenpitrixClient = openpitrixv1.NewOpenpitrixClient(informerFactory, apiServer.KubernetesClient.KubeSphere(), s.OpenPitrixOptions, apiServer.ClusterClient)
+
 	server := &http.Server{
 		Addr: fmt.Sprintf(":%d", s.GenericServerRunOptions.InsecurePort),
 	}
@@ -215,17 +234,33 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 		if err != nil {
 			return nil, err
 		}
-		server.TLSConfig.Certificates = []tls.Certificate{certificate}
+
+		server.TLSConfig = &tls.Config{
+			Certificates: []tls.Certificate{certificate},
+		}
+		server.Addr = fmt.Sprintf(":%d", s.GenericServerRunOptions.SecurePort)
 	}
 
 	sch := scheme.Scheme
-	if err := apis.AddToScheme(sch); err != nil {
-		klog.Fatalf("unable add APIs to scheme: %v", err)
-	}
+	s.schemeOnce.Do(func() {
+		if err := apis.AddToScheme(sch); err != nil {
+			klog.Fatalf("unable add APIs to scheme: %v", err)
+		}
+	})
 
 	apiServer.RuntimeCache, err = runtimecache.New(apiServer.KubernetesClient.Config(), runtimecache.Options{Scheme: sch})
 	if err != nil {
-		klog.Fatalf("unable to create runtime cache: %v", err)
+		klog.Fatalf("unable to create controller runtime cache: %v", err)
+	}
+
+	apiServer.RuntimeClient, err = runtimeclient.New(apiServer.KubernetesClient.Config(), runtimeclient.Options{Scheme: sch})
+	if err != nil {
+		klog.Fatalf("unable to create controller runtime client: %v", err)
+	}
+
+	apiServer.Issuer, err = token.NewIssuer(s.AuthenticationOptions)
+	if err != nil {
+		klog.Fatalf("unable to create issuer: %v", err)
 	}
 
 	apiServer.Server = server

cmd/ks-apiserver/app/server.go

@@ -17,20 +17,22 @@ limitations under the License.
 package app
 
 import (
+	"context"
 	"fmt"
+	"net/http"
 
+	"github.com/google/gops/agent"
 	"github.com/spf13/cobra"
 	utilerrors "k8s.io/apimachinery/pkg/util/errors"
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
 
+	"sigs.k8s.io/controller-runtime/pkg/manager/signals"
+
 	"kubesphere.io/kubesphere/cmd/ks-apiserver/app/options"
 	apiserverconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
-	"kubesphere.io/kubesphere/pkg/utils/signals"
 	"kubesphere.io/kubesphere/pkg/utils/term"
 	"kubesphere.io/kubesphere/pkg/version"
-
-	tracing "kubesphere.io/kubesphere/pkg/kapis/servicemesh/metrics/v1alpha2"
 )
 
 func NewAPIServerCommand() *cobra.Command {
@@ -57,7 +59,15 @@ cluster's shared state through which all other components interact.`,
 				return utilerrors.NewAggregate(errs)
 			}
 
-			return Run(s, signals.SetupSignalHandler())
+			if s.GOPSEnabled {
+				// Add agent to report additional information such as the current stack trace, Go version, memory stats, etc.
+				// Bind to a random port on address 127.0.0.1.
+				if err := agent.Listen(agent.Options{}); err != nil {
+					klog.Fatal(err)
+				}
+			}
+
+			return Run(s, apiserverconfig.WatchConfigChange(), signals.SetupSignalHandler())
 		},
 		SilenceUsage: true,
 	}
@@ -88,31 +98,54 @@ cluster's shared state through which all other components interact.`,
 	return cmd
 }
 
-func Run(s *options.ServerRunOptions, stopCh <-chan struct{}) error {
+func Run(s *options.ServerRunOptions, configCh <-chan apiserverconfig.Config, ctx context.Context) error {
+	ictx, cancelFunc := context.WithCancel(context.TODO())
+	errCh := make(chan error)
+	defer close(errCh)
+	go func() {
+		if err := run(s, ictx); err != nil {
+			errCh <- err
+		}
+	}()
 
-	initializeServicemeshConfig(s)
+	// The ctx (signals.SetupSignalHandler()) is to control the entire program life cycle,
+	// The ictx(internal context)  is created here to control the life cycle of the ks-apiserver(http server, sharedInformer etc.)
+	// when config change, stop server and renew context, start new server
+	for {
+		select {
+		case <-ctx.Done():
+			cancelFunc()
+			return nil
+		case cfg := <-configCh:
+			cancelFunc()
+			s.Config = &cfg
+			ictx, cancelFunc = context.WithCancel(context.TODO())
+			go func() {
+				if err := run(s, ictx); err != nil {
+					errCh <- err
+				}
+			}()
+		case err := <-errCh:
+			cancelFunc()
+			return err
+		}
+	}
+}
 
-	apiserver, err := s.NewAPIServer(stopCh)
+func run(s *options.ServerRunOptions, ctx context.Context) error {
+	apiserver, err := s.NewAPIServer(ctx.Done())
 	if err != nil {
 		return err
 	}
 
-	err = apiserver.PrepareRun(stopCh)
+	err = apiserver.PrepareRun(ctx.Done())
 	if err != nil {
+		return err
+	}
+
+	err = apiserver.Run(ctx)
+	if err == http.ErrServerClosed {
 		return nil
 	}
-
-	return apiserver.Run(stopCh)
-}
-
-func initializeServicemeshConfig(s *options.ServerRunOptions) {
-	// Config jaeger query endpoint address
-	if s.ServiceMeshOptions != nil && len(s.ServiceMeshOptions.JaegerQueryHost) != 0 {
-		tracing.JaegerQueryUrl = s.ServiceMeshOptions.JaegerQueryHost
-	}
-
-	// Set the kiali query endpoint address
-	if s.ServiceMeshOptions != nil && len(s.ServiceMeshOptions.KialiQueryHost) != 0 {
-		tracing.KialiQueryUrl = s.ServiceMeshOptions.KialiQueryHost
-	}
+	return err
 }

config/crds/application.kubesphere.io_helmapplications.yaml

@@ -37,10 +37,14 @@ spec:
         description: HelmApplication is the Schema for the helmapplications API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -73,10 +77,13 @@ spec:
             description: HelmApplicationStatus defines the observed state of HelmApplication
             properties:
               latestVersion:
-                description: If this application belong to appStore, latestVersion is the the latest version of the active application version. otherwise latestVersion is the latest version of all application version
+                description: If this application belong to appStore, latestVersion
+                  is the the latest version of the active application version. otherwise
+                  latestVersion is the latest version of all application version
                 type: string
               state:
-                description: 'the state of the helm application: draft, submitted, passed, rejected, suspended, active'
+                description: 'the state of the helm application: draft, submitted,
+                  passed, rejected, suspended, active'
                 type: string
               statusTime:
                 format: date-time

config/crds/application.kubesphere.io_helmapplicationversions.yaml

@@ -31,13 +31,18 @@ spec:
     name: v1alpha1
     schema:
       openAPIV3Schema:
-        description: HelmApplicationVersion is the Schema for the helmapplicationversions API
+        description: HelmApplicationVersion is the Schema for the helmapplicationversions
+          API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -47,13 +52,15 @@ spec:
               annotations:
                 additionalProperties:
                   type: string
-                description: Annotations are additional mappings uninterpreted by Helm, made available for inspection by other applications.
+                description: Annotations are additional mappings uninterpreted by
+                  Helm, made available for inspection by other applications.
                 type: object
               apiVersion:
                 description: The API Version of this chart.
                 type: string
               appVersion:
-                description: The version of the application enclosed inside of this chart.
+                description: The version of the application enclosed inside of this
+                  chart.
                 type: string
               condition:
                 description: The condition to check to enable chart
@@ -72,30 +79,39 @@ spec:
               dependencies:
                 description: Dependencies are a list of dependencies for a chart.
                 items:
-                  description: Dependency describes a chart upon which another chart depends. Dependencies can be used to express developer intent, or to capture the state of a chart.
+                  description: Dependency describes a chart upon which another chart
+                    depends. Dependencies can be used to express developer intent,
+                    or to capture the state of a chart.
                   properties:
                     alias:
                       description: Alias usable alias to be used for the chart
                       type: string
                     condition:
-                      description: A yaml path that resolves to a boolean, used for enabling/disabling charts (e.g. subchart1.enabled )
+                      description: A yaml path that resolves to a boolean, used for
+                        enabling/disabling charts (e.g. subchart1.enabled )
                       type: string
                     enabled:
                       description: Enabled bool determines if chart should be loaded
                       type: boolean
                     name:
-                      description: Name is the name of the dependency. This must mach the name in the dependency's Chart.yaml.
+                      description: Name is the name of the dependency. This must mach
+                        the name in the dependency's Chart.yaml.
                       type: string
                     repository:
-                      description: The URL to the repository. Appending `index.yaml` to this string should result in a URL that can be used to fetch the repository index.
+                      description: The URL to the repository. Appending `index.yaml`
+                        to this string should result in a URL that can be used to
+                        fetch the repository index.
                       type: string
                     tags:
-                      description: Tags can be used to group charts for enabling/disabling together
+                      description: Tags can be used to group charts for enabling/disabling
+                        together
                       items:
                         type: string
                       type: array
                     version:
-                      description: Version is the version (range) of this chart. A lock file will always produce a single version, while a dependency may contain a semantic version range.
+                      description: Version is the version (range) of this chart. A
+                        lock file will always produce a single version, while a dependency
+                        may contain a semantic version range.
                       type: string
                   required:
                   - name
@@ -112,7 +128,8 @@ spec:
                 description: chart digest
                 type: string
               home:
-                description: The URL to a relevant project page, git repo, or contact person
+                description: The URL to a relevant project page, git repo, or contact
+                  person
                 type: string
               icon:
                 description: The URL to an icon file.
@@ -123,21 +140,25 @@ spec:
                   type: string
                 type: array
               kubeVersion:
-                description: KubeVersion is a SemVer constraint specifying the version of Kubernetes required.
+                description: KubeVersion is a SemVer constraint specifying the version
+                  of Kubernetes required.
                 type: string
               maintainers:
-                description: A list of name and URL/email address combinations for the maintainer(s)
+                description: A list of name and URL/email address combinations for
+                  the maintainer(s)
                 items:
                   description: Maintainer describes a Chart maintainer.
                   properties:
                     email:
-                      description: Email is an optional email address to contact the named maintainer
+                      description: Email is an optional email address to contact the
+                        named maintainer
                       type: string
                     name:
                       description: Name is a user name or organization name
                       type: string
                     url:
-                      description: URL is an optional URL to an address for the named maintainer
+                      description: URL is an optional URL to an address for the named
+                        maintainer
                       type: string
                   type: object
                 type: array
@@ -165,7 +186,8 @@ spec:
                 type: string
             type: object
           status:
-            description: HelmApplicationVersionStatus defines the observed state of HelmApplicationVersion
+            description: HelmApplicationVersionStatus defines the observed state of
+              HelmApplicationVersion
             properties:
               audit:
                 items:
@@ -179,7 +201,8 @@ spec:
                     operatorType:
                       type: string
                     state:
-                      description: 'audit state: submitted, passed, draft, active, rejected, suspended'
+                      description: 'audit state: submitted, passed, draft, active,
+                        rejected, suspended'
                       type: string
                     time:
                       description: audit time

config/crds/application.kubesphere.io_helmcategories.yaml

@@ -34,10 +34,14 @@ spec:
         description: HelmCategory is the Schema for the helmcategories API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/application.kubesphere.io_helmreleases.yaml

@@ -43,10 +43,14 @@ spec:
         description: HelmRelease is the Schema for the helmreleases API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -66,7 +70,8 @@ spec:
                 description: The name of the chart which will be installed.
                 type: string
               chartVersion:
-                description: Specify the exact chart version to install. If this is not specified, the latest version is installed
+                description: Specify the exact chart version to install. If this is
+                  not specified, the latest version is installed
                 type: string
               description:
                 description: Message got from frontend
@@ -82,7 +87,9 @@ spec:
                 format: byte
                 type: string
               version:
-                description: expected release version, when this version is not equal status.version, the release need upgrade this filed should be modified when any filed of the spec modified.
+                description: expected release version, when this version is not equal
+                  status.version, the release need upgrade this filed should be modified
+                  when any filed of the spec modified.
                 type: integer
             required:
             - chartName
@@ -94,7 +101,8 @@ spec:
             description: HelmReleaseStatus defines the observed state of HelmRelease
             properties:
               deployStatus:
-                description: deploy status list of history, which will store at most 10 state
+                description: deploy status list of history, which will store at most
+                  10 state
                 items:
                   properties:
                     deployTime:
@@ -102,7 +110,8 @@ spec:
                       format: date-time
                       type: string
                     message:
-                      description: A human readable message indicating details about why the release is in this state.
+                      description: A human readable message indicating details about
+                        why the release is in this state.
                       type: string
                     state:
                       description: current state of the release
@@ -121,7 +130,8 @@ spec:
                 format: date-time
                 type: string
               message:
-                description: A human readable message indicating details about why the release is in this state.
+                description: A human readable message indicating details about why
+                  the release is in this state.
                 type: string
               state:
                 description: current state

config/crds/application.kubesphere.io_helmrepos.yaml

@@ -40,10 +40,14 @@ spec:
         description: HelmRepo is the Schema for the helmrepoes API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -56,13 +60,16 @@ spec:
                   accessKeyID:
                     type: string
                   caFile:
-                    description: verify certificates of HTTPS-enabled servers using this CA bundle
+                    description: verify certificates of HTTPS-enabled servers using
+                      this CA bundle
                     type: string
                   certFile:
-                    description: identify HTTPS client using this SSL certificate file
+                    description: identify HTTPS client using this SSL certificate
+                      file
                     type: string
                   insecureSkipTLSVerify:
-                    description: skip tls certificate checks for the repository, default is ture
+                    description: skip tls certificate checks for the repository, default
+                      is ture
                     type: boolean
                   keyFile:
                     description: identify HTTPS client using this SSL key file
@@ -83,13 +90,16 @@ spec:
                 description: name of the repo
                 type: string
               syncPeriod:
-                description: sync period in seconds, no sync when SyncPeriod=0, the minimum SyncPeriod is 180s
+                description: sync period in seconds, no sync when SyncPeriod=0, the
+                  minimum SyncPeriod is 180s
                 type: integer
               url:
                 description: helm repo url
                 type: string
               version:
-                description: expected repo version, when this version is not equal status.version, the repo need upgrade this filed should be modified when any filed of the spec modified.
+                description: expected repo version, when this version is not equal
+                  status.version, the repo need upgrade this filed should be modified
+                  when any filed of the spec modified.
                 type: integer
             required:
             - name
@@ -109,14 +119,17 @@ spec:
                 description: current state of the repo, successful, failed or syncing
                 type: string
               syncState:
-                description: sync state list of history, which will store at most 10 state
+                description: sync state list of history, which will store at most
+                  10 state
                 items:
                   properties:
                     message:
-                      description: A human readable message indicating details about why the repo is in this state.
+                      description: A human readable message indicating details about
+                        why the repo is in this state.
                       type: string
                     state:
-                      description: 'last sync state, valid state are: "failed", "success", and ""'
+                      description: 'last sync state, valid state are: "failed", "success",
+                        and ""'
                       type: string
                     syncTime:
                       format: date-time
@@ -126,7 +139,8 @@ spec:
                   type: object
                 type: array
               version:
-                description: if status.version!=spec.Version, we need sync the repo now
+                description: if status.version!=spec.Version, we need sync the repo
+                  now
                 type: integer
             type: object
         type: object

config/crds/auditing.kubesphere.io_rules.yaml

@@ -1,86 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: rules.auditing.kubesphere.io
-spec:
-  group: auditing.kubesphere.io
-  names:
-    kind: Rule
-    listKind: RuleList
-    plural: rules
-    singular: rule
-  scope: Namespaced
-  versions:
-  - name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: Rule is the Schema for the rules API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: AuditRuleSpec defines the desired state of Rule
-            properties:
-              rules:
-                items:
-                  properties:
-                    alias:
-                      description: This effective When the rule type is alias
-                      type: string
-                    condition:
-                      description: Rule condition This effective When the rule type is rule
-                      type: string
-                    desc:
-                      description: Rule describe
-                      type: string
-                    enable:
-                      description: Is the rule enable
-                      type: boolean
-                    list:
-                      description: This effective When the rule type is list
-                      items:
-                        type: string
-                      type: array
-                    macro:
-                      description: This effective When the rule type is macro
-                      type: string
-                    name:
-                      description: Rule name
-                      type: string
-                    output:
-                      description: The output formater of message which send to user
-                      type: string
-                    priority:
-                      description: Rule priority, DEBUG, INFO, WARNING
-                      type: string
-                    type:
-                      description: Rule type, rule, macro,list,alias
-                      type: string
-                  required:
-                  - enable
-                  type: object
-                type: array
-            type: object
-          status:
-            description: AuditRuleStatus defines the observed state of Rule
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/auditing.kubesphere.io_webhooks.yaml

@@ -1,583 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: webhooks.auditing.kubesphere.io
-spec:
-  group: auditing.kubesphere.io
-  names:
-    kind: Webhook
-    listKind: WebhookList
-    plural: webhooks
-    singular: webhook
-  scope: Namespaced
-  versions:
-  - name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: Webhook is the Schema for the webhooks API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: WebhookSpec defines the desired state of Webhook
-            properties:
-              affinity:
-                description: If specified, the pod's scheduling constraints
-                properties:
-                  nodeAffinity:
-                    description: Describes node affinity scheduling rules for the pod.
-                    properties:
-                      preferredDuringSchedulingIgnoredDuringExecution:
-                        description: The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.
-                        items:
-                          description: An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).
-                          properties:
-                            preference:
-                              description: A node selector term, associated with the corresponding weight.
-                              properties:
-                                matchExpressions:
-                                  description: A list of node selector requirements by node's labels.
-                                  items:
-                                    description: A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                    properties:
-                                      key:
-                                        description: The label key that the selector applies to.
-                                        type: string
-                                      operator:
-                                        description: Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
-                                        type: string
-                                      values:
-                                        description: An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.
-                                        items:
-                                          type: string
-                                        type: array
-                                    required:
-                                    - key
-                                    - operator
-                                    type: object
-                                  type: array
-                                matchFields:
-                                  description: A list of node selector requirements by node's fields.
-                                  items:
-                                    description: A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                    properties:
-                                      key:
-                                        description: The label key that the selector applies to.
-                                        type: string
-                                      operator:
-                                        description: Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
-                                        type: string
-                                      values:
-                                        description: An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.
-                                        items:
-                                          type: string
-                                        type: array
-                                    required:
-                                    - key
-                                    - operator
-                                    type: object
-                                  type: array
-                              type: object
-                            weight:
-                              description: Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.
-                              format: int32
-                              type: integer
-                          required:
-                          - preference
-                          - weight
-                          type: object
-                        type: array
-                      requiredDuringSchedulingIgnoredDuringExecution:
-                        description: If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.
-                        properties:
-                          nodeSelectorTerms:
-                            description: Required. A list of node selector terms. The terms are ORed.
-                            items:
-                              description: A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.
-                              properties:
-                                matchExpressions:
-                                  description: A list of node selector requirements by node's labels.
-                                  items:
-                                    description: A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                    properties:
-                                      key:
-                                        description: The label key that the selector applies to.
-                                        type: string
-                                      operator:
-                                        description: Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
-                                        type: string
-                                      values:
-                                        description: An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.
-                                        items:
-                                          type: string
-                                        type: array
-                                    required:
-                                    - key
-                                    - operator
-                                    type: object
-                                  type: array
-                                matchFields:
-                                  description: A list of node selector requirements by node's fields.
-                                  items:
-                                    description: A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                    properties:
-                                      key:
-                                        description: The label key that the selector applies to.
-                                        type: string
-                                      operator:
-                                        description: Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
-                                        type: string
-                                      values:
-                                        description: An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.
-                                        items:
-                                          type: string
-                                        type: array
-                                    required:
-                                    - key
-                                    - operator
-                                    type: object
-                                  type: array
-                              type: object
-                            type: array
-                        required:
-                        - nodeSelectorTerms
-                        type: object
-                    type: object
-                  podAffinity:
-                    description: Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).
-                    properties:
-                      preferredDuringSchedulingIgnoredDuringExecution:
-                        description: The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.
-                        items:
-                          description: The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)
-                          properties:
-                            podAffinityTerm:
-                              description: Required. A pod affinity term, associated with the corresponding weight.
-                              properties:
-                                labelSelector:
-                                  description: A label query over a set of resources, in this case pods.
-                                  properties:
-                                    matchExpressions:
-                                      description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                      items:
-                                        description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                        properties:
-                                          key:
-                                            description: key is the label key that the selector applies to.
-                                            type: string
-                                          operator:
-                                            description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                            type: string
-                                          values:
-                                            description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                            items:
-                                              type: string
-                                            type: array
-                                        required:
-                                        - key
-                                        - operator
-                                        type: object
-                                      type: array
-                                    matchLabels:
-                                      additionalProperties:
-                                        type: string
-                                      description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                      type: object
-                                  type: object
-                                namespaces:
-                                  description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
-                                  items:
-                                    type: string
-                                  type: array
-                                topologyKey:
-                                  description: This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.
-                                  type: string
-                              required:
-                              - topologyKey
-                              type: object
-                            weight:
-                              description: weight associated with matching the corresponding podAffinityTerm, in the range 1-100.
-                              format: int32
-                              type: integer
-                          required:
-                          - podAffinityTerm
-                          - weight
-                          type: object
-                        type: array
-                      requiredDuringSchedulingIgnoredDuringExecution:
-                        description: If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.
-                        items:
-                          description: Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key <topologyKey> matches that of any node on which a pod of the set of pods is running
-                          properties:
-                            labelSelector:
-                              description: A label query over a set of resources, in this case pods.
-                              properties:
-                                matchExpressions:
-                                  description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                  items:
-                                    description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                    properties:
-                                      key:
-                                        description: key is the label key that the selector applies to.
-                                        type: string
-                                      operator:
-                                        description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                        type: string
-                                      values:
-                                        description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                        items:
-                                          type: string
-                                        type: array
-                                    required:
-                                    - key
-                                    - operator
-                                    type: object
-                                  type: array
-                                matchLabels:
-                                  additionalProperties:
-                                    type: string
-                                  description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                  type: object
-                              type: object
-                            namespaces:
-                              description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
-                              items:
-                                type: string
-                              type: array
-                            topologyKey:
-                              description: This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.
-                              type: string
-                          required:
-                          - topologyKey
-                          type: object
-                        type: array
-                    type: object
-                  podAntiAffinity:
-                    description: Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).
-                    properties:
-                      preferredDuringSchedulingIgnoredDuringExecution:
-                        description: The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.
-                        items:
-                          description: The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)
-                          properties:
-                            podAffinityTerm:
-                              description: Required. A pod affinity term, associated with the corresponding weight.
-                              properties:
-                                labelSelector:
-                                  description: A label query over a set of resources, in this case pods.
-                                  properties:
-                                    matchExpressions:
-                                      description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                      items:
-                                        description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                        properties:
-                                          key:
-                                            description: key is the label key that the selector applies to.
-                                            type: string
-                                          operator:
-                                            description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                            type: string
-                                          values:
-                                            description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                            items:
-                                              type: string
-                                            type: array
-                                        required:
-                                        - key
-                                        - operator
-                                        type: object
-                                      type: array
-                                    matchLabels:
-                                      additionalProperties:
-                                        type: string
-                                      description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                      type: object
-                                  type: object
-                                namespaces:
-                                  description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
-                                  items:
-                                    type: string
-                                  type: array
-                                topologyKey:
-                                  description: This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.
-                                  type: string
-                              required:
-                              - topologyKey
-                              type: object
-                            weight:
-                              description: weight associated with matching the corresponding podAffinityTerm, in the range 1-100.
-                              format: int32
-                              type: integer
-                          required:
-                          - podAffinityTerm
-                          - weight
-                          type: object
-                        type: array
-                      requiredDuringSchedulingIgnoredDuringExecution:
-                        description: If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.
-                        items:
-                          description: Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key <topologyKey> matches that of any node on which a pod of the set of pods is running
-                          properties:
-                            labelSelector:
-                              description: A label query over a set of resources, in this case pods.
-                              properties:
-                                matchExpressions:
-                                  description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                  items:
-                                    description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                    properties:
-                                      key:
-                                        description: key is the label key that the selector applies to.
-                                        type: string
-                                      operator:
-                                        description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                        type: string
-                                      values:
-                                        description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                        items:
-                                          type: string
-                                        type: array
-                                    required:
-                                    - key
-                                    - operator
-                                    type: object
-                                  type: array
-                                matchLabels:
-                                  additionalProperties:
-                                    type: string
-                                  description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                  type: object
-                              type: object
-                            namespaces:
-                              description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
-                              items:
-                                type: string
-                              type: array
-                            topologyKey:
-                              description: This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.
-                              type: string
-                          required:
-                          - topologyKey
-                          type: object
-                        type: array
-                    type: object
-                type: object
-              args:
-                description: Arguments to the entrypoint.. It will be appended to the args and replace the default value.
-                items:
-                  type: string
-                type: array
-              auditLevel:
-                description: 'The Level that all requests are recorded at. available options: None, Metadata, Request, RequestResponse default: Metadata'
-                type: string
-              auditSinkPolicy:
-                description: AuditSinkPolicy is a rule selector, only the rule matched this selector will be taked effect.
-                properties:
-                  alertingRuleSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  archivingRuleSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                type: object
-              auditType:
-                description: Audit type, static or dynamic.
-                type: string
-              image:
-                description: The webhook docker image name.
-                type: string
-              imagePullPolicy:
-                description: 'Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images'
-                type: string
-              imagePullSecrets:
-                description: 'ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images used by this PodSpec. If specified, these secrets will be passed to individual puller implementations for them to use. For example, in the case of docker, only DockerConfig type secrets are honored. More info: https://kubernetes.io/docs/concepts/containers/images#specifying-imagepullsecrets-on-a-pod'
-                items:
-                  description: LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.
-                  properties:
-                    name:
-                      description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?'
-                      type: string
-                  type: object
-                type: array
-              k8sAuditingEnabled:
-                description: K8s auditing is enabled or not.
-                type: boolean
-              nodeSelector:
-                additionalProperties:
-                  type: string
-                description: 'NodeSelector is a selector which must be true for the pod to fit on a node. Selector which must match a node''s labels for the pod to be scheduled on that node. More info: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/'
-                type: object
-              priority:
-                description: Rule priority, DEBUG < INFO < WARNING Audit events will be stored only when the priority of the audit rule matching the audit event is greater than this.
-                type: string
-              receivers:
-                description: ' Receiver contains the information to make a connection with the alertmanager'
-                items:
-                  description: Receiver config which received the audit alert
-                  properties:
-                    config:
-                      description: ClientConfig holds the connection parameters for the webhook
-                      properties:
-                        caBundle:
-                          description: '`caBundle` is a PEM encoded CA bundle which will be used to validate the webhook''s server certificate. If unspecified, system trust roots on the apiserver are used.'
-                          format: byte
-                          type: string
-                        service:
-                          description: "`service` is a reference to the service for this webhook. Either `service` or `url` must be specified. \n If the webhook is running within the cluster, then you should use `service`."
-                          properties:
-                            name:
-                              description: '`name` is the name of the service. Required'
-                              type: string
-                            namespace:
-                              description: '`namespace` is the namespace of the service. Required'
-                              type: string
-                            path:
-                              description: '`path` is an optional URL path which will be sent in any request to this service.'
-                              type: string
-                            port:
-                              description: If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive).
-                              format: int32
-                              type: integer
-                          required:
-                          - name
-                          - namespace
-                          type: object
-                        url:
-                          description: "`url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. \n The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some apiservers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. \n Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. \n The scheme must be \"https\"; the URL must begin with \"https://\". \n A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. \n Attempting to use a user or basic auth e.g. \"user:password@\" is not allowed. Fragments (\"#...\") and query parameters (\"?...\") are not allowed, either."
-                          type: string
-                      type: object
-                    name:
-                      description: Receiver name
-                      type: string
-                    type:
-                      description: Receiver type, alertmanager or webhook
-                      type: string
-                  type: object
-                type: array
-              replicas:
-                description: Number of desired pods. This is a pointer to distinguish between explicit zero and not specified. Defaults to 1.
-                format: int32
-                type: integer
-              resources:
-                description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                properties:
-                  limits:
-                    additionalProperties:
-                      anyOf:
-                      - type: integer
-                      - type: string
-                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                      x-kubernetes-int-or-string: true
-                    description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                    type: object
-                  requests:
-                    additionalProperties:
-                      anyOf:
-                      - type: integer
-                      - type: string
-                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                      x-kubernetes-int-or-string: true
-                    description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                    type: object
-                type: object
-              tolerations:
-                description: If specified, the pod's tolerations.
-                items:
-                  description: The pod this Toleration is attached to tolerates any taint that matches the triple <key,value,effect> using the matching operator <operator>.
-                  properties:
-                    effect:
-                      description: Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
-                      type: string
-                    key:
-                      description: Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.
-                      type: string
-                    operator:
-                      description: Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.
-                      type: string
-                    tolerationSeconds:
-                      description: TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.
-                      format: int64
-                      type: integer
-                    value:
-                      description: Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.
-                      type: string
-                  type: object
-                type: array
-            type: object
-          status:
-            description: WebhookStatus defines the observed state of Webhook
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/cluster.kubesphere.io_clusters.yaml

@@ -35,10 +35,14 @@ spec:
         description: Cluster is the schema for the clusters API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -47,32 +51,58 @@ spec:
               connection:
                 description: Connection holds info to connect to the member cluster
                 properties:
+                  externalKubernetesAPIEndpoint:
+                    description: External Kubernetes API Server endpoint Will be populated
+                      by ks-apiserver if connection type is proxy and ExternalKubeAPIEnabled
+                      is true.
+                    type: string
                   kubeconfig:
-                    description: KubeConfig content used to connect to cluster api server Should provide this field explicitly if connection type is direct. Will be populated by ks-proxy if connection type is proxy.
+                    description: KubeConfig content used to connect to cluster api
+                      server Should provide this field explicitly if connection type
+                      is direct. Will be populated by ks-proxy if connection type
+                      is proxy.
                     format: byte
                     type: string
                   kubernetesAPIEndpoint:
-                    description: 'Kubernetes API Server endpoint. Example: https://10.10.0.1:6443 Should provide this field explicitly if connection type is direct. Will be populated by ks-apiserver if connection type is proxy.'
+                    description: 'Kubernetes API Server endpoint. Example: https://10.10.0.1:6443
+                      Should provide this field explicitly if connection type is direct.
+                      Will be populated by ks-apiserver if connection type is proxy.'
                     type: string
                   kubernetesAPIServerPort:
-                    description: KubeAPIServerPort is the port which listens for forwarding kube-apiserver traffic Only applicable when connection type is proxy.
+                    description: KubeAPIServerPort is the port which listens for forwarding
+                      kube-apiserver traffic Only applicable when connection type
+                      is proxy.
                     type: integer
                   kubesphereAPIEndpoint:
-                    description: 'KubeSphere API Server endpoint. Example: http://10.10.0.11:8080 Should provide this field explicitly if connection type is direct. Will be populated by ks-apiserver if connection type is proxy.'
+                    description: 'KubeSphere API Server endpoint. Example: http://10.10.0.11:8080
+                      Should provide this field explicitly if connection type is direct.
+                      Will be populated by ks-apiserver if connection type is proxy.'
                     type: string
                   kubesphereAPIServerPort:
-                    description: KubeSphereAPIServerPort is the port which listens for forwarding kubesphere apigateway traffic Only applicable when connection type is proxy.
+                    description: KubeSphereAPIServerPort is the port which listens
+                      for forwarding kubesphere apigateway traffic Only applicable
+                      when connection type is proxy.
                     type: integer
                   token:
-                    description: Token used by agents of member cluster to connect to host cluster proxy. This field is populated by apiserver only if connection type is proxy.
+                    description: Token used by agents of member cluster to connect
+                      to host cluster proxy. This field is populated by apiserver
+                      only if connection type is proxy.
                     type: string
                   type:
-                    description: type defines how host cluster will connect to host cluster ConnectionTypeDirect means direct connection, this requires   kubeconfig and kubesphere apiserver endpoint provided ConnectionTypeProxy means using kubesphere proxy, no kubeconfig   or kubesphere apiserver endpoint required
+                    description: type defines how host cluster will connect to host
+                      cluster ConnectionTypeDirect means direct connection, this requires   kubeconfig
+                      and kubesphere apiserver endpoint provided ConnectionTypeProxy
+                      means using kubesphere proxy, no kubeconfig   or kubesphere
+                      apiserver endpoint required
                     type: string
                 type: object
               enable:
                 description: Desired state of the cluster
                 type: boolean
+              externalKubeAPIEnabled:
+                description: ExternalKubeAPIEnabled export kubeapiserver to public
+                  use a lb type service if connection type is proxy
+                type: boolean
               joinFederation:
                 description: Join cluster as a kubefed cluster
                 type: boolean
@@ -83,11 +113,13 @@ spec:
           status:
             properties:
               conditions:
-                description: Represents the latest available observations of a cluster's current state.
+                description: Represents the latest available observations of a cluster's
+                  current state.
                 items:
                   properties:
                     lastTransitionTime:
-                      description: Last time the condition transitioned from one status to another.
+                      description: Last time the condition transitioned from one status
+                        to another.
                       format: date-time
                       type: string
                     lastUpdateTime:
@@ -95,7 +127,8 @@ spec:
                       format: date-time
                       type: string
                     message:
-                      description: A human readable message indicating details about the transition.
+                      description: A human readable message indicating details about
+                        the transition.
                       type: string
                     reason:
                       description: The reason for the condition's last transition.
@@ -114,22 +147,33 @@ spec:
               configz:
                 additionalProperties:
                   type: boolean
-                description: Configz is status of components enabled in the member cluster. This is synchronized with member cluster every amount of time, like 5 minutes.
+                description: Configz is status of components enabled in the member
+                  cluster. This is synchronized with member cluster every amount of
+                  time, like 5 minutes.
                 type: object
               kubeSphereVersion:
-                description: GitVersion of the /kapis/version api response, this field is populated by cluster controller
+                description: GitVersion of the /kapis/version api response, this field
+                  is populated by cluster controller
                 type: string
               kubernetesVersion:
-                description: GitVersion of the kubernetes cluster, this field is populated by cluster controller
+                description: GitVersion of the kubernetes cluster, this field is populated
+                  by cluster controller
                 type: string
               nodeCount:
-                description: Count of the kubernetes cluster nodes This field may not reflect the instant status of the cluster.
+                description: Count of the kubernetes cluster nodes This field may
+                  not reflect the instant status of the cluster.
                 type: integer
               region:
-                description: Region is the name of the region in which all of the nodes in the cluster exist.  e.g. 'us-east1'.
+                description: Region is the name of the region in which all of the
+                  nodes in the cluster exist.  e.g. 'us-east1'.
+                type: string
+              uid:
+                description: UID is the kube-system namespace UID of the cluster,
+                  which represents the unique ID of the cluster.
                 type: string
               zones:
-                description: Zones are the names of availability zones in which the nodes of the cluster exist, e.g. 'us-east1-a'.
+                description: Zones are the names of availability zones in which the
+                  nodes of the cluster exist, e.g. 'us-east1-a'.
                 items:
                   type: string
                 type: array

config/crds/crd.projectcalico.org_blockaffinities.yaml

@@ -1,56 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: blockaffinities.crd.projectcalico.org
-spec:
-  group: crd.projectcalico.org
-  names:
-    kind: BlockAffinity
-    listKind: BlockAffinityList
-    plural: blockaffinities
-    singular: blockaffinity
-  scope: Cluster
-  versions:
-  - name: calicov3
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: BlockAffinitySpec contains the specification for a BlockAffinity resource.
-            properties:
-              cidr:
-                type: string
-              deleted:
-                type: string
-              node:
-                type: string
-              state:
-                type: string
-            required:
-            - cidr
-            - deleted
-            - node
-            - state
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/crd.projectcalico.org_ipamblocks.yaml

@@ -1,81 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: ipamblocks.crd.projectcalico.org
-spec:
-  group: crd.projectcalico.org
-  names:
-    kind: IPAMBlock
-    listKind: IPAMBlockList
-    plural: ipamblocks
-    singular: ipamblock
-  scope: Cluster
-  versions:
-  - name: calicov3
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: IPAMBlockSpec contains the specification for a IPAMBlock resource.
-            properties:
-              affinity:
-                type: string
-              allocations:
-                items:
-                  type: integer
-                type: array
-              attributes:
-                items:
-                  properties:
-                    handle_id:
-                      type: string
-                    secondary:
-                      additionalProperties:
-                        type: string
-                      type: object
-                  required:
-                  - handle_id
-                  - secondary
-                  type: object
-                type: array
-              cidr:
-                type: string
-              deleted:
-                type: boolean
-              strictAffinity:
-                type: boolean
-              unallocated:
-                items:
-                  type: integer
-                type: array
-            required:
-            - affinity
-            - allocations
-            - attributes
-            - cidr
-            - deleted
-            - strictAffinity
-            - unallocated
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/crd.projectcalico.org_ippools.yaml

@@ -1,79 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: ippools.crd.projectcalico.org
-spec:
-  group: crd.projectcalico.org
-  names:
-    kind: IPPool
-    listKind: IPPoolList
-    plural: ippools
-    singular: ippool
-  scope: Cluster
-  versions:
-  - name: calicov3
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: IPPoolSpec contains the specification for an IPPool resource.
-            properties:
-              blockSize:
-                description: The block size to use for IP address assignments from this pool. Defaults to 26 for IPv4 and 112 for IPv6.
-                type: integer
-              cidr:
-                description: The pool CIDR.
-                type: string
-              disabled:
-                description: When disabled is true, Calico IPAM will not assign addresses from this pool.
-                type: boolean
-              ipip:
-                description: 'Deprecated: this field is only used for APIv1 backwards compatibility. Setting this field is not allowed, this field is for internal use only.'
-                properties:
-                  enabled:
-                    description: When enabled is true, ipip tunneling will be used to deliver packets to destinations within this pool.
-                    type: boolean
-                  mode:
-                    description: The IPIP mode.  This can be one of "always" or "cross-subnet".  A mode of "always" will also use IPIP tunneling for routing to destination IP addresses within this pool.  A mode of "cross-subnet" will only use IPIP tunneling when the destination node is on a different subnet to the originating node.  The default value (if not specified) is "always".
-                    type: string
-                type: object
-              ipipMode:
-                description: Contains configuration for IPIP tunneling for this pool. If not specified, then this is defaulted to "Never" (i.e. IPIP tunelling is disabled).
-                type: string
-              nat-outgoing:
-                description: 'Deprecated: this field is only used for APIv1 backwards compatibility. Setting this field is not allowed, this field is for internal use only.'
-                type: boolean
-              natOutgoing:
-                description: When nat-outgoing is true, packets sent from Calico networked containers in this pool to destinations outside of this pool will be masqueraded.
-                type: boolean
-              nodeSelector:
-                description: Allows IPPool to allocate for a specific node by label selector.
-                type: string
-              vxlanMode:
-                description: Contains configuration for VXLAN tunneling for this pool. If not specified, then this is defaulted to "Never" (i.e. VXLAN tunelling is disabled).
-                type: string
-            required:
-            - cidr
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/devops.kubesphere.io_s2ibuilders.yaml

@@ -51,7 +51,7 @@ spec:
                 description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file'
                 properties:
                   addHost:
-                    description: AddHost Add a line to /etc/hosts for test purpose or private use in LAN. Its format is host:IP,muliple hosts can be added  by using multiple --add-host
+                    description: AddHost Add a line to /etc/hosts for test purpose or private use in LAN. Its format is host:IP,multiple hosts can be added  by using multiple --add-host
                     items:
                       type: string
                     type: array

config/crds/gateway.kubesphere.io_gateways.yaml

@@ -0,0 +1,122 @@
+
+---
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: (devel)
+  creationTimestamp: null
+  name: gateways.gateway.kubesphere.io
+spec:
+  group: gateway.kubesphere.io
+  names:
+    kind: Gateway
+    listKind: GatewayList
+    plural: gateways
+    singular: gateway
+  scope: Namespaced
+  versions:
+  - name: v1alpha1
+    schema:
+      openAPIV3Schema:
+        description: Gateway is the Schema for the gateways API
+        properties:
+          apiVersion:
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            type: string
+          kind:
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            type: string
+          metadata:
+            type: object
+          spec:
+            description: GatewaySpec defines the desired state of Gateway
+            properties:
+              controller:
+                properties:
+                  annotations:
+                    additionalProperties:
+                      type: string
+                    type: object
+                  config:
+                    additionalProperties:
+                      type: string
+                    type: object
+                  replicas:
+                    format: int32
+                    type: integer
+                  scope:
+                    properties:
+                      enabled:
+                        type: boolean
+                      namespace:
+                        type: string
+                    type: object
+                type: object
+              deployment:
+                properties:
+                  annotations:
+                    additionalProperties:
+                      type: string
+                    type: object
+                  replicas:
+                    format: int32
+                    type: integer
+                  resources:
+                    description: ResourceRequirements describes the compute resource
+                      requirements.
+                    properties:
+                      limits:
+                        additionalProperties:
+                          anyOf:
+                          - type: integer
+                          - type: string
+                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                          x-kubernetes-int-or-string: true
+                        description: 'Limits describes the maximum amount of compute
+                          resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+                        type: object
+                      requests:
+                        additionalProperties:
+                          anyOf:
+                          - type: integer
+                          - type: string
+                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                          x-kubernetes-int-or-string: true
+                        description: 'Requests describes the minimum amount of compute
+                          resources required. If Requests is omitted for a container,
+                          it defaults to Limits if that is explicitly specified, otherwise
+                          to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+                        type: object
+                    type: object
+                type: object
+              service:
+                properties:
+                  annotations:
+                    additionalProperties:
+                      type: string
+                    type: object
+                  type:
+                    description: Service Type string describes ingress methods for
+                      a service
+                    type: string
+                type: object
+            type: object
+          status:
+            type: object
+            x-kubernetes-preserve-unknown-fields: true
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}
+status:
+  acceptedNames:
+    kind: ""
+    plural: ""
+  conditions: []
+  storedVersions: []

config/crds/gateway.kubesphere.io_nginxes.yaml

@@ -0,0 +1,44 @@
+---
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  name: nginxes.gateway.kubesphere.io
+spec:
+  group: gateway.kubesphere.io
+  names:
+    kind: Nginx
+    listKind: NginxList
+    plural: nginxes
+    singular: nginx
+  scope: Namespaced
+  versions:
+  - name: v1alpha1
+    schema:
+      openAPIV3Schema:
+        description: Nginx is the Schema for the nginxes API
+        properties:
+          apiVersion:
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            type: string
+          kind:
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            type: string
+          metadata:
+            type: object
+          spec:
+            description: Spec defines the desired state of Nginx
+            type: object
+            x-kubernetes-preserve-unknown-fields: true
+          status:
+            description: Status defines the observed state of Nginx
+            type: object
+            x-kubernetes-preserve-unknown-fields: true
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}

config/crds/iam.kubesphere.io_federatedrolebindings.yaml

@@ -21,10 +21,14 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -54,10 +58,12 @@ spec:
                   metadata:
                     type: object
                   roleRef:
-                    description: RoleRef contains information that points to the role being used
+                    description: RoleRef contains information that points to the role
+                      being used
                     properties:
                       apiGroup:
-                        description: APIGroup is the group for the resource being referenced
+                        description: APIGroup is the group for the resource being
+                          referenced
                         type: string
                       kind:
                         description: Kind is the type of resource being referenced
@@ -72,19 +78,30 @@ spec:
                     type: object
                   subjects:
                     items:
-                      description: Subject contains a reference to the object or user identities a role binding applies to.  This can either hold a direct API object reference, or a value for non-objects such as user and group names.
+                      description: Subject contains a reference to the object or user
+                        identities a role binding applies to.  This can either hold
+                        a direct API object reference, or a value for non-objects
+                        such as user and group names.
                       properties:
                         apiGroup:
-                          description: APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io" for User and Group subjects.
+                          description: APIGroup holds the API group of the referenced
+                            subject. Defaults to "" for ServiceAccount subjects. Defaults
+                            to "rbac.authorization.k8s.io" for User and Group subjects.
                           type: string
                         kind:
-                          description: Kind of object being referenced. Values defined by this API group are "User", "Group", and "ServiceAccount". If the Authorizer does not recognized the kind value, the Authorizer should report an error.
+                          description: Kind of object being referenced. Values defined
+                            by this API group are "User", "Group", and "ServiceAccount".
+                            If the Authorizer does not recognized the kind value,
+                            the Authorizer should report an error.
                           type: string
                         name:
                           description: Name of the object being referenced.
                           type: string
                         namespace:
-                          description: Namespace of the referenced object.  If the object kind is non-namespace, such as "User" or "Group", and this value is not empty the Authorizer should report an error.
+                          description: Namespace of the referenced object.  If the
+                            object kind is non-namespace, such as "User" or "Group",
+                            and this value is not empty the Authorizer should report
+                            an error.
                           type: string
                       required:
                       - kind

config/crds/iam.kubesphere.io_federatedroles.yaml

@@ -21,10 +21,14 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -55,30 +59,47 @@ spec:
                     type: object
                   rules:
                     items:
-                      description: PolicyRule holds information that describes a policy rule, but does not contain information about who the rule applies to or which namespace the rule applies to.
+                      description: PolicyRule holds information that describes a policy
+                        rule, but does not contain information about who the rule
+                        applies to or which namespace the rule applies to.
                       properties:
                         apiGroups:
-                          description: APIGroups is the name of the APIGroup that contains the resources.  If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed.
+                          description: APIGroups is the name of the APIGroup that
+                            contains the resources.  If multiple API groups are specified,
+                            any action requested against one of the enumerated resources
+                            in any API group will be allowed.
                           items:
                             type: string
                           type: array
                         nonResourceURLs:
-                          description: NonResourceURLs is a set of partial urls that a user should have access to.  *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"),  but not both.
+                          description: NonResourceURLs is a set of partial urls that
+                            a user should have access to.  *s are allowed, but only
+                            as the full, final step in the path Since non-resource
+                            URLs are not namespaced, this field is only applicable
+                            for ClusterRoles referenced from a ClusterRoleBinding.
+                            Rules can either apply to API resources (such as "pods"
+                            or "secrets") or non-resource URL paths (such as "/api"),  but
+                            not both.
                           items:
                             type: string
                           type: array
                         resourceNames:
-                          description: ResourceNames is an optional white list of names that the rule applies to.  An empty set means that everything is allowed.
+                          description: ResourceNames is an optional white list of
+                            names that the rule applies to.  An empty set means that
+                            everything is allowed.
                           items:
                             type: string
                           type: array
                         resources:
-                          description: Resources is a list of resources this rule applies to.  ResourceAll represents all resources.
+                          description: Resources is a list of resources this rule
+                            applies to.  ResourceAll represents all resources.
                           items:
                             type: string
                           type: array
                         verbs:
-                          description: Verbs is a list of Verbs that apply to ALL the ResourceKinds and AttributeRestrictions contained in this rule.  VerbAll represents all kinds.
+                          description: Verbs is a list of Verbs that apply to ALL
+                            the ResourceKinds and AttributeRestrictions contained
+                            in this rule.  VerbAll represents all kinds.
                           items:
                             type: string
                           type: array

config/crds/iam.kubesphere.io_federatedusers.yaml

@@ -21,10 +21,14 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -69,10 +73,32 @@ spec:
                           type: string
                         type: array
                       lang:
-                        description: The preferred written or spoken language for the user.
+                        description: The preferred written or spoken language for
+                          the user.
                         type: string
                       password:
-                        description: password will be encrypted by mutating admission webhook
+                        description: 'password will be encrypted by mutating admission
+                          webhook Password pattern is tricky here. The rule is simple:
+                          length between [6,64], at least one uppercase letter, one
+                          lowercase letter, one digit. The regexp in console(javascript)
+                          is quite straightforward: ^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)[^]{6,64}$
+                          But in Go, we don''t have ?= (back tracking) capability
+                          in regexp (also in CRD validation pattern) So we adopted
+                          an alternative scheme to achieve. Use 6 different regexp
+                          to combine to achieve the same effect. These six schemes
+                          enumerate the arrangement of numbers, uppercase letters,
+                          and lowercase letters that appear for the first time. -
+                          ^(.*[a-z].*[A-Z].*[0-9].*)$ stands for lowercase letter
+                          comes first, then followed by an uppercase letter, then
+                          a digit. - ^(.*[a-z].*[0-9].*[A-Z].*)$ stands for lowercase
+                          letter comes first, then followed by a digit, then an uppercase
+                          leeter. - ^(.*[A-Z].*[a-z].*[0-9].*)$ ... - ^(.*[A-Z].*[0-9].*[a-z].*)$
+                          ... - ^(.*[0-9].*[a-z].*[A-Z].*)$ ... - ^(.*[0-9].*[A-Z].*[a-z].*)$
+                          ... Last but not least, the bcrypt string is also included
+                          to match the encrypted password. ^(\$2[ayb]\$.{56})$'
+                        maxLength: 64
+                        minLength: 6
+                        pattern: ^(.*[a-z].*[A-Z].*[0-9].*)$|^(.*[a-z].*[0-9].*[A-Z].*)$|^(.*[A-Z].*[a-z].*[0-9].*)$|^(.*[A-Z].*[0-9].*[a-z].*)$|^(.*[0-9].*[a-z].*[A-Z].*)$|^(.*[0-9].*[A-Z].*[a-z].*)$|^(\$2[ayb]\$.{56})$
                         type: string
                     required:
                     - email

config/crds/iam.kubesphere.io_globalrolebindings.yaml

@@ -24,15 +24,20 @@ spec:
         description: GlobalRoleBinding is the Schema for the globalrolebindings API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           roleRef:
-            description: RoleRef can only reference a GlobalRole. If the RoleRef cannot be resolved, the Authorizer must return an error.
+            description: RoleRef can only reference a GlobalRole. If the RoleRef cannot
+              be resolved, the Authorizer must return an error.
             properties:
               apiGroup:
                 description: APIGroup is the group for the resource being referenced
@@ -49,21 +54,31 @@ spec:
             - name
             type: object
           subjects:
-            description: Subjects holds references to the objects the role applies to.
+            description: Subjects holds references to the objects the role applies
+              to.
             items:
-              description: Subject contains a reference to the object or user identities a role binding applies to.  This can either hold a direct API object reference, or a value for non-objects such as user and group names.
+              description: Subject contains a reference to the object or user identities
+                a role binding applies to.  This can either hold a direct API object
+                reference, or a value for non-objects such as user and group names.
               properties:
                 apiGroup:
-                  description: APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io" for User and Group subjects.
+                  description: APIGroup holds the API group of the referenced subject.
+                    Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io"
+                    for User and Group subjects.
                   type: string
                 kind:
-                  description: Kind of object being referenced. Values defined by this API group are "User", "Group", and "ServiceAccount". If the Authorizer does not recognized the kind value, the Authorizer should report an error.
+                  description: Kind of object being referenced. Values defined by
+                    this API group are "User", "Group", and "ServiceAccount". If the
+                    Authorizer does not recognized the kind value, the Authorizer
+                    should report an error.
                   type: string
                 name:
                   description: Name of the object being referenced.
                   type: string
                 namespace:
-                  description: Namespace of the referenced object.  If the object kind is non-namespace, such as "User" or "Group", and this value is not empty the Authorizer should report an error.
+                  description: Namespace of the referenced object.  If the object
+                    kind is non-namespace, such as "User" or "Group", and this value
+                    is not empty the Authorizer should report an error.
                   type: string
               required:
               - kind

config/crds/iam.kubesphere.io_globalroles.yaml

@@ -23,40 +23,59 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           rules:
             description: Rules holds all the PolicyRules for this GlobalRole
             items:
-              description: PolicyRule holds information that describes a policy rule, but does not contain information about who the rule applies to or which namespace the rule applies to.
+              description: PolicyRule holds information that describes a policy rule,
+                but does not contain information about who the rule applies to or
+                which namespace the rule applies to.
               properties:
                 apiGroups:
-                  description: APIGroups is the name of the APIGroup that contains the resources.  If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed.
+                  description: APIGroups is the name of the APIGroup that contains
+                    the resources.  If multiple API groups are specified, any action
+                    requested against one of the enumerated resources in any API group
+                    will be allowed.
                   items:
                     type: string
                   type: array
                 nonResourceURLs:
-                  description: NonResourceURLs is a set of partial urls that a user should have access to.  *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"),  but not both.
+                  description: NonResourceURLs is a set of partial urls that a user
+                    should have access to.  *s are allowed, but only as the full,
+                    final step in the path Since non-resource URLs are not namespaced,
+                    this field is only applicable for ClusterRoles referenced from
+                    a ClusterRoleBinding. Rules can either apply to API resources
+                    (such as "pods" or "secrets") or non-resource URL paths (such
+                    as "/api"),  but not both.
                   items:
                     type: string
                   type: array
                 resourceNames:
-                  description: ResourceNames is an optional white list of names that the rule applies to.  An empty set means that everything is allowed.
+                  description: ResourceNames is an optional white list of names that
+                    the rule applies to.  An empty set means that everything is allowed.
                   items:
                     type: string
                   type: array
                 resources:
-                  description: Resources is a list of resources this rule applies to.  ResourceAll represents all resources.
+                  description: Resources is a list of resources this rule applies
+                    to.  ResourceAll represents all resources.
                   items:
                     type: string
                   type: array
                 verbs:
-                  description: Verbs is a list of Verbs that apply to ALL the ResourceKinds and AttributeRestrictions contained in this rule.  VerbAll represents all kinds.
+                  description: Verbs is a list of Verbs that apply to ALL the ResourceKinds
+                    and AttributeRestrictions contained in this rule.  VerbAll represents
+                    all kinds.
                   items:
                     type: string
                   type: array

config/crds/iam.kubesphere.io_groupbindings.yaml

@@ -31,7 +31,9 @@ spec:
         description: GroupBinding is the Schema for the groupbindings API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           groupRef:
             description: GroupRef defines the desired relation of GroupBinding
@@ -44,7 +46,9 @@ spec:
                 type: string
             type: object
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/iam.kubesphere.io_groups.yaml

@@ -28,10 +28,14 @@ spec:
         description: Group is the Schema for the groups API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/iam.kubesphere.io_loginrecords.yaml

@@ -42,10 +42,14 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/iam.kubesphere.io_rolebases.yaml

@@ -23,15 +23,21 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           role:
             type: object
+            x-kubernetes-embedded-resource: true
+            x-kubernetes-preserve-unknown-fields: true
         required:
         - role
         type: object

config/crds/iam.kubesphere.io_users.yaml

@@ -31,10 +31,14 @@ spec:
         description: User is the Schema for the users API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -57,7 +61,26 @@ spec:
                 description: The preferred written or spoken language for the user.
                 type: string
               password:
-                description: password will be encrypted by mutating admission webhook
+                description: 'password will be encrypted by mutating admission webhook
+                  Password pattern is tricky here. The rule is simple: length between
+                  [6,64], at least one uppercase letter, one lowercase letter, one
+                  digit. The regexp in console(javascript) is quite straightforward:
+                  ^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)[^]{6,64}$ But in Go, we don''t have
+                  ?= (back tracking) capability in regexp (also in CRD validation
+                  pattern) So we adopted an alternative scheme to achieve. Use 6 different
+                  regexp to combine to achieve the same effect. These six schemes
+                  enumerate the arrangement of numbers, uppercase letters, and lowercase
+                  letters that appear for the first time. - ^(.*[a-z].*[A-Z].*[0-9].*)$
+                  stands for lowercase letter comes first, then followed by an uppercase
+                  letter, then a digit. - ^(.*[a-z].*[0-9].*[A-Z].*)$ stands for lowercase
+                  letter comes first, then followed by a digit, then an uppercase
+                  leeter. - ^(.*[A-Z].*[a-z].*[0-9].*)$ ... - ^(.*[A-Z].*[0-9].*[a-z].*)$
+                  ... - ^(.*[0-9].*[a-z].*[A-Z].*)$ ... - ^(.*[0-9].*[A-Z].*[a-z].*)$
+                  ... Last but not least, the bcrypt string is also included to match
+                  the encrypted password. ^(\$2[ayb]\$.{56})$'
+                maxLength: 64
+                minLength: 6
+                pattern: ^(.*[a-z].*[A-Z].*[0-9].*)$|^(.*[a-z].*[0-9].*[A-Z].*)$|^(.*[A-Z].*[a-z].*[0-9].*)$|^(.*[A-Z].*[0-9].*[a-z].*)$|^(.*[0-9].*[a-z].*[A-Z].*)$|^(.*[0-9].*[A-Z].*[a-z].*)$|^(\$2[ayb]\$.{56})$
                 type: string
             required:
             - email

config/crds/iam.kubesphere.io_workspacerolebindings.yaml

@@ -25,18 +25,24 @@ spec:
     name: v1alpha2
     schema:
       openAPIV3Schema:
-        description: WorkspaceRoleBinding is the Schema for the workspacerolebindings API
+        description: WorkspaceRoleBinding is the Schema for the workspacerolebindings
+          API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           roleRef:
-            description: RoleRef can only reference a WorkspaceRole. If the RoleRef cannot be resolved, the Authorizer must return an error.
+            description: RoleRef can only reference a WorkspaceRole. If the RoleRef
+              cannot be resolved, the Authorizer must return an error.
             properties:
               apiGroup:
                 description: APIGroup is the group for the resource being referenced
@@ -53,21 +59,31 @@ spec:
             - name
             type: object
           subjects:
-            description: Subjects holds references to the objects the role applies to.
+            description: Subjects holds references to the objects the role applies
+              to.
             items:
-              description: Subject contains a reference to the object or user identities a role binding applies to.  This can either hold a direct API object reference, or a value for non-objects such as user and group names.
+              description: Subject contains a reference to the object or user identities
+                a role binding applies to.  This can either hold a direct API object
+                reference, or a value for non-objects such as user and group names.
               properties:
                 apiGroup:
-                  description: APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io" for User and Group subjects.
+                  description: APIGroup holds the API group of the referenced subject.
+                    Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io"
+                    for User and Group subjects.
                   type: string
                 kind:
-                  description: Kind of object being referenced. Values defined by this API group are "User", "Group", and "ServiceAccount". If the Authorizer does not recognized the kind value, the Authorizer should report an error.
+                  description: Kind of object being referenced. Values defined by
+                    this API group are "User", "Group", and "ServiceAccount". If the
+                    Authorizer does not recognized the kind value, the Authorizer
+                    should report an error.
                   type: string
                 name:
                   description: Name of the object being referenced.
                   type: string
                 namespace:
-                  description: Namespace of the referenced object.  If the object kind is non-namespace, such as "User" or "Group", and this value is not empty the Authorizer should report an error.
+                  description: Namespace of the referenced object.  If the object
+                    kind is non-namespace, such as "User" or "Group", and this value
+                    is not empty the Authorizer should report an error.
                   type: string
               required:
               - kind

config/crds/iam.kubesphere.io_workspaceroles.yaml

@@ -30,40 +30,59 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           rules:
             description: Rules holds all the PolicyRules for this WorkspaceRole
             items:
-              description: PolicyRule holds information that describes a policy rule, but does not contain information about who the rule applies to or which namespace the rule applies to.
+              description: PolicyRule holds information that describes a policy rule,
+                but does not contain information about who the rule applies to or
+                which namespace the rule applies to.
               properties:
                 apiGroups:
-                  description: APIGroups is the name of the APIGroup that contains the resources.  If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed.
+                  description: APIGroups is the name of the APIGroup that contains
+                    the resources.  If multiple API groups are specified, any action
+                    requested against one of the enumerated resources in any API group
+                    will be allowed.
                   items:
                     type: string
                   type: array
                 nonResourceURLs:
-                  description: NonResourceURLs is a set of partial urls that a user should have access to.  *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"),  but not both.
+                  description: NonResourceURLs is a set of partial urls that a user
+                    should have access to.  *s are allowed, but only as the full,
+                    final step in the path Since non-resource URLs are not namespaced,
+                    this field is only applicable for ClusterRoles referenced from
+                    a ClusterRoleBinding. Rules can either apply to API resources
+                    (such as "pods" or "secrets") or non-resource URL paths (such
+                    as "/api"),  but not both.
                   items:
                     type: string
                   type: array
                 resourceNames:
-                  description: ResourceNames is an optional white list of names that the rule applies to.  An empty set means that everything is allowed.
+                  description: ResourceNames is an optional white list of names that
+                    the rule applies to.  An empty set means that everything is allowed.
                   items:
                     type: string
                   type: array
                 resources:
-                  description: Resources is a list of resources this rule applies to.  ResourceAll represents all resources.
+                  description: Resources is a list of resources this rule applies
+                    to.  ResourceAll represents all resources.
                   items:
                     type: string
                   type: array
                 verbs:
-                  description: Verbs is a list of Verbs that apply to ALL the ResourceKinds and AttributeRestrictions contained in this rule.  VerbAll represents all kinds.
+                  description: Verbs is a list of Verbs that apply to ALL the ResourceKinds
+                    and AttributeRestrictions contained in this rule.  VerbAll represents
+                    all kinds.
                   items:
                     type: string
                   type: array

config/crds/istio_v1alpha3_destinationrule.yaml

@@ -1,763 +0,0 @@
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  creationTimestamp: null
-  labels:
-    controller-tools.k8s.io: "1.0"
-  name: destinationrules.istio.kubesphere.io
-spec:
-  group: istio.kubesphere.io
-  names:
-    kind: DestinationRule
-    plural: destinationrules
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          properties:
-            host:
-              description: 'REQUIRED. The name of a service from the service registry.
-                Service names are looked up from the platform''s service registry
-                (e.g., Kubernetes services, Consul services, etc.) and from the hosts
-                declared by [ServiceEntries](#ServiceEntry). Rules defined for services
-                that do not exist in the service registry will be ignored.  *Note
-                for Kubernetes users*: When short names are used (e.g. "reviews" instead
-                of "reviews.default.svc.cluster.local"), Istio will interpret the
-                short name based on the namespace of the rule, not the service. A
-                rule in the "default" namespace containing a host "reviews will be
-                interpreted as "reviews.default.svc.cluster.local", irrespective of
-                the actual namespace associated with the reviews service. _To avoid
-                potential misconfigurations, it is recommended to always use fully
-                qualified domain names over short names._  Note that the host field
-                applies to both HTTP and TCP services.'
-              type: string
-            subsets:
-              description: One or more named sets that represent individual versions
-                of a service. Traffic policies can be overridden at subset level.
-              items:
-                properties:
-                  labels:
-                    description: REQUIRED. Labels apply a filter over the endpoints
-                      of a service in the service registry. See route rules for examples
-                      of usage.
-                    type: object
-                  name:
-                    description: REQUIRED. Name of the subset. The service name and
-                      the subset name can be used for traffic splitting in a route
-                      rule.
-                    type: string
-                  trafficPolicy:
-                    description: Traffic policies that apply to this subset. Subsets
-                      inherit the traffic policies specified at the DestinationRule
-                      level. Settings specified at the subset level will override
-                      the corresponding settings specified at the DestinationRule
-                      level.
-                    properties:
-                      connectionPool:
-                        description: Settings controlling the volume of connections
-                          to an upstream service
-                        properties:
-                          http:
-                            description: HTTP connection pool settings.
-                            properties:
-                              maxRequestsPerConnection:
-                                description: Maximum number of requests per connection
-                                  to a backend. Setting this parameter to 1 disables
-                                  keep alive.
-                                format: int32
-                                type: integer
-                              maxRetries:
-                                description: Maximum number of retries that can be
-                                  outstanding to all hosts in a cluster at a given
-                                  time. Defaults to 3.
-                                format: int32
-                                type: integer
-                            type: object
-                          tcp:
-                            description: Settings common to both HTTP and TCP upstream
-                              connections.
-                            properties:
-                              connectTimeout:
-                                description: TCP connection timeout.
-                                type: string
-                              maxConnections:
-                                description: Maximum number of HTTP1 /TCP connections
-                                  to a destination host.
-                                format: int32
-                                type: integer
-                            type: object
-                        type: object
-                      loadBalancer:
-                        description: Settings controlling the load balancer algorithms.
-                        properties:
-                          consistentHash:
-                            properties:
-                              httpCookie:
-                                description: Hash based on HTTP cookie.
-                                properties:
-                                  name:
-                                    description: REQUIRED. Name of the cookie.
-                                    type: string
-                                  path:
-                                    description: Path to set for the cookie.
-                                    type: string
-                                  ttl:
-                                    description: REQUIRED. Lifetime of the cookie.
-                                    type: string
-                                required:
-                                - name
-                                - ttl
-                                type: object
-                              httpHeaderName:
-                                description: 'It is required to specify exactly one
-                                  of the fields as hash key: HttpHeaderName, HttpCookie,
-                                  or UseSourceIP. Hash based on a specific HTTP header.'
-                                type: string
-                              minimumRingSize:
-                                description: The minimum number of virtual nodes to
-                                  use for the hash ring. Defaults to 1024. Larger
-                                  ring sizes result in more granular load distributions.
-                                  If the number of hosts in the load balancing pool
-                                  is larger than the ring size, each host will be
-                                  assigned a single virtual node.
-                                format: int64
-                                type: integer
-                              useSourceIp:
-                                description: Hash based on the source IP address.
-                                type: boolean
-                            type: object
-                          simple:
-                            description: 'It is required to specify exactly one of
-                              the fields: Simple or ConsistentHash'
-                            type: string
-                        type: object
-                      outlierDetection:
-                        description: Settings controlling eviction of unhealthy hosts
-                          from the load balancing pool
-                        properties:
-                          baseEjectionTime:
-                            description: 'Minimum ejection duration. A host will remain
-                              ejected for a period equal to the product of minimum
-                              ejection duration and the number of times the host has
-                              been ejected. This technique allows the system to automatically
-                              increase the ejection period for unhealthy upstream
-                              servers. format: 1h/1m/1s/1ms. MUST BE >=1ms. Default
-                              is 30s.'
-                            type: string
-                          consecutiveErrors:
-                            description: Number of errors before a host is ejected
-                              from the connection pool. Defaults to 5. When the upstream
-                              host is accessed over HTTP, a 5xx return code qualifies
-                              as an error. When the upstream host is accessed over
-                              an opaque TCP connection, connect timeouts and connection
-                              error/failure events qualify as an error.
-                            format: int32
-                            type: integer
-                          interval:
-                            description: 'Time interval between ejection sweep analysis.
-                              format: 1h/1m/1s/1ms. MUST BE >=1ms. Default is 10s.'
-                            type: string
-                          maxEjectionPercent:
-                            description: Maximum % of hosts in the load balancing
-                              pool for the upstream service that can be ejected. Defaults
-                              to 10%.
-                            format: int32
-                            type: integer
-                        type: object
-                      portLevelSettings:
-                        description: Traffic policies specific to individual ports.
-                          Note that port level settings will override the destination-level
-                          settings. Traffic settings specified at the destination-level
-                          will not be inherited when overridden by port-level settings,
-                          i.e. default values will be applied to fields omitted in
-                          port-level traffic policies.
-                        items:
-                          properties:
-                            connectionPool:
-                              description: Settings controlling the volume of connections
-                                to an upstream service
-                              properties:
-                                http:
-                                  description: HTTP connection pool settings.
-                                  properties:
-                                    maxRequestsPerConnection:
-                                      description: Maximum number of requests per
-                                        connection to a backend. Setting this parameter
-                                        to 1 disables keep alive.
-                                      format: int32
-                                      type: integer
-                                    maxRetries:
-                                      description: Maximum number of retries that
-                                        can be outstanding to all hosts in a cluster
-                                        at a given time. Defaults to 3.
-                                      format: int32
-                                      type: integer
-                                  type: object
-                                tcp:
-                                  description: Settings common to both HTTP and TCP
-                                    upstream connections.
-                                  properties:
-                                    connectTimeout:
-                                      description: TCP connection timeout.
-                                      type: string
-                                    maxConnections:
-                                      description: Maximum number of HTTP1 /TCP connections
-                                        to a destination host.
-                                      format: int32
-                                      type: integer
-                                  type: object
-                              type: object
-                            loadBalancer:
-                              description: Settings controlling the load balancer
-                                algorithms.
-                              properties:
-                                consistentHash:
-                                  properties:
-                                    httpCookie:
-                                      description: Hash based on HTTP cookie.
-                                      properties:
-                                        name:
-                                          description: REQUIRED. Name of the cookie.
-                                          type: string
-                                        path:
-                                          description: Path to set for the cookie.
-                                          type: string
-                                        ttl:
-                                          description: REQUIRED. Lifetime of the cookie.
-                                          type: string
-                                      required:
-                                      - name
-                                      - ttl
-                                      type: object
-                                    httpHeaderName:
-                                      description: 'It is required to specify exactly
-                                        one of the fields as hash key: HttpHeaderName,
-                                        HttpCookie, or UseSourceIP. Hash based on
-                                        a specific HTTP header.'
-                                      type: string
-                                    minimumRingSize:
-                                      description: The minimum number of virtual nodes
-                                        to use for the hash ring. Defaults to 1024.
-                                        Larger ring sizes result in more granular
-                                        load distributions. If the number of hosts
-                                        in the load balancing pool is larger than
-                                        the ring size, each host will be assigned
-                                        a single virtual node.
-                                      format: int64
-                                      type: integer
-                                    useSourceIp:
-                                      description: Hash based on the source IP address.
-                                      type: boolean
-                                  type: object
-                                simple:
-                                  description: 'It is required to specify exactly
-                                    one of the fields: Simple or ConsistentHash'
-                                  type: string
-                              type: object
-                            outlierDetection:
-                              description: Settings controlling eviction of unhealthy
-                                hosts from the load balancing pool
-                              properties:
-                                baseEjectionTime:
-                                  description: 'Minimum ejection duration. A host
-                                    will remain ejected for a period equal to the
-                                    product of minimum ejection duration and the number
-                                    of times the host has been ejected. This technique
-                                    allows the system to automatically increase the
-                                    ejection period for unhealthy upstream servers.
-                                    format: 1h/1m/1s/1ms. MUST BE >=1ms. Default is
-                                    30s.'
-                                  type: string
-                                consecutiveErrors:
-                                  description: Number of errors before a host is ejected
-                                    from the connection pool. Defaults to 5. When
-                                    the upstream host is accessed over HTTP, a 5xx
-                                    return code qualifies as an error. When the upstream
-                                    host is accessed over an opaque TCP connection,
-                                    connect timeouts and connection error/failure
-                                    events qualify as an error.
-                                  format: int32
-                                  type: integer
-                                interval:
-                                  description: 'Time interval between ejection sweep
-                                    analysis. format: 1h/1m/1s/1ms. MUST BE >=1ms.
-                                    Default is 10s.'
-                                  type: string
-                                maxEjectionPercent:
-                                  description: Maximum % of hosts in the load balancing
-                                    pool for the upstream service that can be ejected.
-                                    Defaults to 10%.
-                                  format: int32
-                                  type: integer
-                              type: object
-                            port:
-                              description: Specifies the port name or number of a
-                                port on the destination service on which this policy
-                                is being applied.  Names must comply with DNS label
-                                syntax (rfc1035) and therefore cannot collide with
-                                numbers. If there are multiple ports on a service
-                                with the same protocol the names should be of the
-                                form <protocol-name>-<DNS label>.
-                              properties:
-                                name:
-                                  description: Valid port name
-                                  type: string
-                                number:
-                                  description: Valid port number
-                                  format: int32
-                                  type: integer
-                              type: object
-                            tls:
-                              description: TLS related settings for connections to
-                                the upstream service.
-                              properties:
-                                caCertificates:
-                                  description: 'OPTIONAL: The path to the file containing
-                                    certificate authority certificates to use in verifying
-                                    a presented server certificate. If omitted, the
-                                    proxy will not verify the server''s certificate.
-                                    Should be empty if mode is `ISTIO_MUTUAL`.'
-                                  type: string
-                                clientCertificate:
-                                  description: REQUIRED if mode is `MUTUAL`. The path
-                                    to the file holding the client-side TLS certificate
-                                    to use. Should be empty if mode is `ISTIO_MUTUAL`.
-                                  type: string
-                                mode:
-                                  description: 'REQUIRED: Indicates whether connections
-                                    to this port should be secured using TLS. The
-                                    value of this field determines how TLS is enforced.'
-                                  type: string
-                                privateKey:
-                                  description: REQUIRED if mode is `MUTUAL`. The path
-                                    to the file holding the client's private key.
-                                    Should be empty if mode is `ISTIO_MUTUAL`.
-                                  type: string
-                                sni:
-                                  description: SNI string to present to the server
-                                    during TLS handshake. Should be empty if mode
-                                    is `ISTIO_MUTUAL`.
-                                  type: string
-                                subjectAltNames:
-                                  description: A list of alternate names to verify
-                                    the subject identity in the certificate. If specified,
-                                    the proxy will verify that the server certificate's
-                                    subject alt name matches one of the specified
-                                    values. Should be empty if mode is `ISTIO_MUTUAL`.
-                                  items:
-                                    type: string
-                                  type: array
-                              required:
-                              - mode
-                              type: object
-                          required:
-                          - port
-                          type: object
-                        type: array
-                      tls:
-                        description: TLS related settings for connections to the upstream
-                          service.
-                        properties:
-                          caCertificates:
-                            description: 'OPTIONAL: The path to the file containing
-                              certificate authority certificates to use in verifying
-                              a presented server certificate. If omitted, the proxy
-                              will not verify the server''s certificate. Should be
-                              empty if mode is `ISTIO_MUTUAL`.'
-                            type: string
-                          clientCertificate:
-                            description: REQUIRED if mode is `MUTUAL`. The path to
-                              the file holding the client-side TLS certificate to
-                              use. Should be empty if mode is `ISTIO_MUTUAL`.
-                            type: string
-                          mode:
-                            description: 'REQUIRED: Indicates whether connections
-                              to this port should be secured using TLS. The value
-                              of this field determines how TLS is enforced.'
-                            type: string
-                          privateKey:
-                            description: REQUIRED if mode is `MUTUAL`. The path to
-                              the file holding the client's private key. Should be
-                              empty if mode is `ISTIO_MUTUAL`.
-                            type: string
-                          sni:
-                            description: SNI string to present to the server during
-                              TLS handshake. Should be empty if mode is `ISTIO_MUTUAL`.
-                            type: string
-                          subjectAltNames:
-                            description: A list of alternate names to verify the subject
-                              identity in the certificate. If specified, the proxy
-                              will verify that the server certificate's subject alt
-                              name matches one of the specified values. Should be
-                              empty if mode is `ISTIO_MUTUAL`.
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - mode
-                        type: object
-                    type: object
-                required:
-                - name
-                - labels
-                type: object
-              type: array
-            trafficPolicy:
-              description: Traffic policies to apply (load balancing policy, connection
-                pool sizes, outlier detection).
-              properties:
-                connectionPool:
-                  description: Settings controlling the volume of connections to an
-                    upstream service
-                  properties:
-                    http:
-                      description: HTTP connection pool settings.
-                      properties:
-                        maxRequestsPerConnection:
-                          description: Maximum number of requests per connection to
-                            a backend. Setting this parameter to 1 disables keep alive.
-                          format: int32
-                          type: integer
-                        maxRetries:
-                          description: Maximum number of retries that can be outstanding
-                            to all hosts in a cluster at a given time. Defaults to
-                            3.
-                          format: int32
-                          type: integer
-                      type: object
-                    tcp:
-                      description: Settings common to both HTTP and TCP upstream connections.
-                      properties:
-                        connectTimeout:
-                          description: TCP connection timeout.
-                          type: string
-                        maxConnections:
-                          description: Maximum number of HTTP1 /TCP connections to
-                            a destination host.
-                          format: int32
-                          type: integer
-                      type: object
-                  type: object
-                loadBalancer:
-                  description: Settings controlling the load balancer algorithms.
-                  properties:
-                    consistentHash:
-                      properties:
-                        httpCookie:
-                          description: Hash based on HTTP cookie.
-                          properties:
-                            name:
-                              description: REQUIRED. Name of the cookie.
-                              type: string
-                            path:
-                              description: Path to set for the cookie.
-                              type: string
-                            ttl:
-                              description: REQUIRED. Lifetime of the cookie.
-                              type: string
-                          required:
-                          - name
-                          - ttl
-                          type: object
-                        httpHeaderName:
-                          description: 'It is required to specify exactly one of the
-                            fields as hash key: HttpHeaderName, HttpCookie, or UseSourceIP.
-                            Hash based on a specific HTTP header.'
-                          type: string
-                        minimumRingSize:
-                          description: The minimum number of virtual nodes to use
-                            for the hash ring. Defaults to 1024. Larger ring sizes
-                            result in more granular load distributions. If the number
-                            of hosts in the load balancing pool is larger than the
-                            ring size, each host will be assigned a single virtual
-                            node.
-                          format: int64
-                          type: integer
-                        useSourceIp:
-                          description: Hash based on the source IP address.
-                          type: boolean
-                      type: object
-                    simple:
-                      description: 'It is required to specify exactly one of the fields:
-                        Simple or ConsistentHash'
-                      type: string
-                  type: object
-                outlierDetection:
-                  description: Settings controlling eviction of unhealthy hosts from
-                    the load balancing pool
-                  properties:
-                    baseEjectionTime:
-                      description: 'Minimum ejection duration. A host will remain
-                        ejected for a period equal to the product of minimum ejection
-                        duration and the number of times the host has been ejected.
-                        This technique allows the system to automatically increase
-                        the ejection period for unhealthy upstream servers. format:
-                        1h/1m/1s/1ms. MUST BE >=1ms. Default is 30s.'
-                      type: string
-                    consecutiveErrors:
-                      description: Number of errors before a host is ejected from
-                        the connection pool. Defaults to 5. When the upstream host
-                        is accessed over HTTP, a 5xx return code qualifies as an error.
-                        When the upstream host is accessed over an opaque TCP connection,
-                        connect timeouts and connection error/failure events qualify
-                        as an error.
-                      format: int32
-                      type: integer
-                    interval:
-                      description: 'Time interval between ejection sweep analysis.
-                        format: 1h/1m/1s/1ms. MUST BE >=1ms. Default is 10s.'
-                      type: string
-                    maxEjectionPercent:
-                      description: Maximum % of hosts in the load balancing pool for
-                        the upstream service that can be ejected. Defaults to 10%.
-                      format: int32
-                      type: integer
-                  type: object
-                portLevelSettings:
-                  description: Traffic policies specific to individual ports. Note
-                    that port level settings will override the destination-level settings.
-                    Traffic settings specified at the destination-level will not be
-                    inherited when overridden by port-level settings, i.e. default
-                    values will be applied to fields omitted in port-level traffic
-                    policies.
-                  items:
-                    properties:
-                      connectionPool:
-                        description: Settings controlling the volume of connections
-                          to an upstream service
-                        properties:
-                          http:
-                            description: HTTP connection pool settings.
-                            properties:
-                              maxRequestsPerConnection:
-                                description: Maximum number of requests per connection
-                                  to a backend. Setting this parameter to 1 disables
-                                  keep alive.
-                                format: int32
-                                type: integer
-                              maxRetries:
-                                description: Maximum number of retries that can be
-                                  outstanding to all hosts in a cluster at a given
-                                  time. Defaults to 3.
-                                format: int32
-                                type: integer
-                            type: object
-                          tcp:
-                            description: Settings common to both HTTP and TCP upstream
-                              connections.
-                            properties:
-                              connectTimeout:
-                                description: TCP connection timeout.
-                                type: string
-                              maxConnections:
-                                description: Maximum number of HTTP1 /TCP connections
-                                  to a destination host.
-                                format: int32
-                                type: integer
-                            type: object
-                        type: object
-                      loadBalancer:
-                        description: Settings controlling the load balancer algorithms.
-                        properties:
-                          consistentHash:
-                            properties:
-                              httpCookie:
-                                description: Hash based on HTTP cookie.
-                                properties:
-                                  name:
-                                    description: REQUIRED. Name of the cookie.
-                                    type: string
-                                  path:
-                                    description: Path to set for the cookie.
-                                    type: string
-                                  ttl:
-                                    description: REQUIRED. Lifetime of the cookie.
-                                    type: string
-                                required:
-                                - name
-                                - ttl
-                                type: object
-                              httpHeaderName:
-                                description: 'It is required to specify exactly one
-                                  of the fields as hash key: HttpHeaderName, HttpCookie,
-                                  or UseSourceIP. Hash based on a specific HTTP header.'
-                                type: string
-                              minimumRingSize:
-                                description: The minimum number of virtual nodes to
-                                  use for the hash ring. Defaults to 1024. Larger
-                                  ring sizes result in more granular load distributions.
-                                  If the number of hosts in the load balancing pool
-                                  is larger than the ring size, each host will be
-                                  assigned a single virtual node.
-                                format: int64
-                                type: integer
-                              useSourceIp:
-                                description: Hash based on the source IP address.
-                                type: boolean
-                            type: object
-                          simple:
-                            description: 'It is required to specify exactly one of
-                              the fields: Simple or ConsistentHash'
-                            type: string
-                        type: object
-                      outlierDetection:
-                        description: Settings controlling eviction of unhealthy hosts
-                          from the load balancing pool
-                        properties:
-                          baseEjectionTime:
-                            description: 'Minimum ejection duration. A host will remain
-                              ejected for a period equal to the product of minimum
-                              ejection duration and the number of times the host has
-                              been ejected. This technique allows the system to automatically
-                              increase the ejection period for unhealthy upstream
-                              servers. format: 1h/1m/1s/1ms. MUST BE >=1ms. Default
-                              is 30s.'
-                            type: string
-                          consecutiveErrors:
-                            description: Number of errors before a host is ejected
-                              from the connection pool. Defaults to 5. When the upstream
-                              host is accessed over HTTP, a 5xx return code qualifies
-                              as an error. When the upstream host is accessed over
-                              an opaque TCP connection, connect timeouts and connection
-                              error/failure events qualify as an error.
-                            format: int32
-                            type: integer
-                          interval:
-                            description: 'Time interval between ejection sweep analysis.
-                              format: 1h/1m/1s/1ms. MUST BE >=1ms. Default is 10s.'
-                            type: string
-                          maxEjectionPercent:
-                            description: Maximum % of hosts in the load balancing
-                              pool for the upstream service that can be ejected. Defaults
-                              to 10%.
-                            format: int32
-                            type: integer
-                        type: object
-                      port:
-                        description: Specifies the port name or number of a port on
-                          the destination service on which this policy is being applied.  Names
-                          must comply with DNS label syntax (rfc1035) and therefore
-                          cannot collide with numbers. If there are multiple ports
-                          on a service with the same protocol the names should be
-                          of the form <protocol-name>-<DNS label>.
-                        properties:
-                          name:
-                            description: Valid port name
-                            type: string
-                          number:
-                            description: Valid port number
-                            format: int32
-                            type: integer
-                        type: object
-                      tls:
-                        description: TLS related settings for connections to the upstream
-                          service.
-                        properties:
-                          caCertificates:
-                            description: 'OPTIONAL: The path to the file containing
-                              certificate authority certificates to use in verifying
-                              a presented server certificate. If omitted, the proxy
-                              will not verify the server''s certificate. Should be
-                              empty if mode is `ISTIO_MUTUAL`.'
-                            type: string
-                          clientCertificate:
-                            description: REQUIRED if mode is `MUTUAL`. The path to
-                              the file holding the client-side TLS certificate to
-                              use. Should be empty if mode is `ISTIO_MUTUAL`.
-                            type: string
-                          mode:
-                            description: 'REQUIRED: Indicates whether connections
-                              to this port should be secured using TLS. The value
-                              of this field determines how TLS is enforced.'
-                            type: string
-                          privateKey:
-                            description: REQUIRED if mode is `MUTUAL`. The path to
-                              the file holding the client's private key. Should be
-                              empty if mode is `ISTIO_MUTUAL`.
-                            type: string
-                          sni:
-                            description: SNI string to present to the server during
-                              TLS handshake. Should be empty if mode is `ISTIO_MUTUAL`.
-                            type: string
-                          subjectAltNames:
-                            description: A list of alternate names to verify the subject
-                              identity in the certificate. If specified, the proxy
-                              will verify that the server certificate's subject alt
-                              name matches one of the specified values. Should be
-                              empty if mode is `ISTIO_MUTUAL`.
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - mode
-                        type: object
-                    required:
-                    - port
-                    type: object
-                  type: array
-                tls:
-                  description: TLS related settings for connections to the upstream
-                    service.
-                  properties:
-                    caCertificates:
-                      description: 'OPTIONAL: The path to the file containing certificate
-                        authority certificates to use in verifying a presented server
-                        certificate. If omitted, the proxy will not verify the server''s
-                        certificate. Should be empty if mode is `ISTIO_MUTUAL`.'
-                      type: string
-                    clientCertificate:
-                      description: REQUIRED if mode is `MUTUAL`. The path to the file
-                        holding the client-side TLS certificate to use. Should be
-                        empty if mode is `ISTIO_MUTUAL`.
-                      type: string
-                    mode:
-                      description: 'REQUIRED: Indicates whether connections to this
-                        port should be secured using TLS. The value of this field
-                        determines how TLS is enforced.'
-                      type: string
-                    privateKey:
-                      description: REQUIRED if mode is `MUTUAL`. The path to the file
-                        holding the client's private key. Should be empty if mode
-                        is `ISTIO_MUTUAL`.
-                      type: string
-                    sni:
-                      description: SNI string to present to the server during TLS
-                        handshake. Should be empty if mode is `ISTIO_MUTUAL`.
-                      type: string
-                    subjectAltNames:
-                      description: A list of alternate names to verify the subject
-                        identity in the certificate. If specified, the proxy will
-                        verify that the server certificate's subject alt name matches
-                        one of the specified values. Should be empty if mode is `ISTIO_MUTUAL`.
-                      items:
-                        type: string
-                      type: array
-                  required:
-                  - mode
-                  type: object
-              type: object
-          required:
-          - host
-          type: object
-      required:
-      - spec
-  version: v1alpha3
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/istio_v1alpha3_gateway.yaml

@@ -1,129 +0,0 @@
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  creationTimestamp: null
-  labels:
-    controller-tools.k8s.io: "1.0"
-  name: gateways.istio.kubesphere.io
-spec:
-  group: istio.kubesphere.io
-  names:
-    kind: Gateway
-    plural: gateways
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          properties:
-            selector:
-              description: One or more labels that indicate a specific set of pods/VMs
-                on which this gateway configuration should be applied. If no selectors
-                are provided, the gateway will be implemented by the default istio-ingress
-                controller.
-              type: object
-            servers:
-              description: 'REQUIRED: A list of server specifications.'
-              items:
-                properties:
-                  hosts:
-                    description: A list of hosts exposed by this gateway. While typically
-                      applicable to HTTP services, it can also be used for TCP services
-                      using TLS with SNI. Standard DNS wildcard prefix syntax is permitted.  A
-                      VirtualService that is bound to a gateway must having a matching
-                      host in its default destination. Specifically one of the VirtualService
-                      destination hosts is a strict suffix of a gateway host or a
-                      gateway host is a suffix of one of the VirtualService hosts.
-                    items:
-                      type: string
-                    type: array
-                  port:
-                    description: 'REQUIRED: The Port on which the proxy should listen
-                      for incoming connections'
-                    properties:
-                      name:
-                        description: Label assigned to the port.
-                        type: string
-                      number:
-                        description: 'REQUIRED: A valid non-negative integer port
-                          number.'
-                        format: int64
-                        type: integer
-                      protocol:
-                        description: 'REQUIRED: The protocol exposed on the port.
-                          MUST BE one of HTTP|HTTPS|GRPC|HTTP2|MONGO|TCP.'
-                        type: string
-                    required:
-                    - number
-                    - protocol
-                    type: object
-                  tls:
-                    description: Set of TLS related options that govern the server's
-                      behavior. Use these options to control if all http requests
-                      should be redirected to https, and the TLS modes to use.
-                    properties:
-                      caCertificates:
-                        description: REQUIRED if mode is "MUTUAL". The path to a file
-                          containing certificate authority certificates to use in
-                          verifying a presented client side certificate.
-                        type: string
-                      httpsRedirect:
-                        description: If set to true, the load balancer will send a
-                          302 redirect for all http connections, asking the clients
-                          to use HTTPS.
-                        type: boolean
-                      mode:
-                        description: 'Optional: Indicates whether connections to this
-                          port should be secured using TLS. The value of this field
-                          determines how TLS is enforced.'
-                        type: string
-                      privateKey:
-                        description: REQUIRED if mode is "SIMPLE" or "MUTUAL". The
-                          path to the file holding the server's private key.
-                        type: string
-                      serverCertificate:
-                        description: REQUIRED if mode is "SIMPLE" or "MUTUAL". The
-                          path to the file holding the server-side TLS certificate
-                          to use.
-                        type: string
-                      subjectAltNames:
-                        description: A list of alternate names to verify the subject
-                          identity in the certificate presented by the client.
-                        items:
-                          type: string
-                        type: array
-                    required:
-                    - httpsRedirect
-                    - serverCertificate
-                    - privateKey
-                    - caCertificates
-                    - subjectAltNames
-                    type: object
-                required:
-                - port
-                type: object
-              type: array
-          required:
-          - servers
-          type: object
-      required:
-      - spec
-  version: v1alpha3
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/istio_v1alpha3_virtualservice.yaml

@@ -1,695 +0,0 @@
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  creationTimestamp: null
-  labels:
-    controller-tools.k8s.io: "1.0"
-  name: virtualservices.istio.kubesphere.io
-spec:
-  group: istio.kubesphere.io
-  names:
-    kind: VirtualService
-    plural: virtualservices
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          properties:
-            gateways:
-              description: The names of gateways and sidecars that should apply these
-                routes. A single VirtualService is used for sidecars inside the mesh
-                as well as for one or more gateways. The selection condition imposed
-                by this field can be overridden using the source field in the match
-                conditions of HTTP/TCP routes. The reserved word "mesh" is used to
-                imply all the sidecars in the mesh. When this field is omitted, the
-                default gateway ("mesh") will be used, which would apply the rule
-                to all sidecars in the mesh. If a list of gateway names is provided,
-                the rules will apply only to the gateways. To apply the rules to both
-                gateways and sidecars, specify "mesh" as one of the gateway names.
-              items:
-                type: string
-              type: array
-            hosts:
-              description: REQUIRED. The destination address for traffic captured
-                by this virtual service. Could be a DNS name with wildcard prefix
-                or a CIDR prefix. Depending on the platform, short-names can also
-                be used instead of a FQDN (i.e. has no dots in the name). In such
-                a scenario, the FQDN of the host would be derived based on the underlying
-                platform.  For example on Kubernetes, when hosts contains a short
-                name, Istio will interpret the short name based on the namespace of
-                the rule. Thus, when a client namespace applies a rule in the "default"
-                namespace containing a name "reviews, Istio will setup routes to the
-                "reviews.default.svc.cluster.local" service. However, if a different
-                name such as "reviews.sales.svc.cluster.local" is used, it would be
-                treated as a FQDN during virtual host matching. In Consul, a plain
-                service name would be resolved to the FQDN "reviews.service.consul".  Note
-                that the hosts field applies to both HTTP and TCP services. Service
-                inside the mesh, i.e., those found in the service registry, must always
-                be referred to using their alphanumeric names. IP addresses or CIDR
-                prefixes are allowed only for services defined via the Gateway.
-              items:
-                type: string
-              type: array
-            http:
-              description: An ordered list of route rules for HTTP traffic. The first
-                rule matching an incoming request is used.
-              items:
-                properties:
-                  appendHeaders:
-                    description: Additional HTTP headers to add before forwarding
-                      a request to the destination service.
-                    type: object
-                  corsPolicy:
-                    description: Cross-Origin Resource Sharing policy
-                    properties:
-                      allowCredentials:
-                        description: Indicates whether the caller is allowed to send
-                          the actual request (not the preflight) using credentials.
-                          Translates to Access-Control-Allow-Credentials header.
-                        type: boolean
-                      allowHeaders:
-                        description: List of HTTP headers that can be used when requesting
-                          the resource. Serialized to Access-Control-Allow-Methods
-                          header.
-                        items:
-                          type: string
-                        type: array
-                      allowMethods:
-                        description: List of HTTP methods allowed to access the resource.
-                          The content will be serialized into the Access-Control-Allow-Methods
-                          header.
-                        items:
-                          type: string
-                        type: array
-                      allowOrigin:
-                        description: The list of origins that are allowed to perform
-                          CORS requests. The content will be serialized into the Access-Control-Allow-Origin
-                          header. Wildcard * will allow all origins.
-                        items:
-                          type: string
-                        type: array
-                      exposeHeaders:
-                        description: A white list of HTTP headers that the browsers
-                          are allowed to access. Serialized into Access-Control-Expose-Headers
-                          header.
-                        items:
-                          type: string
-                        type: array
-                      maxAge:
-                        description: Specifies how long the the results of a preflight
-                          request can be cached. Translates to the Access-Control-Max-Age
-                          header.
-                        type: string
-                    type: object
-                  fault:
-                    description: Fault injection policy to apply on HTTP traffic.
-                    properties:
-                      abort:
-                        description: Abort Http request attempts and return error
-                          codes back to downstream service, giving the impression
-                          that the upstream service is faulty.
-                        properties:
-                          httpStatus:
-                            description: REQUIRED. HTTP status code to use to abort
-                              the Http request.
-                            format: int64
-                            type: integer
-                          percent:
-                            description: Percentage of requests to be aborted with
-                              the error code provided (0-100).
-                            format: int64
-                            type: integer
-                        required:
-                        - httpStatus
-                        type: object
-                      delay:
-                        description: Delay requests before forwarding, emulating various
-                          failures such as network issues, overloaded upstream service,
-                          etc.
-                        properties:
-                          exponentialDelay:
-                            description: (-- Add a delay (based on an exponential
-                              function) before forwarding the request. mean delay
-                              needed to derive the exponential delay values --)
-                            type: string
-                          fixedDelay:
-                            description: 'REQUIRED. Add a fixed delay before forwarding
-                              the request. Format: 1h/1m/1s/1ms. MUST be >=1ms.'
-                            type: string
-                          percent:
-                            description: Percentage of requests on which the delay
-                              will be injected (0-100).
-                            format: int64
-                            type: integer
-                        required:
-                        - fixedDelay
-                        type: object
-                    type: object
-                  match:
-                    description: Match conditions to be satisfied for the rule to
-                      be activated. All conditions inside a single match block have
-                      AND semantics, while the list of match blocks have OR semantics.
-                      The rule is matched if any one of the match blocks succeed.
-                    items:
-                      properties:
-                        authority:
-                          description: 'HTTP Authority values are case-sensitive and
-                            formatted as follows:  - `exact: "value"` for exact string
-                            match  - `prefix: "value"` for prefix-based match  - `regex:
-                            "value"` for ECMAscript style regex-based match'
-                          properties:
-                            exact:
-                              description: exact string match
-                              type: string
-                            prefix:
-                              description: prefix-based match
-                              type: string
-                            regex:
-                              description: ECMAscript style regex-based match
-                              type: string
-                            suffix:
-                              description: suffix-based match.
-                              type: string
-                          type: object
-                        gateways:
-                          description: Names of gateways where the rule should be
-                            applied to. Gateway names at the top of the VirtualService
-                            (if any) are overridden. The gateway match is independent
-                            of sourceLabels.
-                          items:
-                            type: string
-                          type: array
-                        headers:
-                          description: 'The header keys must be lowercase and use
-                            hyphen as the separator, e.g. _x-request-id_.  Header
-                            values are case-sensitive and formatted as follows:  -
-                            `exact: "value"` for exact string match  - `prefix: "value"`
-                            for prefix-based match  - `regex: "value"` for ECMAscript
-                            style regex-based match  **Note:** The keys `uri`, `scheme`,
-                            `method`, and `authority` will be ignored.'
-                          type: object
-                        method:
-                          description: 'HTTP Method values are case-sensitive and
-                            formatted as follows:  - `exact: "value"` for exact string
-                            match  - `prefix: "value"` for prefix-based match  - `regex:
-                            "value"` for ECMAscript style regex-based match'
-                          properties:
-                            exact:
-                              description: exact string match
-                              type: string
-                            prefix:
-                              description: prefix-based match
-                              type: string
-                            regex:
-                              description: ECMAscript style regex-based match
-                              type: string
-                            suffix:
-                              description: suffix-based match.
-                              type: string
-                          type: object
-                        port:
-                          description: Specifies the ports on the host that is being
-                            addressed. Many services only expose a single port or
-                            label ports with the protocols they support, in these
-                            cases it is not required to explicitly select the port.
-                          format: int32
-                          type: integer
-                        scheme:
-                          description: 'URI Scheme values are case-sensitive and formatted
-                            as follows:  - `exact: "value"` for exact string match  -
-                            `prefix: "value"` for prefix-based match  - `regex: "value"`
-                            for ECMAscript style regex-based match'
-                          properties:
-                            exact:
-                              description: exact string match
-                              type: string
-                            prefix:
-                              description: prefix-based match
-                              type: string
-                            regex:
-                              description: ECMAscript style regex-based match
-                              type: string
-                            suffix:
-                              description: suffix-based match.
-                              type: string
-                          type: object
-                        sourceLabels:
-                          description: One or more labels that constrain the applicability
-                            of a rule to workloads with the given labels. If the VirtualService
-                            has a list of gateways specified at the top, it should
-                            include the reserved gateway `mesh` in order for this
-                            field to be applicable.
-                          type: object
-                        uri:
-                          description: 'URI to match values are case-sensitive and
-                            formatted as follows:  - `exact: "value"` for exact string
-                            match  - `prefix: "value"` for prefix-based match  - `regex:
-                            "value"` for ECMAscript style regex-based match'
-                          properties:
-                            exact:
-                              description: exact string match
-                              type: string
-                            prefix:
-                              description: prefix-based match
-                              type: string
-                            regex:
-                              description: ECMAscript style regex-based match
-                              type: string
-                            suffix:
-                              description: suffix-based match.
-                              type: string
-                          type: object
-                      type: object
-                    type: array
-                  mirror:
-                    description: Mirror HTTP traffic to a another destination in addition
-                      to forwarding the requests to the intended destination. Mirrored
-                      traffic is on a best effort basis where the sidecar/gateway
-                      will not wait for the mirrored cluster to respond before returning
-                      the response from the original destination.  Statistics will
-                      be generated for the mirrored destination.
-                    properties:
-                      host:
-                        description: 'REQUIRED. The name of a service from the service
-                          registry. Service names are looked up from the platform''s
-                          service registry (e.g., Kubernetes services, Consul services,
-                          etc.) and from the hosts declared by [ServiceEntry](#ServiceEntry).
-                          Traffic forwarded to destinations that are not found in
-                          either of the two, will be dropped.  *Note for Kubernetes
-                          users*: When short names are used (e.g. "reviews" instead
-                          of "reviews.default.svc.cluster.local"), Istio will interpret
-                          the short name based on the namespace of the rule, not the
-                          service. A rule in the "default" namespace containing a
-                          host "reviews will be interpreted as "reviews.default.svc.cluster.local",
-                          irrespective of the actual namespace associated with the
-                          reviews service. _To avoid potential misconfigurations,
-                          it is recommended to always use fully qualified domain names
-                          over short names._'
-                        type: string
-                      port:
-                        description: Specifies the port on the host that is being
-                          addressed. If a service exposes only a single port it is
-                          not required to explicitly select the port.
-                        properties:
-                          name:
-                            description: Valid port name
-                            type: string
-                          number:
-                            description: Valid port number
-                            format: int32
-                            type: integer
-                        type: object
-                      subset:
-                        description: The name of a subset within the service. Applicable
-                          only to services within the mesh. The subset must be defined
-                          in a corresponding DestinationRule.
-                        type: string
-                    required:
-                    - host
-                    type: object
-                  redirect:
-                    description: A http rule can either redirect or forward (default)
-                      traffic. If traffic passthrough option is specified in the rule,
-                      route/redirect will be ignored. The redirect primitive can be
-                      used to send a HTTP 302 redirect to a different URI or Authority.
-                    properties:
-                      authority:
-                        description: On a redirect, overwrite the Authority/Host portion
-                          of the URL with this value.
-                        type: string
-                      uri:
-                        description: On a redirect, overwrite the Path portion of
-                          the URL with this value. Note that the entire path will
-                          be replaced, irrespective of the request URI being matched
-                          as an exact path or prefix.
-                        type: string
-                    type: object
-                  removeResponseHeaders:
-                    description: Http headers to remove before returning the response
-                      to the caller
-                    type: object
-                  retries:
-                    description: Retry policy for HTTP requests.
-                    properties:
-                      attempts:
-                        description: REQUIRED. Number of retries for a given request.
-                          The interval between retries will be determined automatically
-                          (25ms+). Actual number of retries attempted depends on the
-                          httpReqTimeout.
-                        format: int64
-                        type: integer
-                      perTryTimeout:
-                        description: 'Timeout per retry attempt for a given request.
-                          format: 1h/1m/1s/1ms. MUST BE >=1ms.'
-                        type: string
-                    required:
-                    - attempts
-                    - perTryTimeout
-                    type: object
-                  rewrite:
-                    description: Rewrite HTTP URIs and Authority headers. Rewrite
-                      cannot be used with Redirect primitive. Rewrite will be performed
-                      before forwarding.
-                    properties:
-                      authority:
-                        description: rewrite the Authority/Host header with this value.
-                        type: string
-                      uri:
-                        description: rewrite the path (or the prefix) portion of the
-                          URI with this value. If the original URI was matched based
-                          on prefix, the value provided in this field will replace
-                          the corresponding matched prefix.
-                        type: string
-                    type: object
-                  route:
-                    description: A http rule can either redirect or forward (default)
-                      traffic. The forwarding target can be one of several versions
-                      of a service (see glossary in beginning of document). Weights
-                      associated with the service version determine the proportion
-                      of traffic it receives.
-                    items:
-                      properties:
-                        destination:
-                          description: REQUIRED. Destination uniquely identifies the
-                            instances of a service to which the request/connection
-                            should be forwarded to.
-                          properties:
-                            host:
-                              description: 'REQUIRED. The name of a service from the
-                                service registry. Service names are looked up from
-                                the platform''s service registry (e.g., Kubernetes
-                                services, Consul services, etc.) and from the hosts
-                                declared by [ServiceEntry](#ServiceEntry). Traffic
-                                forwarded to destinations that are not found in either
-                                of the two, will be dropped.  *Note for Kubernetes
-                                users*: When short names are used (e.g. "reviews"
-                                instead of "reviews.default.svc.cluster.local"), Istio
-                                will interpret the short name based on the namespace
-                                of the rule, not the service. A rule in the "default"
-                                namespace containing a host "reviews will be interpreted
-                                as "reviews.default.svc.cluster.local", irrespective
-                                of the actual namespace associated with the reviews
-                                service. _To avoid potential misconfigurations, it
-                                is recommended to always use fully qualified domain
-                                names over short names._'
-                              type: string
-                            port:
-                              description: Specifies the port on the host that is
-                                being addressed. If a service exposes only a single
-                                port it is not required to explicitly select the port.
-                              properties:
-                                name:
-                                  description: Valid port name
-                                  type: string
-                                number:
-                                  description: Valid port number
-                                  format: int32
-                                  type: integer
-                              type: object
-                            subset:
-                              description: The name of a subset within the service.
-                                Applicable only to services within the mesh. The subset
-                                must be defined in a corresponding DestinationRule.
-                              type: string
-                          required:
-                          - host
-                          type: object
-                        weight:
-                          description: REQUIRED. The proportion of traffic to be forwarded
-                            to the service version. (0-100). Sum of weights across
-                            destinations SHOULD BE == 100. If there is only destination
-                            in a rule, the weight value is assumed to be 100.
-                          format: int64
-                          type: integer
-                      required:
-                      - destination
-                      - weight
-                      type: object
-                    type: array
-                  timeout:
-                    description: Timeout for HTTP requests.
-                    type: string
-                  websocketUpgrade:
-                    description: Indicates that a HTTP/1.1 client connection to this
-                      particular route should be allowed (and expected) to upgrade
-                      to a WebSocket connection. The default is false. Istio's reference
-                      sidecar implementation (Envoy) expects the first request to
-                      this route to contain the WebSocket upgrade headers. Otherwise,
-                      the request will be rejected. Note that Websocket allows secondary
-                      protocol negotiation which may then be subject to further routing
-                      rules based on the protocol selected.
-                    type: boolean
-                type: object
-              type: array
-            tcp:
-              description: An ordered list of route rules for TCP traffic. The first
-                rule matching an incoming request is used.
-              items:
-                properties:
-                  match:
-                    description: Match conditions to be satisfied for the rule to
-                      be activated. All conditions inside a single match block have
-                      AND semantics, while the list of match blocks have OR semantics.
-                      The rule is matched if any one of the match blocks succeed.
-                    items:
-                      properties:
-                        destinationSubnets:
-                          description: IPv4 or IPv6 ip address of destination with
-                            optional subnet.  E.g., a.b.c.d/xx form or just a.b.c.d.
-                          items:
-                            type: string
-                          type: array
-                        gateways:
-                          description: Names of gateways where the rule should be
-                            applied to. Gateway names at the top of the VirtualService
-                            (if any) are overridden. The gateway match is independent
-                            of sourceLabels.
-                          items:
-                            type: string
-                          type: array
-                        port:
-                          description: Specifies the port on the host that is being
-                            addressed. Many services only expose a single port or
-                            label ports with the protocols they support, in these
-                            cases it is not required to explicitly select the port.
-                          format: int64
-                          type: integer
-                        sourceLabels:
-                          description: One or more labels that constrain the applicability
-                            of a rule to workloads with the given labels. If the VirtualService
-                            has a list of gateways specified at the top, it should
-                            include the reserved gateway `mesh` in order for this
-                            field to be applicable.
-                          type: object
-                      type: object
-                    type: array
-                  route:
-                    description: The destinations to which the connection should be
-                      forwarded to. Weights must add to 100%.
-                    items:
-                      properties:
-                        destination:
-                          description: REQUIRED. Destination uniquely identifies the
-                            instances of a service to which the request/connection
-                            should be forwarded to.
-                          properties:
-                            host:
-                              description: 'REQUIRED. The name of a service from the
-                                service registry. Service names are looked up from
-                                the platform''s service registry (e.g., Kubernetes
-                                services, Consul services, etc.) and from the hosts
-                                declared by [ServiceEntry](#ServiceEntry). Traffic
-                                forwarded to destinations that are not found in either
-                                of the two, will be dropped.  *Note for Kubernetes
-                                users*: When short names are used (e.g. "reviews"
-                                instead of "reviews.default.svc.cluster.local"), Istio
-                                will interpret the short name based on the namespace
-                                of the rule, not the service. A rule in the "default"
-                                namespace containing a host "reviews will be interpreted
-                                as "reviews.default.svc.cluster.local", irrespective
-                                of the actual namespace associated with the reviews
-                                service. _To avoid potential misconfigurations, it
-                                is recommended to always use fully qualified domain
-                                names over short names._'
-                              type: string
-                            port:
-                              description: Specifies the port on the host that is
-                                being addressed. If a service exposes only a single
-                                port it is not required to explicitly select the port.
-                              properties:
-                                name:
-                                  description: Valid port name
-                                  type: string
-                                number:
-                                  description: Valid port number
-                                  format: int32
-                                  type: integer
-                              type: object
-                            subset:
-                              description: The name of a subset within the service.
-                                Applicable only to services within the mesh. The subset
-                                must be defined in a corresponding DestinationRule.
-                              type: string
-                          required:
-                          - host
-                          type: object
-                        weight:
-                          description: REQUIRED. The proportion of traffic to be forwarded
-                            to the service version. (0-100). Sum of weights across
-                            destinations SHOULD BE == 100. If there is only destination
-                            in a rule, the weight value is assumed to be 100.
-                          format: int64
-                          type: integer
-                      required:
-                      - destination
-                      - weight
-                      type: object
-                    type: array
-                required:
-                - match
-                - route
-                type: object
-              type: array
-            tls:
-              items:
-                properties:
-                  match:
-                    description: REQUIRED. Match conditions to be satisfied for the
-                      rule to be activated. All conditions inside a single match block
-                      have AND semantics, while the list of match blocks have OR semantics.
-                      The rule is matched if any one of the match blocks succeed.
-                    items:
-                      properties:
-                        destinationSubnets:
-                          description: IPv4 or IPv6 ip addresses of destination with
-                            optional subnet.  E.g., a.b.c.d/xx form or just a.b.c.d.
-                          items:
-                            type: string
-                          type: array
-                        gateways:
-                          description: Names of gateways where the rule should be
-                            applied to. Gateway names at the top of the VirtualService
-                            (if any) are overridden. The gateway match is independent
-                            of sourceLabels.
-                          items:
-                            type: string
-                          type: array
-                        port:
-                          description: Specifies the port on the host that is being
-                            addressed. Many services only expose a single port or
-                            label ports with the protocols they support, in these
-                            cases it is not required to explicitly select the port.
-                          format: int64
-                          type: integer
-                        sniHosts:
-                          description: REQUIRED. SNI (server name indicator) to match
-                            on. Wildcard prefixes can be used in the SNI value, e.g.,
-                            *.com will match foo.example.com as well as example.com.
-                            An SNI value must be a subset (i.e., fall within the domain)
-                            of the corresponding virtual service's hosts
-                          items:
-                            type: string
-                          type: array
-                        sourceLabels:
-                          description: One or more labels that constrain the applicability
-                            of a rule to workloads with the given labels. If the VirtualService
-                            has a list of gateways specified at the top, it should
-                            include the reserved gateway `mesh` in order for this
-                            field to be applicable.
-                          type: object
-                      required:
-                      - sniHosts
-                      type: object
-                    type: array
-                  route:
-                    description: The destination to which the connection should be
-                      forwarded to.
-                    items:
-                      properties:
-                        destination:
-                          description: REQUIRED. Destination uniquely identifies the
-                            instances of a service to which the request/connection
-                            should be forwarded to.
-                          properties:
-                            host:
-                              description: 'REQUIRED. The name of a service from the
-                                service registry. Service names are looked up from
-                                the platform''s service registry (e.g., Kubernetes
-                                services, Consul services, etc.) and from the hosts
-                                declared by [ServiceEntry](#ServiceEntry). Traffic
-                                forwarded to destinations that are not found in either
-                                of the two, will be dropped.  *Note for Kubernetes
-                                users*: When short names are used (e.g. "reviews"
-                                instead of "reviews.default.svc.cluster.local"), Istio
-                                will interpret the short name based on the namespace
-                                of the rule, not the service. A rule in the "default"
-                                namespace containing a host "reviews will be interpreted
-                                as "reviews.default.svc.cluster.local", irrespective
-                                of the actual namespace associated with the reviews
-                                service. _To avoid potential misconfigurations, it
-                                is recommended to always use fully qualified domain
-                                names over short names._'
-                              type: string
-                            port:
-                              description: Specifies the port on the host that is
-                                being addressed. If a service exposes only a single
-                                port it is not required to explicitly select the port.
-                              properties:
-                                name:
-                                  description: Valid port name
-                                  type: string
-                                number:
-                                  description: Valid port number
-                                  format: int32
-                                  type: integer
-                              type: object
-                            subset:
-                              description: The name of a subset within the service.
-                                Applicable only to services within the mesh. The subset
-                                must be defined in a corresponding DestinationRule.
-                              type: string
-                          required:
-                          - host
-                          type: object
-                        weight:
-                          description: REQUIRED. The proportion of traffic to be forwarded
-                            to the service version. (0-100). Sum of weights across
-                            destinations SHOULD BE == 100. If there is only destination
-                            in a rule, the weight value is assumed to be 100.
-                          format: int64
-                          type: integer
-                      required:
-                      - destination
-                      - weight
-                      type: object
-                    type: array
-                required:
-                - match
-                - route
-                type: object
-              type: array
-          required:
-          - hosts
-          type: object
-      required:
-      - spec
-  version: v1alpha3
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/monitoring.kubesphere.io_clusterdashboards.yaml

@@ -1,184 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.4
-  creationTimestamp: null
-  name: clusterdashboards.monitoring.kubesphere.io
-spec:
-  group: monitoring.kubesphere.io
-  names:
-    kind: ClusterDashboard
-    listKind: ClusterDashboardList
-    plural: clusterdashboards
-    singular: clusterdashboard
-  scope: Cluster
-  validation:
-    openAPIV3Schema:
-      description: ClusterDashboard is the Schema for the culsterdashboards API
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: DashboardSpec defines the desired state of Dashboard
-          properties:
-            datasource:
-              description: Dashboard datasource
-              type: string
-            description:
-              description: Dashboard description
-              type: string
-            panels:
-              description: Collection of panels. Panel is one of [Row](row.md), [Singlestat](#singlestat.md)
-                or [Graph](graph.md)
-              items:
-                description: Supported panel type
-                properties:
-                  bars:
-                    description: Display as a bar chart
-                    type: boolean
-                  colors:
-                    description: Set series color
-                    items:
-                      type: string
-                    type: array
-                  decimals:
-                    description: Limit the decimal numbers
-                    format: int64
-                    type: integer
-                  description:
-                    description: Panel description
-                    type: string
-                  format:
-                    description: Display unit
-                    type: string
-                  id:
-                    description: Panel ID
-                    format: int64
-                    type: integer
-                  lines:
-                    description: Display as a line chart
-                    type: boolean
-                  stack:
-                    description: Display as a stacked chart
-                    type: boolean
-                  targets:
-                    allOf:
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    description: A collection of queries
-                    type: array
-                  title:
-                    description: Name of the row panel
-                    type: string
-                  type:
-                    description: Must be `row`
-                    type: string
-                  yaxes:
-                    description: Y-axis options
-                    items:
-                      properties:
-                        decimals:
-                          description: Limit the decimal numbers
-                          format: int64
-                          type: integer
-                        format:
-                          description: Display unit
-                          type: string
-                      type: object
-                    type: array
-                required:
-                - type
-                type: object
-              type: array
-            templating:
-              description: Templating variables
-              items:
-                description: Templating defines a variable, which can be used as a
-                  placeholder in query
-                properties:
-                  name:
-                    description: Variable name
-                    type: string
-                  query:
-                    description: Set variable values to be the return result of the
-                      query
-                    type: string
-                type: object
-              type: array
-            time:
-              description: Time range for display
-              properties:
-                from:
-                  description: Start time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the end time is set to the last month
-                    since now.
-                  type: string
-                to:
-                  description: End time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the start time is set to the last month
-                    since now.
-                  type: string
-              type: object
-            title:
-              description: Dashboard title
-              type: string
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/monitoring.kubesphere.io_dashboards.yaml

@@ -1,184 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.4
-  creationTimestamp: null
-  name: dashboards.monitoring.kubesphere.io
-spec:
-  group: monitoring.kubesphere.io
-  names:
-    kind: Dashboard
-    listKind: DashboardList
-    plural: dashboards
-    singular: dashboard
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      description: Dashboard is the Schema for the dashboards API
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: DashboardSpec defines the desired state of Dashboard
-          properties:
-            datasource:
-              description: Dashboard datasource
-              type: string
-            description:
-              description: Dashboard description
-              type: string
-            panels:
-              description: Collection of panels. Panel is one of [Row](row.md), [Singlestat](#singlestat.md)
-                or [Graph](graph.md)
-              items:
-                description: Supported panel type
-                properties:
-                  bars:
-                    description: Display as a bar chart
-                    type: boolean
-                  colors:
-                    description: Set series color
-                    items:
-                      type: string
-                    type: array
-                  decimals:
-                    description: Limit the decimal numbers
-                    format: int64
-                    type: integer
-                  description:
-                    description: Panel description
-                    type: string
-                  format:
-                    description: Display unit
-                    type: string
-                  id:
-                    description: Panel ID
-                    format: int64
-                    type: integer
-                  lines:
-                    description: Display as a line chart
-                    type: boolean
-                  stack:
-                    description: Display as a stacked chart
-                    type: boolean
-                  targets:
-                    allOf:
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    description: A collection of queries
-                    type: array
-                  title:
-                    description: Name of the row panel
-                    type: string
-                  type:
-                    description: Must be `row`
-                    type: string
-                  yaxes:
-                    description: Y-axis options
-                    items:
-                      properties:
-                        decimals:
-                          description: Limit the decimal numbers
-                          format: int64
-                          type: integer
-                        format:
-                          description: Display unit
-                          type: string
-                      type: object
-                    type: array
-                required:
-                - type
-                type: object
-              type: array
-            templating:
-              description: Templating variables
-              items:
-                description: Templating defines a variable, which can be used as a
-                  placeholder in query
-                properties:
-                  name:
-                    description: Variable name
-                    type: string
-                  query:
-                    description: Set variable values to be the return result of the
-                      query
-                    type: string
-                type: object
-              type: array
-            time:
-              description: Time range for display
-              properties:
-                from:
-                  description: Start time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the end time is set to the last month
-                    since now.
-                  type: string
-                to:
-                  description: End time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the start time is set to the last month
-                    since now.
-                  type: string
-              type: object
-            title:
-              description: Dashboard title
-              type: string
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/network.kubesphere.io_ipamblocks.yaml

@@ -21,10 +21,14 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/network.kubesphere.io_ipamhandles.yaml

@@ -21,10 +21,14 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/network.kubesphere.io_ippools.yaml

@@ -21,23 +21,29 @@ spec:
       openAPIV3Schema:
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           spec:
             properties:
               blockSize:
-                description: The block size to use for IP address assignments from this pool. Defaults to 26 for IPv4 and 112 for IPv6.
+                description: The block size to use for IP address assignments from
+                  this pool. Defaults to 26 for IPv4 and 112 for IPv6.
                 type: integer
               cidr:
                 description: The pool CIDR.
                 type: string
               disabled:
-                description: When disabled is true, IPAM will not assign addresses from this pool.
+                description: When disabled is true, IPAM will not assign addresses
+                  from this pool.
                 type: boolean
               dns:
                 description: DNS contains values interesting for DNS resolvers

config/crds/network.kubesphere.io_namespacenetworkpolicies.yaml

@@ -19,59 +19,109 @@ spec:
     - nsnp
     singular: namespacenetworkpolicy
   scope: Namespaced
-  preserveUnknownFields: false
   versions:
   - name: v1alpha1
     schema:
       openAPIV3Schema:
-        description: NamespaceNetworkPolicy is the Schema for the namespacenetworkpolicies API
+        description: NamespaceNetworkPolicy is the Schema for the namespacenetworkpolicies
+          API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
           spec:
-            description: NamespaceNetworkPolicySpec provides the specification of a NamespaceNetworkPolicy
+            description: NamespaceNetworkPolicySpec provides the specification of
+              a NamespaceNetworkPolicy
             properties:
               egress:
-                description: List of egress rules to be applied to the selected pods. Outgoing traffic is allowed if there are no NetworkPolicies selecting the pod (and cluster policy otherwise allows the traffic), OR if the traffic matches at least one egress rule across all of the NetworkPolicy objects whose podSelector matches the pod. If this field is empty then this NetworkPolicy limits all outgoing traffic (and serves solely to ensure that the pods it selects are isolated by default). This field is beta-level in 1.8
+                description: List of egress rules to be applied to the selected pods.
+                  Outgoing traffic is allowed if there are no NetworkPolicies selecting
+                  the pod (and cluster policy otherwise allows the traffic), OR if
+                  the traffic matches at least one egress rule across all of the NetworkPolicy
+                  objects whose podSelector matches the pod. If this field is empty
+                  then this NetworkPolicy limits all outgoing traffic (and serves
+                  solely to ensure that the pods it selects are isolated by default).
+                  This field is beta-level in 1.8
                 items:
-                  description: NetworkPolicyEgressRule describes a particular set of traffic that is allowed out of pods matched by a NetworkPolicySpec's podSelector. The traffic must match both ports and to. This type is beta-level in 1.8
+                  description: NetworkPolicyEgressRule describes a particular set
+                    of traffic that is allowed out of pods matched by a NetworkPolicySpec's
+                    podSelector. The traffic must match both ports and to. This type
+                    is beta-level in 1.8
                   properties:
                     ports:
-                      description: List of destination ports for outgoing traffic. Each item in this list is combined using a logical OR. If this field is empty or missing, this rule matches all ports (traffic not restricted by port). If this field is present and contains at least one item, then this rule allows traffic only if the traffic matches at least one port in the list.
+                      description: List of destination ports for outgoing traffic.
+                        Each item in this list is combined using a logical OR. If
+                        this field is empty or missing, this rule matches all ports
+                        (traffic not restricted by port). If this field is present
+                        and contains at least one item, then this rule allows traffic
+                        only if the traffic matches at least one port in the list.
                       items:
-                        description: NetworkPolicyPort describes a port to allow traffic on
+                        description: NetworkPolicyPort describes a port to allow traffic
+                          on
                         properties:
+                          endPort:
+                            description: If set, indicates that the range of ports
+                              from port to endPort, inclusive, should be allowed by
+                              the policy. This field cannot be defined if the port
+                              field is not defined or if the port field is defined
+                              as a named (string) port. The endPort must be equal
+                              or greater than port. This feature is in Alpha state
+                              and should be enabled using the Feature Gate "NetworkPolicyEndPort".
+                            format: int32
+                            type: integer
                           port:
                             anyOf:
                             - type: integer
                             - type: string
-                            description: The port on the given protocol. This can either be a numerical or named port on a pod. If this field is not provided, this matches all port names and numbers.
+                            description: The port on the given protocol. This can
+                              either be a numerical or named port on a pod. If this
+                              field is not provided, this matches all port names and
+                              numbers. If present, only traffic on the specified protocol
+                              AND port will be matched.
                             x-kubernetes-int-or-string: true
                           protocol:
                             default: TCP
-                            description: The protocol (TCP, UDP, or SCTP) which traffic must match. If not specified, this field defaults to TCP.
+                            description: The protocol (TCP, UDP, or SCTP) which traffic
+                              must match. If not specified, this field defaults to
+                              TCP.
                             type: string
                         type: object
                       type: array
                     to:
-                      description: List of destinations for outgoing traffic of pods selected for this rule. Items in this list are combined using a logical OR operation. If this field is empty or missing, this rule matches all destinations (traffic not restricted by destination). If this field is present and contains at least one item, this rule allows traffic only if the traffic matches at least one item in the to list.
+                      description: List of destinations for outgoing traffic of pods
+                        selected for this rule. Items in this list are combined using
+                        a logical OR operation. If this field is empty or missing,
+                        this rule matches all destinations (traffic not restricted
+                        by destination). If this field is present and contains at
+                        least one item, this rule allows traffic only if the traffic
+                        matches at least one item in the to list.
                       items:
-                        description: NetworkPolicyPeer describes a peer to allow traffic from. Only certain combinations of fields are allowed
+                        description: NetworkPolicyPeer describes a peer to allow traffic
+                          from. Only certain combinations of fields are allowed
                         properties:
                           ipBlock:
-                            description: IPBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.
+                            description: IPBlock defines policy on a particular IPBlock.
+                              If this field is set then neither of the other fields
+                              can be.
                             properties:
                               cidr:
-                                description: CIDR is a string representing the IP Block Valid examples are "192.168.1.1/24" or "2001:db9::/64"
+                                description: CIDR is a string representing the IP
+                                  Block Valid examples are "192.168.1.1/24" or "2001:db9::/64"
                                 type: string
                               except:
-                                description: Except is a slice of CIDRs that should not be included within an IP Block Valid examples are "192.168.1.1/24" or "2001:db9::/64" Except values will be rejected if they are outside the CIDR range
+                                description: Except is a slice of CIDRs that should
+                                  not be included within an IP Block Valid examples
+                                  are "192.168.1.1/24" or "2001:db9::/64" Except values
+                                  will be rejected if they are outside the CIDR range
                                 items:
                                   type: string
                                 type: array
@@ -100,23 +150,45 @@ spec:
                   type: object
                 type: array
               ingress:
-                description: List of ingress rules to be applied to the selected pods. Traffic is allowed to a pod if there are no NetworkPolicies selecting the pod (and cluster policy otherwise allows the traffic), OR if the traffic source is the pod's local node, OR if the traffic matches at least one ingress rule across all of the NetworkPolicy objects whose podSelector matches the pod. If this field is empty then this NetworkPolicy does not allow any traffic (and serves solely to ensure that the pods it selects are isolated by default)
+                description: List of ingress rules to be applied to the selected pods.
+                  Traffic is allowed to a pod if there are no NetworkPolicies selecting
+                  the pod (and cluster policy otherwise allows the traffic), OR if
+                  the traffic source is the pod's local node, OR if the traffic matches
+                  at least one ingress rule across all of the NetworkPolicy objects
+                  whose podSelector matches the pod. If this field is empty then this
+                  NetworkPolicy does not allow any traffic (and serves solely to ensure
+                  that the pods it selects are isolated by default)
                 items:
-                  description: NetworkPolicyIngressRule describes a particular set of traffic that is allowed to the pods matched by a NetworkPolicySpec's podSelector. The traffic must match both ports and from.
+                  description: NetworkPolicyIngressRule describes a particular set
+                    of traffic that is allowed to the pods matched by a NetworkPolicySpec's
+                    podSelector. The traffic must match both ports and from.
                   properties:
                     from:
-                      description: List of sources which should be able to access the pods selected for this rule. Items in this list are combined using a logical OR operation. If this field is empty or missing, this rule matches all sources (traffic not restricted by source). If this field is present and contains at least one item, this rule allows traffic only if the traffic matches at least one item in the from list.
+                      description: List of sources which should be able to access
+                        the pods selected for this rule. Items in this list are combined
+                        using a logical OR operation. If this field is empty or missing,
+                        this rule matches all sources (traffic not restricted by source).
+                        If this field is present and contains at least one item, this
+                        rule allows traffic only if the traffic matches at least one
+                        item in the from list.
                       items:
-                        description: NetworkPolicyPeer describes a peer to allow traffic from. Only certain combinations of fields are allowed
+                        description: NetworkPolicyPeer describes a peer to allow traffic
+                          from. Only certain combinations of fields are allowed
                         properties:
                           ipBlock:
-                            description: IPBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.
+                            description: IPBlock defines policy on a particular IPBlock.
+                              If this field is set then neither of the other fields
+                              can be.
                             properties:
                               cidr:
-                                description: CIDR is a string representing the IP Block Valid examples are "192.168.1.1/24" or "2001:db9::/64"
+                                description: CIDR is a string representing the IP
+                                  Block Valid examples are "192.168.1.1/24" or "2001:db9::/64"
                                 type: string
                               except:
-                                description: Except is a slice of CIDRs that should not be included within an IP Block Valid examples are "192.168.1.1/24" or "2001:db9::/64" Except values will be rejected if they are outside the CIDR range
+                                description: Except is a slice of CIDRs that should
+                                  not be included within an IP Block Valid examples
+                                  are "192.168.1.1/24" or "2001:db9::/64" Except values
+                                  will be rejected if they are outside the CIDR range
                                 items:
                                   type: string
                                 type: array
@@ -143,28 +215,63 @@ spec:
                         type: object
                       type: array
                     ports:
-                      description: List of ports which should be made accessible on the pods selected for this rule. Each item in this list is combined using a logical OR. If this field is empty or missing, this rule matches all ports (traffic not restricted by port). If this field is present and contains at least one item, then this rule allows traffic only if the traffic matches at least one port in the list.
+                      description: List of ports which should be made accessible on
+                        the pods selected for this rule. Each item in this list is
+                        combined using a logical OR. If this field is empty or missing,
+                        this rule matches all ports (traffic not restricted by port).
+                        If this field is present and contains at least one item, then
+                        this rule allows traffic only if the traffic matches at least
+                        one port in the list.
                       items:
-                        description: NetworkPolicyPort describes a port to allow traffic on
+                        description: NetworkPolicyPort describes a port to allow traffic
+                          on
                         properties:
+                          endPort:
+                            description: If set, indicates that the range of ports
+                              from port to endPort, inclusive, should be allowed by
+                              the policy. This field cannot be defined if the port
+                              field is not defined or if the port field is defined
+                              as a named (string) port. The endPort must be equal
+                              or greater than port. This feature is in Alpha state
+                              and should be enabled using the Feature Gate "NetworkPolicyEndPort".
+                            format: int32
+                            type: integer
                           port:
                             anyOf:
                             - type: integer
                             - type: string
-                            description: The port on the given protocol. This can either be a numerical or named port on a pod. If this field is not provided, this matches all port names and numbers.
+                            description: The port on the given protocol. This can
+                              either be a numerical or named port on a pod. If this
+                              field is not provided, this matches all port names and
+                              numbers. If present, only traffic on the specified protocol
+                              AND port will be matched.
                             x-kubernetes-int-or-string: true
                           protocol:
                             default: TCP
-                            description: The protocol (TCP, UDP, or SCTP) which traffic must match. If not specified, this field defaults to TCP.
+                            description: The protocol (TCP, UDP, or SCTP) which traffic
+                              must match. If not specified, this field defaults to
+                              TCP.
                             type: string
                         type: object
                       type: array
                   type: object
                 type: array
               policyTypes:
-                description: List of rule types that the NetworkPolicy relates to. Valid options are "Ingress", "Egress", or "Ingress,Egress". If this field is not specified, it will default based on the existence of Ingress or Egress rules; policies that contain an Egress section are assumed to affect Egress, and all policies (whether or not they contain an Ingress section) are assumed to affect Ingress. If you want to write an egress-only policy, you must explicitly specify policyTypes [ "Egress" ]. Likewise, if you want to write a policy that specifies that no egress is allowed, you must specify a policyTypes value that include "Egress" (since such a policy would not include an Egress section and would otherwise default to just [ "Ingress" ]). This field is beta-level in 1.8
+                description: List of rule types that the NetworkPolicy relates to.
+                  Valid options are "Ingress", "Egress", or "Ingress,Egress". If this
+                  field is not specified, it will default based on the existence of
+                  Ingress or Egress rules; policies that contain an Egress section
+                  are assumed to affect Egress, and all policies (whether or not they
+                  contain an Ingress section) are assumed to affect Ingress. If you
+                  want to write an egress-only policy, you must explicitly specify
+                  policyTypes [ "Egress" ]. Likewise, if you want to write a policy
+                  that specifies that no egress is allowed, you must specify a policyTypes
+                  value that include "Egress" (since such a policy would not include
+                  an Egress section and would otherwise default to just [ "Ingress"
+                  ]). This field is beta-level in 1.8
                 items:
-                  description: Policy Type string describes the NetworkPolicy type This type is beta-level in 1.8
+                  description: PolicyType string describes the NetworkPolicy type
+                    This type is beta-level in 1.8
                   type: string
                 type: array
             type: object

config/crds/notification.kubesphere.io_configs.yaml

@@ -1,283 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: configs.notification.kubesphere.io
-spec:
-  group: notification.kubesphere.io
-  names:
-    categories:
-    - notification-manager
-    kind: Config
-    listKind: ConfigList
-    plural: configs
-    shortNames:
-    - nc
-    singular: config
-  scope: Cluster
-  versions:
-  - name: v2beta1
-    schema:
-      openAPIV3Schema:
-        description: DingTalkConfig is the Schema for the dingtalkconfigs API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: ConfigSpec defines the desired state of Config
-            properties:
-              dingtalk:
-                properties:
-                  conversation:
-                    description: Only needed when send alerts to the conversation.
-                    properties:
-                      appkey:
-                        description: The key of the application with which to send messages.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      appsecret:
-                        description: The key in the secret to be used. Must be a valid secret key.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                    type: object
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                type: object
-              email:
-                properties:
-                  authIdentify:
-                    description: The identity for PLAIN authentication.
-                    type: string
-                  authPassword:
-                    description: The secret contains the SMTP password for LOGIN and PLAIN authentications.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                  authSecret:
-                    description: The secret contains the SMTP secret for CRAM-MD5 authentication.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                  authUsername:
-                    description: The username for CRAM-MD5, LOGIN and PLAIN authentications.
-                    type: string
-                  from:
-                    description: The sender address.
-                    type: string
-                  hello:
-                    description: The hostname to use when identifying to the SMTP server.
-                    type: string
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  requireTLS:
-                    description: The default SMTP TLS requirement.
-                    type: boolean
-                  smartHost:
-                    description: The address of the SMTP server.
-                    properties:
-                      host:
-                        type: string
-                      port:
-                        type: integer
-                    required:
-                    - host
-                    - port
-                    type: object
-                  tls:
-                    description: TLSConfig configures the options for TLS connections.
-                    properties:
-                      clientCertificate:
-                        description: The certificate of the client.
-                        properties:
-                          cert:
-                            description: The client cert file for the targets.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          key:
-                            description: The client key file for the targets.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                        type: object
-                      insecureSkipVerify:
-                        description: Disable target certificate validation.
-                        type: boolean
-                      rootCA:
-                        description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      serverName:
-                        description: Used to verify the hostname for the targets.
-                        type: string
-                    type: object
-                required:
-                - from
-                - smartHost
-                type: object
-              slack:
-                properties:
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  slackTokenSecret:
-                    description: The token of user or bot.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                type: object
-              webhook:
-                properties:
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                type: object
-              wechat:
-                properties:
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  wechatApiAgentId:
-                    description: The id of the application which sending message.
-                    type: string
-                  wechatApiCorpId:
-                    description: The corp id for authentication.
-                    type: string
-                  wechatApiSecret:
-                    description: The API key to use when talking to the WeChat API.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                  wechatApiUrl:
-                    description: The WeChat API URL.
-                    type: string
-                required:
-                - wechatApiAgentId
-                - wechatApiCorpId
-                - wechatApiSecret
-                type: object
-            type: object
-          status:
-            description: ConfigStatus defines the observed state of Config
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/notification.kubesphere.io_receivers.yaml

@@ -1,590 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: receivers.notification.kubesphere.io
-spec:
-  group: notification.kubesphere.io
-  names:
-    categories:
-    - notification-manager
-    kind: Receiver
-    listKind: ReceiverList
-    plural: receivers
-    shortNames:
-    - nr
-    singular: receiver
-  scope: Cluster
-  versions:
-  - name: v2beta1
-    schema:
-      openAPIV3Schema:
-        description: Receiver is the Schema for the receivers API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: ReceiverSpec defines the desired state of Receiver
-            properties:
-              dingtalk:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  chatbot:
-                    description: Be careful, a ChatBot only can send 20 message per minute.
-                    properties:
-                      keywords:
-                        description: Custom keywords of ChatBot
-                        items:
-                          type: string
-                        type: array
-                      secret:
-                        description: Secret of ChatBot, you can get it after enabled Additional Signature of ChatBot.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      webhook:
-                        description: The webhook of ChatBot which the message will send to.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                    required:
-                    - webhook
-                    type: object
-                  conversation:
-                    description: The conversation which message will send to.
-                    properties:
-                      chatids:
-                        items:
-                          type: string
-                        type: array
-                    required:
-                    - chatids
-                    type: object
-                  dingtalkConfigSelector:
-                    description: DingTalkConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                type: object
-              email:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  emailConfigSelector:
-                    description: EmailConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  to:
-                    description: Receivers' email addresses
-                    items:
-                      type: string
-                    type: array
-                required:
-                - to
-                type: object
-              slack:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  channels:
-                    description: The channel or user to send notifications to.
-                    items:
-                      type: string
-                    type: array
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  slackConfigSelector:
-                    description: SlackConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                required:
-                - channels
-                type: object
-              webhook:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  httpConfig:
-                    description: HTTPClientConfig configures an HTTP client.
-                    properties:
-                      basicAuth:
-                        description: The HTTP basic authentication credentials for the targets.
-                        properties:
-                          password:
-                            description: SecretKeySelector selects a key of a Secret.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          username:
-                            type: string
-                        required:
-                        - username
-                        type: object
-                      bearerToken:
-                        description: The bearer token for the targets.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      proxyUrl:
-                        description: HTTP proxy server to use to connect to the targets.
-                        type: string
-                      tlsConfig:
-                        description: TLSConfig to use to connect to the targets.
-                        properties:
-                          clientCertificate:
-                            description: The certificate of the client.
-                            properties:
-                              cert:
-                                description: The client cert file for the targets.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              key:
-                                description: The client key file for the targets.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                            type: object
-                          insecureSkipVerify:
-                            description: Disable target certificate validation.
-                            type: boolean
-                          rootCA:
-                            description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          serverName:
-                            description: Used to verify the hostname for the targets.
-                            type: string
-                        type: object
-                    type: object
-                  service:
-                    description: "`service` is a reference to the service for this webhook. Either `service` or `url` must be specified. \n If the webhook is running within the cluster, then you should use `service`."
-                    properties:
-                      name:
-                        description: '`name` is the name of the service. Required'
-                        type: string
-                      namespace:
-                        description: '`namespace` is the namespace of the service. Required'
-                        type: string
-                      path:
-                        description: '`path` is an optional URL path which will be sent in any request to this service.'
-                        type: string
-                      port:
-                        description: If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive).
-                        format: int32
-                        type: integer
-                      scheme:
-                        description: Http scheme, default is http.
-                        type: string
-                    required:
-                    - name
-                    - namespace
-                    type: object
-                  url:
-                    description: "`url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. \n The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some api servers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. \n Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. \n A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. \n Attempting to use a user or basic auth e.g. \"user:password@\" is not allowed. Fragments (\"#...\") and query parameters (\"?...\") are not allowed, either."
-                    type: string
-                  webhookConfigSelector:
-                    description: WebhookConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                type: object
-              wechat:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  toParty:
-                    items:
-                      type: string
-                    type: array
-                  toTag:
-                    items:
-                      type: string
-                    type: array
-                  toUser:
-                    items:
-                      type: string
-                    type: array
-                  wechatConfigSelector:
-                    description: WechatConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                type: object
-            type: object
-          status:
-            description: ReceiverStatus defines the observed state of Receiver
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/quota.kubesphere.io_resourcequotas.yaml

@@ -21,13 +21,18 @@ spec:
   - name: v1alpha2
     schema:
       openAPIV3Schema:
-        description: WorkspaceResourceQuota sets aggregate quota restrictions enforced per workspace
+        description: WorkspaceResourceQuota sets aggregate quota restrictions enforced
+          per workspace
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -44,24 +49,39 @@ spec:
                       - type: string
                       pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                       x-kubernetes-int-or-string: true
-                    description: 'hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
+                    description: 'hard is the set of desired hard limits for each
+                      named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
                     type: object
                   scopeSelector:
-                    description: scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.
+                    description: scopeSelector is also a collection of filters like
+                      scopes that must match each object tracked by a quota but expressed
+                      using ScopeSelectorOperator in combination with possible values.
+                      For a resource to match, both scopes AND scopeSelector (if specified
+                      in spec), must be matched.
                     properties:
                       matchExpressions:
-                        description: A list of scope selector requirements by scope of the resources.
+                        description: A list of scope selector requirements by scope
+                          of the resources.
                         items:
-                          description: A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.
+                          description: A scoped-resource selector requirement is a
+                            selector that contains values, a scope name, and an operator
+                            that relates the scope name and values.
                           properties:
                             operator:
-                              description: Represents a scope's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.
+                              description: Represents a scope's relationship to a
+                                set of values. Valid operators are In, NotIn, Exists,
+                                DoesNotExist.
                               type: string
                             scopeName:
-                              description: The name of the scope that the selector applies to.
+                              description: The name of the scope that the selector
+                                applies to.
                               type: string
                             values:
-                              description: An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
+                              description: An array of string values. If the operator
+                                is In or NotIn, the values array must be non-empty.
+                                If the operator is Exists or DoesNotExist, the values
+                                array must be empty. This array is replaced during
+                                a strategic merge patch.
                               items:
                                 type: string
                               type: array
@@ -72,9 +92,12 @@ spec:
                         type: array
                     type: object
                   scopes:
-                    description: A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.
+                    description: A collection of filters that must match each object
+                      tracked by a quota. If not specified, the quota matches all
+                      objects.
                     items:
-                      description: A ResourceQuotaScope defines a filter that must match each object tracked by a quota
+                      description: A ResourceQuotaScope defines a filter that must
+                        match each object tracked by a quota
                       type: string
                     type: array
                 type: object
@@ -88,12 +111,14 @@ spec:
             - selector
             type: object
           status:
-            description: Status defines the actual enforced quota and its current usage
+            description: Status defines the actual enforced quota and its current
+              usage
             properties:
               namespaces:
                 description: Namespaces slices the usage by project.
                 items:
-                  description: ResourceQuotaStatusByNamespace gives status for a particular project
+                  description: ResourceQuotaStatusByNamespace gives status for a particular
+                    project
                   properties:
                     hard:
                       additionalProperties:
@@ -102,7 +127,8 @@ spec:
                         - type: string
                         pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                         x-kubernetes-int-or-string: true
-                      description: 'Hard is the set of enforced hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
+                      description: 'Hard is the set of enforced hard limits for each
+                        named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
                       type: object
                     namespace:
                       description: Namespace the project this status applies to
@@ -114,14 +140,16 @@ spec:
                         - type: string
                         pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                         x-kubernetes-int-or-string: true
-                      description: Used is the current observed total usage of the resource in the namespace.
+                      description: Used is the current observed total usage of the
+                        resource in the namespace.
                       type: object
                   required:
                   - namespace
                   type: object
                 type: array
               total:
-                description: Total defines the actual enforced quota and its current usage across all projects
+                description: Total defines the actual enforced quota and its current
+                  usage across all projects
                 properties:
                   hard:
                     additionalProperties:
@@ -130,7 +158,8 @@ spec:
                       - type: string
                       pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                       x-kubernetes-int-or-string: true
-                    description: 'Hard is the set of enforced hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
+                    description: 'Hard is the set of enforced hard limits for each
+                      named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
                     type: object
                   used:
                     additionalProperties:
@@ -139,7 +168,8 @@ spec:
                       - type: string
                       pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                       x-kubernetes-int-or-string: true
-                    description: Used is the current observed total usage of the resource in the namespace.
+                    description: Used is the current observed total usage of the resource
+                      in the namespace.
                     type: object
                 type: object
             required:

config/crds/servicemesh.kubesphere.io_strategies.yaml

@@ -91,453 +91,690 @@ spec:
                     description: Metadata of the virtual services created from this template
                     type: object
                   spec:
-                    description: Spec indicates the behavior of a virtual service.
+                    description: 'Configuration affecting label/content routing, sni routing,
+                      etc. See more details at: https://istio.io/docs/reference/config/networking/virtual-service.html'
                     properties:
-                      export_to:
-                        description: "A list of namespaces to which this virtual service is exported. Exporting a virtual service allows it to be used by sidecars and gateways defined in other namespaces. This feature provides a mechanism for service owners and mesh administrators to control the visibility of virtual services across namespace boundaries. \n If no namespaces are specified then the virtual service is exported to all namespaces by default. \n The value \".\" is reserved and defines an export to the same namespace that the virtual service is declared in. Similarly the value \"*\" is reserved and defines an export to all namespaces. \n NOTE: in the current release, the `exportTo` value is restricted to \".\" or \"*\" (i.e., the current namespace or all namespaces)."
+                      exportTo:
+                        description: A list of namespaces to which this virtual service is
+                          exported.
                         items:
+                          format: string
                           type: string
                         type: array
                       gateways:
-                        description: The names of gateways and sidecars that should apply these routes. Gateways in other namespaces may be referred to by `<gateway namespace>/<gateway name>`; specifying a gateway with no namespace qualifier is the same as specifying the VirtualService's namespace. A single VirtualService is used for sidecars inside the mesh as well as for one or more gateways. The selection condition imposed by this field can be overridden using the source field in the match conditions of protocol-specific routes. The reserved word `mesh` is used to imply all the sidecars in the mesh. When this field is omitted, the default gateway (`mesh`) will be used, which would apply the rule to all sidecars in the mesh. If a list of gateway names is provided, the rules will apply only to the gateways. To apply the rules to both gateways and sidecars, specify `mesh` as one of the gateway names.
+                        description: The names of gateways and sidecars that should apply
+                          these routes.
                         items:
+                          format: string
                           type: string
                         type: array
                       hosts:
-                        description: "The destination hosts to which traffic is being sent. Could be a DNS name with wildcard prefix or an IP address.  Depending on the platform, short-names can also be used instead of a FQDN (i.e. has no dots in the name). In such a scenario, the FQDN of the host would be derived based on the underlying platform. \n A single VirtualService can be used to describe all the traffic properties of the corresponding hosts, including those for multiple HTTP and TCP ports. Alternatively, the traffic properties of a host can be defined using more than one VirtualService, with certain caveats. Refer to the [Operations Guide](https://istio.io/docs/ops/best-practices/traffic-management/#split-virtual-services) for details. \n *Note for Kubernetes users*: When short names are used (e.g. \"reviews\" instead of \"reviews.default.svc.cluster.local\"), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the \"default\" namespace containing a host \"reviews\" will be interpreted as \"reviews.default.svc.cluster.local\", irrespective of the actual namespace associated with the reviews service. _To avoid potential misconfigurations, it is recommended to always use fully qualified domain names over short names._ \n The hosts field applies to both HTTP and TCP services. Service inside the mesh, i.e., those found in the service registry, must always be referred to using their alphanumeric names. IP addresses are allowed only for services defined via the Gateway. \n *Note*: It must be empty for a delegate VirtualService."
+                        description: The destination hosts to which traffic is being sent.
                         items:
+                          format: string
                           type: string
                         type: array
                       http:
-                        description: An ordered list of route rules for HTTP traffic. HTTP routes will be applied to platform service ports named 'http-*'/'http2-*'/'grpc-*', gateway ports with protocol HTTP/HTTP2/GRPC/ TLS-terminated-HTTPS and service entry ports using HTTP/HTTP2/GRPC protocols.  The first rule matching an incoming request is used.
+                        description: An ordered list of route rules for HTTP traffic.
                         items:
-                          description: Describes match conditions and actions for routing HTTP/1.1, HTTP2, and gRPC traffic. See VirtualService for usage examples.
                           properties:
-                            cors_policy:
-                              description: Cross-Origin Resource Sharing policy (CORS). Refer to [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) for further details about cross origin resource sharing.
+                            corsPolicy:
+                              description: Cross-Origin Resource Sharing policy (CORS).
                               properties:
-                                allow_credentials:
-                                  description: Indicates whether the caller is allowed to send the actual request (not the preflight) using credentials. Translates to `Access-Control-Allow-Credentials` header.
-                                  properties:
-                                    value:
-                                      description: The bool value.
-                                      type: boolean
-                                  type: object
-                                allow_headers:
-                                  description: List of HTTP headers that can be used when requesting the resource. Serialized to Access-Control-Allow-Headers header.
+                                allowCredentials:
+                                  nullable: true
+                                  type: boolean
+                                allowHeaders:
                                   items:
+                                    format: string
                                     type: string
                                   type: array
-                                allow_methods:
-                                  description: List of HTTP methods allowed to access the resource. The content will be serialized into the Access-Control-Allow-Methods header.
+                                allowMethods:
+                                  description: List of HTTP methods allowed to access the
+                                    resource.
                                   items:
+                                    format: string
                                     type: string
                                   type: array
-                                allow_origin:
-                                  description: The list of origins that are allowed to perform CORS requests. The content will be serialized into the Access-Control-Allow-Origin header. Wildcard * will allow all origins. $hide_from_docs
+                                allowOrigin:
+                                  description: The list of origins that are allowed to perform
+                                    CORS requests.
                                   items:
+                                    format: string
                                     type: string
                                   type: array
-                                allow_origins:
-                                  description: String patterns that match allowed origins. An origin is allowed if any of the string matchers match. If a match is found, then the outgoing Access-Control-Allow-Origin would be set to the origin as provided by the client.
+                                allowOrigins:
+                                  description: String patterns that match allowed origins.
                                   items:
-                                    description: Describes how to match a given string in HTTP headers. Match is case-sensitive.
+                                    oneOf:
+                                      - not:
+                                          anyOf:
+                                            - required:
+                                                - exact
+                                            - required:
+                                                - prefix
+                                            - required:
+                                                - regex
+                                      - required:
+                                          - exact
+                                      - required:
+                                          - prefix
+                                      - required:
+                                          - regex
+                                    properties:
+                                      exact:
+                                        format: string
+                                        type: string
+                                      prefix:
+                                        format: string
+                                        type: string
+                                      regex:
+                                        description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                        format: string
+                                        type: string
                                     type: object
                                   type: array
-                                expose_headers:
-                                  description: A white list of HTTP headers that the browsers are allowed to access. Serialized into Access-Control-Expose-Headers header.
+                                exposeHeaders:
                                   items:
+                                    format: string
                                     type: string
                                   type: array
-                                max_age:
-                                  description: Specifies how long the results of a preflight request can be cached. Translates to the `Access-Control-Max-Age` header.
-                                  properties:
-                                    nanos:
-                                      description: Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 `seconds` field and a positive or negative `nanos` field. For durations of one second or more, a non-zero value for the `nanos` field must be of the same sign as the `seconds` field. Must be from -999,999,999 to +999,999,999 inclusive.
-                                      format: int32
-                                      type: integer
-                                    seconds:
-                                      description: 'Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years'
-                                      format: int64
-                                      type: integer
-                                  type: object
+                                maxAge:
+                                  type: string
                               type: object
                             delegate:
-                              description: 'Delegate is used to specify the particular VirtualService which can be used to define delegate HTTPRoute. It can be set only when `Route` and `Redirect` are empty, and the route rules of the delegate VirtualService will be merged with that in the current one. **NOTE**:    1. Only one level delegation is supported.    2. The delegate''s HTTPMatchRequest must be a strict subset of the root''s,       otherwise there is a conflict and the HTTPRoute will not take effect.'
                               properties:
                                 name:
                                   description: Name specifies the name of the delegate VirtualService.
+                                  format: string
                                   type: string
                                 namespace:
-                                  description: Namespace specifies the namespace where the delegate VirtualService resides. By default, it is same to the root's.
+                                  description: Namespace specifies the namespace where the
+                                    delegate VirtualService resides.
+                                  format: string
                                   type: string
                               type: object
                             fault:
-                              description: Fault injection policy to apply on HTTP traffic at the client side. Note that timeouts or retries will not be enabled when faults are enabled on the client side.
+                              description: Fault injection policy to apply on HTTP traffic
+                                at the client side.
                               properties:
                                 abort:
-                                  description: Abort Http request attempts and return error codes back to downstream service, giving the impression that the upstream service is faulty.
+                                  oneOf:
+                                    - not:
+                                        anyOf:
+                                          - required:
+                                              - httpStatus
+                                          - required:
+                                              - grpcStatus
+                                          - required:
+                                              - http2Error
+                                    - required:
+                                        - httpStatus
+                                    - required:
+                                        - grpcStatus
+                                    - required:
+                                        - http2Error
                                   properties:
+                                    grpcStatus:
+                                      format: string
+                                      type: string
+                                    http2Error:
+                                      format: string
+                                      type: string
+                                    httpStatus:
+                                      description: HTTP status code to use to abort the Http
+                                        request.
+                                      format: int32
+                                      type: integer
                                     percentage:
-                                      description: Percentage of requests to be aborted with the error code provided.
+                                      description: Percentage of requests to be aborted with
+                                        the error code provided.
                                       properties:
                                         value:
+                                          format: double
                                           type: number
                                       type: object
                                   type: object
                                 delay:
-                                  description: Delay requests before forwarding, emulating various failures such as network issues, overloaded upstream service, etc.
+                                  oneOf:
+                                    - not:
+                                        anyOf:
+                                          - required:
+                                              - fixedDelay
+                                          - required:
+                                              - exponentialDelay
+                                    - required:
+                                        - fixedDelay
+                                    - required:
+                                        - exponentialDelay
                                   properties:
+                                    exponentialDelay:
+                                      type: string
+                                    fixedDelay:
+                                      description: Add a fixed delay before forwarding the
+                                        request.
+                                      type: string
                                     percent:
-                                      description: Percentage of requests on which the delay will be injected (0-100). Use of integer `percent` value is deprecated. Use the double `percentage` field instead.
+                                      description: Percentage of requests on which the delay
+                                        will be injected (0-100).
                                       format: int32
                                       type: integer
                                     percentage:
-                                      description: Percentage of requests on which the delay will be injected.
+                                      description: Percentage of requests on which the delay
+                                        will be injected.
                                       properties:
                                         value:
+                                          format: double
                                           type: number
                                       type: object
                                   type: object
                               type: object
                             headers:
-                              description: Header manipulation rules
                               properties:
                                 request:
-                                  description: Header manipulation rules to apply before forwarding a request to the destination service
                                   properties:
                                     add:
                                       additionalProperties:
+                                        format: string
                                         type: string
-                                      description: Append the given values to the headers specified by keys (will create a comma-separated list of values)
                                       type: object
                                     remove:
-                                      description: Remove a the specified headers
                                       items:
+                                        format: string
                                         type: string
                                       type: array
                                     set:
                                       additionalProperties:
+                                        format: string
                                         type: string
-                                      description: Overwrite the headers specified by key with the given values
                                       type: object
                                   type: object
                                 response:
-                                  description: Header manipulation rules to apply before returning a response to the caller
                                   properties:
                                     add:
                                       additionalProperties:
+                                        format: string
                                         type: string
-                                      description: Append the given values to the headers specified by keys (will create a comma-separated list of values)
                                       type: object
                                     remove:
-                                      description: Remove a the specified headers
                                       items:
+                                        format: string
                                         type: string
                                       type: array
                                     set:
                                       additionalProperties:
+                                        format: string
                                         type: string
-                                      description: Overwrite the headers specified by key with the given values
                                       type: object
                                   type: object
                               type: object
                             match:
-                              description: Match conditions to be satisfied for the rule to be activated. All conditions inside a single match block have AND semantics, while the list of match blocks have OR semantics. The rule is matched if any one of the match blocks succeed.
                               items:
-                                description: "HttpMatchRequest specifies a set of criterion to be met in order for the rule to be applied to the HTTP request. For example, the following restricts the rule to match only requests where the URL path starts with /ratings/v2/ and the request contains a custom `end-user` header with value `jason`. \n {{<tabset category-name=\"example\">}} {{<tab name=\"v1alpha3\" category-value=\"v1alpha3\">}} ```yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata:   name: ratings-route spec:   hosts:   - ratings.prod.svc.cluster.local   http:   - match:     - headers:         end-user:           exact: jason       uri:         prefix: \"/ratings/v2/\"       ignoreUriCase: true     route:     - destination:         host: ratings.prod.svc.cluster.local ``` {{</tab>}} \n {{<tab name=\"v1beta1\" category-value=\"v1beta1\">}} ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata:   name: ratings-route spec:   hosts:   - ratings.prod.svc.cluster.local   http:   - match:     - headers:         end-user:           exact: jason       uri:         prefix: \"/ratings/v2/\"       ignoreUriCase: true     route:     - destination:         host: ratings.prod.svc.cluster.local ``` {{</tab>}} {{</tabset>}} \n HTTPMatchRequest CANNOT be empty. **Note:** No regex string match can be set when delegate VirtualService is specified."
                                 properties:
                                   authority:
-                                    description: "HTTP Authority values are case-sensitive and formatted as follows: \n - `exact: \"value\"` for exact string match \n - `prefix: \"value\"` for prefix-based match \n - `regex: \"value\"` for ECMAscript style regex-based match"
+                                    oneOf:
+                                      - not:
+                                          anyOf:
+                                            - required:
+                                                - exact
+                                            - required:
+                                                - prefix
+                                            - required:
+                                                - regex
+                                      - required:
+                                          - exact
+                                      - required:
+                                          - prefix
+                                      - required:
+                                          - regex
+                                    properties:
+                                      exact:
+                                        format: string
+                                        type: string
+                                      prefix:
+                                        format: string
+                                        type: string
+                                      regex:
+                                        description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                        format: string
+                                        type: string
                                     type: object
                                   gateways:
-                                    description: Names of gateways where the rule should be applied. Gateway names in the top-level `gateways` field of the VirtualService (if any) are overridden. The gateway match is independent of sourceLabels.
+                                    description: Names of gateways where the rule should be
+                                      applied.
                                     items:
+                                      format: string
                                       type: string
                                     type: array
                                   headers:
                                     additionalProperties:
-                                      description: Describes how to match a given string in HTTP headers. Match is case-sensitive.
+                                      oneOf:
+                                        - not:
+                                            anyOf:
+                                              - required:
+                                                  - exact
+                                              - required:
+                                                  - prefix
+                                              - required:
+                                                  - regex
+                                        - required:
+                                            - exact
+                                        - required:
+                                            - prefix
+                                        - required:
+                                            - regex
+                                      properties:
+                                        exact:
+                                          format: string
+                                          type: string
+                                        prefix:
+                                          format: string
+                                          type: string
+                                        regex:
+                                          description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                          format: string
+                                          type: string
                                       type: object
-                                    description: "The header keys must be lowercase and use hyphen as the separator, e.g. _x-request-id_. \n Header values are case-sensitive and formatted as follows: \n - `exact: \"value\"` for exact string match \n - `prefix: \"value\"` for prefix-based match \n - `regex: \"value\"` for ECMAscript style regex-based match \n If the value is empty and only the name of header is specfied, presence of the header is checked. **Note:** The keys `uri`, `scheme`, `method`, and `authority` will be ignored."
                                     type: object
-                                  ignore_uri_case:
-                                    description: "Flag to specify whether the URI matching should be case-insensitive. \n **Note:** The case will be ignored only in the case of `exact` and `prefix` URI matches."
+                                  ignoreUriCase:
+                                    description: Flag to specify whether the URI matching
+                                      should be case-insensitive.
                                     type: boolean
                                   method:
-                                    description: "HTTP Method values are case-sensitive and formatted as follows: \n - `exact: \"value\"` for exact string match \n - `prefix: \"value\"` for prefix-based match \n - `regex: \"value\"` for ECMAscript style regex-based match"
+                                    oneOf:
+                                      - not:
+                                          anyOf:
+                                            - required:
+                                                - exact
+                                            - required:
+                                                - prefix
+                                            - required:
+                                                - regex
+                                      - required:
+                                          - exact
+                                      - required:
+                                          - prefix
+                                      - required:
+                                          - regex
+                                    properties:
+                                      exact:
+                                        format: string
+                                        type: string
+                                      prefix:
+                                        format: string
+                                        type: string
+                                      regex:
+                                        description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                        format: string
+                                        type: string
                                     type: object
                                   name:
-                                    description: The name assigned to a match. The match's name will be concatenated with the parent route's name and will be logged in the access logs for requests matching this route.
+                                    description: The name assigned to a match.
+                                    format: string
                                     type: string
                                   port:
-                                    description: Specifies the ports on the host that is being addressed. Many services only expose a single port or label ports with the protocols they support, in these cases it is not required to explicitly select the port.
-                                    format: int32
+                                    description: Specifies the ports on the host that is being
+                                      addressed.
                                     type: integer
-                                  query_params:
+                                  queryParams:
                                     additionalProperties:
-                                      description: Describes how to match a given string in HTTP headers. Match is case-sensitive.
+                                      oneOf:
+                                        - not:
+                                            anyOf:
+                                              - required:
+                                                  - exact
+                                              - required:
+                                                  - prefix
+                                              - required:
+                                                  - regex
+                                        - required:
+                                            - exact
+                                        - required:
+                                            - prefix
+                                        - required:
+                                            - regex
+                                      properties:
+                                        exact:
+                                          format: string
+                                          type: string
+                                        prefix:
+                                          format: string
+                                          type: string
+                                        regex:
+                                          description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                          format: string
+                                          type: string
                                       type: object
-                                    description: "Query parameters for matching. \n Ex: - For a query parameter like \"?key=true\", the map key would be \"key\" and   the string match could be defined as `exact: \"true\"`. - For a query parameter like \"?key\", the map key would be \"key\" and the   string match could be defined as `exact: \"\"`. - For a query parameter like \"?key=123\", the map key would be \"key\" and the   string match could be defined as `regex: \"\\d+$\"`. Note that this   configuration will only match values like \"123\" but not \"a123\" or \"123a\". \n **Note:** `prefix` matching is currently not supported."
+                                    description: Query parameters for matching.
                                     type: object
                                   scheme:
-                                    description: "URI Scheme values are case-sensitive and formatted as follows: \n - `exact: \"value\"` for exact string match \n - `prefix: \"value\"` for prefix-based match \n - `regex: \"value\"` for ECMAscript style regex-based match"
+                                    oneOf:
+                                      - not:
+                                          anyOf:
+                                            - required:
+                                                - exact
+                                            - required:
+                                                - prefix
+                                            - required:
+                                                - regex
+                                      - required:
+                                          - exact
+                                      - required:
+                                          - prefix
+                                      - required:
+                                          - regex
+                                    properties:
+                                      exact:
+                                        format: string
+                                        type: string
+                                      prefix:
+                                        format: string
+                                        type: string
+                                      regex:
+                                        description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                        format: string
+                                        type: string
                                     type: object
-                                  source_labels:
+                                  sourceLabels:
                                     additionalProperties:
+                                      format: string
                                       type: string
-                                    description: One or more labels that constrain the applicability of a rule to workloads with the given labels. If the VirtualService has a list of gateways specified in the top-level `gateways` field, it must include the reserved gateway `mesh` for this field to be applicable.
                                     type: object
-                                  source_namespace:
-                                    description: Source namespace constraining the applicability of a rule to workloads in that namespace. If the VirtualService has a list of gateways specified in the top-level `gateways` field, it must include the reserved gateway `mesh` for this field to be applicable.
+                                  sourceNamespace:
+                                    description: Source namespace constraining the applicability
+                                      of a rule to workloads in that namespace.
+                                    format: string
                                     type: string
                                   uri:
-                                    description: "URI to match values are case-sensitive and formatted as follows: \n - `exact: \"value\"` for exact string match \n - `prefix: \"value\"` for prefix-based match \n - `regex: \"value\"` for ECMAscript style regex-based match \n **Note:** Case-insensitive matching could be enabled via the `ignore_uri_case` flag."
+                                    oneOf:
+                                      - not:
+                                          anyOf:
+                                            - required:
+                                                - exact
+                                            - required:
+                                                - prefix
+                                            - required:
+                                                - regex
+                                      - required:
+                                          - exact
+                                      - required:
+                                          - prefix
+                                      - required:
+                                          - regex
+                                    properties:
+                                      exact:
+                                        format: string
+                                        type: string
+                                      prefix:
+                                        format: string
+                                        type: string
+                                      regex:
+                                        description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                        format: string
+                                        type: string
                                     type: object
-                                  without_headers:
+                                  withoutHeaders:
                                     additionalProperties:
-                                      description: Describes how to match a given string in HTTP headers. Match is case-sensitive.
+                                      oneOf:
+                                        - not:
+                                            anyOf:
+                                              - required:
+                                                  - exact
+                                              - required:
+                                                  - prefix
+                                              - required:
+                                                  - regex
+                                        - required:
+                                            - exact
+                                        - required:
+                                            - prefix
+                                        - required:
+                                            - regex
+                                      properties:
+                                        exact:
+                                          format: string
+                                          type: string
+                                        prefix:
+                                          format: string
+                                          type: string
+                                        regex:
+                                          description: RE2 style regex-based match (https://github.com/google/re2/wiki/Syntax).
+                                          format: string
+                                          type: string
                                       type: object
-                                    description: withoutHeader has the same syntax with the header, but has opposite meaning. If a header is matched with a matching rule among withoutHeader, the traffic becomes not matched one.
+                                    description: withoutHeader has the same syntax with the
+                                      header, but has opposite meaning.
                                     type: object
                                 type: object
                               type: array
                             mirror:
-                              description: Mirror HTTP traffic to a another destination in addition to forwarding the requests to the intended destination. Mirrored traffic is on a best effort basis where the sidecar/gateway will not wait for the mirrored cluster to respond before returning the response from the original destination.  Statistics will be generated for the mirrored destination.
                               properties:
                                 host:
-                                  description: "The name of a service from the service registry. Service names are looked up from the platform's service registry (e.g., Kubernetes services, Consul services, etc.) and from the hosts declared by [ServiceEntry](https://istio.io/docs/reference/config/networking/service-entry/#ServiceEntry). Traffic forwarded to destinations that are not found in either of the two, will be dropped. \n *Note for Kubernetes users*: When short names are used (e.g. \"reviews\" instead of \"reviews.default.svc.cluster.local\"), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the \"default\" namespace containing a host \"reviews will be interpreted as \"reviews.default.svc.cluster.local\", irrespective of the actual namespace associated with the reviews service. To avoid potential misconfiguration, it is recommended to always use fully qualified domain names over short names."
+                                  description: The name of a service from the service registry.
+                                  format: string
                                   type: string
                                 port:
-                                  description: Specifies the port on the host that is being addressed. If a service exposes only a single port it is not required to explicitly select the port.
+                                  description: Specifies the port on the host that is being
+                                    addressed.
                                   properties:
                                     number:
-                                      description: Valid port number
-                                      format: int32
                                       type: integer
                                   type: object
                                 subset:
-                                  description: The name of a subset within the service. Applicable only to services within the mesh. The subset must be defined in a corresponding DestinationRule.
+                                  description: The name of a subset within the service.
+                                  format: string
                                   type: string
                               type: object
                             mirror_percent:
-                              description: Percentage of the traffic to be mirrored by the `mirror` field. Use of integer `mirror_percent` value is deprecated. Use the double `mirror_percentage` field instead
-                              properties:
-                                value:
-                                  description: The uint32 value.
-                                  format: int32
-                                  type: integer
-                              type: object
-                            mirror_percentage:
-                              description: Percentage of the traffic to be mirrored by the `mirror` field. If this field is absent, all the traffic (100%) will be mirrored. Max value is 100.
+                              description: Percentage of the traffic to be mirrored by the
+                                `mirror` field.
+                              nullable: true
+                              type: integer
+                            mirrorPercent:
+                              description: Percentage of the traffic to be mirrored by the
+                                `mirror` field.
+                              nullable: true
+                              type: integer
+                            mirrorPercentage:
+                              description: Percentage of the traffic to be mirrored by the
+                                `mirror` field.
                               properties:
                                 value:
+                                  format: double
                                   type: number
                               type: object
                             name:
-                              description: The name assigned to the route for debugging purposes. The route's name will be concatenated with the match's name and will be logged in the access logs for requests matching this route/match.
+                              description: The name assigned to the route for debugging purposes.
+                              format: string
                               type: string
                             redirect:
-                              description: A HTTP rule can either redirect or forward (default) traffic. If traffic passthrough option is specified in the rule, route/redirect will be ignored. The redirect primitive can be used to send a HTTP 301 redirect to a different URI or Authority.
+                              description: A HTTP rule can either redirect or forward (default)
+                                traffic.
                               properties:
                                 authority:
-                                  description: On a redirect, overwrite the Authority/Host portion of the URL with this value.
+                                  format: string
                                   type: string
-                                redirect_code:
-                                  description: On a redirect, Specifies the HTTP status code to use in the redirect response. The default response code is MOVED_PERMANENTLY (301).
-                                  format: int32
+                                redirectCode:
                                   type: integer
                                 uri:
-                                  description: On a redirect, overwrite the Path portion of the URL with this value. Note that the entire path will be replaced, irrespective of the request URI being matched as an exact path or prefix.
+                                  format: string
                                   type: string
                               type: object
                             retries:
                               description: Retry policy for HTTP requests.
                               properties:
                                 attempts:
-                                  description: Number of retries for a given request. The interval between retries will be determined automatically (25ms+). Actual number of retries attempted depends on the request `timeout` of the [HTTP route](https://istio.io/docs/reference/config/networking/virtual-service/#HTTPRoute).
+                                  description: Number of retries to be allowed for a given
+                                    request.
                                   format: int32
                                   type: integer
-                                per_try_timeout:
-                                  description: 'Timeout per retry attempt for a given request. format: 1h/1m/1s/1ms. MUST BE >=1ms.'
-                                  properties:
-                                    nanos:
-                                      description: Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 `seconds` field and a positive or negative `nanos` field. For durations of one second or more, a non-zero value for the `nanos` field must be of the same sign as the `seconds` field. Must be from -999,999,999 to +999,999,999 inclusive.
-                                      format: int32
-                                      type: integer
-                                    seconds:
-                                      description: 'Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years'
-                                      format: int64
-                                      type: integer
-                                  type: object
-                                retry_on:
-                                  description: Specifies the conditions under which retry takes place. One or more policies can be specified using a ‘,’ delimited list. See the [retry policies](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on) and [gRPC retry policies](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-grpc-on) for more details.
+                                perTryTimeout:
+                                  description: Timeout per attempt for a given request, including
+                                    the initial call and any retries.
                                   type: string
-                                retry_remote_localities:
-                                  description: Flag to specify whether the retries should retry to other localities. See the [retry plugin configuration](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/http_connection_management#retry-plugin-configuration) for more details.
-                                  properties:
-                                    value:
-                                      description: The bool value.
-                                      type: boolean
-                                  type: object
+                                retryOn:
+                                  description: Specifies the conditions under which retry
+                                    takes place.
+                                  format: string
+                                  type: string
+                                retryRemoteLocalities:
+                                  description: Flag to specify whether the retries should
+                                    retry to other localities.
+                                  nullable: true
+                                  type: boolean
                               type: object
                             rewrite:
-                              description: Rewrite HTTP URIs and Authority headers. Rewrite cannot be used with Redirect primitive. Rewrite will be performed before forwarding.
+                              description: Rewrite HTTP URIs and Authority headers.
                               properties:
                                 authority:
-                                  description: rewrite the Authority/Host header with this value.
+                                  description: rewrite the Authority/Host header with this
+                                    value.
+                                  format: string
                                   type: string
                                 uri:
-                                  description: rewrite the path (or the prefix) portion of the URI with this value. If the original URI was matched based on prefix, the value provided in this field will replace the corresponding matched prefix.
+                                  format: string
                                   type: string
                               type: object
                             route:
-                              description: A HTTP rule can either redirect or forward (default) traffic. The forwarding target can be one of several versions of a service (see glossary in beginning of document). Weights associated with the service version determine the proportion of traffic it receives.
+                              description: A HTTP rule can either redirect or forward (default)
+                                traffic.
                               items:
-                                description: "Each routing rule is associated with one or more service versions (see glossary in beginning of document). Weights associated with the version determine the proportion of traffic it receives. For example, the following rule will route 25% of traffic for the \"reviews\" service to instances with the \"v2\" tag and the remaining traffic (i.e., 75%) to \"v1\". \n {{<tabset category-name=\"example\">}} {{<tab name=\"v1alpha3\" category-value=\"v1alpha3\">}} ```yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata:   name: reviews-route spec:   hosts:   - reviews.prod.svc.cluster.local   http:   - route:     - destination:         host: reviews.prod.svc.cluster.local         subset: v2       weight: 25     - destination:         host: reviews.prod.svc.cluster.local         subset: v1       weight: 75 ``` {{</tab>}} \n {{<tab name=\"v1beta1\" category-value=\"v1beta1\">}} ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata:   name: reviews-route spec:   hosts:   - reviews.prod.svc.cluster.local   http:   - route:     - destination:         host: reviews.prod.svc.cluster.local         subset: v2       weight: 25     - destination:         host: reviews.prod.svc.cluster.local         subset: v1       weight: 75 ``` {{</tab>}} {{</tabset>}} \n And the associated DestinationRule \n {{<tabset category-name=\"example\">}} {{<tab name=\"v1alpha3\" category-value=\"v1alpha3\">}} ```yaml apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata:   name: reviews-destination spec:   host: reviews.prod.svc.cluster.local   subsets:   - name: v1     labels:       version: v1   - name: v2     labels:       version: v2 ``` {{</tab>}} \n {{<tab name=\"v1beta1\" category-value=\"v1beta1\">}} ```yaml apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata:   name: reviews-destination spec:   host: reviews.prod.svc.cluster.local   subsets:   - name: v1     labels:       version: v1   - name: v2     labels:       version: v2 ``` {{</tab>}} {{</tabset>}} \n Traffic can also be split across two entirely different services without having to define new subsets. For example, the following rule forwards 25% of traffic to reviews.com to dev.reviews.com \n {{<tabset category-name=\"example\">}} {{<tab name=\"v1alpha3\" category-value=\"v1alpha3\">}} ```yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata:   name: reviews-route-two-domains spec:   hosts:   - reviews.com   http:   - route:     - destination:         host: dev.reviews.com       weight: 25     - destination:         host: reviews.com       weight: 75 ``` {{</tab>}} \n {{<tab name=\"v1beta1\" category-value=\"v1beta1\">}} ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata:   name: reviews-route-two-domains spec:   hosts:   - reviews.com   http:   - route:     - destination:         host: dev.reviews.com       weight: 25     - destination:         host: reviews.com       weight: 75 ``` {{</tab>}} {{</tabset>}}"
                                 properties:
                                   destination:
-                                    description: Destination uniquely identifies the instances of a service to which the request/connection should be forwarded to.
                                     properties:
                                       host:
-                                        description: "The name of a service from the service registry. Service names are looked up from the platform's service registry (e.g., Kubernetes services, Consul services, etc.) and from the hosts declared by [ServiceEntry](https://istio.io/docs/reference/config/networking/service-entry/#ServiceEntry). Traffic forwarded to destinations that are not found in either of the two, will be dropped. \n *Note for Kubernetes users*: When short names are used (e.g. \"reviews\" instead of \"reviews.default.svc.cluster.local\"), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the \"default\" namespace containing a host \"reviews will be interpreted as \"reviews.default.svc.cluster.local\", irrespective of the actual namespace associated with the reviews service. To avoid potential misconfiguration, it is recommended to always use fully qualified domain names over short names."
+                                        description: The name of a service from the service
+                                          registry.
+                                        format: string
                                         type: string
                                       port:
-                                        description: Specifies the port on the host that is being addressed. If a service exposes only a single port it is not required to explicitly select the port.
+                                        description: Specifies the port on the host that is
+                                          being addressed.
                                         properties:
                                           number:
-                                            description: Valid port number
-                                            format: int32
                                             type: integer
                                         type: object
                                       subset:
-                                        description: The name of a subset within the service. Applicable only to services within the mesh. The subset must be defined in a corresponding DestinationRule.
+                                        description: The name of a subset within the service.
+                                        format: string
                                         type: string
                                     type: object
                                   headers:
-                                    description: Header manipulation rules
                                     properties:
                                       request:
-                                        description: Header manipulation rules to apply before forwarding a request to the destination service
                                         properties:
                                           add:
                                             additionalProperties:
+                                              format: string
                                               type: string
-                                            description: Append the given values to the headers specified by keys (will create a comma-separated list of values)
                                             type: object
                                           remove:
-                                            description: Remove a the specified headers
                                             items:
+                                              format: string
                                               type: string
                                             type: array
                                           set:
                                             additionalProperties:
+                                              format: string
                                               type: string
-                                            description: Overwrite the headers specified by key with the given values
                                             type: object
                                         type: object
                                       response:
-                                        description: Header manipulation rules to apply before returning a response to the caller
                                         properties:
                                           add:
                                             additionalProperties:
+                                              format: string
                                               type: string
-                                            description: Append the given values to the headers specified by keys (will create a comma-separated list of values)
                                             type: object
                                           remove:
-                                            description: Remove a the specified headers
                                             items:
+                                              format: string
                                               type: string
                                             type: array
                                           set:
                                             additionalProperties:
+                                              format: string
                                               type: string
-                                            description: Overwrite the headers specified by key with the given values
                                             type: object
                                         type: object
                                     type: object
                                   weight:
-                                    description: The proportion of traffic to be forwarded to the service version. (0-100). Sum of weights across destinations SHOULD BE == 100. If there is only one destination in a rule, the weight value is assumed to be 100.
                                     format: int32
                                     type: integer
                                 type: object
                               type: array
                             timeout:
-                              description: Timeout for HTTP requests.
-                              properties:
-                                nanos:
-                                  description: Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 `seconds` field and a positive or negative `nanos` field. For durations of one second or more, a non-zero value for the `nanos` field must be of the same sign as the `seconds` field. Must be from -999,999,999 to +999,999,999 inclusive.
-                                  format: int32
-                                  type: integer
-                                seconds:
-                                  description: 'Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years'
-                                  format: int64
-                                  type: integer
-                              type: object
+                              description: Timeout for HTTP requests, default is disabled.
+                              type: string
                           type: object
                         type: array
                       tcp:
-                        description: An ordered list of route rules for opaque TCP traffic. TCP routes will be applied to any port that is not a HTTP or TLS port. The first rule matching an incoming request is used.
+                        description: An ordered list of route rules for opaque TCP traffic.
                         items:
-                          description: "Describes match conditions and actions for routing TCP traffic. The following routing rule forwards traffic arriving at port 27017 for mongo.prod.svc.cluster.local to another Mongo server on port 5555. \n {{<tabset category-name=\"example\">}} {{<tab name=\"v1alpha3\" category-value=\"v1alpha3\">}} ```yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata:   name: bookinfo-Mongo spec:   hosts:   - mongo.prod.svc.cluster.local   tcp:   - match:     - port: 27017     route:     - destination:         host: mongo.backup.svc.cluster.local         port:           number: 5555 ``` {{</tab>}} \n {{<tab name=\"v1beta1\" category-value=\"v1beta1\">}} ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata:   name: bookinfo-Mongo spec:   hosts:   - mongo.prod.svc.cluster.local   tcp:   - match:     - port: 27017     route:     - destination:         host: mongo.backup.svc.cluster.local         port:           number: 5555 ``` {{</tab>}} {{</tabset>}}"
                           properties:
                             match:
-                              description: Match conditions to be satisfied for the rule to be activated. All conditions inside a single match block have AND semantics, while the list of match blocks have OR semantics. The rule is matched if any one of the match blocks succeed.
                               items:
-                                description: L4 connection match attributes. Note that L4 connection matching support is incomplete.
                                 properties:
-                                  destination_subnets:
-                                    description: IPv4 or IPv6 ip addresses of destination with optional subnet.  E.g., a.b.c.d/xx form or just a.b.c.d.
+                                  destinationSubnets:
+                                    description: IPv4 or IPv6 ip addresses of destination
+                                      with optional subnet.
                                     items:
+                                      format: string
                                       type: string
                                     type: array
                                   gateways:
-                                    description: Names of gateways where the rule should be applied. Gateway names in the top-level `gateways` field of the VirtualService (if any) are overridden. The gateway match is independent of sourceLabels.
+                                    description: Names of gateways where the rule should be
+                                      applied.
                                     items:
+                                      format: string
                                       type: string
                                     type: array
                                   port:
-                                    description: Specifies the port on the host that is being addressed. Many services only expose a single port or label ports with the protocols they support, in these cases it is not required to explicitly select the port.
-                                    format: int32
+                                    description: Specifies the port on the host that is being
+                                      addressed.
                                     type: integer
-                                  source_labels:
+                                  sourceLabels:
                                     additionalProperties:
+                                      format: string
                                       type: string
-                                    description: One or more labels that constrain the applicability of a rule to workloads with the given labels. If the VirtualService has a list of gateways specified in the top-level `gateways` field, it should include the reserved gateway `mesh` in order for this field to be applicable.
                                     type: object
-                                  source_namespace:
-                                    description: Source namespace constraining the applicability of a rule to workloads in that namespace. If the VirtualService has a list of gateways specified in the top-level `gateways` field, it must include the reserved gateway `mesh` for this field to be applicable.
+                                  sourceNamespace:
+                                    description: Source namespace constraining the applicability
+                                      of a rule to workloads in that namespace.
+                                    format: string
                                     type: string
-                                  source_subnet:
-                                    description: IPv4 or IPv6 ip address of source with optional subnet. E.g., a.b.c.d/xx form or just a.b.c.d $hide_from_docs
+                                  sourceSubnet:
+                                    description: IPv4 or IPv6 ip address of source with optional
+                                      subnet.
+                                    format: string
                                     type: string
                                 type: object
                               type: array
                             route:
-                              description: The destination to which the connection should be forwarded to.
+                              description: The destination to which the connection should
+                                be forwarded to.
                               items:
-                                description: L4 routing rule weighted destination.
                                 properties:
                                   destination:
-                                    description: Destination uniquely identifies the instances of a service to which the request/connection should be forwarded to.
                                     properties:
                                       host:
-                                        description: "The name of a service from the service registry. Service names are looked up from the platform's service registry (e.g., Kubernetes services, Consul services, etc.) and from the hosts declared by [ServiceEntry](https://istio.io/docs/reference/config/networking/service-entry/#ServiceEntry). Traffic forwarded to destinations that are not found in either of the two, will be dropped. \n *Note for Kubernetes users*: When short names are used (e.g. \"reviews\" instead of \"reviews.default.svc.cluster.local\"), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the \"default\" namespace containing a host \"reviews will be interpreted as \"reviews.default.svc.cluster.local\", irrespective of the actual namespace associated with the reviews service. To avoid potential misconfiguration, it is recommended to always use fully qualified domain names over short names."
+                                        description: The name of a service from the service
+                                          registry.
+                                        format: string
                                         type: string
                                       port:
-                                        description: Specifies the port on the host that is being addressed. If a service exposes only a single port it is not required to explicitly select the port.
+                                        description: Specifies the port on the host that is
+                                          being addressed.
                                         properties:
                                           number:
-                                            description: Valid port number
-                                            format: int32
                                             type: integer
                                         type: object
                                       subset:
-                                        description: The name of a subset within the service. Applicable only to services within the mesh. The subset must be defined in a corresponding DestinationRule.
+                                        description: The name of a subset within the service.
+                                        format: string
                                         type: string
                                     type: object
                                   weight:
-                                    description: The proportion of traffic to be forwarded to the service version. If there is only one destination in a rule, all traffic will be routed to it irrespective of the weight.
                                     format: int32
                                     type: integer
                                 type: object
@@ -545,69 +782,72 @@ spec:
                           type: object
                         type: array
                       tls:
-                        description: 'An ordered list of route rule for non-terminated TLS & HTTPS traffic. Routing is typically performed using the SNI value presented by the ClientHello message. TLS routes will be applied to platform service ports named ''https-*'', ''tls-*'', unterminated gateway ports using HTTPS/TLS protocols (i.e. with "passthrough" TLS mode) and service entry ports using HTTPS/TLS protocols.  The first rule matching an incoming request is used.  NOTE: Traffic ''https-*'' or ''tls-*'' ports without associated virtual service will be treated as opaque TCP traffic.'
                         items:
-                          description: "Describes match conditions and actions for routing unterminated TLS traffic (TLS/HTTPS) The following routing rule forwards unterminated TLS traffic arriving at port 443 of gateway called \"mygateway\" to internal services in the mesh based on the SNI value. \n {{<tabset category-name=\"example\">}} {{<tab name=\"v1alpha3\" category-value=\"v1alpha3\">}} ```yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata:   name: bookinfo-sni spec:   hosts:   - \"*.bookinfo.com\"   gateways:   - mygateway   tls:   - match:     - port: 443       sniHosts:       - login.bookinfo.com     route:     - destination:         host: login.prod.svc.cluster.local   - match:     - port: 443       sniHosts:       - reviews.bookinfo.com     route:     - destination:         host: reviews.prod.svc.cluster.local ``` {{</tab>}} \n {{<tab name=\"v1beta1\" category-value=\"v1beta1\">}} ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata:   name: bookinfo-sni spec:   hosts:   - \"*.bookinfo.com\"   gateways:   - mygateway   tls:   - match:     - port: 443       sniHosts:       - login.bookinfo.com     route:     - destination:         host: login.prod.svc.cluster.local   - match:     - port: 443       sniHosts:       - reviews.bookinfo.com     route:     - destination:         host: reviews.prod.svc.cluster.local ``` {{</tab>}} {{</tabset>}}"
                           properties:
                             match:
-                              description: Match conditions to be satisfied for the rule to be activated. All conditions inside a single match block have AND semantics, while the list of match blocks have OR semantics. The rule is matched if any one of the match blocks succeed.
                               items:
-                                description: TLS connection match attributes.
                                 properties:
-                                  destination_subnets:
-                                    description: IPv4 or IPv6 ip addresses of destination with optional subnet.  E.g., a.b.c.d/xx form or just a.b.c.d.
+                                  destinationSubnets:
+                                    description: IPv4 or IPv6 ip addresses of destination
+                                      with optional subnet.
                                     items:
+                                      format: string
                                       type: string
                                     type: array
                                   gateways:
-                                    description: Names of gateways where the rule should be applied. Gateway names in the top-level `gateways` field of the VirtualService (if any) are overridden. The gateway match is independent of sourceLabels.
+                                    description: Names of gateways where the rule should be
+                                      applied.
                                     items:
+                                      format: string
                                       type: string
                                     type: array
                                   port:
-                                    description: Specifies the port on the host that is being addressed. Many services only expose a single port or label ports with the protocols they support, in these cases it is not required to explicitly select the port.
-                                    format: int32
+                                    description: Specifies the port on the host that is being
+                                      addressed.
                                     type: integer
-                                  sni_hosts:
-                                    description: SNI (server name indicator) to match on. Wildcard prefixes can be used in the SNI value, e.g., *.com will match foo.example.com as well as example.com. An SNI value must be a subset (i.e., fall within the domain) of the corresponding virtual serivce's hosts.
+                                  sniHosts:
+                                    description: SNI (server name indicator) to match on.
                                     items:
+                                      format: string
                                       type: string
                                     type: array
-                                  source_labels:
+                                  sourceLabels:
                                     additionalProperties:
+                                      format: string
                                       type: string
-                                    description: One or more labels that constrain the applicability of a rule to workloads with the given labels. If the VirtualService has a list of gateways specified in the top-level `gateways` field, it should include the reserved gateway `mesh` in order for this field to be applicable.
                                     type: object
-                                  source_namespace:
-                                    description: Source namespace constraining the applicability of a rule to workloads in that namespace. If the VirtualService has a list of gateways specified in the top-level `gateways` field, it must include the reserved gateway `mesh` for this field to be applicable.
+                                  sourceNamespace:
+                                    description: Source namespace constraining the applicability
+                                      of a rule to workloads in that namespace.
+                                    format: string
                                     type: string
                                 type: object
                               type: array
                             route:
-                              description: The destination to which the connection should be forwarded to.
+                              description: The destination to which the connection should
+                                be forwarded to.
                               items:
-                                description: L4 routing rule weighted destination.
                                 properties:
                                   destination:
-                                    description: Destination uniquely identifies the instances of a service to which the request/connection should be forwarded to.
                                     properties:
                                       host:
-                                        description: "The name of a service from the service registry. Service names are looked up from the platform's service registry (e.g., Kubernetes services, Consul services, etc.) and from the hosts declared by [ServiceEntry](https://istio.io/docs/reference/config/networking/service-entry/#ServiceEntry). Traffic forwarded to destinations that are not found in either of the two, will be dropped. \n *Note for Kubernetes users*: When short names are used (e.g. \"reviews\" instead of \"reviews.default.svc.cluster.local\"), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the \"default\" namespace containing a host \"reviews will be interpreted as \"reviews.default.svc.cluster.local\", irrespective of the actual namespace associated with the reviews service. To avoid potential misconfiguration, it is recommended to always use fully qualified domain names over short names."
+                                        description: The name of a service from the service
+                                          registry.
+                                        format: string
                                         type: string
                                       port:
-                                        description: Specifies the port on the host that is being addressed. If a service exposes only a single port it is not required to explicitly select the port.
+                                        description: Specifies the port on the host that is
+                                          being addressed.
                                         properties:
                                           number:
-                                            description: Valid port number
-                                            format: int32
                                             type: integer
                                         type: object
                                       subset:
-                                        description: The name of a subset within the service. Applicable only to services within the mesh. The subset must be defined in a corresponding DestinationRule.
+                                        description: The name of a subset within the service.
+                                        format: string
                                         type: string
                                     type: object
                                   weight:
-                                    description: The proportion of traffic to be forwarded to the service version. If there is only one destination in a rule, all traffic will be routed to it irrespective of the weight.
                                     format: int32
                                     type: integer
                                 type: object

config/crds/snapshot.storage.k8s.io_volumesnapshot.yaml

@@ -1,503 +0,0 @@
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.5
-    api-approved.kubernetes.io: "https://github.com/kubernetes-csi/external-snapshotter/pull/260"
-  creationTimestamp: null
-  name: volumesnapshotclasses.snapshot.storage.k8s.io
-spec:
-  additionalPrinterColumns:
-    - JSONPath: .driver
-      name: Driver
-      type: string
-    - JSONPath: .deletionPolicy
-      description: Determines whether a VolumeSnapshotContent created through the VolumeSnapshotClass
-        should be deleted when its bound VolumeSnapshot is deleted.
-      name: DeletionPolicy
-      type: string
-    - JSONPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-  group: snapshot.storage.k8s.io
-  names:
-    kind: VolumeSnapshotClass
-    listKind: VolumeSnapshotClassList
-    plural: volumesnapshotclasses
-    singular: volumesnapshotclass
-  preserveUnknownFields: false
-  scope: Cluster
-  subresources: {}
-  validation:
-    openAPIV3Schema:
-      description: VolumeSnapshotClass specifies parameters that a underlying storage
-        system uses when creating a volume snapshot. A specific VolumeSnapshotClass
-        is used by specifying its name in a VolumeSnapshot object. VolumeSnapshotClasses
-        are non-namespaced
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        deletionPolicy:
-          description: deletionPolicy determines whether a VolumeSnapshotContent created
-            through the VolumeSnapshotClass should be deleted when its bound VolumeSnapshot
-            is deleted. Supported values are "Retain" and "Delete". "Retain" means
-            that the VolumeSnapshotContent and its physical snapshot on underlying
-            storage system are kept. "Delete" means that the VolumeSnapshotContent
-            and its physical snapshot on underlying storage system are deleted. Required.
-          enum:
-            - Delete
-            - Retain
-          type: string
-        driver:
-          description: driver is the name of the storage driver that handles this
-            VolumeSnapshotClass. Required.
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        parameters:
-          additionalProperties:
-            type: string
-          description: parameters is a key-value map with storage driver specific
-            parameters for creating snapshots. These values are opaque to Kubernetes.
-          type: object
-      required:
-        - deletionPolicy
-        - driver
-      type: object
-  version: v1beta1
-  versions:
-    - name: v1beta1
-      served: true
-      storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.5
-    api-approved.kubernetes.io: "https://github.com/kubernetes-csi/external-snapshotter/pull/260"
-  creationTimestamp: null
-  name: volumesnapshotcontents.snapshot.storage.k8s.io
-spec:
-  additionalPrinterColumns:
-    - JSONPath: .status.readyToUse
-      description: Indicates if a snapshot is ready to be used to restore a volume.
-      name: ReadyToUse
-      type: boolean
-    - JSONPath: .status.restoreSize
-      description: Represents the complete size of the snapshot in bytes
-      name: RestoreSize
-      type: integer
-    - JSONPath: .spec.deletionPolicy
-      description: Determines whether this VolumeSnapshotContent and its physical snapshot
-        on the underlying storage system should be deleted when its bound VolumeSnapshot
-        is deleted.
-      name: DeletionPolicy
-      type: string
-    - JSONPath: .spec.driver
-      description: Name of the CSI driver used to create the physical snapshot on the
-        underlying storage system.
-      name: Driver
-      type: string
-    - JSONPath: .spec.volumeSnapshotClassName
-      description: Name of the VolumeSnapshotClass to which this snapshot belongs.
-      name: VolumeSnapshotClass
-      type: string
-    - JSONPath: .spec.volumeSnapshotRef.name
-      description: Name of the VolumeSnapshot object to which this VolumeSnapshotContent
-        object is bound.
-      name: VolumeSnapshot
-      type: string
-    - JSONPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-  group: snapshot.storage.k8s.io
-  names:
-    kind: VolumeSnapshotContent
-    listKind: VolumeSnapshotContentList
-    plural: volumesnapshotcontents
-    singular: volumesnapshotcontent
-  preserveUnknownFields: false
-  scope: Cluster
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      description: VolumeSnapshotContent represents the actual "on-disk" snapshot
-        object in the underlying storage system
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        spec:
-          description: spec defines properties of a VolumeSnapshotContent created
-            by the underlying storage system. Required.
-          properties:
-            deletionPolicy:
-              description: deletionPolicy determines whether this VolumeSnapshotContent
-                and its physical snapshot on the underlying storage system should
-                be deleted when its bound VolumeSnapshot is deleted. Supported values
-                are "Retain" and "Delete". "Retain" means that the VolumeSnapshotContent
-                and its physical snapshot on underlying storage system are kept. "Delete"
-                means that the VolumeSnapshotContent and its physical snapshot on
-                underlying storage system are deleted. In dynamic snapshot creation
-                case, this field will be filled in with the "DeletionPolicy" field
-                defined in the VolumeSnapshotClass the VolumeSnapshot refers to. For
-                pre-existing snapshots, users MUST specify this field when creating
-                the VolumeSnapshotContent object. Required.
-              enum:
-                - Delete
-                - Retain
-              type: string
-            driver:
-              description: driver is the name of the CSI driver used to create the
-                physical snapshot on the underlying storage system. This MUST be the
-                same as the name returned by the CSI GetPluginName() call for that
-                driver. Required.
-              type: string
-            source:
-              description: source specifies from where a snapshot will be created.
-                This field is immutable after creation. Required.
-              properties:
-                snapshotHandle:
-                  description: snapshotHandle specifies the CSI "snapshot_id" of a
-                    pre-existing snapshot on the underlying storage system. This field
-                    is immutable.
-                  type: string
-                volumeHandle:
-                  description: volumeHandle specifies the CSI "volume_id" of the volume
-                    from which a snapshot should be dynamically taken from. This field
-                    is immutable.
-                  type: string
-              type: object
-            volumeSnapshotClassName:
-              description: name of the VolumeSnapshotClass to which this snapshot
-                belongs.
-              type: string
-            volumeSnapshotRef:
-              description: volumeSnapshotRef specifies the VolumeSnapshot object to
-                which this VolumeSnapshotContent object is bound. VolumeSnapshot.Spec.VolumeSnapshotContentName
-                field must reference to this VolumeSnapshotContent's name for the
-                bidirectional binding to be valid. For a pre-existing VolumeSnapshotContent
-                object, name and namespace of the VolumeSnapshot object MUST be provided
-                for binding to happen. This field is immutable after creation. Required.
-              properties:
-                apiVersion:
-                  description: API version of the referent.
-                  type: string
-                fieldPath:
-                  description: 'If referring to a piece of an object instead of an
-                    entire object, this string should contain a valid JSON/Go field
-                    access statement, such as desiredState.manifest.containers[2].
-                    For example, if the object reference is to a container within
-                    a pod, this would take on a value like: "spec.containers{name}"
-                    (where "name" refers to the name of the container that triggered
-                    the event) or if no container name is specified "spec.containers[2]"
-                    (container with index 2 in this pod). This syntax is chosen only
-                    to have some well-defined way of referencing a part of an object.
-                    TODO: this design is not final and this field is subject to change
-                    in the future.'
-                  type: string
-                kind:
-                  description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                  type: string
-                name:
-                  description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                  type: string
-                namespace:
-                  description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                  type: string
-                resourceVersion:
-                  description: 'Specific resourceVersion to which this reference is
-                    made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                  type: string
-                uid:
-                  description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                  type: string
-              type: object
-          required:
-            - deletionPolicy
-            - driver
-            - source
-            - volumeSnapshotRef
-          type: object
-        status:
-          description: status represents the current information of a snapshot.
-          properties:
-            creationTime:
-              description: creationTime is the timestamp when the point-in-time snapshot
-                is taken by the underlying storage system. In dynamic snapshot creation
-                case, this field will be filled in with the "creation_time" value
-                returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing
-                snapshot, this field will be filled with the "creation_time" value
-                returned from the CSI "ListSnapshots" gRPC call if the driver supports
-                it. If not specified, it indicates the creation time is unknown. The
-                format of this field is a Unix nanoseconds time encoded as an int64.
-                On Unix, the command `date +%s%N` returns the current time in nanoseconds
-                since 1970-01-01 00:00:00 UTC.
-              format: int64
-              type: integer
-            error:
-              description: error is the latest observed error during snapshot creation,
-                if any.
-              properties:
-                message:
-                  description: 'message is a string detailing the encountered error
-                    during snapshot creation if specified. NOTE: message may be logged,
-                    and it should not contain sensitive information.'
-                  type: string
-                time:
-                  description: time is the timestamp when the error was encountered.
-                  format: date-time
-                  type: string
-              type: object
-            readyToUse:
-              description: readyToUse indicates if a snapshot is ready to be used
-                to restore a volume. In dynamic snapshot creation case, this field
-                will be filled in with the "ready_to_use" value returned from CSI
-                "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this
-                field will be filled with the "ready_to_use" value returned from the
-                CSI "ListSnapshots" gRPC call if the driver supports it, otherwise,
-                this field will be set to "True". If not specified, it means the readiness
-                of a snapshot is unknown.
-              type: boolean
-            restoreSize:
-              description: restoreSize represents the complete size of the snapshot
-                in bytes. In dynamic snapshot creation case, this field will be filled
-                in with the "size_bytes" value returned from CSI "CreateSnapshotRequest"
-                gRPC call. For a pre-existing snapshot, this field will be filled
-                with the "size_bytes" value returned from the CSI "ListSnapshots"
-                gRPC call if the driver supports it. When restoring a volume from
-                this snapshot, the size of the volume MUST NOT be smaller than the
-                restoreSize if it is specified, otherwise the restoration will fail.
-                If not specified, it indicates that the size is unknown.
-              format: int64
-              minimum: 0
-              type: integer
-            snapshotHandle:
-              description: snapshotHandle is the CSI "snapshot_id" of a snapshot on
-                the underlying storage system. If not specified, it indicates that
-                dynamic snapshot creation has either failed or it is still in progress.
-              type: string
-          type: object
-      required:
-        - spec
-      type: object
-  version: v1beta1
-  versions:
-    - name: v1beta1
-      served: true
-      storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.5
-    api-approved.kubernetes.io: "https://github.com/kubernetes-csi/external-snapshotter/pull/260"
-  creationTimestamp: null
-  name: volumesnapshots.snapshot.storage.k8s.io
-spec:
-  additionalPrinterColumns:
-    - JSONPath: .status.readyToUse
-      description: Indicates if a snapshot is ready to be used to restore a volume.
-      name: ReadyToUse
-      type: boolean
-    - JSONPath: .spec.source.persistentVolumeClaimName
-      description: Name of the source PVC from where a dynamically taken snapshot will
-        be created.
-      name: SourcePVC
-      type: string
-    - JSONPath: .spec.source.volumeSnapshotContentName
-      description: Name of the VolumeSnapshotContent which represents a pre-provisioned
-        snapshot.
-      name: SourceSnapshotContent
-      type: string
-    - JSONPath: .status.restoreSize
-      description: Represents the complete size of the snapshot.
-      name: RestoreSize
-      type: string
-    - JSONPath: .spec.volumeSnapshotClassName
-      description: The name of the VolumeSnapshotClass requested by the VolumeSnapshot.
-      name: SnapshotClass
-      type: string
-    - JSONPath: .status.boundVolumeSnapshotContentName
-      description: The name of the VolumeSnapshotContent to which this VolumeSnapshot
-        is bound.
-      name: SnapshotContent
-      type: string
-    - JSONPath: .status.creationTime
-      description: Timestamp when the point-in-time snapshot is taken by the underlying
-        storage system.
-      name: CreationTime
-      type: date
-    - JSONPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-  group: snapshot.storage.k8s.io
-  names:
-    kind: VolumeSnapshot
-    listKind: VolumeSnapshotList
-    plural: volumesnapshots
-    singular: volumesnapshot
-  preserveUnknownFields: false
-  scope: Namespaced
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      description: VolumeSnapshot is a user's request for either creating a point-in-time
-        snapshot of a persistent volume, or binding to a pre-existing snapshot.
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        spec:
-          description: 'spec defines the desired characteristics of a snapshot requested
-            by a user. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshots#volumesnapshots
-            Required.'
-          properties:
-            source:
-              description: source specifies where a snapshot will be created from.
-                This field is immutable after creation. Required.
-              properties:
-                persistentVolumeClaimName:
-                  description: persistentVolumeClaimName specifies the name of the
-                    PersistentVolumeClaim object in the same namespace as the VolumeSnapshot
-                    object where the snapshot should be dynamically taken from. This
-                    field is immutable.
-                  type: string
-                volumeSnapshotContentName:
-                  description: volumeSnapshotContentName specifies the name of a pre-existing
-                    VolumeSnapshotContent object. This field is immutable.
-                  type: string
-              type: object
-            volumeSnapshotClassName:
-              description: 'volumeSnapshotClassName is the name of the VolumeSnapshotClass
-                requested by the VolumeSnapshot. If not specified, the default snapshot
-                class will be used if one exists. If not specified, and there is no
-                default snapshot class, dynamic snapshot creation will fail. Empty
-                string is not allowed for this field. TODO(xiangqian): a webhook validation
-                on empty string. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes'
-              type: string
-          required:
-            - source
-          type: object
-        status:
-          description: 'status represents the current information of a snapshot. NOTE:
-            status can be modified by sources other than system controllers, and must
-            not be depended upon for accuracy. Controllers should only use information
-            from the VolumeSnapshotContent object after verifying that the binding
-            is accurate and complete.'
-          properties:
-            boundVolumeSnapshotContentName:
-              description: 'boundVolumeSnapshotContentName represents the name of
-                the VolumeSnapshotContent object to which the VolumeSnapshot object
-                is bound. If not specified, it indicates that the VolumeSnapshot object
-                has not been successfully bound to a VolumeSnapshotContent object
-                yet. NOTE: Specified boundVolumeSnapshotContentName alone does not
-                mean binding       is valid. Controllers MUST always verify bidirectional
-                binding between       VolumeSnapshot and VolumeSnapshotContent to
-                avoid possible security issues.'
-              type: string
-            creationTime:
-              description: creationTime is the timestamp when the point-in-time snapshot
-                is taken by the underlying storage system. In dynamic snapshot creation
-                case, this field will be filled in with the "creation_time" value
-                returned from CSI "CreateSnapshotRequest" gRPC call. For a pre-existing
-                snapshot, this field will be filled with the "creation_time" value
-                returned from the CSI "ListSnapshots" gRPC call if the driver supports
-                it. If not specified, it indicates that the creation time of the snapshot
-                is unknown.
-              format: date-time
-              type: string
-            error:
-              description: error is the last observed error during snapshot creation,
-                if any. This field could be helpful to upper level controllers(i.e.,
-                application controller) to decide whether they should continue on
-                waiting for the snapshot to be created based on the type of error
-                reported.
-              properties:
-                message:
-                  description: 'message is a string detailing the encountered error
-                    during snapshot creation if specified. NOTE: message may be logged,
-                    and it should not contain sensitive information.'
-                  type: string
-                time:
-                  description: time is the timestamp when the error was encountered.
-                  format: date-time
-                  type: string
-              type: object
-            readyToUse:
-              description: readyToUse indicates if a snapshot is ready to be used
-                to restore a volume. In dynamic snapshot creation case, this field
-                will be filled in with the "ready_to_use" value returned from CSI
-                "CreateSnapshotRequest" gRPC call. For a pre-existing snapshot, this
-                field will be filled with the "ready_to_use" value returned from the
-                CSI "ListSnapshots" gRPC call if the driver supports it, otherwise,
-                this field will be set to "True". If not specified, it means the readiness
-                of a snapshot is unknown.
-              type: boolean
-            restoreSize:
-              anyOf:
-                - type: integer
-                - type: string
-              description: restoreSize represents the complete size of the snapshot
-                in bytes. In dynamic snapshot creation case, this field will be filled
-                in with the "size_bytes" value returned from CSI "CreateSnapshotRequest"
-                gRPC call. For a pre-existing snapshot, this field will be filled
-                with the "size_bytes" value returned from the CSI "ListSnapshots"
-                gRPC call if the driver supports it. When restoring a volume from
-                this snapshot, the size of the volume MUST NOT be smaller than the
-                restoreSize if it is specified, otherwise the restoration will fail.
-                If not specified, it indicates that the size is unknown.
-              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-              x-kubernetes-int-or-string: true
-          type: object
-      required:
-        - spec
-      type: object
-  version: v1beta1
-  versions:
-    - name: v1beta1
-      served: true
-      storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/storage.kubesphere.io_storageclasseraccessor.yaml

@@ -0,0 +1,180 @@
+---
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: v0.4.1
+  creationTimestamp: null
+  name: accessors.storage.kubesphere.io
+spec:
+  group: storage.kubesphere.io
+  names:
+    kind: Accessor
+    listKind: AccessorList
+    plural: accessors
+    singular: accessor
+  scope: Cluster
+  versions:
+    - additionalPrinterColumns:
+        - jsonPath: .spec.storageClassName
+          name: StorageClass
+          type: string
+        - jsonPath: .metadata.creationTimestamp
+          name: Age
+          type: date
+      name: v1alpha1
+      schema:
+        openAPIV3Schema:
+          description: Accessor is the Schema for the accessors API
+          properties:
+            apiVersion:
+              description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+              type: string
+            kind:
+              description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+              type: string
+            metadata:
+              type: object
+            spec:
+              description: AccessorSpec defines the desired state of Accessor
+              properties:
+                namespaceSelector:
+                  properties:
+                    fieldSelector:
+                      items:
+                        properties:
+                          fieldExpressions:
+                            items:
+                              properties:
+                                field:
+                                  enum:
+                                    - Name
+                                    - Status
+                                  type: string
+                                operator:
+                                  enum:
+                                    - In
+                                    - NotIn
+                                  type: string
+                                values:
+                                  items:
+                                    type: string
+                                  type: array
+                              required:
+                                - field
+                                - operator
+                                - values
+                              type: object
+                            type: array
+                        required:
+                          - fieldExpressions
+                        type: object
+                      type: array
+                    labelSelector:
+                      items:
+                        properties:
+                          matchExpressions:
+                            items:
+                              properties:
+                                key:
+                                  type: string
+                                operator:
+                                  enum:
+                                    - In
+                                    - NotIn
+                                  type: string
+                                values:
+                                  items:
+                                    type: string
+                                  type: array
+                              required:
+                                - key
+                                - operator
+                                - values
+                              type: object
+                            type: array
+                        required:
+                          - matchExpressions
+                        type: object
+                      type: array
+                  type: object
+                storageClassName:
+                  type: string
+                workspaceSelector:
+                  properties:
+                    fieldSelector:
+                      items:
+                        properties:
+                          fieldExpressions:
+                            items:
+                              properties:
+                                field:
+                                  enum:
+                                    - Name
+                                    - Status
+                                  type: string
+                                operator:
+                                  enum:
+                                    - In
+                                    - NotIn
+                                  type: string
+                                values:
+                                  items:
+                                    type: string
+                                  type: array
+                              required:
+                                - field
+                                - operator
+                                - values
+                              type: object
+                            type: array
+                        required:
+                          - fieldExpressions
+                        type: object
+                      type: array
+                    labelSelector:
+                      items:
+                        properties:
+                          matchExpressions:
+                            items:
+                              properties:
+                                key:
+                                  type: string
+                                operator:
+                                  enum:
+                                    - In
+                                    - NotIn
+                                  type: string
+                                values:
+                                  items:
+                                    type: string
+                                  type: array
+                              required:
+                                - key
+                                - operator
+                                - values
+                              type: object
+                            type: array
+                        required:
+                          - matchExpressions
+                        type: object
+                      type: array
+                  type: object
+              required:
+                - storageClassName
+              type: object
+          type: object
+      served: true
+      storage: true
+      subresources:
+        status: {}
+status:
+  acceptedNames:
+    kind: ""
+    plural: ""
+  conditions: []
+  storedVersions: []

config/crds/tenant.kubesphere.io_workspaces.yaml

@@ -24,10 +24,14 @@ spec:
         description: Workspace is the Schema for the workspaces API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object

config/crds/tenant.kubesphere.io_workspacetemplates.yaml

@@ -24,10 +24,14 @@ spec:
         description: WorkspaceTemplate is the Schema for the workspacetemplates API
         properties:
           apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
             type: string
           kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
             type: string
           metadata:
             type: object
@@ -47,6 +51,7 @@ spec:
                             type: string
                           value:
                             type: object
+                            x-kubernetes-preserve-unknown-fields: true
                         required:
                         - path
                         type: object
@@ -58,21 +63,34 @@ spec:
               placement:
                 properties:
                   clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
+                    description: A label selector is a label query over a set of resources.
+                      The result of matchLabels and matchExpressions are ANDed. An
+                      empty label selector matches all objects. A null label selector
+                      matches no objects.
                     properties:
                       matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
+                        description: matchExpressions is a list of label selector
+                          requirements. The requirements are ANDed.
                         items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
+                          description: A label selector requirement is a selector
+                            that contains values, a key, and an operator that relates
+                            the key and values.
                           properties:
                             key:
-                              description: key is the label key that the selector applies to.
+                              description: key is the label key that the selector
+                                applies to.
                               type: string
                             operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
+                              description: operator represents a key's relationship
+                                to a set of values. Valid operators are In, NotIn,
+                                Exists and DoesNotExist.
                               type: string
                             values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
+                              description: values is an array of string values. If
+                                the operator is In or NotIn, the values array must
+                                be non-empty. If the operator is Exists or DoesNotExist,
+                                the values array must be empty. This array is replaced
+                                during a strategic merge patch.
                               items:
                                 type: string
                               type: array
@@ -84,7 +102,11 @@ spec:
                       matchLabels:
                         additionalProperties:
                           type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
+                        description: matchLabels is a map of {key,value} pairs. A
+                          single {key,value} in the matchLabels map is equivalent
+                          to an element of matchExpressions, whose key field is "key",
+                          the operator is "In", and the values array contains only
+                          "value". The requirements are ANDed.
                         type: object
                     type: object
                   clusters:

config/crds/types.kubefed.io_federatedapplications.yaml

@@ -1,440 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedapplications.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedApplication
-    listKind: FederatedApplicationList
-    plural: federatedapplications
-    singular: federatedapplication
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: ApplicationSpec defines the specification for an Application.
-                    properties:
-                      addOwnerRef:
-                        description: AddOwnerRef objects - flag to indicate if we need to add OwnerRefs to matching objects Matching is done by using Selector to query all ComponentGroupKinds
-                        type: boolean
-                      assemblyPhase:
-                        description: AssemblyPhase represents the current phase of the application's assembly. An empty value is equivalent to "Succeeded".
-                        type: string
-                      componentKinds:
-                        description: ComponentGroupKinds is a list of Kinds for Application's components (e.g. Deployments, Pods, Services, CRDs). It can be used in conjunction with the Application's Selector to list or watch the Applications components.
-                        items:
-                          description: GroupKind specifies a Group and a Kind, but does not force a version.  This is useful for identifying concepts during lookup stages without having partially valid types
-                          properties:
-                            group:
-                              type: string
-                            kind:
-                              type: string
-                          required:
-                          - group
-                          - kind
-                          type: object
-                        type: array
-                      descriptor:
-                        description: Descriptor regroups information and metadata about an application.
-                        properties:
-                          description:
-                            description: Description is a brief string description of the Application.
-                            type: string
-                          icons:
-                            description: Icons is an optional list of icons for an application. Icon information includes the source, size, and mime type.
-                            items:
-                              description: ImageSpec contains information about an image used as an icon.
-                              properties:
-                                size:
-                                  description: (optional) The size of the image in pixels (e.g., 25x25).
-                                  type: string
-                                src:
-                                  description: The source for image represented as either an absolute URL to the image or a Data URL containing the image. Data URLs are defined in RFC 2397.
-                                  type: string
-                                type:
-                                  description: (optional) The mine type of the image (e.g., "image/png").
-                                  type: string
-                              required:
-                              - src
-                              type: object
-                            type: array
-                          keywords:
-                            description: Keywords is an optional list of key words associated with the application (e.g. MySQL, RDBMS, database).
-                            items:
-                              type: string
-                            type: array
-                          links:
-                            description: Links are a list of descriptive URLs intended to be used to surface additional documentation, dashboards, etc.
-                            items:
-                              description: Link contains information about an URL to surface documentation, dashboards, etc.
-                              properties:
-                                description:
-                                  description: Description is human readable content explaining the purpose of the link.
-                                  type: string
-                                url:
-                                  description: Url typically points at a website address.
-                                  type: string
-                              type: object
-                            type: array
-                          maintainers:
-                            description: Maintainers is an optional list of maintainers of the application. The maintainers in this list maintain the the source code, images, and package for the application.
-                            items:
-                              description: ContactData contains information about an individual or organization.
-                              properties:
-                                email:
-                                  description: Email is the email address.
-                                  type: string
-                                name:
-                                  description: Name is the descriptive name.
-                                  type: string
-                                url:
-                                  description: Url could typically be a website address.
-                                  type: string
-                              type: object
-                            type: array
-                          notes:
-                            description: Notes contain a human readable snippets intended as a quick start for the users of the Application. CommonMark markdown syntax may be used for rich text representation.
-                            type: string
-                          owners:
-                            description: Owners is an optional list of the owners of the installed application. The owners of the application should be contacted in the event of a planned or unplanned disruption affecting the application.
-                            items:
-                              description: ContactData contains information about an individual or organization.
-                              properties:
-                                email:
-                                  description: Email is the email address.
-                                  type: string
-                                name:
-                                  description: Name is the descriptive name.
-                                  type: string
-                                url:
-                                  description: Url could typically be a website address.
-                                  type: string
-                              type: object
-                            type: array
-                          type:
-                            description: Type is the type of the application (e.g. WordPress, MySQL, Cassandra).
-                            type: string
-                          version:
-                            description: Version is an optional version indicator for the Application.
-                            type: string
-                        type: object
-                      info:
-                        description: Info contains human readable key,value pairs for the Application.
-                        items:
-                          description: InfoItem is a human readable key,value pair containing important information about how to access the Application.
-                          properties:
-                            name:
-                              description: Name is a human readable title for this piece of information.
-                              type: string
-                            type:
-                              description: Type of the value for this InfoItem.
-                              type: string
-                            value:
-                              description: Value is human readable content.
-                              type: string
-                            valueFrom:
-                              description: ValueFrom defines a reference to derive the value from another source.
-                              properties:
-                                configMapKeyRef:
-                                  description: Selects a key of a ConfigMap.
-                                  properties:
-                                    apiVersion:
-                                      description: API version of the referent.
-                                      type: string
-                                    fieldPath:
-                                      description: 'If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.'
-                                      type: string
-                                    key:
-                                      description: The key to select.
-                                      type: string
-                                    kind:
-                                      description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                                      type: string
-                                    name:
-                                      description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                                      type: string
-                                    namespace:
-                                      description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                                      type: string
-                                    resourceVersion:
-                                      description: 'Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                                      type: string
-                                    uid:
-                                      description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                                      type: string
-                                  type: object
-                                ingressRef:
-                                  description: Select an Ingress.
-                                  properties:
-                                    apiVersion:
-                                      description: API version of the referent.
-                                      type: string
-                                    fieldPath:
-                                      description: 'If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.'
-                                      type: string
-                                    host:
-                                      description: The optional host to select.
-                                      type: string
-                                    kind:
-                                      description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                                      type: string
-                                    name:
-                                      description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                                      type: string
-                                    namespace:
-                                      description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                                      type: string
-                                    path:
-                                      description: The optional HTTP path.
-                                      type: string
-                                    protocol:
-                                      description: Protocol for the ingress
-                                      type: string
-                                    resourceVersion:
-                                      description: 'Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                                      type: string
-                                    uid:
-                                      description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                                      type: string
-                                  type: object
-                                secretKeyRef:
-                                  description: Selects a key of a Secret.
-                                  properties:
-                                    apiVersion:
-                                      description: API version of the referent.
-                                      type: string
-                                    fieldPath:
-                                      description: 'If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.'
-                                      type: string
-                                    key:
-                                      description: The key to select.
-                                      type: string
-                                    kind:
-                                      description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                                      type: string
-                                    name:
-                                      description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                                      type: string
-                                    namespace:
-                                      description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                                      type: string
-                                    resourceVersion:
-                                      description: 'Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                                      type: string
-                                    uid:
-                                      description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                                      type: string
-                                  type: object
-                                serviceRef:
-                                  description: Select a Service.
-                                  properties:
-                                    apiVersion:
-                                      description: API version of the referent.
-                                      type: string
-                                    fieldPath:
-                                      description: 'If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.'
-                                      type: string
-                                    kind:
-                                      description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                                      type: string
-                                    name:
-                                      description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                                      type: string
-                                    namespace:
-                                      description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                                      type: string
-                                    path:
-                                      description: The optional HTTP path.
-                                      type: string
-                                    port:
-                                      description: The optional port to select.
-                                      format: int32
-                                      type: integer
-                                    protocol:
-                                      description: Protocol for the service
-                                      type: string
-                                    resourceVersion:
-                                      description: 'Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                                      type: string
-                                    uid:
-                                      description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                                      type: string
-                                  type: object
-                                type:
-                                  description: Type of source.
-                                  type: string
-                              type: object
-                          type: object
-                        type: array
-                      selector:
-                        description: 'Selector is a label query over kinds that created by the application. It must match the component objects'' labels. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors'
-                        properties:
-                          matchExpressions:
-                            description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                            items:
-                              description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                              properties:
-                                key:
-                                  description: key is the label key that the selector applies to.
-                                  type: string
-                                operator:
-                                  description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                  type: string
-                                values:
-                                  description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                  items:
-                                    type: string
-                                  type: array
-                              required:
-                              - key
-                              - operator
-                              type: object
-                            type: array
-                          matchLabels:
-                            additionalProperties:
-                              type: string
-                            description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                            type: object
-                        type: object
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedclusterrolebindings.yaml

@@ -1,194 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedclusterrolebindings.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedClusterRoleBinding
-    listKind: FederatedClusterRoleBindingList
-    plural: federatedclusterrolebindings
-    singular: federatedclusterrolebinding
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  roleRef:
-                    description: RoleRef can only reference a ClusterRole in the global namespace. If the RoleRef cannot be resolved, the Authorizer must return an error.
-                    properties:
-                      apiGroup:
-                        description: APIGroup is the group for the resource being referenced
-                        type: string
-                      kind:
-                        description: Kind is the type of resource being referenced
-                        type: string
-                      name:
-                        description: Name is the name of resource being referenced
-                        type: string
-                    required:
-                    - apiGroup
-                    - kind
-                    - name
-                    type: object
-                  subjects:
-                    items:
-                      description: Subject contains a reference to the object or user identities a role binding applies to.  This can either hold a direct API object reference, or a value for non-objects such as user and group names.
-                      properties:
-                        apiGroup:
-                          description: APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io" for User and Group subjects.
-                          type: string
-                        kind:
-                          description: Kind of object being referenced. Values defined by this API group are "User", "Group", and "ServiceAccount". If the Authorizer does not recognized the kind value, the Authorizer should report an error.
-                          type: string
-                        name:
-                          description: Name of the object being referenced.
-                          type: string
-                        namespace:
-                          description: Namespace of the referenced object.  If the object kind is non-namespace, such as "User" or "Group", and this value is not empty the Authorizer should report an error.
-                          type: string
-                      required:
-                      - kind
-                      - name
-                      type: object
-                    type: array
-                required:
-                - roleRef
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedclusterroles.yaml

@@ -1,224 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedclusterroles.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedClusterRole
-    listKind: FederatedClusterRoleList
-    plural: federatedclusterroles
-    singular: federatedclusterrole
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  aggregationRule:
-                    description: AggregationRule describes how to locate ClusterRoles to aggregate into the ClusterRole
-                    properties:
-                      clusterRoleSelectors:
-                        description: ClusterRoleSelectors holds a list of selectors which will be used to find ClusterRoles and create the rules. If any of the selectors match, then the ClusterRole's permissions will be added
-                        items:
-                          description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                          properties:
-                            matchExpressions:
-                              description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                              items:
-                                description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                properties:
-                                  key:
-                                    description: key is the label key that the selector applies to.
-                                    type: string
-                                  operator:
-                                    description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                    type: string
-                                  values:
-                                    description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                    items:
-                                      type: string
-                                    type: array
-                                required:
-                                - key
-                                - operator
-                                type: object
-                              type: array
-                            matchLabels:
-                              additionalProperties:
-                                type: string
-                              description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                              type: object
-                          type: object
-                        type: array
-                    type: object
-                  rules:
-                    items:
-                      description: PolicyRule holds information that describes a policy rule, but does not contain information about who the rule applies to or which namespace the rule applies to.
-                      properties:
-                        apiGroups:
-                          description: APIGroups is the name of the APIGroup that contains the resources.  If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed.
-                          items:
-                            type: string
-                          type: array
-                        nonResourceURLs:
-                          description: NonResourceURLs is a set of partial urls that a user should have access to.  *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"),  but not both.
-                          items:
-                            type: string
-                          type: array
-                        resourceNames:
-                          description: ResourceNames is an optional white list of names that the rule applies to.  An empty set means that everything is allowed.
-                          items:
-                            type: string
-                          type: array
-                        resources:
-                          description: Resources is a list of resources this rule applies to.  ResourceAll represents all resources.
-                          items:
-                            type: string
-                          type: array
-                        verbs:
-                          description: Verbs is a list of Verbs that apply to ALL the ResourceKinds and AttributeRestrictions contained in this rule.  VerbAll represents all kinds.
-                          items:
-                            type: string
-                          type: array
-                      required:
-                      - verbs
-                      type: object
-                    type: array
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedconfigmaps.yaml

@@ -1,163 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedconfigmaps.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedConfigMap
-    listKind: FederatedConfigMapList
-    plural: federatedconfigmaps
-    singular: federatedconfigmap
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  binaryData:
-                    additionalProperties:
-                      format: byte
-                      type: string
-                    type: object
-                  data:
-                    additionalProperties:
-                      type: string
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedgroupbindings.yaml

@@ -1,170 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedgroupbindings.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedGroupBinding
-    listKind: FederatedGroupBindingList
-    plural: federatedgroupbindings
-    singular: federatedgroupbinding
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  groupRef:
-                    description: GroupRef defines the desired relation of GroupBinding
-                    properties:
-                      apiGroup:
-                        type: string
-                      kind:
-                        type: string
-                      name:
-                        type: string
-                    type: object
-                  metadata:
-                    type: object
-                  users:
-                    items:
-                      type: string
-                    type: array
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedgroups.yaml

@@ -1,159 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedgroups.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedGroup
-    listKind: FederatedGroupList
-    plural: federatedgroups
-    singular: federatedgroup
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                  spec:
-                    description: GroupSpec defines the desired state of Group
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedingresses.yaml

@@ -1,265 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedingresses.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedIngress
-    listKind: FederatedIngressList
-    plural: federatedingresses
-    singular: federatedingress
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: IngressSpec describes the Ingress the user wishes to exist.
-                    properties:
-                      backend:
-                        description: A default backend capable of servicing requests that don't match any rule. At least one of 'backend' or 'rules' must be specified. This field is optional to allow the loadbalancer controller or defaulting logic to specify a global default.
-                        properties:
-                          resource:
-                            description: Resource is an ObjectRef to another Kubernetes resource in the namespace of the Ingress object. If resource is specified, serviceName and servicePort must not be specified.
-                            properties:
-                              apiGroup:
-                                description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
-                                type: string
-                              kind:
-                                description: Kind is the type of resource being referenced
-                                type: string
-                              name:
-                                description: Name is the name of resource being referenced
-                                type: string
-                            required:
-                            - kind
-                            - name
-                            type: object
-                          serviceName:
-                            description: Specifies the name of the referenced service.
-                            type: string
-                          servicePort:
-                            anyOf:
-                            - type: integer
-                            - type: string
-                            description: Specifies the port of the referenced service.
-                            x-kubernetes-int-or-string: true
-                        type: object
-                      ingressClassName:
-                        description: IngressClassName is the name of the IngressClass cluster resource. The associated IngressClass defines which controller will implement the resource. This replaces the deprecated `kubernetes.io/ingress.class` annotation. For backwards compatibility, when that annotation is set, it must be given precedence over this field. The controller may emit a warning if the field and annotation have different values. Implementations of this API should ignore Ingresses without a class specified. An IngressClass resource may be marked as default, which can be used to set a default value for this field. For more information, refer to the IngressClass documentation.
-                        type: string
-                      rules:
-                        description: A list of host rules used to configure the Ingress. If unspecified, or no rule matches, all traffic is sent to the default backend.
-                        items:
-                          description: IngressRule represents the rules mapping the paths under a specified host to the related backend services. Incoming requests are first evaluated for a host match, then routed to the backend associated with the matching IngressRuleValue.
-                          properties:
-                            host:
-                              description: "Host is the fully qualified domain name of a network host, as defined by RFC 3986. Note the following deviations from the \"host\" part of the URI as defined in RFC 3986: 1. IPs are not allowed. Currently an IngressRuleValue can only apply to    the IP in the Spec of the parent Ingress. 2. The `:` delimiter is not respected because ports are not allowed. \t  Currently the port of an Ingress is implicitly :80 for http and \t  :443 for https. Both these may change in the future. Incoming requests are matched against the host before the IngressRuleValue. If the host is unspecified, the Ingress routes all traffic based on the specified IngressRuleValue. \n Host can be \"precise\" which is a domain name without the terminating dot of a network host (e.g. \"foo.bar.com\") or \"wildcard\", which is a domain name prefixed with a single wildcard label (e.g. \"*.foo.com\"). The wildcard character '*' must appear by itself as the first DNS label and matches only a single label. You cannot have a wildcard label by itself (e.g. Host == \"*\"). Requests will be matched against the Host field in the following way: 1. If Host is precise, the request matches this rule if the http host header is equal to Host. 2. If Host is a wildcard, then the request matches this rule if the http host header is to equal to the suffix (removing the first label) of the wildcard rule."
-                              type: string
-                            http:
-                              description: 'HTTPIngressRuleValue is a list of http selectors pointing to backends. In the example: http://<host>/<path>?<searchpart> -> backend where where parts of the url correspond to RFC 3986, this resource will be used to match against everything after the last ''/'' and before the first ''?'' or ''#''.'
-                              properties:
-                                paths:
-                                  description: A collection of paths that map requests to backends.
-                                  items:
-                                    description: HTTPIngressPath associates a path with a backend. Incoming urls matching the path are forwarded to the backend.
-                                    properties:
-                                      backend:
-                                        description: Backend defines the referenced service endpoint to which the traffic will be forwarded to.
-                                        properties:
-                                          resource:
-                                            description: Resource is an ObjectRef to another Kubernetes resource in the namespace of the Ingress object. If resource is specified, serviceName and servicePort must not be specified.
-                                            properties:
-                                              apiGroup:
-                                                description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
-                                                type: string
-                                              kind:
-                                                description: Kind is the type of resource being referenced
-                                                type: string
-                                              name:
-                                                description: Name is the name of resource being referenced
-                                                type: string
-                                            required:
-                                            - kind
-                                            - name
-                                            type: object
-                                          serviceName:
-                                            description: Specifies the name of the referenced service.
-                                            type: string
-                                          servicePort:
-                                            anyOf:
-                                            - type: integer
-                                            - type: string
-                                            description: Specifies the port of the referenced service.
-                                            x-kubernetes-int-or-string: true
-                                        type: object
-                                      path:
-                                        description: Path is matched against the path of an incoming request. Currently it can contain characters disallowed from the conventional "path" part of a URL as defined by RFC 3986. Paths must begin with a '/'. When unspecified, all paths from incoming requests are matched.
-                                        type: string
-                                      pathType:
-                                        description: 'PathType determines the interpretation of the Path matching. PathType can be one of the following values: * Exact: Matches the URL path exactly. * Prefix: Matches based on a URL path prefix split by ''/''. Matching is   done on a path element by element basis. A path element refers is the   list of labels in the path split by the ''/'' separator. A request is a   match for path p if every p is an element-wise prefix of p of the   request path. Note that if the last element of the path is a substring   of the last element in request path, it is not a match (e.g. /foo/bar   matches /foo/bar/baz, but does not match /foo/barbaz). * ImplementationSpecific: Interpretation of the Path matching is up to   the IngressClass. Implementations can treat this as a separate PathType   or treat it identically to Prefix or Exact path types. Implementations are required to support all path types. Defaults to ImplementationSpecific.'
-                                        type: string
-                                    required:
-                                    - backend
-                                    type: object
-                                  type: array
-                              required:
-                              - paths
-                              type: object
-                          type: object
-                        type: array
-                      tls:
-                        description: TLS configuration. Currently the Ingress only supports a single TLS port, 443. If multiple members of this list specify different hosts, they will be multiplexed on the same port according to the hostname specified through the SNI TLS extension, if the ingress controller fulfilling the ingress supports SNI.
-                        items:
-                          description: IngressTLS describes the transport layer security associated with an Ingress.
-                          properties:
-                            hosts:
-                              description: Hosts are a list of hosts included in the TLS certificate. The values in this list must match the name/s used in the tlsSecret. Defaults to the wildcard host setting for the loadbalancer controller fulfilling this Ingress, if left unspecified.
-                              items:
-                                type: string
-                              type: array
-                            secretName:
-                              description: SecretName is the name of the secret used to terminate SSL traffic on 443. Field is left optional to allow SSL routing based on SNI hostname alone. If the SNI host in a listener conflicts with the "Host" header field used by an IngressRule, the SNI host is used for termination and value of the Host header is used for routing.
-                              type: string
-                          type: object
-                        type: array
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedlimitranges.yaml

@@ -1,217 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedlimitranges.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedLimitRange
-    listKind: FederatedLimitRangeList
-    plural: federatedlimitranges
-    singular: federatedlimitrange
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: LimitRangeSpec defines a min/max usage limit for resources that match on kind.
-                    properties:
-                      limits:
-                        description: Limits is the list of LimitRangeItem objects that are enforced.
-                        items:
-                          description: LimitRangeItem defines a min/max usage limit for any resource that matches on kind.
-                          properties:
-                            default:
-                              additionalProperties:
-                                anyOf:
-                                - type: integer
-                                - type: string
-                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                                x-kubernetes-int-or-string: true
-                              description: Default resource requirement limit value by resource name if resource limit is omitted.
-                              type: object
-                            defaultRequest:
-                              additionalProperties:
-                                anyOf:
-                                - type: integer
-                                - type: string
-                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                                x-kubernetes-int-or-string: true
-                              description: DefaultRequest is the default resource requirement request value by resource name if resource request is omitted.
-                              type: object
-                            max:
-                              additionalProperties:
-                                anyOf:
-                                - type: integer
-                                - type: string
-                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                                x-kubernetes-int-or-string: true
-                              description: Max usage constraints on this kind by resource name.
-                              type: object
-                            maxLimitRequestRatio:
-                              additionalProperties:
-                                anyOf:
-                                - type: integer
-                                - type: string
-                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                                x-kubernetes-int-or-string: true
-                              description: MaxLimitRequestRatio if specified, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value; this represents the max burst for the named resource.
-                              type: object
-                            min:
-                              additionalProperties:
-                                anyOf:
-                                - type: integer
-                                - type: string
-                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                                x-kubernetes-int-or-string: true
-                              description: Min usage constraints on this kind by resource name.
-                              type: object
-                            type:
-                              description: Type of resource that this limit applies to.
-                              type: string
-                          required:
-                          - type
-                          type: object
-                        type: array
-                    required:
-                    - limits
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatednamespaces.yaml

@@ -1,164 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatednamespaces.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedNamespace
-    listKind: FederatedNamespaceList
-    plural: federatednamespaces
-    singular: federatednamespace
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: NamespaceSpec describes the attributes on a Namespace.
-                    properties:
-                      finalizers:
-                        description: 'Finalizers is an opaque list of values that must be empty to permanently remove object from storage. More info: https://kubernetes.io/docs/tasks/administer-cluster/namespaces/'
-                        items:
-                          description: FinalizerName is the name identifying a finalizer during namespace lifecycle.
-                          type: string
-                        type: array
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatednotificationconfigs.yaml

@@ -1,393 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatednotificationconfigs.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedNotificationConfig
-    listKind: FederatedNotificationConfigList
-    plural: federatednotificationconfigs
-    singular: federatednotificationconfig
-  scope: Cluster
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                    x-kubernetes-preserve-unknown-fields: true
-                  spec:
-                    description: ConfigSpec defines the desired state of Config
-                    properties:
-                      dingtalk:
-                        properties:
-                          conversation:
-                            description: Only needed when send alerts to the conversation.
-                            properties:
-                              appkey:
-                                description: The key of the application with which to send messages.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              appsecret:
-                                description: The key in the secret to be used. Must be a valid secret key.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                            type: object
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                        type: object
-                      email:
-                        properties:
-                          authIdentify:
-                            description: The identity for PLAIN authentication.
-                            type: string
-                          authPassword:
-                            description: The secret contains the SMTP password for LOGIN and PLAIN authentications.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          authSecret:
-                            description: The secret contains the SMTP secret for CRAM-MD5 authentication.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          authUsername:
-                            description: The username for CRAM-MD5, LOGIN and PLAIN authentications.
-                            type: string
-                          from:
-                            description: The sender address.
-                            type: string
-                          hello:
-                            description: The hostname to use when identifying to the SMTP server.
-                            type: string
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                          requireTLS:
-                            description: The default SMTP TLS requirement.
-                            type: boolean
-                          smartHost:
-                            description: The address of the SMTP server.
-                            properties:
-                              host:
-                                type: string
-                              port:
-                                type: integer
-                            required:
-                            - host
-                            - port
-                            type: object
-                          tls:
-                            description: TLSConfig configures the options for TLS connections.
-                            properties:
-                              clientCertificate:
-                                description: The certificate of the client.
-                                properties:
-                                  cert:
-                                    description: The client cert file for the targets.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                  key:
-                                    description: The client key file for the targets.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                type: object
-                              insecureSkipVerify:
-                                description: Disable target certificate validation.
-                                type: boolean
-                              rootCA:
-                                description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              serverName:
-                                description: Used to verify the hostname for the targets.
-                                type: string
-                            type: object
-                        required:
-                        - from
-                        - smartHost
-                        type: object
-                      slack:
-                        properties:
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                          slackTokenSecret:
-                            description: The token of user or bot.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                        type: object
-                      webhook:
-                        properties:
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                        type: object
-                      wechat:
-                        properties:
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                          wechatApiAgentId:
-                            description: The id of the application which sending message.
-                            type: string
-                          wechatApiCorpId:
-                            description: The corp id for authentication.
-                            type: string
-                          wechatApiSecret:
-                            description: The API key to use when talking to the WeChat API.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          wechatApiUrl:
-                            description: The WeChat API URL.
-                            type: string
-                        required:
-                        - wechatApiAgentId
-                        - wechatApiCorpId
-                        - wechatApiSecret
-                        type: object
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatednotificationreceivers.yaml

@@ -1,700 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatednotificationreceivers.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedNotificationReceiver
-    listKind: FederatedNotificationReceiverList
-    plural: federatednotificationreceivers
-    singular: federatednotificationreceiver
-  scope: Cluster
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                    x-kubernetes-preserve-unknown-fields: true
-                  spec:
-                    description: ReceiverSpec defines the desired state of Receiver
-                    properties:
-                      dingtalk:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          chatbot:
-                            description: Be careful, a ChatBot only can send 20 message per minute.
-                            properties:
-                              keywords:
-                                description: Custom keywords of ChatBot
-                                items:
-                                  type: string
-                                type: array
-                              secret:
-                                description: Secret of ChatBot, you can get it after enabled Additional Signature of ChatBot.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              webhook:
-                                description: The webhook of ChatBot which the message will send to.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                            required:
-                            - webhook
-                            type: object
-                          conversation:
-                            description: The conversation which message will send to.
-                            properties:
-                              chatids:
-                                items:
-                                  type: string
-                                type: array
-                            required:
-                            - chatids
-                            type: object
-                          dingtalkConfigSelector:
-                            description: DingTalkConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                        type: object
-                      email:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          emailConfigSelector:
-                            description: EmailConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          to:
-                            description: Receivers' email addresses
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - to
-                        type: object
-                      slack:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          channels:
-                            description: The channel or user to send notifications to.
-                            items:
-                              type: string
-                            type: array
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          slackConfigSelector:
-                            description: SlackConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                        required:
-                        - channels
-                        type: object
-                      webhook:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          httpConfig:
-                            description: HTTPClientConfig configures an HTTP client.
-                            properties:
-                              basicAuth:
-                                description: The HTTP basic authentication credentials for the targets.
-                                properties:
-                                  password:
-                                    description: SecretKeySelector selects a key of a Secret.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                  username:
-                                    type: string
-                                required:
-                                - username
-                                type: object
-                              bearerToken:
-                                description: The bearer token for the targets.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              proxyUrl:
-                                description: HTTP proxy server to use to connect to the targets.
-                                type: string
-                              tlsConfig:
-                                description: TLSConfig to use to connect to the targets.
-                                properties:
-                                  clientCertificate:
-                                    description: The certificate of the client.
-                                    properties:
-                                      cert:
-                                        description: The client cert file for the targets.
-                                        properties:
-                                          key:
-                                            description: The key of the secret to select from.  Must be a valid secret key.
-                                            type: string
-                                          name:
-                                            description: Name of the secret.
-                                            type: string
-                                          namespace:
-                                            description: The namespace of the secret, default to the pod's namespace.
-                                            type: string
-                                        required:
-                                        - key
-                                        type: object
-                                      key:
-                                        description: The client key file for the targets.
-                                        properties:
-                                          key:
-                                            description: The key of the secret to select from.  Must be a valid secret key.
-                                            type: string
-                                          name:
-                                            description: Name of the secret.
-                                            type: string
-                                          namespace:
-                                            description: The namespace of the secret, default to the pod's namespace.
-                                            type: string
-                                        required:
-                                        - key
-                                        type: object
-                                    type: object
-                                  insecureSkipVerify:
-                                    description: Disable target certificate validation.
-                                    type: boolean
-                                  rootCA:
-                                    description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                  serverName:
-                                    description: Used to verify the hostname for the targets.
-                                    type: string
-                                type: object
-                            type: object
-                          service:
-                            description: "`service` is a reference to the service for this webhook. Either `service` or `url` must be specified. \n If the webhook is running within the cluster, then you should use `service`."
-                            properties:
-                              name:
-                                description: '`name` is the name of the service. Required'
-                                type: string
-                              namespace:
-                                description: '`namespace` is the namespace of the service. Required'
-                                type: string
-                              path:
-                                description: '`path` is an optional URL path which will be sent in any request to this service.'
-                                type: string
-                              port:
-                                description: If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive).
-                                format: int32
-                                type: integer
-                              scheme:
-                                description: Http scheme, default is http.
-                                type: string
-                            required:
-                            - name
-                            - namespace
-                            type: object
-                          url:
-                            description: "`url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. \n The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some api servers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. \n Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. \n A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. \n Attempting to use a user or basic auth e.g. \"user:password@\" is not allowed. Fragments (\"#...\") and query parameters (\"?...\") are not allowed, either."
-                            type: string
-                          webhookConfigSelector:
-                            description: WebhookConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                        type: object
-                      wechat:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          toParty:
-                            items:
-                              type: string
-                            type: array
-                          toTag:
-                            items:
-                              type: string
-                            type: array
-                          toUser:
-                            items:
-                              type: string
-                            type: array
-                          wechatConfigSelector:
-                            description: WechatConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                        type: object
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedpersistentvolumeclaims.yaml

@@ -1,248 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedpersistentvolumeclaims.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedPersistentVolumeClaim
-    listKind: FederatedPersistentVolumeClaimList
-    plural: federatedpersistentvolumeclaims
-    singular: federatedpersistentvolumeclaim
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  apiVersion:
-                    description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-                    type: string
-                  kind:
-                    description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                    type: string
-                  metadata:
-                    type: object
-                  spec:
-                    description: PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes
-                    properties:
-                      accessModes:
-                        description: 'AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1'
-                        items:
-                          type: string
-                        type: array
-                      dataSource:
-                        description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.'
-                        properties:
-                          apiGroup:
-                            description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
-                            type: string
-                          kind:
-                            description: Kind is the type of resource being referenced
-                            type: string
-                          name:
-                            description: Name is the name of resource being referenced
-                            type: string
-                        required:
-                        - kind
-                        - name
-                        type: object
-                      resources:
-                        description: 'Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
-                        properties:
-                          limits:
-                            additionalProperties:
-                              anyOf:
-                              - type: integer
-                              - type: string
-                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                              x-kubernetes-int-or-string: true
-                            description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                            type: object
-                          requests:
-                            additionalProperties:
-                              anyOf:
-                              - type: integer
-                              - type: string
-                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                              x-kubernetes-int-or-string: true
-                            description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                            type: object
-                        type: object
-                      selector:
-                        description: A label query over volumes to consider for binding.
-                        properties:
-                          matchExpressions:
-                            description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                            items:
-                              description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                              properties:
-                                key:
-                                  description: key is the label key that the selector applies to.
-                                  type: string
-                                operator:
-                                  description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                  type: string
-                                values:
-                                  description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                  items:
-                                    type: string
-                                  type: array
-                              required:
-                              - key
-                              - operator
-                              type: object
-                            type: array
-                          matchLabels:
-                            additionalProperties:
-                              type: string
-                            description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                            type: object
-                        type: object
-                      storageClassName:
-                        description: 'Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1'
-                        type: string
-                      volumeMode:
-                        description: volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec.
-                        type: string
-                      volumeName:
-                        description: VolumeName is the binding reference to the PersistentVolume backing this claim.
-                        type: string
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedresourcequota.yaml

@@ -1,198 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedresourcequota.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedResourceQuota
-    listKind: FederatedResourceQuotaList
-    plural: federatedresourcequota
-    singular: federatedresourcequota
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: ResourceQuotaSpec defines the desired hard limits to enforce for Quota.
-                    properties:
-                      hard:
-                        additionalProperties:
-                          anyOf:
-                          - type: integer
-                          - type: string
-                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                          x-kubernetes-int-or-string: true
-                        description: 'hard is the set of desired hard limits for each named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
-                        type: object
-                      scopeSelector:
-                        description: scopeSelector is also a collection of filters like scopes that must match each object tracked by a quota but expressed using ScopeSelectorOperator in combination with possible values. For a resource to match, both scopes AND scopeSelector (if specified in spec), must be matched.
-                        properties:
-                          matchExpressions:
-                            description: A list of scope selector requirements by scope of the resources.
-                            items:
-                              description: A scoped-resource selector requirement is a selector that contains values, a scope name, and an operator that relates the scope name and values.
-                              properties:
-                                operator:
-                                  description: Represents a scope's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist.
-                                  type: string
-                                scopeName:
-                                  description: The name of the scope that the selector applies to.
-                                  type: string
-                                values:
-                                  description: An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                  items:
-                                    type: string
-                                  type: array
-                              required:
-                              - operator
-                              - scopeName
-                              type: object
-                            type: array
-                        type: object
-                      scopes:
-                        description: A collection of filters that must match each object tracked by a quota. If not specified, the quota matches all objects.
-                        items:
-                          description: A ResourceQuotaScope defines a filter that must match each object tracked by a quota
-                          type: string
-                        type: array
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedsecrets.yaml

@@ -1,165 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedsecrets.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedSecret
-    listKind: FederatedSecretList
-    plural: federatedsecrets
-    singular: federatedsecret
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  data:
-                    additionalProperties:
-                      format: byte
-                      type: string
-                    type: object
-                  stringData:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  type:
-                    type: string
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedservices.yaml

@@ -1,263 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedservices.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedService
-    listKind: FederatedServiceList
-    plural: federatedservices
-    singular: federatedservice
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  apiVersion:
-                    description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-                    type: string
-                  kind:
-                    description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                    type: string
-                  metadata:
-                    type: object
-                  spec:
-                    description: ServiceSpec describes the attributes that a user creates on a service.
-                    properties:
-                      clusterIP:
-                        description: 'clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
-                        type: string
-                      externalIPs:
-                        description: externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service.  These IPs are not managed by Kubernetes.  The user is responsible for ensuring that traffic arrives at a node with this IP.  A common example is external load-balancers that are not part of the Kubernetes system.
-                        items:
-                          type: string
-                        type: array
-                      externalName:
-                        description: externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.
-                        type: string
-                      externalTrafficPolicy:
-                        description: externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. "Local" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. "Cluster" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.
-                        type: string
-                      healthCheckNodePort:
-                        description: healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.
-                        format: int32
-                        type: integer
-                      ipFamily:
-                        description: ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6).  If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster.  If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family.  Endpoints for this Service will be of this family.  This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.
-                        type: string
-                      loadBalancerIP:
-                        description: 'Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.'
-                        type: string
-                      loadBalancerSourceRanges:
-                        description: 'If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/'
-                        items:
-                          type: string
-                        type: array
-                      ports:
-                        description: 'The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
-                        items:
-                          description: ServicePort contains information on service's port.
-                          properties:
-                            appProtocol:
-                              description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and http://www.iana.org/assignments/service-names). Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. Field can be enabled with ServiceAppProtocol feature gate.
-                              type: string
-                            name:
-                              description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.
-                              type: string
-                            nodePort:
-                              description: 'The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport'
-                              format: int32
-                              type: integer
-                            port:
-                              description: The port that will be exposed by this service.
-                              format: int32
-                              type: integer
-                            protocol:
-                              default: TCP
-                              description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP.
-                              type: string
-                            targetPort:
-                              anyOf:
-                              - type: integer
-                              - type: string
-                              description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service'
-                              x-kubernetes-int-or-string: true
-                          required:
-                          - port
-                          type: object
-                        type: array
-                        x-kubernetes-list-map-keys:
-                        - port
-                        - protocol
-                        x-kubernetes-list-type: map
-                      publishNotReadyAddresses:
-                        description: publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.
-                        type: boolean
-                      selector:
-                        additionalProperties:
-                          type: string
-                        description: 'Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/'
-                        type: object
-                      sessionAffinity:
-                        description: 'Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
-                        type: string
-                      sessionAffinityConfig:
-                        description: sessionAffinityConfig contains the configurations of session affinity.
-                        properties:
-                          clientIP:
-                            description: clientIP contains the configurations of Client IP based session affinity.
-                            properties:
-                              timeoutSeconds:
-                                description: timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == "ClientIP". Default value is 10800(for 3 hours).
-                                format: int32
-                                type: integer
-                            type: object
-                        type: object
-                      topologyKeys:
-                        description: topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value "*" may be used to mean "any topology". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.
-                        items:
-                          type: string
-                        type: array
-                      type:
-                        description: 'type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types'
-                        type: string
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedusers.yaml

@@ -1,178 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedusers.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedUser
-    listKind: FederatedUserList
-    plural: federatedusers
-    singular: federateduser
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: UserSpec defines the desired state of User
-                    properties:
-                      description:
-                        description: Description of the user.
-                        type: string
-                      displayName:
-                        type: string
-                      email:
-                        description: Unique email address(https://www.ietf.org/rfc/rfc5322.txt).
-                        type: string
-                      groups:
-                        items:
-                          type: string
-                        type: array
-                      lang:
-                        description: The preferred written or spoken language for the user.
-                        type: string
-                      password:
-                        description: password will be encrypted by mutating admission webhook
-                        type: string
-                    required:
-                    - email
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedworkspacerolebindings.yaml

@@ -1,197 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedworkspacerolebindings.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedWorkspaceRoleBinding
-    listKind: FederatedWorkspaceRoleBindingList
-    plural: federatedworkspacerolebindings
-    singular: federatedworkspacerolebinding
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                  roleRef:
-                    description: RoleRef can only reference a WorkspaceRole. If the RoleRef cannot be resolved, the Authorizer must return an error.
-                    properties:
-                      apiGroup:
-                        description: APIGroup is the group for the resource being referenced
-                        type: string
-                      kind:
-                        description: Kind is the type of resource being referenced
-                        type: string
-                      name:
-                        description: Name is the name of resource being referenced
-                        type: string
-                    required:
-                    - apiGroup
-                    - kind
-                    - name
-                    type: object
-                  subjects:
-                    description: Subjects holds references to the objects the role applies to.
-                    items:
-                      description: Subject contains a reference to the object or user identities a role binding applies to.  This can either hold a direct API object reference, or a value for non-objects such as user and group names.
-                      properties:
-                        apiGroup:
-                          description: APIGroup holds the API group of the referenced subject. Defaults to "" for ServiceAccount subjects. Defaults to "rbac.authorization.k8s.io" for User and Group subjects.
-                          type: string
-                        kind:
-                          description: Kind of object being referenced. Values defined by this API group are "User", "Group", and "ServiceAccount". If the Authorizer does not recognized the kind value, the Authorizer should report an error.
-                          type: string
-                        name:
-                          description: Name of the object being referenced.
-                          type: string
-                        namespace:
-                          description: Namespace of the referenced object.  If the object kind is non-namespace, such as "User" or "Group", and this value is not empty the Authorizer should report an error.
-                          type: string
-                      required:
-                      - kind
-                      - name
-                      type: object
-                    type: array
-                required:
-                - roleRef
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedworkspaceroles.yaml

@@ -1,190 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedworkspaceroles.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedWorkspaceRole
-    listKind: FederatedWorkspaceRoleList
-    plural: federatedworkspaceroles
-    singular: federatedworkspacerole
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                  rules:
-                    description: Rules holds all the PolicyRules for this WorkspaceRole
-                    items:
-                      description: PolicyRule holds information that describes a policy rule, but does not contain information about who the rule applies to or which namespace the rule applies to.
-                      properties:
-                        apiGroups:
-                          description: APIGroups is the name of the APIGroup that contains the resources.  If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed.
-                          items:
-                            type: string
-                          type: array
-                        nonResourceURLs:
-                          description: NonResourceURLs is a set of partial urls that a user should have access to.  *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"),  but not both.
-                          items:
-                            type: string
-                          type: array
-                        resourceNames:
-                          description: ResourceNames is an optional white list of names that the rule applies to.  An empty set means that everything is allowed.
-                          items:
-                            type: string
-                          type: array
-                        resources:
-                          description: Resources is a list of resources this rule applies to.  ResourceAll represents all resources.
-                          items:
-                            type: string
-                          type: array
-                        verbs:
-                          description: Verbs is a list of Verbs that apply to ALL the ResourceKinds and AttributeRestrictions contained in this rule.  VerbAll represents all kinds.
-                          items:
-                            type: string
-                          type: array
-                      required:
-                      - verbs
-                      type: object
-                    type: array
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_federatedworkspaces.yaml

@@ -1,164 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedworkspaces.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedWorkspace
-    listKind: FederatedWorkspaceList
-    plural: federatedworkspaces
-    singular: federatedworkspace
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                  spec:
-                    description: WorkspaceSpec defines the desired state of Workspace
-                    properties:
-                      manager:
-                        type: string
-                      networkIsolation:
-                        type: boolean
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_genericfederatedresources.yaml

@@ -1,79 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: genericfederatedresources.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: GenericFederatedResource
-    listKind: GenericFederatedResourceList
-    plural: genericfederatedresources
-    singular: genericfederatedresource
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_genericoverrides.yaml

@@ -1,64 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: genericoverrides.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: GenericOverride
-    listKind: GenericOverrideList
-    plural: genericoverrides
-    singular: genericoverride
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_genericplacements.yaml

@@ -1,84 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: genericplacements.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: GenericPlacement
-    listKind: GenericPlacementList
-    plural: genericplacements
-    singular: genericplacement
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crds/types.kubefed.io_persistentvolumeclaimtemplates.yaml

@@ -1,125 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: persistentvolumeclaimtemplates.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: PersistentVolumeClaimTemplate
-    listKind: PersistentVolumeClaimTemplateList
-    plural: persistentvolumeclaimtemplates
-    singular: persistentvolumeclaimtemplate
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes
-            properties:
-              accessModes:
-                description: 'AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1'
-                items:
-                  type: string
-                type: array
-              dataSource:
-                description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.'
-                properties:
-                  apiGroup:
-                    description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
-                    type: string
-                  kind:
-                    description: Kind is the type of resource being referenced
-                    type: string
-                  name:
-                    description: Name is the name of resource being referenced
-                    type: string
-                required:
-                - kind
-                - name
-                type: object
-              resources:
-                description: 'Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
-                properties:
-                  limits:
-                    additionalProperties:
-                      anyOf:
-                      - type: integer
-                      - type: string
-                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                      x-kubernetes-int-or-string: true
-                    description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                    type: object
-                  requests:
-                    additionalProperties:
-                      anyOf:
-                      - type: integer
-                      - type: string
-                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
-                      x-kubernetes-int-or-string: true
-                    description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
-                    type: object
-                type: object
-              selector:
-                description: A label query over volumes to consider for binding.
-                properties:
-                  matchExpressions:
-                    description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                    items:
-                      description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                      properties:
-                        key:
-                          description: key is the label key that the selector applies to.
-                          type: string
-                        operator:
-                          description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                          type: string
-                        values:
-                          description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                          items:
-                            type: string
-                          type: array
-                      required:
-                      - key
-                      - operator
-                      type: object
-                    type: array
-                  matchLabels:
-                    additionalProperties:
-                      type: string
-                    description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                    type: object
-                type: object
-              storageClassName:
-                description: 'Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1'
-                type: string
-              volumeMode:
-                description: volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec.
-                type: string
-              volumeName:
-                description: VolumeName is the binding reference to the PersistentVolume backing this claim.
-                type: string
-            type: object
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []