Merge pull request #3350 from benyp/release-3.0

fix duplicate args error
2021-02-18 10:47:26 +08:00 · 2021-02-07 15:43:37 +08:00 · 2020-09-10 10:46:12 +08:00 · 2020-09-08 17:32:26 +08:00 · 2020-09-08 16:59:30 +08:00 · 2020-09-08 16:59:30 +08:00
4981 changed files with 241256 additions and 510356 deletions
--- a/.dockerignore
+++ b/.dockerignore
@@ -1,3 +1,2 @@
-# exclude all files and folders except bin folder
-**
-!bin
+tmp/
+.github
--- a/.github/ISSUE_TEMPLATE/bug_report.md
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -3,16 +3,16 @@ name: Bug report
 about: Create a report to help us improve
 ---

-<!--
-You don't need to remove this comment section, it's invisible on the issues page.
+## English only!

-## General remarks
+**注意！GitHub Issue 仅支持英文，中文 Issue 请在 [论坛](https://kubesphere.com.cn/forum/) 提交。**

-* Attention, please fill out this issues form using English only!
-* 注意！GitHub Issue 仅支持英文，中文 Issue 请在 [论坛](https://kubesphere.com.cn/forum/) 提交。
-* This form is to report bugs. For general usage questions you can join our Slack channel
-       [KubeSphere-users](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
-->
+**General remarks**
+
+> Please delete this section including header before submitting
+>
+> This form is to report bugs. For general usage questions refer to our Slack channel
+>        [KubeSphere-users](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTdkNTc3OTdmNzdiODViZjViNTU5ZDY3M2I2MzY4MTI4OGZlOTJmMDg5ZTFiMDAwYzNlZDY5NjA0NzZlNDU5NmY)

 **Describe the Bug**
 A clear and concise description of what the bug is.
--- a/.github/ISSUE_TEMPLATE/feature_request.md
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -1,69 +0,0 @@
---
-name: Feature Request
-about: Have a good idea? Please don't hesitate to write it down, describe the new feature.
---
-
-<!--
-You don't need to remove this comment section, it's invisible on the issues page.
-
-## General remarks
-
-* Attention, please fill out this issues form using English only!
-* 注意！GitHub Issue 仅支持英文，中文 Issue 请在 [论坛](https://kubesphere.com.cn/forum/) 提交。
-* This form is to report bugs. For general usage questions, you can join our Slack channel
-       [KubeSphere-users](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
-->
-
-**What's it about?**
-<!--
-A clear and concise description of what this feature request is.
-->
-
-**What's the reason why we need it?**
-<!--
-Please tell us if you think it's a necessary feature for Kubesphere. Give us as many details about it as you can. 
-Two or more use cases might be very helpful when other contributors try to go through this request. If you have some references,
-please just add it below.
-->
-
-I believe this is an important feature for Kubesphere. There're a few use cases:
-
-* case one
-* case two
-* ...
-
-Please leave your comments below if there's anyone agrees with me. Or just give me a thumb up.
-
-**Area Suggestion**
-<!--
-In order to have a clear issue list, giving an accuracy area is necessary. If you are not sure about it, please just leave it alone.
-
-You can find some possible areas below. Please attention, sometimes crossing multiple areas might be possible. So, you 
-can keep one or more areas in this issue.
-
-> /area alerting
-> /area api
-> /area apiserver
-> /area app-management
-> /area audit
-> /area console
-> /area devops
-> /area documentation
-> /area edge
-> /area iam
-> /area installation
-> /area logging
-> /area microservice
-> /area monitoring
-> /area multicluster
-> /area networking
-> /area notification
-> /area observability
-> /area performance
-> /area security
-> /area storage
-> /area test
-> /area upgrade
-->
-
-/kind feature-request
--- a/.github/ISSUE_TEMPLATE/installation_failure.md
+++ b/.github/ISSUE_TEMPLATE/installation_failure.md
@@ -7,13 +7,6 @@ about: Create an issue to help us improve installation

 **注意！GitHub Issue 仅支持英文，中文 Issue 请在 [论坛](https://kubesphere.com.cn/forum/) 提交。**

-**General remarks**
-
-> Please delete this section including header before submitting
->
-> This form is to report installation issues. For general usage questions you can refer to [KubeSphere Documentation](https://kubesphere.io/docs) or join our Slack channel
->        [KubeSphere-users](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
-
 **What's your question**


@@ -27,4 +20,4 @@ about: Create an issue to help us improve installation

 **Installer Version**

-> e.g. v2.1.0, v2.1.1, v3.0.0
+> e.g. v2.1.0, v2.1.1
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -19,8 +19,9 @@ jobs:
    env:
      GO111MODULE: on
    steps:
+
      - name: Set up Go 1.13
-        uses: actions/setup-go@v2
+        uses: actions/setup-go@v1
        with:
          go-version: 1.13
        id: go
@@ -31,9 +32,6 @@ jobs:
      - name: Check pr is properly formatted
        run: diff -u <(echo -n) <(gofmt -d ./pkg ./cmd ./tools ./test)

-      - name: Verify goimports
-        run: go get -u golang.org/x/tools/cmd/goimports && bash hack/verify-goimports.sh
-
      - name: Downloading go dependencies
        run: go mod vendor

@@ -65,6 +63,4 @@ jobs:
          DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
          DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
        if: github.event_name == 'push'
-        run: |
-          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
-          bash hack/docker_build.sh ${{ steps.extract_branch.outputs.branch }}
+        run: bash hack/docker_build.sh ${{ steps.extract_branch.outputs.branch }}
--- a/.github/workflows/e2e-test.yml
+++ b/.github/workflows/e2e-test.yml
@@ -1,37 +0,0 @@
-name: e2e
-
-on:
-  schedule:
-    # run e2e test every 4 hours
-    - cron: 0 */4 * * *
-  workflow_dispatch:
-jobs:
-  build:
-    name: Test
-    runs-on: ubuntu-latest
-    env:
-      GO111MODULE: on
-    steps:
-
-      - name: Set up Go 1.13
-        uses: actions/setup-go@v1
-        with:
-          go-version: 1.13
-        id: go
-
-      - name: Check out code into the Go module directory
-        uses: actions/checkout@v2
-
-      - name: Downloading go dependencies
-        run: go mod vendor
-
-      - name: Create kind cluster
-        uses: helm/kind-action@v1.0.0-rc.1
-        with:
-          config: .github/workflows/kind/kind.yaml
-
-      - name: Deploy KubeSphere to Kind
-        run: KIND_CLUSTER_NAME=chart-testing hack/deploy-kubesphere.sh
-
-      - name: Run e2e testing
-        run: go test ./test/e2e
--- a/.github/workflows/kind/kind.yaml
+++ b/.github/workflows/kind/kind.yaml
@@ -1,11 +0,0 @@
-kind: Cluster
-apiVersion: kind.x-k8s.io/v1alpha4
-nodes:
- role: control-plane
-  image: kindest/node:v1.19.7
-  extraMounts:
-  - hostPath: /etc/localtime
-    containerPath: /etc/localtime
-  extraPortMappings:
-  - containerPort: 30881
-    hostPort: 9090
--- a/.github/workflows/nightly-builds.yml
+++ b/.github/workflows/nightly-builds.yml
@@ -1,66 +0,0 @@
-name: NightlyBuild
-
-on:
-  schedule:
-    # This is a UTC time
-    - cron: "0 16 * * *"
-  # Keep it only for test purpose, comment it once everything is ok
-  workflow_dispatch:
-
-jobs:
-  build:
-    name: Build
-    runs-on: ubuntu-latest
-    env:
-      GO111MODULE: on
-    steps:
-
-      - name: Set up Go 1.13
-        uses: actions/setup-go@v1
-        with:
-          go-version: 1.13
-        id: go
-
-      - name: Check out code into the Go module directory
-        uses: actions/checkout@v2
-
-      - name: Downloading go dependencies
-        run: go mod vendor
-
-      - name: Install kubebuilder
-        run: bash hack/install_kubebuilder.sh
-
-      - name: Build
-        run: make all
-
-      - name: Make OpenAPI Spec
-        run: make openapi
-
-      - name: Build and push docker images
-        env:
-          DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
-          DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
-        run: |
-          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
-          bash hack/docker_build.sh master
-
-          if [[ $? == 0 ]]; then
-            tag=nightly-$(date '+%Y%m%d')
-
-            docker tag kubespheredev/ks-apiserver kubespheredev/ks-apiserver:${tag}
-            docker tag kubespheredev/ks-controller-manager kubespheredev/ks-controller-manager:${tag}
-
-            docker push kubespheredev/ks-apiserver:${tag}
-            docker push kubespheredev/ks-controller-manager:${tag}
-          else
-            exit -1
-          fi
-
-      - name: slack
-        uses: 8398a7/action-slack@v3
-        env:
-          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
-        with:
-          status: ${{ job.status }}
-          fields: repo,message,commit,author,action,eventName,ref,workflow,job,took
-        if: always()
--- a/.gitignore
+++ b/.gitignore
@@ -19,7 +19,6 @@ bin/

 # Vscode files
 .vscode/
-__debug_bin

 # OSX trash
 .DS_Store
@@ -31,4 +30,4 @@ coverage.txt
 kustomize/network/etcd
 apiserver.local.config
 tmp/
-kubesphere.yaml
+
--- a/26
+++ b/26
@@ -34,29 +34,22 @@ define ALL_HELP_INFO
 #           debugging tools like delve.
 endef
 .PHONY: all
-all: test ks-apiserver ks-controller-manager
+all: test ks-apiserver controller-manager

 # Build ks-apiserver binary
 ks-apiserver: fmt vet
 	hack/gobuild.sh cmd/ks-apiserver

-# Build ks-controller-manager binary
-ks-controller-manager: fmt vet
+# Build controller-manager binary
+controller-manager: fmt vet
 	hack/gobuild.sh cmd/controller-manager

-# Build e2e binary
-e2e: fmt vet
-	hack/build_e2e.sh test/e2e
-
 # Run go fmt against code 
-fmt:
+fmt: generate
 	gofmt -w ./pkg ./cmd ./tools ./api

-goimports:
-	@hack/update-goimports.sh
-
 # Run go vet against code
-vet:
+vet: generate
 	go vet ./pkg/... ./cmd/...

 # Generate manifests e.g. CRD, RBAC etc.
@@ -67,6 +60,11 @@ deploy: manifests
 	kubectl apply -f config/crds
 	kustomize build config/default | kubectl apply -f -

+# generate will generate crds' deepcopy & go openapi structs
+# Futher more about go:genreate . https://blog.golang.org/generate
+generate:
+	go generate ./pkg/... ./cmd/...
+
 mockgen:
 	mockgen -package=openpitrix -source=pkg/simple/client/openpitrix/openpitrix.go -destination=pkg/simple/client/openpitrix/mock.go

@@ -86,12 +84,10 @@ openapi:
 # Build the docker image
 docker-build: all
 	hack/docker_build.sh
-docker-build-no-test: ks-apiserver ks-controller-manager
-	hack/docker_build.sh

 # Run tests
 test: fmt vet
-	export KUBEBUILDER_CONTROLPLANE_START_TIMEOUT=2m; go test ./pkg/... ./cmd/... -covermode=atomic -coverprofile=coverage.txt
+	export KUBEBUILDER_CONTROLPLANE_START_TIMEOUT=1m; go test ./pkg/... ./cmd/... -covermode=atomic -coverprofile=coverage.txt

 .PHONY: clean
 clean:
--- a/2
+++ b/2
@@ -19,5 +19,3 @@ reviewers:
    - zheng1
    - soulseen
    - shaowenchen
-    - stoneshi-yunify
-    - linuxsuren
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
 [![License](http://img.shields.io/badge/license-apache%20v2-blue.svg)](https://github.com/KubeSphere/KubeSphere/blob/master/LICENSE)
 [![Build Status](https://travis-ci.org/kubesphere/kubesphere.svg?branch=master)](https://travis-ci.org/kubesphere/kubesphere)
 [![Go Report Card](https://goreportcard.com/badge/github.com/kubesphere/kubesphere)](https://goreportcard.com/report/github.com/kubesphere/kubesphere)
-[![KubeSphere release](https://img.shields.io/github/release/kubesphere/kubesphere.svg?color=release&label=release&logo=release&logoColor=release)](https://github.com/kubesphere/kubesphere/releases/tag/v3.0.0)
+[![KubeSphere release](https://img.shields.io/github/release/kubesphere/kubesphere.svg?color=release&label=release&logo=release&logoColor=release)](https://github.com/kubesphere/kubesphere/releases/tag/v2.1.1)

 ![logo](docs/images/kubesphere-logo.png)

@@ -51,65 +51,112 @@ KubeSphere uses a loosely-coupled architecture that separates the [frontend](htt
 |Feature|Description|
 |---|---|
 | Provisioning Kubernetes Cluster|Support deploy Kubernetes on your infrastructure out of box, including online and air gapped installation|
-| Multi-cluster Management | Provide a centralized control plane to manage multiple Kubernetes Clusters, support application distribution across multiple clusters and cloud providers|
 | Kubernetes Resource Management | Provide web console for creating and managing Kubernetes resources, with powerful observability including monitoring, logging, events, alerting and notification |
 | DevOps System | Provide out-of-box CI/CD based on Jenkins, and offers automated workflow tools including binary-to-image (B2I) and source-to-image (S2I) |
 | Application Store | Provide application store for Helm-based applications, and offers application lifecycle management |
 | Service Mesh (Istio-based) | Provide fine-grained traffic management, observability and tracing for distributed microservice applications, provides visualization for traffic topology |
-| Rich Observability | Provide multi-dimensional monitoring metrics, and provides multi-tenant logging, events and [auditing](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) management, support alerting and notification for both application and infrastructure |
+| Rich Observability | Provide multi-dimensional monitoring metrics, and provides multi-tenant log query and collection, support alerting and notification for both application and infrastructure |
 | Multi-tenant Management | Provide unified authentication with fine-grained roles and three-tier authorization system, supports AD/LDAP authentication |
 | Infrastructure Management | Support node management and monitoring, and supports adding new nodes for Kubernetes cluster |
-| Storage Support | Support GlusterFS, CephRBD, NFS, LocalPV (default), etc. open source storage solutions, provides CSI plugins to consume storage from cloud providers |
-| Network Support | Support Calico, Flannel, etc., provides [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) management, and load balancer plug-in [Porter](https://github.com/kubesphere/porter) for bare metal.|
+| Storage Support | Support GlusterFS, CephRBD, NFS, Local (default) etc. open source storage solutions, provide CSI plugins to consume storage from cloud providers |
+| Network Support | Support Calico, Flannel, etc. open source network solutions, provides load balancer plug-in [Porter](https://github.com/kubesphere/porter) for Kubernetes installed on physical machines |
 | GPU Support | Support add GPU node, support vGPU, enables running ML applications on Kubernetes, e.g. TensorFlow |

-Please see the [Feature and Benefits](https://kubesphere.io/docs/introduction/features/) for further information.
+Please See the [Feature and Benefits](https://kubesphere.io/docs/introduction/features/) for further information.

 ----

 ## Latest Release

-KubeSphere 3.0.0 is now generally available! See the [Release Notes For 3.0.0](https://kubesphere.io/docs/release/release-v300/) for the updates.
+KubeSphere 2.1.1 was released on **February 23rd, 2020**. See the [Release Notes For 2.1.1](https://kubesphere.io/docs/release/release-v211/) for the updates.

 ## Installation

 KubeSphere can run anywhere from on-premise datacenter to any cloud to edge. In addition, it can be deployed on any version-compatible running Kubernetes cluster.

-### QuickStarts
+### Deploy on Existing Kubernetes Cluster

-[Quickstarts](https://kubesphere.io/docs/quick-start/) include six hands-on lab exercises that help you quickly get started with KubeSphere.
+#### Prerequisites

-### Installing on Existing Kubernetes Cluster
+- `Kubernetes version`： `1.15.x, 1.16.x, 1.17.x`
+- `2.10.0 ≤ Helm Version ＜ 3.0.0` excluding 2.16.0 because of [#6894](https://github.com/helm/helm/issues/6894). Please see [Install and Configure Helm in Kubernetes](https://devopscube.com/install-configure-helm-kubernetes/). Helm v3 will be supported in KubeSphere 3.0.0.
+- An existing Storage Class in your Kubernetes cluster, use `kubectl get sc` to verify it
+- The CSR signing feature is activated in kube-apiserver, see [RKE installation issue](https://github.com/kubesphere/kubesphere/issues/1925#issuecomment-591698309).

- [Installing KubeSphere on Amazon EKS](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-eks/)
- [Installing KubeSphere on Azure AKS](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-aks/)
- [Installing KubeSphere on Google GKE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-gke/)
- [Installing KubeSphere on DigitalOcean Kubernetes](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-do/)
- [Installing KubeSphere on Oracle OKE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-oke/)
- [Installing KubeSphere on Tencent TKE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-tencent-tke/)
- [Installing KubeSphere on Huaweicloud CCE](https://v3-0.docs.kubesphere.io/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-huawei-cce/)
+Install KubeSphere using kubectl.

-### Installing on Linux
+- If there are 1 Core and 2 GB RAM available in your cluster, use the command below to set up a default minimal installation only. You can enable other components after installation if more resource added in later on. See [Pluggable Components Installation](https://kubesphere.io/docs/installation/pluggable-components/) for detailed information.

- [Installing KubeSphere on Azure VM](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/install-ks-on-azure-vms/)
- [Installing KubeSphere on VMware vSphere](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/on-premises/install-kubesphere-on-vmware-vsphere/)
- [Installing KubeSphere on QingCloud Instance](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/kubesphere-on-qingcloud-instance/)
- [Installing on Alibaba Cloud ECS](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/install-kubesphere-on-ali-ecs/)
- [Installing on Huaweicloud VM](https://v3-0.docs.kubesphere.io/docs/installing-on-linux/public-cloud/install-ks-on-huaweicloud-ecs/)
+```bash
+kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-minimal.yaml
+```
+
+- If there are 8 Cores and 16 GB RAM available in your cluster, use the command below to install a complete KubeSphere, i.e. with all components enabled:
+
+```bash
+kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-complete-setup.yaml
+```
+
+Wait the installation logs using the following command till showing `"Successful"`, then you can log in the console using the default username and password.
+
+```bash
+kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
+```
+
+### Deploy on Linux
+
+KubeSphere Installer can help you to install KubeSphere and Kubernetes on your linux machines. It provides [All-in-One](https://kubesphere.io/docs/installation/all-in-one/) and [Multi-Node](https://kubesphere.io/docs/installation/multi-node/) installation options.
+
+#### Minimum Requirements
+
+- Operating Systems
+  - CentOS 7.4~7.7 (64 bit)
+  - Ubuntu 16.04/18.04 LTS (64 bit)
+  - Red Hat Enterprise Linux Server 7.4 (64 bit)
+  - Debian Stretch 9.5 (64 bit)
+- Hardware
+  - CPU：2 Cores,  Memory：4 GB, Disk Space：100 GB
+
+> Note:  Please disable the firewall, or ensure your firewall meets the [port requirements](https://kubesphere.io/docs/installation/port-firewall/).
+
+#### All-in-One (QuickStart)
+
+```bash
+curl -L https://kubesphere.io/download/stable/latest > installer.tar.gz \
+&& tar -zxf installer.tar.gz && cd kubesphere-all-v2.1.1/scripts
+$ ./install.sh
+```
+
+Choose `"1) All-in-one"` to start the default minimal installation.
+
+You can enable other components after installation, see [Pluggable Components Installation](https://kubesphere.io/docs/installation/pluggable-components/).
+
+## To start using KubeSphere
+
+- KubeSphere Documentation ([En](https://kubesphere.io/docs/)/[中](https://kubesphere.io/docs/zh-CN/)）
+- [API Documentation](https://kubesphere.io/docs/api-reference/api-docs/)

 ## Contributing, Support, Discussion, and Community

-We :heart: your contribution. The [community](https://github.com/kubesphere/community) walks you through how to get started contributing KubeSphere. The [development guide](https://github.com/kubesphere/community/tree/master/developer-guide/development) explains how to set up development environment.
+This [community](https://github.com/kubesphere/community) walks you through how to get started contributing KubeSphere. The [development guide](https://github.com/kubesphere/community/tree/master/developer-guide/development) explains how to set up development environment.

- [Slack Channel](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
- [Youtube](https://www.youtube.com/channel/UCyTdUQUYjf7XLjxECx63Hpw)
- [Follow us on Twitter](https://twitter.com/KubeSphere)
+If you need any help with KubeSphere, please join us at [Slack Channel](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE).
+
+We also communicate through [Google Group](https://groups.google.com/forum/#!forum/kubesphere).

 Please submit any KubeSphere bugs, issues, and feature requests to [KubeSphere GitHub Issue](https://github.com/kubesphere/kubesphere/issues).

 ## Who are using KubeSphere

-The [user case studies](https://kubesphere.io/case/) page includes the user list of the project. You can [submit a PR](https://github.com/kubesphere/kubesphere/blob/master/docs/powered-by-kubesphere.md) to add your institution name and homepage if you are using KubeSphere.
+The [Powered by KubeSphere](https://kubesphere.io/case/) page includes the user list of the project. You can submit your institution name and homepage if you are using KubeSphere.
+
+## RoadMap
+
+Currently, KubeSphere has released the following 5 major editions. The future releases include multicluster, big data, AI, SDN, etc. See [Plans for 2.1.1 and 3.0.0](https://github.com/kubesphere/kubesphere/issues/1368) for more details.
+
+**Express Edition** => **v1.0.x** => **v2.0.x**  => **v2.1.0** => **v2.1.1** => **v3.0.0**
+
+![Roadmap](https://pek3b.qingstor.com/kubesphere-docs/png/20190926000413.png)

 ## Landscapes

--- a/README_zh.md
+++ b/README_zh.md
@@ -2,7 +2,7 @@

 [![License](http://img.shields.io/badge/license-apache%20v2-blue.svg)](https://github.com/KubeSphere/KubeSphere/blob/master/LICENSE)
 [![Build Status](https://travis-ci.org/kubesphere/kubesphere.svg?branch=master)](https://travis-ci.org/kubesphere/kubesphere)
-[![KubeSphere release](https://img.shields.io/github/release/kubesphere/kubesphere.svg?color=release&label=release&logo=release&logoColor=release)](https://github.com/kubesphere/kubesphere/releases/tag/v3.0.0)
+[![KubeSphere release](https://img.shields.io/github/release/kubesphere/kubesphere.svg?color=release&label=release&logo=release&logoColor=release)](https://github.com/kubesphere/kubesphere/releases/tag/v2.1.1)

 ![logo](docs/images/kubesphere-logo.png)

@@ -12,9 +12,9 @@

 > [English](README.md) | 中文

-[KubeSphere](https://kubesphere.com.cn) 是在 [Kubernetes](https://kubernetes.io) 之上构建的面向云原生应用的 **容器混合云**，支持多云与多集群管理，提供全栈的 IT 自动化运维的能力，简化企业的 DevOps 工作流。KubeSphere 提供了运维友好的向导式操作界面，帮助企业快速构建一个强大和功能丰富的容器云平台。KubeSphere 愿景是打造一个基于 Kubernetes 的云原生分布式操作系统，它的架构可以很方便地与云原生生态进行即插即用（plug-and-play）的集成。
+[KubeSphere](https://kubesphere.com.cn) 是在 [Kubernetes](https://kubernetes.io) 之上构建的以**应用为中心的**多租户容器平台，提供全栈的 IT 自动化运维的能力，简化企业的 DevOps 工作流。KubeSphere 提供了运维友好的向导式操作界面，帮助企业快速构建一个强大和功能丰富的容器云平台。KubeSphere 愿景是打造一个基于 Kubernetes 的云原生分布式操作系统，它的架构可以很方便地与云原生生态进行即插即用（plug-and-play）的集成。

-KubeSphere 目前最新的版本为 3.0.0，所有版本 100% 开源，关于 KubeSphere 更详细的介绍与说明请参阅 [什么是 KubeSphere](https://kubesphere.com.cn/docs/zh-CN/introduction/what-is-kubesphere/)。
+KubeSphere 目前最新的版本为 2.1.1，所有版本 100% 开源，关于 KubeSphere 更详细的介绍与说明请参阅 [什么是 KubeSphere](https://kubesphere.com.cn/docs/zh-CN/introduction/what-is-kubesphere/)。

 <table>
  <tr>
@@ -49,17 +49,16 @@ KubeSphere 采用了前后端分离的架构设计，后端的各个功能组件

 |功能 |介绍 |
 | --- | ---|
-|多云与多集群管理|提供多云与多集群的中央管理面板，支持集群导入，支持应用在多云与多集群一键分发|
 | Kubernetes 集群搭建与运维 | 支持在线 & 离线安装、升级与扩容 K8s 集群，支持安装 “云原生全家桶” |
 | Kubernetes 资源可视化管理 | 可视化纳管原生 Kubernetes 资源，支持向导式创建与管理 K8s 资源 |
 | 基于 Jenkins 的 DevOps 系统 | 支持图形化与脚本两种方式构建 CI/CD 流水线，内置 Source to Image（S2I）和 Binary to Image（B2I）等 CD 工具 |
-| 应用商店与应用生命周期管理 | 提供应用商店，内置 Redis、MySQL 等 15 个常用应用，支持应用的生命周期管理 |
+| 应用商店与应用生命周期管理 | 提供应用商店，内置 Redis、MySQL 等九个常用应用，支持应用的生命周期管理 |
 | 基于 Istio 的微服务治理 (Service Mesh) | 提供可视化无代码侵入的 **灰度发布、熔断、流量治理与流量拓扑、分布式 Tracing** |
 | 多租户管理 | 提供基于角色的细粒度多租户统一认证，支持 **对接企业 LDAP/AD**，提供多层级的权限管理 |
 | 丰富的可观察性功能 | 提供集群/工作负载/Pod/容器等多维度的监控，提供基于多租户的日志查询与日志收集，支持节点与应用层级的告警与通知 |
 |基础设施管理|支持 Kubernetes 节点管理，支持节点扩容与集群升级，提供基于节点的多项监控指标与告警规则 |
-| 存储管理 | 支持对接 Ceph、GlusterFS、NFS、Local PV，支持可视化运维管理 PVC、StorageClass，提供 CSI 插件对接云平台存储 |
-| 网络管理 | 提供租户网络隔离与 K8s [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) 管理，支持 Calico、Flannel，提供 [Porter LB](https://github.com/kubesphere/porter) 用于暴露物理环境 K8s 集群的 LoadBalancer 服务 |
+| 存储管理 | 支持对接 Ceph、GlusterFS、NFS、Local PV，支持可视化管理 PVC、PV、StorageClass，提供 CSI 插件对接云平台存储 |
+| 网络管理 | 支持 Calico、Flannel，提供 Porter LB 插件用于暴露物理环境 K8s 集群的 LoadBalancer 服务 |
 | GPU support | 集群支持添加 GPU 与 vGPU，可运行 TensorFlow 等 ML 框架 |

 以上功能说明详见 [产品功能](https://kubesphere.com.cn/docs/zh-CN/introduction/features/)。
@@ -68,48 +67,92 @@ KubeSphere 采用了前后端分离的架构设计，后端的各个功能组件

 ## 最新发布

-KubeSphere 3.0.0 已于 2020 年 8 月 31 日正式 GA！点击 [Release Notes For 3.0.0](https://kubesphere.com.cn/docs/release/release-v300/) 查看 3.0.0 版本的更新详情。
+KubeSphere 2.1.1 已于 2020 年 02 月 23 日 正式发布，点击 [Release Notes For 2.1.1](https://kubesphere.com.cn/docs/zh-CN/release/release-v211/) 查看 2.1.1 版本的更新详情。

-## 安装 3.0.0
+## 快速安装

-### 快速入门
+### 部署在 Linux

-[快速入门系列](https://kubesphere.com.cn/docs/quick-start/) 提供了快速安装与入门示例，供初次安装体验参考。
+- 操作系统
+  - CentOS 7.5 (64 bit)
+  - Ubuntu 16.04/18.04 LTS (64 bit)
+  - Red Hat Enterprise Linux Server 7.4 (64 bit)
+  - Debian Stretch 9.5 (64 bit)
+- 配置规格（最低）
+  - CPU：2 Cores， 内存：4 GB， 硬盘：100 GB

-### 在已有 Kubernetes 之上安装 KubeSphere
+#### All-in-One

- [基于 Kubernetes 的安装介绍](https://kubesphere.com.cn/docs/installing-on-kubernetes/introduction/overview/)
- [在阿里云 ACK 安装 KubeSphere](https://kubesphere.com.cn/forum/d/1745-kubesphere-v3-0-0-dev-on-ack)
- [在腾讯云 TKE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-tencent-tke/)
- [在华为云 CCE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-huawei-cce/)
- [在 AWS EKS 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-eks/)
- [在 Google GKE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-gke/)
- [在 Azure AKS 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-aks/)
- [在 DigitalOcean 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-do/)
- [在 Oracle OKE 安装 KubeSphere](https://kubesphere.com.cn/docs/installing-on-kubernetes/hosted-kubernetes/install-kubesphere-on-oke/)
+执行以下命令下载 Installer，请关闭防火墙或 [开放指定的端口](https://kubesphere.com.cn/docs/zh-CN/installation/port-firewall/)，建议使用干净的机器并使用 `root` 用户安装：

-### 基于 Linux 安装 KubeSphere
+```bash
+curl -L https://kubesphere.io/download/stable/latest > installer.tar.gz \
+&& tar -zxf installer.tar.gz && cd kubesphere-all-v2.1.1/scripts

- [多节点安装介绍（以三节点为例）](https://kubesphere.com.cn/docs/installing-on-linux/introduction/multioverview/)
- [在 VMware vSphere 安装高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/on-premises/install-kubesphere-on-vmware-vsphere/)
- [在青云QingCloud 安装高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/kubesphere-on-qingcloud-instance/)
- [在阿里云 ECS 部署高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/install-kubesphere-on-ali-ecs/)
+./install.sh
+```

- [在华为云 VM 部署高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/install-kubesphere-on-huaweicloud-ecs/)
- [在 Azure VM 安装高可用集群](https://kubesphere.com.cn/docs/installing-on-linux/public-cloud/install-kubesphere-on-azure-vms/)
+直接选择 `"1) All-in-one"` 即可开始快速安装。默认仅开启最小安装，请参考 [开启可插拔功能功能组件](https://kubesphere.com.cn/docs/zh-CN/installation/pluggable-components/) 按需开启其它功能组件。
+
+> 注意：All-in-One 仅适用于**测试环境**，**正式环境** 安装和使用请参考 [安装说明](https://kubesphere.com.cn/docs/zh-CN/installation/intro/)。
+
+### 部署在已有 Kubernetes 集群上
+
+可参考 [前提条件](https://kubesphere.com.cn/docs/zh-CN/installation/prerequisites/) 验证是否满足以下条件：
+
+#### 前提条件
+
+- `Kubernetes` 版本： `1.15.x、1.16.x、1.17.x`；
+- `Helm`版本： `2.10.0 ≤ Helm Version ＜ 3.0.0`（不支持 helm 2.16.0 [#6894](https://github.com/helm/helm/issues/6894)），且已安装了 Tiller，参考 [如何安装与配置 Helm](https://devopscube.com/install-configure-helm-kubernetes/) （预计 3.0 支持 Helm v3）；
+- 集群已有默认的存储类型（StorageClass），若还没有准备存储请参考 [安装 OpenEBS 创建 LocalPV 存储类型](../../appendix/install-openebs) 用作开发测试环境。
+
+用 kubectl 安装
+
+- 若您的集群可用的资源符合 CPU >= 1 Core，可用内存 >= 2 G，可参考以下命令开启 KubeSphere 最小化安装。后续如果您的集群资源足够可以参考 [开启可插拔功能功能组件](https://kubesphere.com.cn/docs/zh-CN/installation/pluggable-components/) 按需开启其它功能组件。
+
+```yaml
+kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-minimal.yaml
+```
+
+- 若您的集群可用的资源符合 CPU ≥ 8 Core，可用内存 ≥ 16 G，建议参考以下命令开启 KubeSphere 完整安装，即开启所有功能组件的安装：
+
+```yaml
+kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-complete-setup.yaml
+```
+
+查看滚动刷新的安装日志，请耐心等待安装成功。当看到 `"Successful"` 的日志与登录信息提示，则说明 KubeSphere 安装成功，请使用日志提示的管理员账号登陆控制台。
+
+```bash
+kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
+```
+
+## 开始使用 KubeSphere
+
+- KubeSphere 文档中心 ([En](https://kubesphere.com.cn/docs/)/[中](https://kubesphere.com.cn/docs/zh-CN/))
+- [API 文档](https://kubesphere.com.cn/docs/zh-CN/api-reference/api-docs/)

 ## 技术社区

-[KubeSphere 社区](https://github.com/kubesphere/community) 包含所有社区的信息，包括如何开发，兴趣小组(SIG)等。比如[开发指南](https://github.com/kubesphere/community/tree/master/developer-guide/development) 详细说明了如何从源码编译、KubeSphere 的 GitHub 工作流、如何贡献代码以及如何测试等。
+[KubeSphere 社区](https://github.com/kubesphere/community)包含所有社区的信息，包括如何开发，兴趣小组(SIG)等。比如[开发指南](https://github.com/kubesphere/community/tree/master/developer-guide/development) 详细说明了如何从源码编译、KubeSphere 的 GitHub 工作流、如何贡献代码以及如何测试等。

- [中文论坛](https://kubesphere.com.cn/forum/)
+- [论坛](https://kubesphere.com.cn/forum/)
 - [Slack Channel](https://join.slack.com/t/kubesphere/shared_invite/enQtNTE3MDIxNzUxNzQ0LTZkNTdkYWNiYTVkMTM5ZThhODY1MjAyZmVlYWEwZmQ3ODQ1NmM1MGVkNWEzZTRhNzk0MzM5MmY4NDc3ZWVhMjE)
- [社区微信群（见官网底部）](https://kubesphere.com.cn/)
- [Bug 与建议反馈（GitHub Issue）](https://github.com/kubesphere/kubesphere/issues)
+
+## Bug 与建议反馈
+
+欢迎在 [GitHub Issue](https://github.com/kubesphere/kubesphere/issues) 提交 Issue。

 ## 谁在使用 KubeSphere

-[Powered by KubeSphere](https://kubesphere.com.cn/case/) 列出了哪些企业在使用 KubeSphere，如果您所在的企业已安装使用了 KubeSphere，欢迎[提交 PR](https://github.com/kubesphere/kubesphere/blob/master/docs/powered-by-kubesphere.md)。
+[Powered by KubeSphere](docs/powered-by-kubesphere.md) 列出了哪些企业在使用 KubeSphere，如果您所在的企业已安装使用了 KubeSphere，欢迎提交 PR 至该页面。
+
+## 路线图
+
+目前，KubeSphere 已发布了 3 个大版本和 1 个小版本和 4 个 fixpack，所有版本都是完全开源的，参考 [Plans for 2.1.1 and 3.0.0](https://github.com/kubesphere/kubesphere/issues/1368) 了解后续版本的规划，欢迎在 GitHub issue 中提交需求。
+
+**Express Edition** => **v1.0.x** => **v2.0.x**  => **v2.1.0** => **v2.1.1** => **v3.0.0**
+
+![Roadmap](https://pek3b.qingstor.com/kubesphere-docs/png/20190926000514.png)

 ## Landscapes

--- a/api/api-rules/violation_exceptions.list
+++ b/api/api-rules/violation_exceptions.list
@@ -1,6 +1,3 @@
-API rule violation: list_type_missing,./pkg/apis/devops/v1alpha3,DevOpsProjectList,Items
-API rule violation: list_type_missing,./pkg/apis/devops/v1alpha3,NoScmPipeline,Parameters
-API rule violation: list_type_missing,./pkg/apis/devops/v1alpha3,PipelineList,Items
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIGroup,ServerAddressByClientCIDRs
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIGroup,Versions
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,APIGroupList,Groups
@@ -29,63 +26,9 @@ API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,Table
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/apis/meta/v1,UpdateOptions,DryRun
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/runtime,RawExtension,Raw
 API rule violation: list_type_missing,k8s.io/apimachinery/pkg/runtime,Unknown,Raw
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,ApiUri
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromForks
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromOrigin
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,BitbucketServerSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,DiscarderProperty,DaysToKeep
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,DiscarderProperty,NumToKeep
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,ApiUri
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverPRFromForks
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverPRFromOrigin
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GithubSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,ApiUri
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,CloneOption
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverBranches
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverPRFromForks
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverPRFromOrigin
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,DiscoverTags
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,RegexFilter
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,GitlabSource,ServerName
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchJobTrigger,CreateActionJobsToTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchJobTrigger,DeleteActionJobsToTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,BitbucketServerSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitHubSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitlabSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,MultiBranchJobTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,ScriptPath
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,SingleSvnSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,SourceType
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,SvnSource
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,MultiBranchPipeline,TimerTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,NoScmPipeline,DisableConcurrent
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,NoScmPipeline,RemoteTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,NoScmPipeline,TimerTrigger
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,Parameter,DefaultValue
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,PipelineSpec,MultiBranchPipeline
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SingleSvnSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SingleSvnSource,ScmId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SvnSource,CredentialId
-API rule violation: names_match,./pkg/apis/devops/v1alpha3,SvnSource,ScmId
+API rule violation: list_type_missing,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,DevOpsProjectList,Items
+API rule violation: list_type_missing,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,NoScmPipeline,Parameters
+API rule violation: list_type_missing,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,PipelineList,Items
 API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,APIResourceList,APIResources
 API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,Duration,Duration
 API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,InternalEvent,Object
@@ -96,3 +39,46 @@ API rule violation: names_match,k8s.io/apimachinery/pkg/apis/meta/v1,Time,Time
 API rule violation: names_match,k8s.io/apimachinery/pkg/runtime,Unknown,ContentEncoding
 API rule violation: names_match,k8s.io/apimachinery/pkg/runtime,Unknown,ContentType
 API rule violation: names_match,k8s.io/apimachinery/pkg/runtime,Unknown,Raw
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,ApiUri
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,CloneOption
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,CredentialId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromForks
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,DiscoverPRFromOrigin
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,RegexFilter
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,BitbucketServerSource,ScmId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,DiscarderProperty,DaysToKeep
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,DiscarderProperty,NumToKeep
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GitSource,CloneOption
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GitSource,CredentialId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GitSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GitSource,RegexFilter
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GitSource,ScmId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,ApiUri
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,CloneOption
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,CredentialId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,DiscoverBranches
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,DiscoverPRFromForks
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,DiscoverPRFromOrigin
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,RegexFilter
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,GithubSource,ScmId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchJobTrigger,CreateActionJobsToTrigger
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchJobTrigger,DeleteActionJobsToTrigger
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,BitbucketServerSource
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitHubSource
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,GitSource
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,MultiBranchJobTrigger
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,ScriptPath
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,SingleSvnSource
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,SourceType
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,SvnSource
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,MultiBranchPipeline,TimerTrigger
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,NoScmPipeline,DisableConcurrent
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,NoScmPipeline,RemoteTrigger
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,NoScmPipeline,TimerTrigger
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,Parameter,DefaultValue
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,PipelineSpec,MultiBranchPipeline
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,SingleSvnSource,CredentialId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,SingleSvnSource,ScmId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,SvnSource,CredentialId
+API rule violation: names_match,kubesphere.io/kubesphere/pkg/apis/devops/v1alpha3,SvnSource,ScmId
--- a/api/ks-openapi-spec/swagger.json
+++ b/api/ks-openapi-spec/swagger.json
--- a/api/openapi-spec/swagger.json
+++ b/api/openapi-spec/swagger.json
--- a/build/ks-apiserver/Dockerfile
+++ b/build/ks-apiserver/Dockerfile
@@ -3,15 +3,8 @@
 # that can be found in the LICENSE file.
 FROM alpine:3.11

-ARG HELM_VERSION=v3.5.2
+RUN apk add --no-cache ca-certificates 

-RUN apk add --no-cache ca-certificates
-# install helm
-RUN wget https://get.helm.sh/helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    tar xvf helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    rm helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    mv linux-amd64/helm /usr/bin/ && \
-    rm -rf linux-amd64
 # To speed up building process, we copy binary directly from make
 # result instead of building it again, so make sure you run the 
 # following command first before building docker image
--- a/build/ks-controller-manager/Dockerfile
+++ b/build/ks-controller-manager/Dockerfile
@@ -3,23 +3,9 @@
 # that can be found in the LICENSE file.
 FROM alpine:3.11

-ARG HELM_VERSION=v3.5.2
-ARG KUSTOMIZE_VERSION=v4.0.5
+COPY  /bin/cmd/controller-manager /usr/local/bin/

 RUN apk add --no-cache ca-certificates
-# install helm
-RUN wget https://get.helm.sh/helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    tar xvf helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    rm helm-${HELM_VERSION}-linux-amd64.tar.gz && \
-    mv linux-amd64/helm /usr/bin/ && \
-    rm -rf linux-amd64
-# install kustomize
-RUN wget https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F${KUSTOMIZE_VERSION}/kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
-    tar xvf kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
-    rm kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz && \
-    mv kustomize /usr/bin
-    
-COPY  /bin/cmd/controller-manager /usr/local/bin/

 EXPOSE 8443 8080

--- a/cmd/controller-manager/app/controllers.go
+++ b/cmd/controller-manager/app/controllers.go
@@ -20,11 +20,9 @@ import (
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/client-go/tools/cache"
 	"k8s.io/klog"
-	"sigs.k8s.io/controller-runtime/pkg/manager"
-	"sigs.k8s.io/kubefed/pkg/controller/util"
-
 	iamv1alpha2 "kubesphere.io/kubesphere/pkg/apis/iam/v1alpha2"
 	authoptions "kubesphere.io/kubesphere/pkg/apiserver/authentication/options"
+	"kubesphere.io/kubesphere/pkg/controller/application"
 	"kubesphere.io/kubesphere/pkg/controller/certificatesigningrequest"
 	"kubesphere.io/kubesphere/pkg/controller/cluster"
 	"kubesphere.io/kubesphere/pkg/controller/clusterrolebinding"
@@ -33,14 +31,9 @@ import (
 	"kubesphere.io/kubesphere/pkg/controller/devopsproject"
 	"kubesphere.io/kubesphere/pkg/controller/globalrole"
 	"kubesphere.io/kubesphere/pkg/controller/globalrolebinding"
-	"kubesphere.io/kubesphere/pkg/controller/group"
-	"kubesphere.io/kubesphere/pkg/controller/groupbinding"
 	"kubesphere.io/kubesphere/pkg/controller/job"
-	"kubesphere.io/kubesphere/pkg/controller/loginrecord"
-	"kubesphere.io/kubesphere/pkg/controller/network/ippool"
 	"kubesphere.io/kubesphere/pkg/controller/network/nsnetworkpolicy"
-	"kubesphere.io/kubesphere/pkg/controller/network/nsnetworkpolicy/provider"
-	"kubesphere.io/kubesphere/pkg/controller/notification"
+	"kubesphere.io/kubesphere/pkg/controller/network/provider"
 	"kubesphere.io/kubesphere/pkg/controller/pipeline"
 	"kubesphere.io/kubesphere/pkg/controller/s2ibinary"
 	"kubesphere.io/kubesphere/pkg/controller/s2irun"
@@ -48,14 +41,18 @@ import (
 	"kubesphere.io/kubesphere/pkg/controller/storage/expansion"
 	"kubesphere.io/kubesphere/pkg/controller/user"
 	"kubesphere.io/kubesphere/pkg/controller/virtualservice"
+	"kubesphere.io/kubesphere/pkg/controller/workspacerole"
+	"kubesphere.io/kubesphere/pkg/controller/workspacerolebinding"
+	"kubesphere.io/kubesphere/pkg/controller/workspacetemplate"
 	"kubesphere.io/kubesphere/pkg/informers"
 	"kubesphere.io/kubesphere/pkg/simple/client/devops"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
 	ldapclient "kubesphere.io/kubesphere/pkg/simple/client/ldap"
-	"kubesphere.io/kubesphere/pkg/simple/client/multicluster"
 	"kubesphere.io/kubesphere/pkg/simple/client/network"
-	ippoolclient "kubesphere.io/kubesphere/pkg/simple/client/network/ippool"
+	"kubesphere.io/kubesphere/pkg/simple/client/openpitrix"
 	"kubesphere.io/kubesphere/pkg/simple/client/s3"
+	"sigs.k8s.io/controller-runtime/pkg/manager"
+	"sigs.k8s.io/kubefed/pkg/controller/util"
 )

 func addControllers(
@@ -65,9 +62,9 @@ func addControllers(
 	devopsClient devops.Interface,
 	s3Client s3.Interface,
 	ldapClient ldapclient.Interface,
-	options *k8s.KubernetesOptions,
 	authenticationOptions *authoptions.AuthenticationOptions,
-	multiClusterOptions *multicluster.Options,
+	openpitrixClient openpitrix.Client,
+	multiClusterEnabled bool,
 	networkOptions *network.Options,
 	serviceMeshEnabled bool,
 	kubectlImage string,
@@ -76,8 +73,7 @@ func addControllers(
 	kubernetesInformer := informerFactory.KubernetesSharedInformerFactory()
 	istioInformer := informerFactory.IstioSharedInformerFactory()
 	kubesphereInformer := informerFactory.KubeSphereSharedInformerFactory()
-
-	multiClusterEnabled := multiClusterOptions.Enable
+	applicationInformer := informerFactory.ApplicationSharedInformerFactory()

 	var vsController, drController manager.Runnable
 	if serviceMeshEnabled {
@@ -98,6 +94,15 @@ func addControllers(
 			client.KubeSphere())
 	}

+	apController := application.NewApplicationController(kubernetesInformer.Core().V1().Services(),
+		kubernetesInformer.Apps().V1().Deployments(),
+		kubernetesInformer.Apps().V1().StatefulSets(),
+		kubesphereInformer.Servicemesh().V1alpha2().Strategies(),
+		kubesphereInformer.Servicemesh().V1alpha2().ServicePolicies(),
+		applicationInformer.App().V1beta1().Applications(),
+		client.Kubernetes(),
+		client.Application())
+
 	jobController := job.NewJobController(kubernetesInformer.Batch().V1().Jobs(), client.Kubernetes())

 	var s2iBinaryController, s2iRunController, devopsProjectController, devopsPipelineController, devopsCredentialController manager.Runnable
@@ -152,8 +157,10 @@ func addControllers(
 		kubernetesInformer.Apps().V1().ReplicaSets(),
 		kubernetesInformer.Apps().V1().StatefulSets())

-	var fedUserCache, fedGlobalRoleBindingCache, fedGlobalRoleCache cache.Store
-	var fedUserCacheController, fedGlobalRoleBindingCacheController, fedGlobalRoleCacheController cache.Controller
+	var fedUserCache, fedGlobalRoleBindingCache, fedGlobalRoleCache,
+		fedWorkspaceRoleCache, fedWorkspaceRoleBindingCache cache.Store
+	var fedUserCacheController, fedGlobalRoleBindingCacheController, fedGlobalRoleCacheController,
+		fedWorkspaceRoleCacheController, fedWorkspaceRoleBindingCacheController cache.Controller

 	if multiClusterEnabled {
 		fedUserClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedUserResource)
@@ -171,31 +178,44 @@ func addControllers(
 			klog.Error(err)
 			return err
 		}
+		fedWorkspaceRoleClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedWorkspaceRoleResource)
+		if err != nil {
+			klog.Error(err)
+			return err
+		}
+		fedWorkspaceRoleBindingClient, err := util.NewResourceClient(client.Config(), &iamv1alpha2.FedWorkspaceRoleBindingResource)
+		if err != nil {
+			klog.Error(err)
+			return err
+		}

 		fedUserCache, fedUserCacheController = util.NewResourceInformer(fedUserClient, "", &iamv1alpha2.FedUserResource, func(object runtime.Object) {})
 		fedGlobalRoleCache, fedGlobalRoleCacheController = util.NewResourceInformer(fedGlobalRoleClient, "", &iamv1alpha2.FedGlobalRoleResource, func(object runtime.Object) {})
 		fedGlobalRoleBindingCache, fedGlobalRoleBindingCacheController = util.NewResourceInformer(fedGlobalRoleBindingClient, "", &iamv1alpha2.FedGlobalRoleBindingResource, func(object runtime.Object) {})
+		fedWorkspaceRoleCache, fedWorkspaceRoleCacheController = util.NewResourceInformer(fedWorkspaceRoleClient, "", &iamv1alpha2.FedWorkspaceRoleResource, func(object runtime.Object) {})
+		fedWorkspaceRoleBindingCache, fedWorkspaceRoleBindingCacheController = util.NewResourceInformer(fedWorkspaceRoleBindingClient, "", &iamv1alpha2.FedWorkspaceRoleBindingResource, func(object runtime.Object) {})

 		go fedUserCacheController.Run(stopCh)
 		go fedGlobalRoleCacheController.Run(stopCh)
 		go fedGlobalRoleBindingCacheController.Run(stopCh)
+		go fedWorkspaceRoleCacheController.Run(stopCh)
+		go fedWorkspaceRoleBindingCacheController.Run(stopCh)
 	}

-	userController := user.NewUserController(client.Kubernetes(), client.KubeSphere(), client.Config(),
+	userController := user.NewUserController(client.Kubernetes(), client.KubeSphere(),
+		client.Config(),
 		kubesphereInformer.Iam().V1alpha2().Users(),
-		kubesphereInformer.Iam().V1alpha2().LoginRecords(),
 		fedUserCache, fedUserCacheController,
+		kubesphereInformer.Iam().V1alpha2().LoginRecords(),
 		kubernetesInformer.Core().V1().ConfigMaps(),
 		ldapClient, devopsClient,
 		authenticationOptions, multiClusterEnabled)

-	loginRecordController := loginrecord.NewLoginRecordController(
+	loginRecordController := user.NewLoginRecordController(
 		client.Kubernetes(),
 		client.KubeSphere(),
 		kubesphereInformer.Iam().V1alpha2().LoginRecords(),
-		kubesphereInformer.Iam().V1alpha2().Users(),
-		authenticationOptions.LoginHistoryRetentionPeriod,
-		authenticationOptions.LoginHistoryMaximumEntries)
+		authenticationOptions.LoginHistoryRetentionPeriod)

 	csrController := certificatesigningrequest.NewController(client.Kubernetes(),
 		kubernetesInformer.Certificates().V1beta1().CertificateSigningRequests(),
@@ -211,19 +231,27 @@ func addControllers(
 	globalRoleController := globalrole.NewController(client.Kubernetes(), client.KubeSphere(),
 		kubesphereInformer.Iam().V1alpha2().GlobalRoles(), fedGlobalRoleCache, fedGlobalRoleCacheController)

+	workspaceRoleController := workspacerole.NewController(client.Kubernetes(), client.KubeSphere(),
+		kubesphereInformer.Iam().V1alpha2().WorkspaceRoles(),
+		fedWorkspaceRoleCache, fedWorkspaceRoleCacheController,
+		kubesphereInformer.Tenant().V1alpha2().WorkspaceTemplates(), multiClusterEnabled)
+
 	globalRoleBindingController := globalrolebinding.NewController(client.Kubernetes(), client.KubeSphere(),
 		kubesphereInformer.Iam().V1alpha2().GlobalRoleBindings(),
 		fedGlobalRoleBindingCache, fedGlobalRoleBindingCacheController,
 		multiClusterEnabled)

-	groupBindingController := groupbinding.NewController(client.Kubernetes(), client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().GroupBindings(),
-		kubesphereInformer.Types().V1beta1().FederatedGroupBindings(),
-		multiClusterEnabled)
+	workspaceRoleBindingController := workspacerolebinding.NewController(client.Kubernetes(), client.KubeSphere(),
+		kubesphereInformer.Iam().V1alpha2().WorkspaceRoleBindings(),
+		fedWorkspaceRoleBindingCache, fedWorkspaceRoleBindingCacheController,
+		kubesphereInformer.Tenant().V1alpha2().WorkspaceTemplates(), multiClusterEnabled)

-	groupController := group.NewController(client.Kubernetes(), client.KubeSphere(),
-		kubesphereInformer.Iam().V1alpha2().Groups(),
-		kubesphereInformer.Types().V1beta1().FederatedGroups(),
+	workspaceTemplateController := workspacetemplate.NewController(client.Kubernetes(), client.KubeSphere(),
+		kubesphereInformer.Tenant().V1alpha2().WorkspaceTemplates(),
+		kubesphereInformer.Tenant().V1alpha1().Workspaces(),
+		kubesphereInformer.Iam().V1alpha2().RoleBases(),
+		kubesphereInformer.Iam().V1alpha2().WorkspaceRoles(),
+		kubesphereInformer.Types().V1beta1().FederatedWorkspaces(),
 		multiClusterEnabled)

 	var clusterController manager.Runnable
@@ -233,7 +261,7 @@ func addControllers(
 			client.Config(),
 			kubesphereInformer.Cluster().V1alpha1().Clusters(),
 			client.KubeSphere().ClusterV1alpha1().Clusters(),
-			multiClusterOptions.ClusterControllerResyncSecond)
+			openpitrixClient)
 	}

 	var nsnpController manager.Runnable
@@ -252,30 +280,25 @@ func addControllers(
 			kubernetesInformer.Core().V1().Namespaces(), nsnpProvider, networkOptions.NSNPOptions)
 	}

-	var ippoolController manager.Runnable
-	ippoolProvider := ippoolclient.NewProvider(kubernetesInformer, client.KubeSphere(), client.Kubernetes(), networkOptions.IPPoolType, options)
-	if ippoolProvider != nil {
-		ippoolController = ippool.NewIPPoolController(kubesphereInformer, kubernetesInformer, client.Kubernetes(), client.KubeSphere(), ippoolProvider)
-	}
-
 	controllers := map[string]manager.Runnable{
-		"virtualservice-controller":     vsController,
-		"destinationrule-controller":    drController,
-		"job-controller":                jobController,
-		"s2ibinary-controller":          s2iBinaryController,
-		"s2irun-controller":             s2iRunController,
-		"storagecapability-controller":  storageCapabilityController,
-		"volumeexpansion-controller":    volumeExpansionController,
-		"user-controller":               userController,
-		"loginrecord-controller":        loginRecordController,
-		"cluster-controller":            clusterController,
-		"nsnp-controller":               nsnpController,
-		"csr-controller":                csrController,
-		"clusterrolebinding-controller": clusterRoleBindingController,
-		"globalrolebinding-controller":  globalRoleBindingController,
-		"ippool-controller":             ippoolController,
-		"groupbinding-controller":       groupBindingController,
-		"group-controller":              groupController,
+		"virtualservice-controller":       vsController,
+		"destinationrule-controller":      drController,
+		"application-controller":          apController,
+		"job-controller":                  jobController,
+		"s2ibinary-controller":            s2iBinaryController,
+		"s2irun-controller":               s2iRunController,
+		"storagecapability-controller":    storageCapabilityController,
+		"volumeexpansion-controller":      volumeExpansionController,
+		"user-controller":                 userController,
+		"loginrecord-controller":          loginRecordController,
+		"cluster-controller":              clusterController,
+		"nsnp-controller":                 nsnpController,
+		"csr-controller":                  csrController,
+		"clusterrolebinding-controller":   clusterRoleBindingController,
+		"globalrolebinding-controller":    globalRoleBindingController,
+		"workspacetemplate-controller":    workspaceTemplateController,
+		"workspacerole-controller":        workspaceRoleController,
+		"workspacerolebinding-controller": workspaceRoleBindingController,
 	}

 	if devopsClient != nil {
@@ -286,11 +309,6 @@ func addControllers(

 	if multiClusterEnabled {
 		controllers["globalrole-controller"] = globalRoleController
-		notificationController, err := notification.NewController(client.Kubernetes(), mgr.GetClient(), mgr.GetCache())
-		if err != nil {
-			return err
-		}
-		controllers["notification-controller"] = notificationController
 	}

 	for name, ctrl := range controllers {
--- a/cmd/controller-manager/app/helper.go
+++ b/cmd/controller-manager/app/helper.go
@@ -17,14 +17,12 @@ limitations under the License.
 package app

 import (
-	"context"
 	"fmt"
-	"net/http"
-	"time"
-
 	"k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset"
 	"k8s.io/apimachinery/pkg/util/wait"
 	"k8s.io/klog"
+	"net/http"
+	"time"
 )

 // WaitForAPIServer waits for the API Server's /healthz endpoint to report "ok" before timeout.
@@ -33,7 +31,7 @@ func WaitForAPIServer(client clientset.Interface, timeout time.Duration) error {

 	err := wait.PollImmediate(time.Second, timeout, func() (bool, error) {
 		healthStatus := 0
-		result := client.Discovery().RESTClient().Get().AbsPath("/healthz").Do(context.Background()).StatusCode(&healthStatus)
+		result := client.Discovery().RESTClient().Get().AbsPath("/healthz").Do().StatusCode(&healthStatus)
 		if result.Error() != nil {
 			lastErr = fmt.Errorf("failed to get apiserver /healthz status: %v", result.Error())
 			return false, nil
--- a/cmd/controller-manager/app/options/options.go
+++ b/cmd/controller-manager/app/options/options.go
@@ -18,16 +18,10 @@ package options

 import (
 	"flag"
-	"strings"
-	"time"
-
-	"k8s.io/apimachinery/pkg/labels"
-
 	"github.com/spf13/pflag"
 	"k8s.io/client-go/tools/leaderelection"
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
-
 	authoptions "kubesphere.io/kubesphere/pkg/apiserver/authentication/options"
 	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
@@ -37,6 +31,8 @@ import (
 	"kubesphere.io/kubesphere/pkg/simple/client/openpitrix"
 	"kubesphere.io/kubesphere/pkg/simple/client/s3"
 	"kubesphere.io/kubesphere/pkg/simple/client/servicemesh"
+	"strings"
+	"time"
 )

 type KubeSphereControllerManagerOptions struct {
@@ -52,15 +48,6 @@ type KubeSphereControllerManagerOptions struct {
 	LeaderElect           bool
 	LeaderElection        *leaderelection.LeaderElectionConfig
 	WebhookCertDir        string
-
-	// KubeSphere is using sigs.k8s.io/application as fundamental object to implement Application Management.
-	// There are other projects also built on sigs.k8s.io/application, when KubeSphere installed along side
-	// them, conflicts happen. So we leave an option to only reconcile applications  matched with the given
-	// selector. Default will reconcile all applications.
-	//    For example
-	//      "kubesphere.io/creator=" means reconcile applications with this label key
-	//      "!kubesphere.io/creator" means exclude applications with this key
-	ApplicationSelector string
 }

 func NewKubeSphereControllerManagerOptions() *KubeSphereControllerManagerOptions {
@@ -79,9 +66,8 @@ func NewKubeSphereControllerManagerOptions() *KubeSphereControllerManagerOptions
 			RenewDeadline: 15 * time.Second,
 			RetryPeriod:   5 * time.Second,
 		},
-		LeaderElect:         false,
-		WebhookCertDir:      "",
-		ApplicationSelector: "",
+		LeaderElect:    false,
+		WebhookCertDir: "",
 	}

 	return s
@@ -112,11 +98,6 @@ func (s *KubeSphereControllerManagerOptions) Flags() cliflag.NamedFlagSets {
 		"if not set, webhook server would look up the server key and certificate in"+
 		"{TempDir}/k8s-webhook-server/serving-certs")

-	gfs := fss.FlagSet("generic")
-	gfs.StringVar(&s.ApplicationSelector, "application-selector", s.ApplicationSelector, ""+
-		"Only reconcile application(sigs.k8s.io/application) objects match given selector, this could avoid conflicts with "+
-		"other projects built on top of sig-application. Default behavior is to reconcile all of application objects.")
-
 	kfs := fss.FlagSet("klog")
 	local := flag.NewFlagSet("klog", flag.ExitOnError)
 	klog.InitFlags(local)
@@ -136,14 +117,6 @@ func (s *KubeSphereControllerManagerOptions) Validate() []error {
 	errs = append(errs, s.OpenPitrixOptions.Validate()...)
 	errs = append(errs, s.NetworkOptions.Validate()...)
 	errs = append(errs, s.LdapOptions.Validate()...)
-
-	if len(s.ApplicationSelector) != 0 {
-		_, err := labels.Parse(s.ApplicationSelector)
-		if err != nil {
-			errs = append(errs, err)
-		}
-	}
-
 	return errs
 }

--- a/cmd/controller-manager/app/server.go
+++ b/cmd/controller-manager/app/server.go
@@ -18,45 +18,31 @@ package app

 import (
 	"fmt"
-	"os"
-
 	"github.com/spf13/cobra"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/labels"
 	utilerrors "k8s.io/apimachinery/pkg/util/errors"
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
 	"k8s.io/klog/klogr"
-	ctrl "sigs.k8s.io/controller-runtime"
-	"sigs.k8s.io/controller-runtime/pkg/manager"
-	"sigs.k8s.io/controller-runtime/pkg/runtime/signals"
-	"sigs.k8s.io/controller-runtime/pkg/webhook"
-
 	"kubesphere.io/kubesphere/cmd/controller-manager/app/options"
 	"kubesphere.io/kubesphere/pkg/apis"
 	controllerconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
-	"kubesphere.io/kubesphere/pkg/controller/application"
 	"kubesphere.io/kubesphere/pkg/controller/namespace"
-	"kubesphere.io/kubesphere/pkg/controller/network/webhooks"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmapplication"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmcategory"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmrelease"
-	"kubesphere.io/kubesphere/pkg/controller/openpitrix/helmrepo"
-	"kubesphere.io/kubesphere/pkg/controller/quota"
-	"kubesphere.io/kubesphere/pkg/controller/serviceaccount"
+	"kubesphere.io/kubesphere/pkg/controller/network/nsnetworkpolicy"
 	"kubesphere.io/kubesphere/pkg/controller/user"
 	"kubesphere.io/kubesphere/pkg/controller/workspace"
-	"kubesphere.io/kubesphere/pkg/controller/workspacerole"
-	"kubesphere.io/kubesphere/pkg/controller/workspacerolebinding"
-	"kubesphere.io/kubesphere/pkg/controller/workspacetemplate"
 	"kubesphere.io/kubesphere/pkg/informers"
 	"kubesphere.io/kubesphere/pkg/simple/client/devops"
 	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
 	ldapclient "kubesphere.io/kubesphere/pkg/simple/client/ldap"
+	"kubesphere.io/kubesphere/pkg/simple/client/openpitrix"
 	"kubesphere.io/kubesphere/pkg/simple/client/s3"
-	"kubesphere.io/kubesphere/pkg/utils/metrics"
 	"kubesphere.io/kubesphere/pkg/utils/term"
+	"os"
+	application "sigs.k8s.io/application/controllers"
+	"sigs.k8s.io/controller-runtime/pkg/manager"
+	"sigs.k8s.io/controller-runtime/pkg/runtime/signals"
+	"sigs.k8s.io/controller-runtime/pkg/webhook"
 )

 func NewControllerManagerCommand() *cobra.Command {
@@ -116,7 +102,6 @@ func NewControllerManagerCommand() *cobra.Command {
 }

 func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{}) error {
-
 	kubernetesClient, err := k8s.NewKubernetesClient(s.KubernetesOptions)
 	if err != nil {
 		klog.Errorf("Failed to create kubernetes clientset %v", err)
@@ -132,8 +117,9 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 	}

 	var ldapClient ldapclient.Interface
-	// when there is no ldapOption, we set ldapClient as nil, which means we don't need to sync user info into ldap.
-	if s.LdapOptions != nil && len(s.LdapOptions.Host) != 0 {
+	if s.LdapOptions == nil || len(s.LdapOptions.Host) == 0 {
+		return fmt.Errorf("ldap service address MUST not be empty")
+	} else {
 		if s.LdapOptions.Host == ldapclient.FAKE_HOST { // for debug only
 			ldapClient = ldapclient.NewSimpleLdap()
 		} else {
@@ -142,8 +128,14 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 				return fmt.Errorf("failed to connect to ldap service, please check ldap status, error: %v", err)
 			}
 		}
-	} else {
-		klog.Warning("ks-controller-manager starts without ldap provided, it will not sync user into ldap")
+	}
+
+	var openpitrixClient openpitrix.Client
+	if s.OpenPitrixOptions != nil && !s.OpenPitrixOptions.IsEmpty() {
+		openpitrixClient, err = openpitrix.NewClient(s.OpenPitrixOptions)
+		if err != nil {
+			return fmt.Errorf("failed to connect to openpitrix, please check openpitrix status, error: %v", err)
+		}
 	}

 	var s3Client s3.Interface
@@ -158,9 +150,9 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 		kubernetesClient.Kubernetes(),
 		kubernetesClient.KubeSphere(),
 		kubernetesClient.Istio(),
+		kubernetesClient.Application(),
 		kubernetesClient.Snapshot(),
-		kubernetesClient.ApiExtensions(),
-		kubernetesClient.Prometheus())
+		kubernetesClient.ApiExtensions())

 	mgrOptions := manager.Options{
 		CertDir: s.WebhookCertDir,
@@ -181,7 +173,7 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 	}

 	klog.V(0).Info("setting up manager")
-	ctrl.SetLogger(klogr.New())
+
 	// Use 8443 instead of 443 cause we need root permission to bind port 443
 	mgr, err := manager.New(kubernetesClient.Config(), mgrOptions)
 	if err != nil {
@@ -192,91 +184,24 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 		klog.Fatalf("unable add APIs to scheme: %v", err)
 	}

-	// register common meta types into schemas.
-	metav1.AddToGroupVersion(mgr.GetScheme(), metav1.SchemeGroupVersion)
-
-	workspaceTemplateReconciler := &workspacetemplate.Reconciler{MultiClusterEnabled: s.MultiClusterOptions.Enable}
-	if err = workspaceTemplateReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace template controller: %v", err)
-	}
-
-	workspaceReconciler := &workspace.Reconciler{}
-	if err = workspaceReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace controller: %v", err)
-	}
-
-	workspaceRoleReconciler := &workspacerole.Reconciler{MultiClusterEnabled: s.MultiClusterOptions.Enable}
-	if err = workspaceRoleReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace role controller: %v", err)
-	}
-
-	workspaceRoleBindingReconciler := &workspacerolebinding.Reconciler{MultiClusterEnabled: s.MultiClusterOptions.Enable}
-	if err = workspaceRoleBindingReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create workspace role binding controller: %v", err)
-	}
-
-	namespaceReconciler := &namespace.Reconciler{}
-	if err = namespaceReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create namespace controller: %v", err)
-	}
-
-	err = helmrepo.Add(mgr)
+	err = workspace.Add(mgr)
 	if err != nil {
-		klog.Fatal("Unable to create helm repo controller")
+		klog.Fatal("Unable to create workspace controller")
 	}

-	err = helmcategory.Add(mgr)
+	err = namespace.Add(mgr)
 	if err != nil {
-		klog.Fatal("Unable to create helm category controller")
+		klog.Fatal("Unable to create namespace controller")
 	}

-	var opS3Client s3.Interface
-	if !s.OpenPitrixOptions.AppStoreConfIsEmpty() {
-		opS3Client, err = s3.NewS3Client(s.OpenPitrixOptions.S3Options)
-		if err != nil {
-			klog.Fatalf("failed to connect to s3, please check openpitrix s3 service status, error: %v", err)
-		}
-		err = (&helmapplication.ReconcileHelmApplication{}).SetupWithManager(mgr)
-		if err != nil {
-			klog.Fatalf("Unable to create helm application controller, error: %s", err)
-		}
-
-		err = (&helmapplication.ReconcileHelmApplicationVersion{}).SetupWithManager(mgr)
-		if err != nil {
-			klog.Fatalf("Unable to create helm application version controller, error: %s ", err)
-		}
-	}
-
-	err = (&helmrelease.ReconcileHelmRelease{
-		// nil interface is valid value.
-		StorageClient:      opS3Client,
-		KsFactory:          informerFactory.KubeSphereSharedInformerFactory(),
-		MultiClusterEnable: s.MultiClusterOptions.Enable,
+	err = (&application.ApplicationReconciler{
+		Scheme: mgr.GetScheme(),
+		Client: mgr.GetClient(),
+		Mapper: mgr.GetRESTMapper(),
+		Log:    klogr.New(),
 	}).SetupWithManager(mgr)
-
 	if err != nil {
-		klog.Fatalf("Unable to create helm release controller, error: %s", err)
-	}
-
-	selector, _ := labels.Parse(s.ApplicationSelector)
-	applicationReconciler := &application.ApplicationReconciler{
-		Scheme:              mgr.GetScheme(),
-		Client:              mgr.GetClient(),
-		Mapper:              mgr.GetRESTMapper(),
-		ApplicationSelector: selector,
-	}
-	if err = applicationReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create application controller: %v", err)
-	}
-
-	saReconciler := &serviceaccount.Reconciler{}
-	if err = saReconciler.SetupWithManager(mgr); err != nil {
-		klog.Fatalf("Unable to create ServiceAccount controller: %v", err)
-	}
-
-	resourceQuotaReconciler := quota.Reconciler{}
-	if err := resourceQuotaReconciler.SetupWithManager(mgr, quota.DefaultMaxConcurrentReconciles, quota.DefaultResyncPeriod, informerFactory.KubernetesSharedInformerFactory()); err != nil {
-		klog.Fatalf("Unable to create ResourceQuota controller: %v", err)
+		klog.Fatal("Unable to create application controller")
 	}

 	// TODO(jeff): refactor config with CRD
@@ -287,9 +212,9 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 		devopsClient,
 		s3Client,
 		ldapClient,
-		s.KubernetesOptions,
 		s.AuthenticationOptions,
-		s.MultiClusterOptions,
+		openpitrixClient,
+		s.MultiClusterOptions.Enable,
 		s.NetworkOptions,
 		servicemeshEnabled,
 		s.AuthenticationOptions.KubectlImage, stopCh); err != nil {
@@ -305,20 +230,8 @@ func run(s *options.KubeSphereControllerManagerOptions, stopCh <-chan struct{})
 	hookServer := mgr.GetWebhookServer()

 	klog.V(2).Info("registering webhooks to the webhook server")
-	hookServer.Register("/validate-email-iam-kubesphere-io-v1alpha2", &webhook.Admission{Handler: &user.EmailValidator{Client: mgr.GetClient()}})
-	hookServer.Register("/validate-network-kubesphere-io-v1alpha1", &webhook.Admission{Handler: &webhooks.ValidatingHandler{C: mgr.GetClient()}})
-	hookServer.Register("/mutate-network-kubesphere-io-v1alpha1", &webhook.Admission{Handler: &webhooks.MutatingHandler{C: mgr.GetClient()}})
-
-	resourceQuotaAdmission, err := quota.NewResourceQuotaAdmission(mgr.GetClient(), mgr.GetScheme())
-	if err != nil {
-		klog.Fatalf("unable to create resource quota admission: %v", err)
-	}
-	hookServer.Register("/validate-quota-kubesphere-io-v1alpha2", &webhook.Admission{Handler: resourceQuotaAdmission})
-
-	klog.V(2).Info("registering metrics to the webhook server")
-	// Add an extra metric endpoint, so we can use the the same metric definition with ks-apiserver
-	// /kapis/metrics is independent of controller-manager's built-in /metrics
-	mgr.AddMetricsExtraHandler("/kapis/metrics", metrics.Handler())
+	hookServer.Register("/validate-email-iam-kubesphere-io-v1alpha2-user", &webhook.Admission{Handler: &user.EmailValidator{Client: mgr.GetClient()}})
+	hookServer.Register("/validate-nsnp-kubesphere-io-v1alpha1-network", &webhook.Admission{Handler: &nsnetworkpolicy.NSNPValidator{Client: mgr.GetClient()}})

 	klog.V(0).Info("Starting the controllers.")
 	if err = mgr.Start(stopCh); err != nil {
--- a/cmd/controller-manager/controller-manager.go
+++ b/cmd/controller-manager/controller-manager.go
@@ -17,9 +17,8 @@ limitations under the License.
 package main

 import (
-	"os"
-
 	"kubesphere.io/kubesphere/cmd/controller-manager/app"
+	"os"
 )

 func main() {
--- a/cmd/ks-apiserver/apiserver.go
+++ b/cmd/ks-apiserver/apiserver.go
@@ -17,9 +17,8 @@ limitations under the License.
 package main

 import (
-	"log"
-
 	"kubesphere.io/kubesphere/cmd/ks-apiserver/app"
+	"log"
 )

 func main() {
--- a/cmd/ks-apiserver/app/options/options.go
+++ b/cmd/ks-apiserver/app/options/options.go
@@ -20,33 +20,25 @@ import (
 	"crypto/tls"
 	"flag"
 	"fmt"
-
 	cliflag "k8s.io/component-base/cli/flag"
 	"k8s.io/klog"
-	runtimecache "sigs.k8s.io/controller-runtime/pkg/cache"
-
-	"kubesphere.io/kubesphere/pkg/apis"
 	"kubesphere.io/kubesphere/pkg/apiserver"
 	apiserverconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
-	"kubesphere.io/kubesphere/pkg/client/clientset/versioned/scheme"
 	"kubesphere.io/kubesphere/pkg/informers"
 	genericoptions "kubesphere.io/kubesphere/pkg/server/options"
-	"kubesphere.io/kubesphere/pkg/simple/client/alerting"
 	auditingclient "kubesphere.io/kubesphere/pkg/simple/client/auditing/elasticsearch"
 	"kubesphere.io/kubesphere/pkg/simple/client/cache"
-
-	"net/http"
-	"strings"
-
 	"kubesphere.io/kubesphere/pkg/simple/client/devops/jenkins"
 	eventsclient "kubesphere.io/kubesphere/pkg/simple/client/events/elasticsearch"
 	"kubesphere.io/kubesphere/pkg/simple/client/k8s"
 	esclient "kubesphere.io/kubesphere/pkg/simple/client/logging/elasticsearch"
-	"kubesphere.io/kubesphere/pkg/simple/client/monitoring/metricsserver"
 	"kubesphere.io/kubesphere/pkg/simple/client/monitoring/prometheus"
+	"kubesphere.io/kubesphere/pkg/simple/client/openpitrix"
 	"kubesphere.io/kubesphere/pkg/simple/client/s3"
 	fakes3 "kubesphere.io/kubesphere/pkg/simple/client/s3/fake"
 	"kubesphere.io/kubesphere/pkg/simple/client/sonarqube"
+	"net/http"
+	"strings"
 )

 type ServerRunOptions struct {
@@ -86,7 +78,6 @@ func (s *ServerRunOptions) Flags() (fss cliflag.NamedFlagSets) {
 	s.MultiClusterOptions.AddFlags(fss.FlagSet("multicluster"), s.MultiClusterOptions)
 	s.EventsOptions.AddFlags(fss.FlagSet("events"), s.EventsOptions)
 	s.AuditingOptions.AddFlags(fss.FlagSet("auditing"), s.AuditingOptions)
-	s.AlertingOptions.AddFlags(fss.FlagSet("alerting"), s.AlertingOptions)

 	fs = fss.FlagSet("klog")
 	local := flag.NewFlagSet("klog", flag.ExitOnError)
@@ -114,7 +105,7 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 	apiServer.KubernetesClient = kubernetesClient

 	informerFactory := informers.NewInformerFactories(kubernetesClient.Kubernetes(), kubernetesClient.KubeSphere(),
-		kubernetesClient.Istio(), kubernetesClient.Snapshot(), kubernetesClient.ApiExtensions(), kubernetesClient.Prometheus())
+		kubernetesClient.Istio(), kubernetesClient.Application(), kubernetesClient.Snapshot(), kubernetesClient.ApiExtensions())
 	apiServer.InformerFactory = informerFactory

 	if s.MonitoringOptions == nil || len(s.MonitoringOptions.Endpoint) == 0 {
@@ -127,10 +118,8 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 		apiServer.MonitoringClient = monitoringClient
 	}

-	apiServer.MetricsClient = metricsserver.NewMetricsClient(kubernetesClient.Kubernetes(), s.KubernetesOptions)
-
 	if s.LoggingOptions.Host != "" {
-		loggingClient, err := esclient.NewClient(s.LoggingOptions)
+		loggingClient, err := esclient.NewElasticsearch(s.LoggingOptions)
 		if err != nil {
 			return nil, fmt.Errorf("failed to connect to elasticsearch, please check elasticsearch status, error: %v", err)
 		}
@@ -166,7 +155,9 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 	}

 	var cacheClient cache.Interface
-	if s.RedisOptions != nil && len(s.RedisOptions.Host) != 0 {
+	if s.RedisOptions == nil || len(s.RedisOptions.Host) == 0 {
+		return nil, fmt.Errorf("redis service address MUST not be empty, please check configmap/kubesphere-config in kubesphere-system namespace")
+	} else {
 		if s.RedisOptions.Host == fakeInterface && s.DebugMode {
 			apiServer.CacheClient = cache.NewSimpleCache()
 		} else {
@@ -176,10 +167,6 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 			}
 			apiServer.CacheClient = cacheClient
 		}
-	} else {
-		klog.Warning("ks-apiserver starts without redis provided, it will use in memory cache. " +
-			"This may cause inconsistencies when running ks-apiserver with multiple replicas.")
-		apiServer.CacheClient = cache.NewSimpleCache()
 	}

 	if s.EventsOptions.Host != "" {
@@ -198,12 +185,12 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 		apiServer.AuditingClient = auditingClient
 	}

-	if s.AlertingOptions != nil && (s.AlertingOptions.PrometheusEndpoint != "" || s.AlertingOptions.ThanosRulerEndpoint != "") {
-		alertingClient, err := alerting.NewRuleClient(s.AlertingOptions)
+	if s.OpenPitrixOptions != nil && !s.OpenPitrixOptions.IsEmpty() {
+		opClient, err := openpitrix.NewClient(s.OpenPitrixOptions)
 		if err != nil {
-			return nil, fmt.Errorf("failed to init alerting client: %v", err)
+			return nil, fmt.Errorf("failed to connect to openpitrix, please check openpitrix status, error: %v", err)
 		}
-		apiServer.AlertingClient = alertingClient
+		apiServer.OpenpitrixClient = opClient
 	}

 	server := &http.Server{
@@ -218,16 +205,6 @@ func (s *ServerRunOptions) NewAPIServer(stopCh <-chan struct{}) (*apiserver.APIS
 		server.TLSConfig.Certificates = []tls.Certificate{certificate}
 	}

-	sch := scheme.Scheme
-	if err := apis.AddToScheme(sch); err != nil {
-		klog.Fatalf("unable add APIs to scheme: %v", err)
-	}
-
-	apiServer.RuntimeCache, err = runtimecache.New(apiServer.KubernetesClient.Config(), runtimecache.Options{Scheme: sch})
-	if err != nil {
-		klog.Fatalf("unable to create runtime cache: %v", err)
-	}
-
 	apiServer.Server = server

 	return apiServer, nil
--- a/cmd/ks-apiserver/app/options/validation.go
+++ b/cmd/ks-apiserver/app/options/validation.go
@@ -31,11 +31,9 @@ func (s *ServerRunOptions) Validate() []error {
 	errors = append(errors, s.OpenPitrixOptions.Validate()...)
 	errors = append(errors, s.NetworkOptions.Validate()...)
 	errors = append(errors, s.LoggingOptions.Validate()...)
-	errors = append(errors, s.AuthenticationOptions.Validate()...)
 	errors = append(errors, s.AuthorizationOptions.Validate()...)
 	errors = append(errors, s.EventsOptions.Validate()...)
 	errors = append(errors, s.AuditingOptions.Validate()...)
-	errors = append(errors, s.AlertingOptions.Validate()...)

 	return errors
 }
--- a/cmd/ks-apiserver/app/server.go
+++ b/cmd/ks-apiserver/app/server.go
@@ -18,12 +18,10 @@ package app

 import (
 	"fmt"
-
+	kconfig "github.com/kiali/kiali/config"
 	"github.com/spf13/cobra"
 	utilerrors "k8s.io/apimachinery/pkg/util/errors"
 	cliflag "k8s.io/component-base/cli/flag"
-	"k8s.io/klog"
-
 	"kubesphere.io/kubesphere/cmd/ks-apiserver/app/options"
 	apiserverconfig "kubesphere.io/kubesphere/pkg/apiserver/config"
 	"kubesphere.io/kubesphere/pkg/utils/signals"
@@ -42,8 +40,6 @@ func NewAPIServerCommand() *cobra.Command {
 			GenericServerRunOptions: s.GenericServerRunOptions,
 			Config:                  conf,
 		}
-	} else {
-		klog.Fatal("Failed to load configuration from disk", err)
 	}

 	cmd := &cobra.Command{
@@ -94,13 +90,24 @@ func Run(s *options.ServerRunOptions, stopCh <-chan struct{}) error {
 }

 func initializeServicemeshConfig(s *options.ServerRunOptions) {
+	// Initialize kiali config
+	config := kconfig.NewConfig()
+
 	// Config jaeger query endpoint address
 	if s.ServiceMeshOptions != nil && len(s.ServiceMeshOptions.JaegerQueryHost) != 0 {
 		tracing.JaegerQueryUrl = s.ServiceMeshOptions.JaegerQueryHost
 	}

-	// Set the kiali query endpoint address
-	if s.ServiceMeshOptions != nil && len(s.ServiceMeshOptions.KialiQueryHost) != 0 {
-		tracing.KialiQueryUrl = s.ServiceMeshOptions.KialiQueryHost
-	}
+	// Exclude system namespaces
+	config.API.Namespaces.Exclude = []string{"istio-system", "kubesphere*", "kube*"}
+	config.InCluster = true
+
+	// Set default prometheus service url
+	config.ExternalServices.PrometheusServiceURL = s.ServiceMeshOptions.ServicemeshPrometheusHost
+	config.ExternalServices.PrometheusCustomMetricsURL = config.ExternalServices.PrometheusServiceURL
+
+	// Set istio pilot discovery service url
+	config.ExternalServices.Istio.UrlServiceVersion = s.ServiceMeshOptions.IstioPilotHost
+
+	kconfig.Set(config)
 }
--- a/config/crds/app_v1beta1_application.yaml
+++ b/config/crds/app_v1beta1_application.yaml
@@ -1,529 +1,236 @@
-# Copyright 2020 The Kubernetes Authors.
-# SPDX-License-Identifier: Apache-2.0
-
-apiVersion: apiextensions.k8s.io/v1
+apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
-  annotations:
-    api-approved.kubernetes.io: https://github.com/kubernetes-sigs/application/pull/2
-    controller-gen.kubebuilder.io/version: v0.4.0
  creationTimestamp: null
+  labels:
+    controller-tools.k8s.io: "1.0"
  name: applications.app.k8s.io
 spec:
  group: app.k8s.io
  names:
-    categories:
-    - all
    kind: Application
-    listKind: ApplicationList
    plural: applications
-    shortNames:
-    - app
-    singular: application
  scope: Namespaced
-  versions:
-  - additionalPrinterColumns:
-    - description: The type of the application
-      jsonPath: .spec.descriptor.type
-      name: Type
-      type: string
-    - description: The creation date
-      jsonPath: .spec.descriptor.version
-      name: Version
-      type: string
-    - description: The application object owns the matched resources
-      jsonPath: .spec.addOwnerRef
-      name: Owner
-      type: boolean
-    - description: Numbers of components ready
-      jsonPath: .status.componentsReady
-      name: Ready
-      type: string
-    - description: The creation date
-      jsonPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-    name: v1beta1
-    schema:
-      openAPIV3Schema:
-        description: Application is the Schema for the applications API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation
-              of an object. Servers should convert recognized schemas to the latest
-              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this
-              object represents. Servers may infer this from the endpoint the client
-              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: ApplicationSpec defines the specification for an Application.
-            properties:
-              addOwnerRef:
-                description: AddOwnerRef objects - flag to indicate if we need to
-                  add OwnerRefs to matching objects Matching is done by using Selector
-                  to query all ComponentGroupKinds
-                type: boolean
-              assemblyPhase:
-                description: AssemblyPhase represents the current phase of the application's
-                  assembly. An empty value is equivalent to "Succeeded".
-                type: string
-              componentKinds:
-                description: ComponentGroupKinds is a list of Kinds for Application's
-                  components (e.g. Deployments, Pods, Services, CRDs). It can be used
-                  in conjunction with the Application's Selector to list or watch
-                  the Applications components.
-                items:
-                  description: GroupKind specifies a Group and a Kind, but does not
-                    force a version.  This is useful for identifying concepts during
-                    lookup stages without having partially valid types
-                  properties:
-                    group:
-                      type: string
-                    kind:
-                      type: string
-                  required:
-                  - group
-                  - kind
-                  type: object
-                type: array
-              descriptor:
-                description: Descriptor regroups information and metadata about an
-                  application.
-                properties:
-                  description:
-                    description: Description is a brief string description of the
-                      Application.
-                    type: string
-                  icons:
-                    description: Icons is an optional list of icons for an application.
-                      Icon information includes the source, size, and mime type.
-                    items:
-                      description: ImageSpec contains information about an image used
-                        as an icon.
-                      properties:
-                        size:
-                          description: (optional) The size of the image in pixels
-                            (e.g., 25x25).
-                          type: string
-                        src:
-                          description: The source for image represented as either
-                            an absolute URL to the image or a Data URL containing
-                            the image. Data URLs are defined in RFC 2397.
-                          type: string
-                        type:
-                          description: (optional) The mine type of the image (e.g.,
-                            "image/png").
-                          type: string
-                      required:
-                      - src
-                      type: object
-                    type: array
-                  keywords:
-                    description: Keywords is an optional list of key words associated
-                      with the application (e.g. MySQL, RDBMS, database).
-                    items:
-                      type: string
-                    type: array
-                  links:
-                    description: Links are a list of descriptive URLs intended to
-                      be used to surface additional documentation, dashboards, etc.
-                    items:
-                      description: Link contains information about an URL to surface
-                        documentation, dashboards, etc.
-                      properties:
-                        description:
-                          description: Description is human readable content explaining
-                            the purpose of the link.
-                          type: string
-                        url:
-                          description: Url typically points at a website address.
-                          type: string
-                      type: object
-                    type: array
-                  maintainers:
-                    description: Maintainers is an optional list of maintainers of
-                      the application. The maintainers in this list maintain the the
-                      source code, images, and package for the application.
-                    items:
-                      description: ContactData contains information about an individual
-                        or organization.
-                      properties:
-                        email:
-                          description: Email is the email address.
-                          type: string
-                        name:
-                          description: Name is the descriptive name.
-                          type: string
-                        url:
-                          description: Url could typically be a website address.
-                          type: string
-                      type: object
-                    type: array
-                  notes:
-                    description: Notes contain a human readable snippets intended
-                      as a quick start for the users of the Application. CommonMark
-                      markdown syntax may be used for rich text representation.
-                    type: string
-                  owners:
-                    description: Owners is an optional list of the owners of the installed
-                      application. The owners of the application should be contacted
-                      in the event of a planned or unplanned disruption affecting
-                      the application.
-                    items:
-                      description: ContactData contains information about an individual
-                        or organization.
-                      properties:
-                        email:
-                          description: Email is the email address.
-                          type: string
-                        name:
-                          description: Name is the descriptive name.
-                          type: string
-                        url:
-                          description: Url could typically be a website address.
-                          type: string
-                      type: object
-                    type: array
-                  type:
-                    description: Type is the type of the application (e.g. WordPress,
-                      MySQL, Cassandra).
-                    type: string
-                  version:
-                    description: Version is an optional version indicator for the
-                      Application.
-                    type: string
+  validation:
+    openAPIV3Schema:
+      properties:
+        apiVersion:
+          type: string
+        kind:
+          type: string
+        metadata:
+          type: object
+        spec:
+          properties:
+            assemblyPhase:
+              type: string
+            componentKinds:
+              items:
                type: object
-              info:
-                description: Info contains human readable key,value pairs for the
-                  Application.
-                items:
-                  description: InfoItem is a human readable key,value pair containing
-                    important information about how to access the Application.
-                  properties:
-                    name:
-                      description: Name is a human readable title for this piece of
-                        information.
-                      type: string
-                    type:
-                      description: Type of the value for this InfoItem.
-                      type: string
-                    value:
-                      description: Value is human readable content.
-                      type: string
-                    valueFrom:
-                      description: ValueFrom defines a reference to derive the value
-                        from another source.
-                      properties:
-                        configMapKeyRef:
-                          description: Selects a key of a ConfigMap.
-                          properties:
-                            apiVersion:
-                              description: API version of the referent.
-                              type: string
-                            fieldPath:
-                              description: 'If referring to a piece of an object instead
-                                of an entire object, this string should contain a
-                                valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
-                                For example, if the object reference is to a container
-                                within a pod, this would take on a value like: "spec.containers{name}"
-                                (where "name" refers to the name of the container
-                                that triggered the event) or if no container name
-                                is specified "spec.containers[2]" (container with
-                                index 2 in this pod). This syntax is chosen only to
-                                have some well-defined way of referencing a part of
-                                an object. TODO: this design is not final and this
-                                field is subject to change in the future.'
-                              type: string
-                            key:
-                              description: The key to select.
-                              type: string
-                            kind:
-                              description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                              type: string
-                            name:
-                              description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                              type: string
-                            namespace:
-                              description: 'Namespace of the referent. More info:
-                                https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                              type: string
-                            resourceVersion:
-                              description: 'Specific resourceVersion to which this
-                                reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                              type: string
-                            uid:
-                              description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                              type: string
-                          type: object
-                        ingressRef:
-                          description: Select an Ingress.
-                          properties:
-                            apiVersion:
-                              description: API version of the referent.
-                              type: string
-                            fieldPath:
-                              description: 'If referring to a piece of an object instead
-                                of an entire object, this string should contain a
-                                valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
-                                For example, if the object reference is to a container
-                                within a pod, this would take on a value like: "spec.containers{name}"
-                                (where "name" refers to the name of the container
-                                that triggered the event) or if no container name
-                                is specified "spec.containers[2]" (container with
-                                index 2 in this pod). This syntax is chosen only to
-                                have some well-defined way of referencing a part of
-                                an object. TODO: this design is not final and this
-                                field is subject to change in the future.'
-                              type: string
-                            host:
-                              description: The optional host to select.
-                              type: string
-                            kind:
-                              description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                              type: string
-                            name:
-                              description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                              type: string
-                            namespace:
-                              description: 'Namespace of the referent. More info:
-                                https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                              type: string
-                            path:
-                              description: The optional HTTP path.
-                              type: string
-                            protocol:
-                              description: Protocol for the ingress
-                              type: string
-                            resourceVersion:
-                              description: 'Specific resourceVersion to which this
-                                reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                              type: string
-                            uid:
-                              description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                              type: string
-                          type: object
-                        secretKeyRef:
-                          description: Selects a key of a Secret.
-                          properties:
-                            apiVersion:
-                              description: API version of the referent.
-                              type: string
-                            fieldPath:
-                              description: 'If referring to a piece of an object instead
-                                of an entire object, this string should contain a
-                                valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
-                                For example, if the object reference is to a container
-                                within a pod, this would take on a value like: "spec.containers{name}"
-                                (where "name" refers to the name of the container
-                                that triggered the event) or if no container name
-                                is specified "spec.containers[2]" (container with
-                                index 2 in this pod). This syntax is chosen only to
-                                have some well-defined way of referencing a part of
-                                an object. TODO: this design is not final and this
-                                field is subject to change in the future.'
-                              type: string
-                            key:
-                              description: The key to select.
-                              type: string
-                            kind:
-                              description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                              type: string
-                            name:
-                              description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                              type: string
-                            namespace:
-                              description: 'Namespace of the referent. More info:
-                                https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                              type: string
-                            resourceVersion:
-                              description: 'Specific resourceVersion to which this
-                                reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                              type: string
-                            uid:
-                              description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                              type: string
-                          type: object
-                        serviceRef:
-                          description: Select a Service.
-                          properties:
-                            apiVersion:
-                              description: API version of the referent.
-                              type: string
-                            fieldPath:
-                              description: 'If referring to a piece of an object instead
-                                of an entire object, this string should contain a
-                                valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
-                                For example, if the object reference is to a container
-                                within a pod, this would take on a value like: "spec.containers{name}"
-                                (where "name" refers to the name of the container
-                                that triggered the event) or if no container name
-                                is specified "spec.containers[2]" (container with
-                                index 2 in this pod). This syntax is chosen only to
-                                have some well-defined way of referencing a part of
-                                an object. TODO: this design is not final and this
-                                field is subject to change in the future.'
-                              type: string
-                            kind:
-                              description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                              type: string
-                            name:
-                              description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                              type: string
-                            namespace:
-                              description: 'Namespace of the referent. More info:
-                                https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                              type: string
-                            path:
-                              description: The optional HTTP path.
-                              type: string
-                            port:
-                              description: The optional port to select.
-                              format: int32
-                              type: integer
-                            protocol:
-                              description: Protocol for the service
-                              type: string
-                            resourceVersion:
-                              description: 'Specific resourceVersion to which this
-                                reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                              type: string
-                            uid:
-                              description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                              type: string
-                          type: object
-                        type:
-                          description: Type of source.
-                          type: string
-                      type: object
-                  type: object
-                type: array
-              selector:
-                description: 'Selector is a label query over kinds that created by
-                  the application. It must match the component objects'' labels. More
-                  info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors'
+              type: array
+            descriptor:
+              properties:
+                description:
+                  type: string
+                icons:
+                  items:
+                    properties:
+                      size:
+                        type: string
+                      src:
+                        type: string
+                      type:
+                        type: string
+                    required:
+                    - src
+                    type: object
+                  type: array
+                keywords:
+                  items:
+                    type: string
+                  type: array
+                links:
+                  items:
+                    properties:
+                      description:
+                        type: string
+                      url:
+                        type: string
+                    type: object
+                  type: array
+                maintainers:
+                  items:
+                    properties:
+                      email:
+                        type: string
+                      name:
+                        type: string
+                      url:
+                        type: string
+                    type: object
+                  type: array
+                notes:
+                  type: string
+                owners:
+                  items:
+                    properties:
+                      email:
+                        type: string
+                      name:
+                        type: string
+                      url:
+                        type: string
+                    type: object
+                  type: array
+                type:
+                  type: string
+                version:
+                  type: string
+              type: object
+            info:
+              items:
                properties:
-                  matchExpressions:
-                    description: matchExpressions is a list of label selector requirements.
-                      The requirements are ANDed.
-                    items:
-                      description: A label selector requirement is a selector that
-                        contains values, a key, and an operator that relates the key
-                        and values.
-                      properties:
-                        key:
-                          description: key is the label key that the selector applies
-                            to.
-                          type: string
-                        operator:
-                          description: operator represents a key's relationship to
-                            a set of values. Valid operators are In, NotIn, Exists
-                            and DoesNotExist.
-                          type: string
-                        values:
-                          description: values is an array of string values. If the
-                            operator is In or NotIn, the values array must be non-empty.
-                            If the operator is Exists or DoesNotExist, the values
-                            array must be empty. This array is replaced during a strategic
-                            merge patch.
-                          items:
+                  name:
+                    type: string
+                  type:
+                    type: string
+                  value:
+                    type: string
+                  valueFrom:
+                    properties:
+                      configMapKeyRef:
+                        properties:
+                          apiVersion:
                            type: string
-                          type: array
-                      required:
-                      - key
-                      - operator
-                      type: object
-                    type: array
-                  matchLabels:
-                    additionalProperties:
-                      type: string
-                    description: matchLabels is a map of {key,value} pairs. A single
-                      {key,value} in the matchLabels map is equivalent to an element
-                      of matchExpressions, whose key field is "key", the operator
-                      is "In", and the values array contains only "value". The requirements
-                      are ANDed.
+                          fieldPath:
+                            type: string
+                          key:
+                            type: string
+                          kind:
+                            type: string
+                          name:
+                            type: string
+                          namespace:
+                            type: string
+                          resourceVersion:
+                            type: string
+                          uid:
+                            type: string
+                        type: object
+                      ingressRef:
+                        properties:
+                          apiVersion:
+                            type: string
+                          fieldPath:
+                            type: string
+                          host:
+                            type: string
+                          kind:
+                            type: string
+                          name:
+                            type: string
+                          namespace:
+                            type: string
+                          path:
+                            type: string
+                          resourceVersion:
+                            type: string
+                          uid:
+                            type: string
+                        type: object
+                      secretKeyRef:
+                        properties:
+                          apiVersion:
+                            type: string
+                          fieldPath:
+                            type: string
+                          key:
+                            type: string
+                          kind:
+                            type: string
+                          name:
+                            type: string
+                          namespace:
+                            type: string
+                          resourceVersion:
+                            type: string
+                          uid:
+                            type: string
+                        type: object
+                      serviceRef:
+                        properties:
+                          apiVersion:
+                            type: string
+                          fieldPath:
+                            type: string
+                          kind:
+                            type: string
+                          name:
+                            type: string
+                          namespace:
+                            type: string
+                          path:
+                            type: string
+                          port:
+                            format: int32
+                            type: integer
+                          resourceVersion:
+                            type: string
+                          uid:
+                            type: string
+                        type: object
+                      type:
+                        type: string
                    type: object
                type: object
-            type: object
-          status:
-            description: ApplicationStatus defines controller's the observed state
-              of Application
-            properties:
-              components:
-                description: Object status array for all matching objects
-                items:
-                  description: ObjectStatus is a generic status holder for objects
-                  properties:
-                    group:
-                      description: Object group
-                      type: string
-                    kind:
-                      description: Kind of object
-                      type: string
-                    link:
-                      description: Link to object
-                      type: string
-                    name:
-                      description: Name of object
-                      type: string
-                    status:
-                      description: 'Status. Values: InProgress, Ready, Unknown'
-                      type: string
-                  type: object
-                type: array
-              componentsReady:
-                description: 'ComponentsReady: status of the components in the format
-                  ready/total'
-                type: string
-              conditions:
-                description: Conditions represents the latest state of the object
-                items:
-                  description: Condition describes the state of an object at a certain
-                    point.
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transitioned from one status
-                        to another.
-                      format: date-time
-                      type: string
-                    lastUpdateTime:
-                      description: Last time the condition was probed
-                      format: date-time
-                      type: string
-                    message:
-                      description: A human readable message indicating details about
-                        the transition.
-                      type: string
-                    reason:
-                      description: The reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of condition.
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                description: ObservedGeneration is the most recent generation observed.
-                  It corresponds to the Object's generation, which is updated on mutation
-                  by the API Server.
-                format: int64
-                type: integer
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
+              type: array
+            selector:
+              type: object
+          type: object
+        status:
+          properties:
+            components:
+              items:
+                properties:
+                  group:
+                    type: string
+                  kind:
+                    type: string
+                  link:
+                    type: string
+                  name:
+                    type: string
+                  status:
+                    type: string
+                type: object
+              type: array
+            conditions:
+              items:
+                properties:
+                  lastTransitionTime:
+                    format: date-time
+                    type: string
+                  lastUpdateTime:
+                    format: date-time
+                    type: string
+                  message:
+                    type: string
+                  reason:
+                    type: string
+                  status:
+                    type: string
+                  type:
+                    type: string
+                required:
+                - type
+                - status
+                type: object
+              type: array
+            observedGeneration:
+              format: int64
+              type: integer
+          type: object
+  version: v1beta1
 status:
  acceptedNames:
    kind: ""
--- a/config/crds/application.kubesphere.io_helmapplications.yaml
+++ b/config/crds/application.kubesphere.io_helmapplications.yaml
@@ -1,101 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: helmapplications.application.kubesphere.io
-spec:
-  group: application.kubesphere.io
-  names:
-    kind: HelmApplication
-    listKind: HelmApplicationList
-    plural: helmapplications
-    shortNames:
-    - happ
-    singular: helmapplication
-  scope: Cluster
-  versions:
-  - additionalPrinterColumns:
-    - jsonPath: .spec.name
-      name: application name
-      type: string
-    - jsonPath: .metadata.labels.kubesphere\.io/workspace
-      name: workspace
-      type: string
-    - jsonPath: .status.state
-      name: State
-      type: string
-    - jsonPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-    name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: HelmApplication is the Schema for the helmapplications API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: HelmApplicationSpec defines the desired state of HelmApplication
-            properties:
-              abstraction:
-                description: info from frontend
-                type: string
-              appHome:
-                type: string
-              attachments:
-                description: attachments id
-                items:
-                  type: string
-                type: array
-              description:
-                description: description from chart's description or frontend
-                type: string
-              icon:
-                description: The attachment id of the icon
-                type: string
-              name:
-                description: the name of the helm application
-                type: string
-            required:
-            - name
-            type: object
-          status:
-            description: HelmApplicationStatus defines the observed state of HelmApplication
-            properties:
-              latestVersion:
-                description: If this application belong to appStore, latestVersion is the the latest version of the active application version. otherwise latestVersion is the latest version of all application version
-                type: string
-              state:
-                description: 'the state of the helm application: draft, submitted, passed, rejected, suspended, active'
-                type: string
-              statusTime:
-                format: date-time
-                type: string
-              updateTime:
-                format: date-time
-                type: string
-            required:
-            - statusTime
-            - updateTime
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/application.kubesphere.io_helmapplicationversions.yaml
+++ b/config/crds/application.kubesphere.io_helmapplicationversions.yaml
@@ -1,205 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: helmapplicationversions.application.kubesphere.io
-spec:
-  group: application.kubesphere.io
-  names:
-    kind: HelmApplicationVersion
-    listKind: HelmApplicationVersionList
-    plural: helmapplicationversions
-    shortNames:
-    - happver
-    singular: helmapplicationversion
-  scope: Cluster
-  versions:
-  - additionalPrinterColumns:
-    - jsonPath: .spec.name
-      name: application name
-      type: string
-    - jsonPath: .status.state
-      name: State
-      type: string
-    - jsonPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-    name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: HelmApplicationVersion is the Schema for the helmapplicationversions API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: HelmApplicationVersionSpec defines the desired state of HelmApplicationVersion
-            properties:
-              annotations:
-                additionalProperties:
-                  type: string
-                description: Annotations are additional mappings uninterpreted by Helm, made available for inspection by other applications.
-                type: object
-              apiVersion:
-                description: The API Version of this chart.
-                type: string
-              appVersion:
-                description: The version of the application enclosed inside of this chart.
-                type: string
-              condition:
-                description: The condition to check to enable chart
-                type: string
-              created:
-                description: chart create time
-                format: date-time
-                type: string
-              data:
-                description: raw data of chart, it will !!!NOT!!! be save to etcd
-                format: byte
-                type: string
-              dataKey:
-                description: dataKey in the storage
-                type: string
-              dependencies:
-                description: Dependencies are a list of dependencies for a chart.
-                items:
-                  description: Dependency describes a chart upon which another chart depends. Dependencies can be used to express developer intent, or to capture the state of a chart.
-                  properties:
-                    alias:
-                      description: Alias usable alias to be used for the chart
-                      type: string
-                    condition:
-                      description: A yaml path that resolves to a boolean, used for enabling/disabling charts (e.g. subchart1.enabled )
-                      type: string
-                    enabled:
-                      description: Enabled bool determines if chart should be loaded
-                      type: boolean
-                    name:
-                      description: Name is the name of the dependency. This must mach the name in the dependency's Chart.yaml.
-                      type: string
-                    repository:
-                      description: The URL to the repository. Appending `index.yaml` to this string should result in a URL that can be used to fetch the repository index.
-                      type: string
-                    tags:
-                      description: Tags can be used to group charts for enabling/disabling together
-                      items:
-                        type: string
-                      type: array
-                    version:
-                      description: Version is the version (range) of this chart. A lock file will always produce a single version, while a dependency may contain a semantic version range.
-                      type: string
-                  required:
-                  - name
-                  - repository
-                  type: object
-                type: array
-              deprecated:
-                description: Whether or not this chart is deprecated
-                type: boolean
-              description:
-                description: A one-sentence description of the chart
-                type: string
-              digest:
-                description: chart digest
-                type: string
-              home:
-                description: The URL to a relevant project page, git repo, or contact person
-                type: string
-              icon:
-                description: The URL to an icon file.
-                type: string
-              keywords:
-                description: A list of string keywords
-                items:
-                  type: string
-                type: array
-              kubeVersion:
-                description: KubeVersion is a SemVer constraint specifying the version of Kubernetes required.
-                type: string
-              maintainers:
-                description: A list of name and URL/email address combinations for the maintainer(s)
-                items:
-                  description: Maintainer describes a Chart maintainer.
-                  properties:
-                    email:
-                      description: Email is an optional email address to contact the named maintainer
-                      type: string
-                    name:
-                      description: Name is a user name or organization name
-                      type: string
-                    url:
-                      description: URL is an optional URL to an address for the named maintainer
-                      type: string
-                  type: object
-                type: array
-              name:
-                description: The name of the chart
-                type: string
-              sources:
-                description: Source is the URL to the source code of this chart
-                items:
-                  type: string
-                type: array
-              tags:
-                description: The tags to check to enable chart
-                type: string
-              type:
-                description: 'Specifies the chart type: application or library'
-                type: string
-              urls:
-                description: chart url
-                items:
-                  type: string
-                type: array
-              version:
-                description: A SemVer 2 conformant version string of the chart
-                type: string
-            type: object
-          status:
-            description: HelmApplicationVersionStatus defines the observed state of HelmApplicationVersion
-            properties:
-              audit:
-                items:
-                  properties:
-                    message:
-                      description: audit message
-                      type: string
-                    operator:
-                      description: audit operator
-                      type: string
-                    operatorType:
-                      type: string
-                    state:
-                      description: 'audit state: submitted, passed, draft, active, rejected, suspended'
-                      type: string
-                    time:
-                      description: audit time
-                      format: date-time
-                      type: string
-                  required:
-                  - time
-                  type: object
-                type: array
-              state:
-                type: string
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/application.kubesphere.io_helmcategories.yaml
+++ b/config/crds/application.kubesphere.io_helmcategories.yaml
@@ -1,76 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: helmcategories.application.kubesphere.io
-spec:
-  group: application.kubesphere.io
-  names:
-    kind: HelmCategory
-    listKind: HelmCategoryList
-    plural: helmcategories
-    shortNames:
-    - hctg
-    singular: helmcategory
-  scope: Cluster
-  versions:
-  - additionalPrinterColumns:
-    - jsonPath: .spec.name
-      name: name
-      type: string
-    - jsonPath: .status.total
-      name: total
-      type: string
-    - jsonPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-    name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: HelmCategory is the Schema for the helmcategories API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: HelmCategorySpec defines the desired state of HelmRepo
-            properties:
-              description:
-                description: info from frontend
-                type: string
-              locale:
-                type: string
-              name:
-                description: name of the category
-                type: string
-            required:
-            - name
-            type: object
-          status:
-            properties:
-              total:
-                description: total helmapplications belong to this category
-                type: integer
-            required:
-            - total
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/application.kubesphere.io_helmreleases.yaml
+++ b/config/crds/application.kubesphere.io_helmreleases.yaml
@@ -1,145 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: helmreleases.application.kubesphere.io
-spec:
-  group: application.kubesphere.io
-  names:
-    kind: HelmRelease
-    listKind: HelmReleaseList
-    plural: helmreleases
-    shortNames:
-    - hrls
-    singular: helmrelease
-  scope: Cluster
-  versions:
-  - additionalPrinterColumns:
-    - jsonPath: .spec.name
-      name: Release Name
-      type: string
-    - jsonPath: .metadata.labels.kubesphere\.io/workspace
-      name: Workspace
-      type: string
-    - jsonPath: .metadata.labels.kubesphere\.io/cluster
-      name: Cluster
-      type: string
-    - jsonPath: .metadata.labels.kubesphere\.io/namespace
-      name: Namespace
-      type: string
-    - jsonPath: .status.state
-      name: State
-      type: string
-    - jsonPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-    name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: HelmRelease is the Schema for the helmreleases API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: HelmReleaseSpec defines the desired state of HelmRelease
-            properties:
-              appId:
-                description: id of the helmapplication
-                type: string
-              appVerId:
-                description: application version id
-                type: string
-              chartAppVer:
-                description: appVersion from Chart.yaml
-                type: string
-              chartName:
-                description: The name of the chart which will be installed.
-                type: string
-              chartVersion:
-                description: Specify the exact chart version to install. If this is not specified, the latest version is installed
-                type: string
-              description:
-                description: Message got from frontend
-                type: string
-              name:
-                description: Name of the release
-                type: string
-              repoId:
-                description: id of  the repo
-                type: string
-              values:
-                description: helm release values.yaml
-                format: byte
-                type: string
-              version:
-                description: expected release version, when this version is not equal status.version, the release need upgrade this filed should be modified when any filed of the spec modified.
-                type: integer
-            required:
-            - chartName
-            - chartVersion
-            - name
-            - version
-            type: object
-          status:
-            description: HelmReleaseStatus defines the observed state of HelmRelease
-            properties:
-              deployStatus:
-                description: deploy status list of history, which will store at most 10 state
-                items:
-                  properties:
-                    deployTime:
-                      description: deploy time
-                      format: date-time
-                      type: string
-                    message:
-                      description: A human readable message indicating details about why the release is in this state.
-                      type: string
-                    state:
-                      description: deploy state
-                      type: string
-                  required:
-                  - deployTime
-                  - state
-                  type: object
-                type: array
-              lastDeployed:
-                description: last successful deploy time
-                format: date-time
-                type: string
-              lastUpdate:
-                description: last update time
-                format: date-time
-                type: string
-              message:
-                description: A human readable message indicating details about why the release is in this state.
-                type: string
-              state:
-                description: current state
-                type: string
-              version:
-                description: current release version
-                type: integer
-            required:
-            - state
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/application.kubesphere.io_helmrepos.yaml
+++ b/config/crds/application.kubesphere.io_helmrepos.yaml
@@ -1,142 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: helmrepos.application.kubesphere.io
-spec:
-  group: application.kubesphere.io
-  names:
-    kind: HelmRepo
-    listKind: HelmRepoList
-    plural: helmrepos
-    shortNames:
-    - hrepo
-    singular: helmrepo
-  scope: Cluster
-  versions:
-  - additionalPrinterColumns:
-    - jsonPath: .spec.name
-      name: name
-      type: string
-    - jsonPath: .metadata.labels.kubesphere\.io/workspace
-      name: Workspace
-      type: string
-    - jsonPath: .spec.url
-      name: url
-      type: string
-    - jsonPath: .status.state
-      name: State
-      type: string
-    - jsonPath: .metadata.creationTimestamp
-      name: Age
-      type: date
-    name: v1alpha1
-    schema:
-      openAPIV3Schema:
-        description: HelmRepo is the Schema for the helmrepoes API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: HelmRepoSpec defines the desired state of HelmRepo
-            properties:
-              credential:
-                description: helm repo credential
-                properties:
-                  accessKeyID:
-                    type: string
-                  caFile:
-                    description: verify certificates of HTTPS-enabled servers using this CA bundle
-                    type: string
-                  certFile:
-                    description: identify HTTPS client using this SSL certificate file
-                    type: string
-                  insecureSkipTLSVerify:
-                    description: skip tls certificate checks for the repository, default is ture
-                    type: boolean
-                  keyFile:
-                    description: identify HTTPS client using this SSL key file
-                    type: string
-                  password:
-                    description: chart repository password
-                    type: string
-                  secretAccessKey:
-                    type: string
-                  username:
-                    description: chart repository username
-                    type: string
-                type: object
-              description:
-                description: chart repo description from frontend
-                type: string
-              name:
-                description: name of the repo
-                type: string
-              syncPeriod:
-                description: sync period in seconds, no sync when SyncPeriod=0, the minimum SyncPeriod is 180s
-                type: integer
-              url:
-                description: helm repo url
-                type: string
-              version:
-                description: expected repo version, when this version is not equal status.version, the repo need upgrade this filed should be modified when any filed of the spec modified.
-                type: integer
-            required:
-            - name
-            - url
-            type: object
-          status:
-            description: HelmRepoStatus defines the observed state of HelmRepo
-            properties:
-              data:
-                description: repo index
-                type: string
-              lastUpdateTime:
-                description: status last update time
-                format: date-time
-                type: string
-              state:
-                description: current state of the repo, successful, failed or syncing
-                type: string
-              syncState:
-                description: sync state list of history, which will store at most 10 state
-                items:
-                  properties:
-                    message:
-                      description: A human readable message indicating details about why the repo is in this state.
-                      type: string
-                    state:
-                      description: 'last sync state, valid state are: "failed", "success", and ""'
-                      type: string
-                    syncTime:
-                      format: date-time
-                      type: string
-                  required:
-                  - syncTime
-                  type: object
-                type: array
-              version:
-                description: if status.version!=spec.Version, we need sync the repo now
-                type: integer
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/devops.kubesphere.io_pipelines.yaml
+++ b/config/crds/devops.kubesphere.io_pipelines.yaml
@@ -53,8 +53,6 @@ spec:
                      type: object
                    discover_pr_from_origin:
                      type: integer
-                    discover_tags:
-                      type: boolean
                    git_clone_option:
                      properties:
                        depth:
@@ -88,8 +86,6 @@ spec:
                      type: string
                    discover_branches:
                      type: boolean
-                    discover_tags:
-                      type: boolean
                    git_clone_option:
                      properties:
                        depth:
@@ -107,8 +103,6 @@ spec:
                      type: string
                  type: object
                github_source:
-                  description: GithubSource and BitbucketServerSource have the same
-                    structure, but we don't use one due to crd errors
                  properties:
                    api_uri:
                      type: string
@@ -125,8 +119,6 @@ spec:
                      type: object
                    discover_pr_from_origin:
                      type: integer
-                    discover_tags:
-                      type: boolean
                    git_clone_option:
                      properties:
                        depth:
@@ -145,45 +137,6 @@ spec:
                    scm_id:
                      type: string
                  type: object
-                gitlab_source:
-                  properties:
-                    api_uri:
-                      type: string
-                    credential_id:
-                      type: string
-                    discover_branches:
-                      type: integer
-                    discover_pr_from_forks:
-                      properties:
-                        strategy:
-                          type: integer
-                        trust:
-                          type: integer
-                      type: object
-                    discover_pr_from_origin:
-                      type: integer
-                    discover_tags:
-                      type: boolean
-                    git_clone_option:
-                      properties:
-                        depth:
-                          type: integer
-                        shallow:
-                          type: boolean
-                        timeout:
-                          type: integer
-                      type: object
-                    owner:
-                      type: string
-                    regex_filter:
-                      type: string
-                    repo:
-                      type: string
-                    scm_id:
-                      type: string
-                    server_name:
-                      type: string
-                  type: object
                multibranch_job_trigger:
                  properties:
                    create_action_job_to_trigger:
--- a/config/crds/iam.kubesphere.io_groupbindings.yaml
+++ b/config/crds/iam.kubesphere.io_groupbindings.yaml
@@ -1,69 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: groupbindings.iam.kubesphere.io
-spec:
-  additionalPrinterColumns:
-  - JSONPath: .groupRef.name
-    name: Group
-    type: string
-  - JSONPath: .users
-    name: Users
-    type: string
-  group: iam.kubesphere.io
-  names:
-    categories:
-    - group
-    kind: GroupBinding
-    listKind: GroupBindingList
-    plural: groupbindings
-    singular: groupbinding
-  scope: Cluster
-  subresources: {}
-  validation:
-    openAPIV3Schema:
-      description: GroupBinding is the Schema for the groupbindings API
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        groupRef:
-          description: GroupRef defines the desired relation of GroupBinding
-          properties:
-            apiGroup:
-              type: string
-            kind:
-              type: string
-            name:
-              type: string
-          type: object
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        users:
-          items:
-            type: string
-          type: array
-      type: object
-  version: v1alpha2
-  versions:
-  - name: v1alpha2
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/iam.kubesphere.io_groups.yaml
+++ b/config/crds/iam.kubesphere.io_groups.yaml
@@ -1,58 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: groups.iam.kubesphere.io
-spec:
-  additionalPrinterColumns:
-  - JSONPath: .metadata.labels.kubesphere\.io/workspace
-    name: Workspace
-    type: string
-  group: iam.kubesphere.io
-  names:
-    categories:
-    - group
-    kind: Group
-    listKind: GroupList
-    plural: groups
-    singular: group
-  scope: Cluster
-  subresources: {}
-  validation:
-    openAPIV3Schema:
-      description: Group is the Schema for the groups API
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: GroupSpec defines the desired state of Group
-          type: object
-        status:
-          description: GroupStatus defines the observed state of Group
-          type: object
-      type: object
-  version: v1alpha2
-  versions:
-  - name: v1alpha2
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/iam.kubesphere.io_users.yaml
+++ b/config/crds/iam.kubesphere.io_users.yaml
@@ -1,6 +1,6 @@

 ---
-apiVersion: apiextensions.k8s.io/v1
+apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
  annotations:
@@ -8,6 +8,13 @@ metadata:
  creationTimestamp: null
  name: users.iam.kubesphere.io
 spec:
+  additionalPrinterColumns:
+  - JSONPath: .spec.email
+    name: Email
+    type: string
+  - JSONPath: .status.state
+    name: Status
+    type: string
  group: iam.kubesphere.io
  names:
    categories:
@@ -17,73 +24,72 @@ spec:
    plural: users
    singular: user
  scope: Cluster
+  subresources: {}
+  validation:
+    openAPIV3Schema:
+      description: User is the Schema for the users API
+      properties:
+        apiVersion:
+          description: 'APIVersion defines the versioned schema of this representation
+            of an object. Servers should convert recognized schemas to the latest
+            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+          type: string
+        kind:
+          description: 'Kind is a string value representing the REST resource this
+            object represents. Servers may infer this from the endpoint the client
+            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+          type: string
+        metadata:
+          description: Standard object's metadata.
+          type: object
+        spec:
+          description: UserSpec defines the desired state of User
+          properties:
+            description:
+              description: Description of the user.
+              type: string
+            displayName:
+              type: string
+            email:
+              description: Unique email address(https://www.ietf.org/rfc/rfc5322.txt).
+              type: string
+            groups:
+              items:
+                type: string
+              type: array
+            lang:
+              description: The preferred written or spoken language for the user.
+              type: string
+            password:
+              description: password will be encrypted by mutating admission webhook
+              type: string
+          required:
+          - email
+          type: object
+        status:
+          description: UserStatus defines the observed state of User
+          properties:
+            lastLoginTime:
+              description: Last login attempt timestamp
+              format: date-time
+              type: string
+            lastTransitionTime:
+              format: date-time
+              type: string
+            reason:
+              type: string
+            state:
+              description: The user status
+              type: string
+          type: object
+      required:
+      - spec
+      type: object
+  version: v1alpha2
  versions:
-  - additionalPrinterColumns:
-    - jsonPath: .spec.email
-      name: Email
-      type: string
-    - jsonPath: .status.state
-      name: Status
-      type: string
-    name: v1alpha2
-    schema:
-      openAPIV3Schema:
-        description: User is the Schema for the users API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: UserSpec defines the desired state of User
-            properties:
-              description:
-                description: Description of the user.
-                type: string
-              displayName:
-                type: string
-              email:
-                description: Unique email address(https://www.ietf.org/rfc/rfc5322.txt).
-                type: string
-              groups:
-                items:
-                  type: string
-                type: array
-              lang:
-                description: The preferred written or spoken language for the user.
-                type: string
-              password:
-                description: password will be encrypted by mutating admission webhook
-                type: string
-            required:
-            - email
-            type: object
-          status:
-            description: UserStatus defines the observed state of User
-            properties:
-              lastLoginTime:
-                description: Last login attempt timestamp
-                format: date-time
-                type: string
-              lastTransitionTime:
-                format: date-time
-                type: string
-              reason:
-                type: string
-              state:
-                description: The user status
-                type: string
-            type: object
-        required:
-        - spec
-        type: object
+  - name: v1alpha2
    served: true
    storage: true
-    subresources: {}
 status:
  acceptedNames:
    kind: ""
--- a/config/crds/monitoring.kubesphere.io_clusterdashboards.yaml
+++ b/config/crds/monitoring.kubesphere.io_clusterdashboards.yaml
@@ -1,184 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.4
-  creationTimestamp: null
-  name: clusterdashboards.monitoring.kubesphere.io
-spec:
-  group: monitoring.kubesphere.io
-  names:
-    kind: ClusterDashboard
-    listKind: ClusterDashboardList
-    plural: clusterdashboards
-    singular: clusterdashboard
-  scope: Cluster
-  validation:
-    openAPIV3Schema:
-      description: ClusterDashboard is the Schema for the culsterdashboards API
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: DashboardSpec defines the desired state of Dashboard
-          properties:
-            datasource:
-              description: Dashboard datasource
-              type: string
-            description:
-              description: Dashboard description
-              type: string
-            panels:
-              description: Collection of panels. Panel is one of [Row](row.md), [Singlestat](#singlestat.md)
-                or [Graph](graph.md)
-              items:
-                description: Supported panel type
-                properties:
-                  bars:
-                    description: Display as a bar chart
-                    type: boolean
-                  colors:
-                    description: Set series color
-                    items:
-                      type: string
-                    type: array
-                  decimals:
-                    description: Limit the decimal numbers
-                    format: int64
-                    type: integer
-                  description:
-                    description: Panel description
-                    type: string
-                  format:
-                    description: Display unit
-                    type: string
-                  id:
-                    description: Panel ID
-                    format: int64
-                    type: integer
-                  lines:
-                    description: Display as a line chart
-                    type: boolean
-                  stack:
-                    description: Display as a stacked chart
-                    type: boolean
-                  targets:
-                    allOf:
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    description: A collection of queries
-                    type: array
-                  title:
-                    description: Name of the row panel
-                    type: string
-                  type:
-                    description: Must be `row`
-                    type: string
-                  yaxes:
-                    description: Y-axis options
-                    items:
-                      properties:
-                        decimals:
-                          description: Limit the decimal numbers
-                          format: int64
-                          type: integer
-                        format:
-                          description: Display unit
-                          type: string
-                      type: object
-                    type: array
-                required:
-                - type
-                type: object
-              type: array
-            templating:
-              description: Templating variables
-              items:
-                description: Templating defines a variable, which can be used as a
-                  placeholder in query
-                properties:
-                  name:
-                    description: Variable name
-                    type: string
-                  query:
-                    description: Set variable values to be the return result of the
-                      query
-                    type: string
-                type: object
-              type: array
-            time:
-              description: Time range for display
-              properties:
-                from:
-                  description: Start time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the end time is set to the last month
-                    since now.
-                  type: string
-                to:
-                  description: End time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the start time is set to the last month
-                    since now.
-                  type: string
-              type: object
-            title:
-              description: Dashboard title
-              type: string
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/monitoring.kubesphere.io_dashboards.yaml
+++ b/config/crds/monitoring.kubesphere.io_dashboards.yaml
@@ -1,184 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.2.4
-  creationTimestamp: null
-  name: dashboards.monitoring.kubesphere.io
-spec:
-  group: monitoring.kubesphere.io
-  names:
-    kind: Dashboard
-    listKind: DashboardList
-    plural: dashboards
-    singular: dashboard
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      description: Dashboard is the Schema for the dashboards API
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: DashboardSpec defines the desired state of Dashboard
-          properties:
-            datasource:
-              description: Dashboard datasource
-              type: string
-            description:
-              description: Dashboard description
-              type: string
-            panels:
-              description: Collection of panels. Panel is one of [Row](row.md), [Singlestat](#singlestat.md)
-                or [Graph](graph.md)
-              items:
-                description: Supported panel type
-                properties:
-                  bars:
-                    description: Display as a bar chart
-                    type: boolean
-                  colors:
-                    description: Set series color
-                    items:
-                      type: string
-                    type: array
-                  decimals:
-                    description: Limit the decimal numbers
-                    format: int64
-                    type: integer
-                  description:
-                    description: Panel description
-                    type: string
-                  format:
-                    description: Display unit
-                    type: string
-                  id:
-                    description: Panel ID
-                    format: int64
-                    type: integer
-                  lines:
-                    description: Display as a line chart
-                    type: boolean
-                  stack:
-                    description: Display as a stacked chart
-                    type: boolean
-                  targets:
-                    allOf:
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    - items:
-                        description: Query editor options
-                        properties:
-                          expr:
-                            description: Input for fetching metrics.
-                            type: string
-                          legendFormat:
-                            description: Legend format for outputs. You can make a
-                              dynamic legend with templating variables.
-                            type: string
-                          refId:
-                            description: Reference ID
-                            format: int64
-                            type: integer
-                          step:
-                            description: Set series time interval
-                            type: string
-                        type: object
-                    description: A collection of queries
-                    type: array
-                  title:
-                    description: Name of the row panel
-                    type: string
-                  type:
-                    description: Must be `row`
-                    type: string
-                  yaxes:
-                    description: Y-axis options
-                    items:
-                      properties:
-                        decimals:
-                          description: Limit the decimal numbers
-                          format: int64
-                          type: integer
-                        format:
-                          description: Display unit
-                          type: string
-                      type: object
-                    type: array
-                required:
-                - type
-                type: object
-              type: array
-            templating:
-              description: Templating variables
-              items:
-                description: Templating defines a variable, which can be used as a
-                  placeholder in query
-                properties:
-                  name:
-                    description: Variable name
-                    type: string
-                  query:
-                    description: Set variable values to be the return result of the
-                      query
-                    type: string
-                type: object
-              type: array
-            time:
-              description: Time range for display
-              properties:
-                from:
-                  description: Start time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the end time is set to the last month
-                    since now.
-                  type: string
-                to:
-                  description: End time in the format of `^now([+-][0-9]+[smhdwMy])?$`,
-                    eg. `now-1M`. It denotes the start time is set to the last month
-                    since now.
-                  type: string
-              type: object
-            title:
-              description: Dashboard title
-              type: string
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/network.kubesphere.io_ipamblocks.yaml
+++ b/config/crds/network.kubesphere.io_ipamblocks.yaml
@@ -1,79 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  creationTimestamp: null
-  name: ipamblocks.network.kubesphere.io
-spec:
-  group: network.kubesphere.io
-  names:
-    kind: IPAMBlock
-    plural: ipamblocks
-  scope: Cluster
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          description: Standard object's metadata.
-          type: object
-        spec:
-          description: Specification of the IPAMBlock.
-          properties:
-            allocations:
-              items:
-                type: integer
-                nullable: true
-              type: array
-            attributes:
-              items:
-                properties:
-                  handle_id:
-                    type: string
-                  secondary:
-                    additionalProperties:
-                      type: string
-                    type: object
-                type: object
-              type: array
-            cidr:
-              type: string
-            deleted:
-              type: boolean
-            id:
-              format: int32
-              type: integer
-            unallocated:
-              items:
-                type: integer
-              type: array
-          required:
-          - allocations
-          - attributes
-          - cidr
-          - deleted
-          - id
-          - unallocated
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/network.kubesphere.io_ipamhandles.yaml
+++ b/config/crds/network.kubesphere.io_ipamhandles.yaml
@@ -1,57 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  creationTimestamp: null
-  name: ipamhandles.network.kubesphere.io
-spec:
-  group: network.kubesphere.io
-  names:
-    kind: IPAMHandle
-    plural: ipamhandles
-  scope: Cluster
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          description: Standard object's metadata.
-          type: object
-        spec:
-          description: Specification of the IPAMHandle.
-          properties:
-            block:
-              additionalProperties:
-                type: integer
-              type: object
-            deleted:
-              type: boolean
-            handleID:
-              type: string
-          required:
-          - block
-          - deleted
-          - handleID
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/network.kubesphere.io_ippools.yaml
+++ b/config/crds/network.kubesphere.io_ippools.yaml
@@ -1,137 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: ippools.network.kubesphere.io
-spec:
-  group: network.kubesphere.io
-  names:
-    kind: IPPool
-    listKind: IPPoolList
-    plural: ippools
-    singular: ippool
-  scope: Cluster
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          properties:
-            blockSize:
-              description: The block size to use for IP address assignments from this
-                pool. Defaults to 26 for IPv4 and 112 for IPv6.
-              type: integer
-            cidr:
-              description: The pool CIDR.
-              type: string
-            disabled:
-              description: When disabled is true, IPAM will not assign addresses from
-                this pool.
-              type: boolean
-            dns:
-              description: DNS contains values interesting for DNS resolvers
-              properties:
-                domain:
-                  type: string
-                nameservers:
-                  items:
-                    type: string
-                  type: array
-                options:
-                  items:
-                    type: string
-                  type: array
-                search:
-                  items:
-                    type: string
-                  type: array
-              type: object
-            gateway:
-              type: string
-            rangeEnd:
-              description: The last ip, inclusive
-              type: string
-            rangeStart:
-              description: The first ip, inclusive
-              type: string
-            routes:
-              items:
-                properties:
-                  dst:
-                    type: string
-                  gateway:
-                    type: string
-                type: object
-              type: array
-            type:
-              type: string
-            vlanConfig:
-              properties:
-                master:
-                  type: string
-                vlanId:
-                  format: int32
-                  type: integer
-              required:
-              - master
-              - vlanId
-              type: object
-          required:
-          - cidr
-          - type
-          type: object
-        status:
-          properties:
-            allocations:
-              type: integer
-            capacity:
-              type: integer
-            reserved:
-              type: integer
-            synced:
-              type: boolean
-            unallocated:
-              type: integer
-            workspaces:
-              additionalProperties:
-                properties:
-                  allocations:
-                    type: integer
-                required:
-                - allocations
-                type: object
-              type: object
-          required:
-          - allocations
-          - capacity
-          - unallocated
-          type: object
-      type: object
-  version: v1alpha1
-  versions:
-  - name: v1alpha1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/notification.kubesphere.io_configs.yaml
+++ b/config/crds/notification.kubesphere.io_configs.yaml
@@ -1,283 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: configs.notification.kubesphere.io
-spec:
-  group: notification.kubesphere.io
-  names:
-    categories:
-    - notification-manager
-    kind: Config
-    listKind: ConfigList
-    plural: configs
-    shortNames:
-    - nc
-    singular: config
-  scope: Cluster
-  versions:
-  - name: v2beta1
-    schema:
-      openAPIV3Schema:
-        description: DingTalkConfig is the Schema for the dingtalkconfigs API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: ConfigSpec defines the desired state of Config
-            properties:
-              dingtalk:
-                properties:
-                  conversation:
-                    description: Only needed when send alerts to the conversation.
-                    properties:
-                      appkey:
-                        description: The key of the application with which to send messages.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      appsecret:
-                        description: The key in the secret to be used. Must be a valid secret key.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                    type: object
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                type: object
-              email:
-                properties:
-                  authIdentify:
-                    description: The identity for PLAIN authentication.
-                    type: string
-                  authPassword:
-                    description: The secret contains the SMTP password for LOGIN and PLAIN authentications.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                  authSecret:
-                    description: The secret contains the SMTP secret for CRAM-MD5 authentication.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                  authUsername:
-                    description: The username for CRAM-MD5, LOGIN and PLAIN authentications.
-                    type: string
-                  from:
-                    description: The sender address.
-                    type: string
-                  hello:
-                    description: The hostname to use when identifying to the SMTP server.
-                    type: string
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  requireTLS:
-                    description: The default SMTP TLS requirement.
-                    type: boolean
-                  smartHost:
-                    description: The address of the SMTP server.
-                    properties:
-                      host:
-                        type: string
-                      port:
-                        type: integer
-                    required:
-                    - host
-                    - port
-                    type: object
-                  tls:
-                    description: TLSConfig configures the options for TLS connections.
-                    properties:
-                      clientCertificate:
-                        description: The certificate of the client.
-                        properties:
-                          cert:
-                            description: The client cert file for the targets.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          key:
-                            description: The client key file for the targets.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                        type: object
-                      insecureSkipVerify:
-                        description: Disable target certificate validation.
-                        type: boolean
-                      rootCA:
-                        description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      serverName:
-                        description: Used to verify the hostname for the targets.
-                        type: string
-                    type: object
-                required:
-                - from
-                - smartHost
-                type: object
-              slack:
-                properties:
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  slackTokenSecret:
-                    description: The token of user or bot.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                type: object
-              webhook:
-                properties:
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                type: object
-              wechat:
-                properties:
-                  labels:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  wechatApiAgentId:
-                    description: The id of the application which sending message.
-                    type: string
-                  wechatApiCorpId:
-                    description: The corp id for authentication.
-                    type: string
-                  wechatApiSecret:
-                    description: The API key to use when talking to the WeChat API.
-                    properties:
-                      key:
-                        description: The key of the secret to select from.  Must be a valid secret key.
-                        type: string
-                      name:
-                        description: Name of the secret.
-                        type: string
-                      namespace:
-                        description: The namespace of the secret, default to the pod's namespace.
-                        type: string
-                    required:
-                    - key
-                    type: object
-                  wechatApiUrl:
-                    description: The WeChat API URL.
-                    type: string
-                required:
-                - wechatApiAgentId
-                - wechatApiCorpId
-                - wechatApiSecret
-                type: object
-            type: object
-          status:
-            description: ConfigStatus defines the observed state of Config
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/notification.kubesphere.io_receivers.yaml
+++ b/config/crds/notification.kubesphere.io_receivers.yaml
@@ -1,590 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: receivers.notification.kubesphere.io
-spec:
-  group: notification.kubesphere.io
-  names:
-    categories:
-    - notification-manager
-    kind: Receiver
-    listKind: ReceiverList
-    plural: receivers
-    shortNames:
-    - nr
-    singular: receiver
-  scope: Cluster
-  versions:
-  - name: v2beta1
-    schema:
-      openAPIV3Schema:
-        description: Receiver is the Schema for the receivers API
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            description: ReceiverSpec defines the desired state of Receiver
-            properties:
-              dingtalk:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  chatbot:
-                    description: Be careful, a ChatBot only can send 20 message per minute.
-                    properties:
-                      keywords:
-                        description: Custom keywords of ChatBot
-                        items:
-                          type: string
-                        type: array
-                      secret:
-                        description: Secret of ChatBot, you can get it after enabled Additional Signature of ChatBot.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      webhook:
-                        description: The webhook of ChatBot which the message will send to.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                    required:
-                    - webhook
-                    type: object
-                  conversation:
-                    description: The conversation which message will send to.
-                    properties:
-                      chatids:
-                        items:
-                          type: string
-                        type: array
-                    required:
-                    - chatids
-                    type: object
-                  dingtalkConfigSelector:
-                    description: DingTalkConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                type: object
-              email:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  emailConfigSelector:
-                    description: EmailConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  to:
-                    description: Receivers' email addresses
-                    items:
-                      type: string
-                    type: array
-                required:
-                - to
-                type: object
-              slack:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  channels:
-                    description: The channel or user to send notifications to.
-                    items:
-                      type: string
-                    type: array
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  slackConfigSelector:
-                    description: SlackConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                required:
-                - channels
-                type: object
-              webhook:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  httpConfig:
-                    description: HTTPClientConfig configures an HTTP client.
-                    properties:
-                      basicAuth:
-                        description: The HTTP basic authentication credentials for the targets.
-                        properties:
-                          password:
-                            description: SecretKeySelector selects a key of a Secret.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          username:
-                            type: string
-                        required:
-                        - username
-                        type: object
-                      bearerToken:
-                        description: The bearer token for the targets.
-                        properties:
-                          key:
-                            description: The key of the secret to select from.  Must be a valid secret key.
-                            type: string
-                          name:
-                            description: Name of the secret.
-                            type: string
-                          namespace:
-                            description: The namespace of the secret, default to the pod's namespace.
-                            type: string
-                        required:
-                        - key
-                        type: object
-                      proxyUrl:
-                        description: HTTP proxy server to use to connect to the targets.
-                        type: string
-                      tlsConfig:
-                        description: TLSConfig to use to connect to the targets.
-                        properties:
-                          clientCertificate:
-                            description: The certificate of the client.
-                            properties:
-                              cert:
-                                description: The client cert file for the targets.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              key:
-                                description: The client key file for the targets.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                            type: object
-                          insecureSkipVerify:
-                            description: Disable target certificate validation.
-                            type: boolean
-                          rootCA:
-                            description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          serverName:
-                            description: Used to verify the hostname for the targets.
-                            type: string
-                        type: object
-                    type: object
-                  service:
-                    description: "`service` is a reference to the service for this webhook. Either `service` or `url` must be specified. \n If the webhook is running within the cluster, then you should use `service`."
-                    properties:
-                      name:
-                        description: '`name` is the name of the service. Required'
-                        type: string
-                      namespace:
-                        description: '`namespace` is the namespace of the service. Required'
-                        type: string
-                      path:
-                        description: '`path` is an optional URL path which will be sent in any request to this service.'
-                        type: string
-                      port:
-                        description: If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive).
-                        format: int32
-                        type: integer
-                      scheme:
-                        description: Http scheme, default is http.
-                        type: string
-                    required:
-                    - name
-                    - namespace
-                    type: object
-                  url:
-                    description: "`url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. \n The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some api servers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. \n Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. \n A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. \n Attempting to use a user or basic auth e.g. \"user:password@\" is not allowed. Fragments (\"#...\") and query parameters (\"?...\") are not allowed, either."
-                    type: string
-                  webhookConfigSelector:
-                    description: WebhookConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                type: object
-              wechat:
-                properties:
-                  alertSelector:
-                    description: Selector to filter alerts.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  enabled:
-                    description: whether the receiver is enabled
-                    type: boolean
-                  toParty:
-                    items:
-                      type: string
-                    type: array
-                  toTag:
-                    items:
-                      type: string
-                    type: array
-                  toUser:
-                    items:
-                      type: string
-                    type: array
-                  wechatConfigSelector:
-                    description: WechatConfig to be selected for this receiver
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                type: object
-            type: object
-          status:
-            description: ReceiverStatus defines the observed state of Receiver
-            type: object
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/quota.kubesphere.io_resourcequotas.yaml
+++ b/config/crds/quota.kubesphere.io_resourcequotas.yaml
@@ -1,170 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: resourcequotas.quota.kubesphere.io
-spec:
-  group: quota.kubesphere.io
-  names:
-    categories:
-    - quota
-    kind: ResourceQuota
-    listKind: ResourceQuotaList
-    plural: resourcequotas
-    singular: resourcequota
-  scope: Cluster
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      description: WorkspaceResourceQuota sets aggregate quota restrictions enforced
-        per workspace
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          description: 'Standard object''s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata'
-          type: object
-        spec:
-          description: Spec defines the desired quota
-          properties:
-            quota:
-              description: Quota defines the desired quota
-              properties:
-                hard:
-                  additionalProperties:
-                    type: string
-                  description: 'hard is the set of desired hard limits for each named
-                    resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
-                  type: object
-                scopeSelector:
-                  description: scopeSelector is also a collection of filters like
-                    scopes that must match each object tracked by a quota but expressed
-                    using ScopeSelectorOperator in combination with possible values.
-                    For a resource to match, both scopes AND scopeSelector (if specified
-                    in spec), must be matched.
-                  properties:
-                    matchExpressions:
-                      description: A list of scope selector requirements by scope
-                        of the resources.
-                      items:
-                        description: A scoped-resource selector requirement is a selector
-                          that contains values, a scope name, and an operator that
-                          relates the scope name and values.
-                        properties:
-                          operator:
-                            description: Represents a scope's relationship to a set
-                              of values. Valid operators are In, NotIn, Exists, DoesNotExist.
-                            type: string
-                          scopeName:
-                            description: The name of the scope that the selector applies
-                              to.
-                            type: string
-                          values:
-                            description: An array of string values. If the operator
-                              is In or NotIn, the values array must be non-empty.
-                              If the operator is Exists or DoesNotExist, the values
-                              array must be empty. This array is replaced during a
-                              strategic merge patch.
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - operator
-                        - scopeName
-                        type: object
-                      type: array
-                  type: object
-                scopes:
-                  description: A collection of filters that must match each object
-                    tracked by a quota. If not specified, the quota matches all objects.
-                  items:
-                    description: A ResourceQuotaScope defines a filter that must match
-                      each object tracked by a quota
-                    type: string
-                  type: array
-              type: object
-            selector:
-              additionalProperties:
-                type: string
-              description: LabelSelector is used to select projects by label.
-              type: object
-          required:
-          - quota
-          - selector
-          type: object
-        status:
-          description: Status defines the actual enforced quota and its current usage
-          properties:
-            namespaces:
-              description: Namespaces slices the usage by project.
-              items:
-                description: ResourceQuotaStatusByNamespace gives status for a particular
-                  project
-                properties:
-                  hard:
-                    additionalProperties:
-                      type: string
-                    description: 'Hard is the set of enforced hard limits for each
-                      named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
-                    type: object
-                  namespace:
-                    description: Namespace the project this status applies to
-                    type: string
-                  used:
-                    additionalProperties:
-                      type: string
-                    description: Used is the current observed total usage of the resource
-                      in the namespace.
-                    type: object
-                required:
-                - namespace
-                type: object
-              type: array
-            total:
-              description: Total defines the actual enforced quota and its current
-                usage across all projects
-              properties:
-                hard:
-                  additionalProperties:
-                    type: string
-                  description: 'Hard is the set of enforced hard limits for each named
-                    resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
-                  type: object
-                used:
-                  additionalProperties:
-                    type: string
-                  description: Used is the current observed total usage of the resource
-                    in the namespace.
-                  type: object
-              type: object
-          required:
-          - namespaces
-          - total
-          type: object
-      required:
-      - spec
-      type: object
-  version: v1alpha2
-  versions:
-  - name: v1alpha2
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/types.kubefed.io_federatedgroupbindings.yaml
+++ b/config/crds/types.kubefed.io_federatedgroupbindings.yaml
@@ -1,212 +0,0 @@
---
-apiVersion: core.kubefed.io/v1beta1
-kind: FederatedTypeConfig
-metadata:
-  name: groupbindings.iam.kubesphere.io
-spec:
-  federatedType:
-    group: types.kubefed.io
-    kind: FederatedGroupBinding
-    pluralName: federatedgroupbindings
-    scope: Cluster
-    version: v1beta1
-  propagation: Enabled
-  targetType:
-    group: iam.kubesphere.io
-    kind: GroupBinding
-    pluralName: groupbindings
-    scope: Cluster
-    version: v1alpha2
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedgroupbindings.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedGroupBinding
-    listKind: FederatedGroupBindingList
-    plural: federatedgroupbindings
-    singular: federatedgroupbinding
-  scope: Cluster
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          properties:
-            overrides:
-              items:
-                properties:
-                  clusterName:
-                    type: string
-                  clusterOverrides:
-                    items:
-                      properties:
-                        op:
-                          type: string
-                        path:
-                          type: string
-                        value:
-                          type: object
-                      required:
-                      - path
-                      type: object
-                    type: array
-                required:
-                - clusterName
-                type: object
-              type: array
-            placement:
-              properties:
-                clusterSelector:
-                  description: A label selector is a label query over a set of resources.
-                    The result of matchLabels and matchExpressions are ANDed. An empty
-                    label selector matches all objects. A null label selector matches
-                    no objects.
-                  properties:
-                    matchExpressions:
-                      description: matchExpressions is a list of label selector requirements.
-                        The requirements are ANDed.
-                      items:
-                        description: A label selector requirement is a selector that
-                          contains values, a key, and an operator that relates the
-                          key and values.
-                        properties:
-                          key:
-                            description: key is the label key that the selector applies
-                              to.
-                            type: string
-                          operator:
-                            description: operator represents a key's relationship
-                              to a set of values. Valid operators are In, NotIn, Exists
-                              and DoesNotExist.
-                            type: string
-                          values:
-                            description: values is an array of string values. If the
-                              operator is In or NotIn, the values array must be non-empty.
-                              If the operator is Exists or DoesNotExist, the values
-                              array must be empty. This array is replaced during a
-                              strategic merge patch.
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - key
-                        - operator
-                        type: object
-                      type: array
-                    matchLabels:
-                      additionalProperties:
-                        type: string
-                      description: matchLabels is a map of {key,value} pairs. A single
-                        {key,value} in the matchLabels map is equivalent to an element
-                        of matchExpressions, whose key field is "key", the operator
-                        is "In", and the values array contains only "value". The requirements
-                        are ANDed.
-                      type: object
-                  type: object
-                clusters:
-                  items:
-                    properties:
-                      name:
-                        type: string
-                    required:
-                    - name
-                    type: object
-                  type: array
-              type: object
-            template:
-              properties:
-                metadata:
-                  type: object
-                groupRef:
-                  description: GroupRef defines the desired relation of GroupBinding
-                  properties:
-                    apiGroup:
-                      type: string
-                    kind:
-                      type: string
-                    name:
-                      type: string
-                  type: object
-                users:
-                  items:
-                    type: string
-                  type: array
-              type: object
-          required:
-          - placement
-          - template
-          type: object
-        status:
-          properties:
-            clusters:
-              items:
-                properties:
-                  name:
-                    type: string
-                  status:
-                    type: string
-                required:
-                - name
-                type: object
-              type: array
-            conditions:
-              items:
-                properties:
-                  lastTransitionTime:
-                    description: Last time the condition transit from one status to
-                      another.
-                    type: string
-                  lastUpdateTime:
-                    description: Last time reconciliation resulted in an error or
-                      the last time a change was propagated to member clusters.
-                    type: string
-                  reason:
-                    description: (brief) reason for the condition's last transition.
-                    type: string
-                  status:
-                    description: Status of the condition, one of True, False, Unknown.
-                    type: string
-                  type:
-                    description: Type of cluster condition
-                    type: string
-                required:
-                - status
-                - type
-                type: object
-              type: array
-            observedGeneration:
-              format: int64
-              type: integer
-          type: object
-      required:
-      - spec
-      type: object
-  version: v1beta1
-  versions:
-  - name: v1beta1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/types.kubefed.io_federatedgroups.yaml
+++ b/config/crds/types.kubefed.io_federatedgroups.yaml
@@ -1,200 +0,0 @@
---
-apiVersion: core.kubefed.io/v1beta1
-kind: FederatedTypeConfig
-metadata:
-  name: groups.iam.kubesphere.io
-spec:
-  federatedType:
-    group: types.kubefed.io
-    kind: FederatedGroup
-    pluralName: federatedgroups
-    scope: Cluster
-    version: v1beta1
-  propagation: Enabled
-  targetType:
-    group: iam.kubesphere.io
-    kind: Group
-    pluralName: groups
-    scope: Cluster
-    version: v1alpha2
---
---
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedgroups.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedGroup
-    listKind: FederatedGroupList
-    plural: federatedgroups
-    singular: federatedgroup
-  scope: Cluster
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          properties:
-            overrides:
-              items:
-                properties:
-                  clusterName:
-                    type: string
-                  clusterOverrides:
-                    items:
-                      properties:
-                        op:
-                          type: string
-                        path:
-                          type: string
-                        value:
-                          type: object
-                      required:
-                      - path
-                      type: object
-                    type: array
-                required:
-                - clusterName
-                type: object
-              type: array
-            placement:
-              properties:
-                clusterSelector:
-                  description: A label selector is a label query over a set of resources.
-                    The result of matchLabels and matchExpressions are ANDed. An empty
-                    label selector matches all objects. A null label selector matches
-                    no objects.
-                  properties:
-                    matchExpressions:
-                      description: matchExpressions is a list of label selector requirements.
-                        The requirements are ANDed.
-                      items:
-                        description: A label selector requirement is a selector that
-                          contains values, a key, and an operator that relates the
-                          key and values.
-                        properties:
-                          key:
-                            description: key is the label key that the selector applies
-                              to.
-                            type: string
-                          operator:
-                            description: operator represents a key's relationship
-                              to a set of values. Valid operators are In, NotIn, Exists
-                              and DoesNotExist.
-                            type: string
-                          values:
-                            description: values is an array of string values. If the
-                              operator is In or NotIn, the values array must be non-empty.
-                              If the operator is Exists or DoesNotExist, the values
-                              array must be empty. This array is replaced during a
-                              strategic merge patch.
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - key
-                        - operator
-                        type: object
-                      type: array
-                    matchLabels:
-                      additionalProperties:
-                        type: string
-                      description: matchLabels is a map of {key,value} pairs. A single
-                        {key,value} in the matchLabels map is equivalent to an element
-                        of matchExpressions, whose key field is "key", the operator
-                        is "In", and the values array contains only "value". The requirements
-                        are ANDed.
-                      type: object
-                  type: object
-                clusters:
-                  items:
-                    properties:
-                      name:
-                        type: string
-                    required:
-                    - name
-                    type: object
-                  type: array
-              type: object
-            template:
-              properties:
-                spec:
-                  description: GroupSpec defines the desired state of Group
-                  type: object
-              type: object
-          required:
-          - placement
-          - template
-          type: object
-        status:
-          properties:
-            clusters:
-              items:
-                properties:
-                  name:
-                    type: string
-                  status:
-                    type: string
-                required:
-                - name
-                type: object
-              type: array
-            conditions:
-              items:
-                properties:
-                  lastTransitionTime:
-                    description: Last time the condition transit from one status to
-                      another.
-                    type: string
-                  lastUpdateTime:
-                    description: Last time reconciliation resulted in an error or
-                      the last time a change was propagated to member clusters.
-                    type: string
-                  reason:
-                    description: (brief) reason for the condition's last transition.
-                    type: string
-                  status:
-                    description: Status of the condition, one of True, False, Unknown.
-                    type: string
-                  type:
-                    description: Type of cluster condition
-                    type: string
-                required:
-                - status
-                - type
-                type: object
-              type: array
-            observedGeneration:
-              format: int64
-              type: integer
-          type: object
-      required:
-      - spec
-      type: object
-  version: v1beta1
-  versions:
-  - name: v1beta1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/types.kubefed.io_federatednamespaces.yaml
+++ b/config/crds/types.kubefed.io_federatednamespaces.yaml
@@ -1,164 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatednamespaces.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedNamespace
-    listKind: FederatedNamespaceList
-    plural: federatednamespaces
-    singular: federatednamespace
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  spec:
-                    description: NamespaceSpec describes the attributes on a Namespace.
-                    properties:
-                      finalizers:
-                        description: 'Finalizers is an opaque list of values that must be empty to permanently remove object from storage. More info: https://kubernetes.io/docs/tasks/administer-cluster/namespaces/'
-                        items:
-                          description: FinalizerName is the name identifying a finalizer during namespace lifecycle.
-                          type: string
-                        type: array
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/types.kubefed.io_federatednotificationconfigs.yaml
+++ b/config/crds/types.kubefed.io_federatednotificationconfigs.yaml
@@ -1,393 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatednotificationconfigs.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedNotificationConfig
-    listKind: FederatedNotificationConfigList
-    plural: federatednotificationconfigs
-    singular: federatednotificationconfig
-  scope: Cluster
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                    x-kubernetes-preserve-unknown-fields: true
-                  spec:
-                    description: ConfigSpec defines the desired state of Config
-                    properties:
-                      dingtalk:
-                        properties:
-                          conversation:
-                            description: Only needed when send alerts to the conversation.
-                            properties:
-                              appkey:
-                                description: The key of the application with which to send messages.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              appsecret:
-                                description: The key in the secret to be used. Must be a valid secret key.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                            type: object
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                        type: object
-                      email:
-                        properties:
-                          authIdentify:
-                            description: The identity for PLAIN authentication.
-                            type: string
-                          authPassword:
-                            description: The secret contains the SMTP password for LOGIN and PLAIN authentications.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          authSecret:
-                            description: The secret contains the SMTP secret for CRAM-MD5 authentication.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          authUsername:
-                            description: The username for CRAM-MD5, LOGIN and PLAIN authentications.
-                            type: string
-                          from:
-                            description: The sender address.
-                            type: string
-                          hello:
-                            description: The hostname to use when identifying to the SMTP server.
-                            type: string
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                          requireTLS:
-                            description: The default SMTP TLS requirement.
-                            type: boolean
-                          smartHost:
-                            description: The address of the SMTP server.
-                            properties:
-                              host:
-                                type: string
-                              port:
-                                type: integer
-                            required:
-                            - host
-                            - port
-                            type: object
-                          tls:
-                            description: TLSConfig configures the options for TLS connections.
-                            properties:
-                              clientCertificate:
-                                description: The certificate of the client.
-                                properties:
-                                  cert:
-                                    description: The client cert file for the targets.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                  key:
-                                    description: The client key file for the targets.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                type: object
-                              insecureSkipVerify:
-                                description: Disable target certificate validation.
-                                type: boolean
-                              rootCA:
-                                description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              serverName:
-                                description: Used to verify the hostname for the targets.
-                                type: string
-                            type: object
-                        required:
-                        - from
-                        - smartHost
-                        type: object
-                      slack:
-                        properties:
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                          slackTokenSecret:
-                            description: The token of user or bot.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                        type: object
-                      webhook:
-                        properties:
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                        type: object
-                      wechat:
-                        properties:
-                          labels:
-                            additionalProperties:
-                              type: string
-                            type: object
-                          wechatApiAgentId:
-                            description: The id of the application which sending message.
-                            type: string
-                          wechatApiCorpId:
-                            description: The corp id for authentication.
-                            type: string
-                          wechatApiSecret:
-                            description: The API key to use when talking to the WeChat API.
-                            properties:
-                              key:
-                                description: The key of the secret to select from.  Must be a valid secret key.
-                                type: string
-                              name:
-                                description: Name of the secret.
-                                type: string
-                              namespace:
-                                description: The namespace of the secret, default to the pod's namespace.
-                                type: string
-                            required:
-                            - key
-                            type: object
-                          wechatApiUrl:
-                            description: The WeChat API URL.
-                            type: string
-                        required:
-                        - wechatApiAgentId
-                        - wechatApiCorpId
-                        - wechatApiSecret
-                        type: object
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/types.kubefed.io_federatednotificationreceivers.yaml
+++ b/config/crds/types.kubefed.io_federatednotificationreceivers.yaml
@@ -1,700 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatednotificationreceivers.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedNotificationReceiver
-    listKind: FederatedNotificationReceiverList
-    plural: federatednotificationreceivers
-    singular: federatednotificationreceiver
-  scope: Cluster
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  metadata:
-                    type: object
-                    x-kubernetes-preserve-unknown-fields: true
-                  spec:
-                    description: ReceiverSpec defines the desired state of Receiver
-                    properties:
-                      dingtalk:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          chatbot:
-                            description: Be careful, a ChatBot only can send 20 message per minute.
-                            properties:
-                              keywords:
-                                description: Custom keywords of ChatBot
-                                items:
-                                  type: string
-                                type: array
-                              secret:
-                                description: Secret of ChatBot, you can get it after enabled Additional Signature of ChatBot.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              webhook:
-                                description: The webhook of ChatBot which the message will send to.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                            required:
-                            - webhook
-                            type: object
-                          conversation:
-                            description: The conversation which message will send to.
-                            properties:
-                              chatids:
-                                items:
-                                  type: string
-                                type: array
-                            required:
-                            - chatids
-                            type: object
-                          dingtalkConfigSelector:
-                            description: DingTalkConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                        type: object
-                      email:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          emailConfigSelector:
-                            description: EmailConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          to:
-                            description: Receivers' email addresses
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - to
-                        type: object
-                      slack:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          channels:
-                            description: The channel or user to send notifications to.
-                            items:
-                              type: string
-                            type: array
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          slackConfigSelector:
-                            description: SlackConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                        required:
-                        - channels
-                        type: object
-                      webhook:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          httpConfig:
-                            description: HTTPClientConfig configures an HTTP client.
-                            properties:
-                              basicAuth:
-                                description: The HTTP basic authentication credentials for the targets.
-                                properties:
-                                  password:
-                                    description: SecretKeySelector selects a key of a Secret.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                  username:
-                                    type: string
-                                required:
-                                - username
-                                type: object
-                              bearerToken:
-                                description: The bearer token for the targets.
-                                properties:
-                                  key:
-                                    description: The key of the secret to select from.  Must be a valid secret key.
-                                    type: string
-                                  name:
-                                    description: Name of the secret.
-                                    type: string
-                                  namespace:
-                                    description: The namespace of the secret, default to the pod's namespace.
-                                    type: string
-                                required:
-                                - key
-                                type: object
-                              proxyUrl:
-                                description: HTTP proxy server to use to connect to the targets.
-                                type: string
-                              tlsConfig:
-                                description: TLSConfig to use to connect to the targets.
-                                properties:
-                                  clientCertificate:
-                                    description: The certificate of the client.
-                                    properties:
-                                      cert:
-                                        description: The client cert file for the targets.
-                                        properties:
-                                          key:
-                                            description: The key of the secret to select from.  Must be a valid secret key.
-                                            type: string
-                                          name:
-                                            description: Name of the secret.
-                                            type: string
-                                          namespace:
-                                            description: The namespace of the secret, default to the pod's namespace.
-                                            type: string
-                                        required:
-                                        - key
-                                        type: object
-                                      key:
-                                        description: The client key file for the targets.
-                                        properties:
-                                          key:
-                                            description: The key of the secret to select from.  Must be a valid secret key.
-                                            type: string
-                                          name:
-                                            description: Name of the secret.
-                                            type: string
-                                          namespace:
-                                            description: The namespace of the secret, default to the pod's namespace.
-                                            type: string
-                                        required:
-                                        - key
-                                        type: object
-                                    type: object
-                                  insecureSkipVerify:
-                                    description: Disable target certificate validation.
-                                    type: boolean
-                                  rootCA:
-                                    description: RootCA defines the root certificate authorities that clients use when verifying server certificates.
-                                    properties:
-                                      key:
-                                        description: The key of the secret to select from.  Must be a valid secret key.
-                                        type: string
-                                      name:
-                                        description: Name of the secret.
-                                        type: string
-                                      namespace:
-                                        description: The namespace of the secret, default to the pod's namespace.
-                                        type: string
-                                    required:
-                                    - key
-                                    type: object
-                                  serverName:
-                                    description: Used to verify the hostname for the targets.
-                                    type: string
-                                type: object
-                            type: object
-                          service:
-                            description: "`service` is a reference to the service for this webhook. Either `service` or `url` must be specified. \n If the webhook is running within the cluster, then you should use `service`."
-                            properties:
-                              name:
-                                description: '`name` is the name of the service. Required'
-                                type: string
-                              namespace:
-                                description: '`namespace` is the namespace of the service. Required'
-                                type: string
-                              path:
-                                description: '`path` is an optional URL path which will be sent in any request to this service.'
-                                type: string
-                              port:
-                                description: If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. `port` should be a valid port number (1-65535, inclusive).
-                                format: int32
-                                type: integer
-                              scheme:
-                                description: Http scheme, default is http.
-                                type: string
-                            required:
-                            - name
-                            - namespace
-                            type: object
-                          url:
-                            description: "`url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. \n The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some api servers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. \n Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. \n A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. \n Attempting to use a user or basic auth e.g. \"user:password@\" is not allowed. Fragments (\"#...\") and query parameters (\"?...\") are not allowed, either."
-                            type: string
-                          webhookConfigSelector:
-                            description: WebhookConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                        type: object
-                      wechat:
-                        properties:
-                          alertSelector:
-                            description: Selector to filter alerts.
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          enabled:
-                            description: whether the receiver is enabled
-                            type: boolean
-                          toParty:
-                            items:
-                              type: string
-                            type: array
-                          toTag:
-                            items:
-                              type: string
-                            type: array
-                          toUser:
-                            items:
-                              type: string
-                            type: array
-                          wechatConfigSelector:
-                            description: WechatConfig to be selected for this receiver
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                        type: object
-                    type: object
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-    subresources:
-      status: {}
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/crds/types.kubefed.io_federatedsecrets.yaml
+++ b/config/crds/types.kubefed.io_federatedsecrets.yaml
@@ -1,165 +0,0 @@
-
---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: (devel)
-  creationTimestamp: null
-  name: federatedsecrets.types.kubefed.io
-spec:
-  group: types.kubefed.io
-  names:
-    kind: FederatedSecret
-    listKind: FederatedSecretList
-    plural: federatedsecrets
-    singular: federatedsecret
-  scope: Namespaced
-  versions:
-  - name: v1beta1
-    schema:
-      openAPIV3Schema:
-        properties:
-          apiVersion:
-            description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-            type: string
-          kind:
-            description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-            type: string
-          metadata:
-            type: object
-          spec:
-            properties:
-              overrides:
-                items:
-                  properties:
-                    clusterName:
-                      type: string
-                    clusterOverrides:
-                      items:
-                        properties:
-                          op:
-                            type: string
-                          path:
-                            type: string
-                          value:
-                            type: object
-                        required:
-                        - path
-                        type: object
-                      type: array
-                  required:
-                  - clusterName
-                  type: object
-                type: array
-              placement:
-                properties:
-                  clusterSelector:
-                    description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
-                    properties:
-                      matchExpressions:
-                        description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
-                        items:
-                          description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
-                          properties:
-                            key:
-                              description: key is the label key that the selector applies to.
-                              type: string
-                            operator:
-                              description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-                              type: string
-                            values:
-                              description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
-                              items:
-                                type: string
-                              type: array
-                          required:
-                          - key
-                          - operator
-                          type: object
-                        type: array
-                      matchLabels:
-                        additionalProperties:
-                          type: string
-                        description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
-                        type: object
-                    type: object
-                  clusters:
-                    items:
-                      properties:
-                        name:
-                          type: string
-                      required:
-                      - name
-                      type: object
-                    type: array
-                type: object
-              template:
-                properties:
-                  data:
-                    additionalProperties:
-                      format: byte
-                      type: string
-                    type: object
-                  stringData:
-                    additionalProperties:
-                      type: string
-                    type: object
-                  type:
-                    type: string
-                type: object
-            required:
-            - placement
-            - template
-            type: object
-          status:
-            properties:
-              clusters:
-                items:
-                  properties:
-                    name:
-                      type: string
-                    status:
-                      type: string
-                  required:
-                  - name
-                  type: object
-                type: array
-              conditions:
-                items:
-                  properties:
-                    lastTransitionTime:
-                      description: Last time the condition transit from one status to another.
-                      type: string
-                    lastUpdateTime:
-                      description: Last time reconciliation resulted in an error or the last time a change was propagated to member clusters.
-                      type: string
-                    reason:
-                      description: (brief) reason for the condition's last transition.
-                      type: string
-                    status:
-                      description: Status of the condition, one of True, False, Unknown.
-                      type: string
-                    type:
-                      description: Type of cluster condition
-                      type: string
-                  required:
-                  - status
-                  - type
-                  type: object
-                type: array
-              observedGeneration:
-                format: int64
-                type: integer
-            type: object
-        required:
-        - spec
-        type: object
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []
--- a/config/samples/iam_v1alpha2_group.yaml
+++ b/config/samples/iam_v1alpha2_group.yaml
@@ -1,4 +0,0 @@
-apiVersion: iam.kubesphere.io/v1alpha2
-kind: Group
-metadata:
-  name: group2
--- a/config/samples/iam_v1alpha2_groupbinding.yaml
+++ b/config/samples/iam_v1alpha2_groupbinding.yaml
@@ -1,12 +0,0 @@
-apiVersion: iam.kubesphere.io/v1alpha2
-kind: GroupBinding
-metadata:
-  name: groupbinding-sample1
-groupRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: Group
-  name: groupdemo
-users:
- user1
- user2
-
--- a/config/samples/iam_v1alpha2_rolebindings.yaml
+++ b/config/samples/iam_v1alpha2_rolebindings.yaml
@@ -1,16 +0,0 @@
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  labels:
-    iam.kubesphere.io/group-ref: admin
-  name: group-group1-admin
-  namespace: proj2
-  
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: Role
-  name: admin
-subjects:
- apiGroup: rbac.authorization.k8s.io
-  kind: Group
-  name: group1
--- a/config/webhook/iam.yaml
+++ b/config/webhook/iam.yaml
@@ -8,9 +8,9 @@ webhooks:
    clientConfig:
      caBundle: <caBundle>
      service:
-        name: ks-controller-manager
+        name: webhook-service
        namespace: kubesphere-system
-        path: /validate-email-iam-kubesphere-io-v1alpha2
+        path: /validate-email-iam-kubesphere-io-v1alpha2-user
    failurePolicy: Fail
    name: vemail.iam.kubesphere.io
    rules:
@@ -22,4 +22,19 @@ webhooks:
          - CREATE
          - UPDATE
        resources:
-          - users
+          - users
+
+---
+
+apiVersion: v1
+kind: Service
+metadata:
+  name: webhook-service
+  namespace: kubesphere-system
+spec:
+  ports:
+    - port: 443
+      targetPort: 443
+  selector:
+    app: ks-controller-manager
+    tier: backend
--- a/config/webhook/ks-controller-manager.yaml
+++ b/config/webhook/ks-controller-manager.yaml
@@ -1,13 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: ks-controller-manager
-  namespace: kubesphere-system
-spec:
-  ports:
-    - port: 443
-      protocol: TCP
-      targetPort: 8443
-  selector:
-    app: ks-controller-manager
-    tier: backend
--- a/config/webhook/network.yaml
+++ b/config/webhook/network.yaml
@@ -1,24 +0,0 @@
-apiVersion: admissionregistration.k8s.io/v1beta1
-kind: ValidatingWebhookConfiguration
-metadata:
-  creationTimestamp: null
-  name: kubesphere-nsnp-validate
-webhooks:
-  - clientConfig:
-      caBundle: <caBundle>
-      service:
-        name: ks-controller-manager
-        namespace: kubesphere-system
-        path: /validate-network-kubesphere-io-v1alpha1
-    failurePolicy: Fail
-    name: validate.nsnp.kubesphere.io
-    rules:
-      - apiGroups:
-          - network.kubesphere.io
-        apiVersions:
-          - v1
-        operations:
-          - CREATE
-          - UPDATE
-        resources:
-          - namespacenetworkpolicies
--- a/config/webhook/nsnp.yaml
+++ b/config/webhook/nsnp.yaml
@@ -0,0 +1,24 @@
+apiVersion: admissionregistration.k8s.io/v1beta1
+kind: ValidatingWebhookConfiguration
+metadata:
+  creationTimestamp: null
+  name: kubesphere-nsnp-validate
+webhooks:
+  - clientConfig:
+      caBundle: <caBundle>
+      service:
+        name: kubesphere-controller-manager-service
+        namespace: kubesphere-system
+        path: /validate-nsnp-kubesphere-io-v1alpha1-network
+    failurePolicy: Fail
+    name: validate.nsnp.kubesphere.io
+    rules:
+      - apiGroups:
+          - network.kubesphere.io
+        apiVersions:
+          - v1
+        operations:
+          - CREATE
+          - UPDATE
+        resources:
+          - namespacenetworkpolicies
--- a/config/webhook/quota.yaml
+++ b/config/webhook/quota.yaml
@@ -1,30 +0,0 @@
-apiVersion: admissionregistration.k8s.io/v1
-kind: ValidatingWebhookConfiguration
-metadata:
-  name: resourcesquotas.quota.kubesphere.io
-webhooks:
-  - admissionReviewVersions:
-      - v1beta1
-    clientConfig:
-      caBundle: <caBundle>
-      service:
-        name: ks-controller-manager
-        namespace: kubesphere-system
-        path: /validate-quota-kubesphere-io-v1alpha2
-        port: 443
-    failurePolicy: Ignore
-    matchPolicy: Exact
-    name: resourcesquotas.quota.kubesphere.io
-    namespaceSelector: {}
-    objectSelector: {}
-    rules:
-      - apiGroups:
-          - '*'
-        apiVersions:
-          - '*'
-        operations:
-          - CREATE
-        resources:
-          - pods
-        scope: '*'
-    sideEffects: None
--- a/docs/powered-by-kubesphere.md
+++ b/docs/powered-by-kubesphere.md
@@ -9,4 +9,3 @@ You can open a Pull Request to submit your institution name and homepage if you
 - 方正株式（武汉）科技开发有限公司
 - 武汉麦尔沃克信息技术
 - [InAccel - Scale-out FPGA-Accelerated Applications](https://inaccel.com)
- [三诺生物](http://www.sinocare.com/)
--- a/docs/roadmap.md
+++ b/docs/roadmap.md
@@ -1,6 +1,8 @@
+# KubeSphere Roadmap
+
 KubeSphere Roadmap demonstrates a list of open source product development plans and features being split by the edition and modules, as well as KubeSphere community's anticipation. Obviously, it details the future's direction of KubeSphere, but may change over time. We hope that can help you to get familiar with the project plans and vision through the Roadmap. Of course, if you have any better ideas, welcome to filing [Issues](https://github.com/kubesphere/kubesphere/issues).

-# Release Goals
+## Release Goals

 | Edition  | Schedule |
 |---|---|
@@ -14,158 +16,20 @@ KubeSphere Roadmap demonstrates a list of open source product development plans
 | Release v2.1.1| Feb, 2020 |
 | Release v3.0.0| June, 2020 |

-# v3.1
-## **Feature:**
+## Features

-### IAM:
+### v2.1.1

- [x] Add account confirmation process, solve problems caused by special characters in username.[#2953](https://github.com/kubesphere/kubesphere/issues/2953)
- [ ] Support [CAS](https://apereo.github.io/cas/5.0.x/protocol/CAS-Protocol-Specification.html) identity provider.[#3047](https://github.com/kubesphere/kubesphere/issues/3047)
- [ ] Support [OIDC](https://openid.net/specs/openid-connect-core-1_0.html) identity provider.[#2941](https://github.com/kubesphere/kubesphere/issues/2941)
- [x] Support IDaaS(Alibaba Cloud Identity as a Service) identity provider.[#2997](https://github.com/kubesphere/kubesphere/pull/2997)
- [x] Improve LDAP identity provider, support LDAPS and search filter.[#2970](https://github.com/kubesphere/kubesphere/issues/2970)
- [x] Improve identity provider plugin, simplify identity provider configuration.[#2970](https://github.com/kubesphere/kubesphere/issues/2970)
- [ ] Limit login record maximum entries.[#3191](https://github.com/kubesphere/kubesphere/issues/3191)
- [ ] Service account management, allows to assign role to Service Account [#3211](https://github.com/kubesphere/kubesphere/issues/3211)
+- [ ] Most of the work will be bugfix
+- [ ] Refactor RBAC in order to support future versions regarding third-party plugins with custom access control.
+- [ ] Refactor installer
+- [ ] FluentBit Operator upgrade

-### Multitenancy:
+### v3.0.0

- [ ] Support transfer namespace to another workspace.[#3028](https://github.com/kubesphere/kubesphere/issues/3028)
- [x] Add user group, now users can be assigned to a group and invite a group to a workspace or a project.[#2940](https://github.com/kubesphere/kubesphere/issues/2940)
- [ ] Add resource quota to a workspace. The resource quota is the same with Kubernetes[ResourceQuota](https://kubernetes.io/docs/concepts/policy/resource-quotas/), providesconstraints that limit aggregate resource consumption by all namespaces within.[  ](https://kubernetes.io/docs/concepts/policy/resource-quotas/)[#2939](https://github.com/kubesphere/kubesphere/issues/2939)
- [ ] Add choose of whether to cascade delete related resources when deleting workspace.[#3192](https://github.com/kubesphere/kubesphere/issues/3192)
+#### Multi-cluster

-### DevOps:
-
- [x] Support Gitlab with Kubesphere multi-branch pipeline. [#3100](https://github.com/kubesphere/kubesphere/issues/3100)
- [x] Run multiple DevOps pipelines at the same time instead of just one.[#1811](https://github.com/kubesphere/kubesphere/issues/1811)
- [x] Clone pipeline. Users now can create exactly same pipeline from an existing one.[#3053](https://github.com/kubesphere/kubesphere/issues/3053)
- [x] Add approval control for pipelines. [#2483](https://github.com/kubesphere/kubesphere/issues/2483) [#3006](https://github.com/kubesphere/kubesphere/issues/3006)
- [ ] Add and display the status of the pipeline on the front page. [#3007](https://github.com/kubesphere/kubesphere/issues/3007)
- [x] Support tag trigger pipeline. [#3051](https://github.com/kubesphere/kubesphere/issues/3051)
- [x] Interactive creation pipeline. [#1283](https://github.com/kubesphere/console/issues/1283)
- [x] Add S2I webhook support. [#6](https://github.com/kubesphere/s2ioperator/issues/6)
-
-### microservice
-
- [x] Support graph traffic direction. It will show outbound/inbound traffic direction in composed application traffic graph. [#3153](https://github.com/kubesphere/kubesphere/issues/3153)
- [x] Support Kiali addon. Users can manage istio by kiali directly. [#3106](https://github.com/kubesphere/kubesphere/issues/3106)
- [x] Support Nginx Ingress Gateway monitor. Add prometheus metrics for nginx ingress controller. [#1205](https://github.com/kubesphere/ks-installer/pull/1205)
- [x] Support add router component. [#1426](https://github.com/kubesphere/console/issues/1426) 
-
-### KubeEdge Integration [#3070](https://github.com/kubesphere/kubesphere/issues/3070)
-
- [x] KubeEdge cloud components setup.
- [ ] KubeEdge edge nodes setup.
- [x] Edge nodes logging and metrics support.
- [x] Automatic network configuration on edge node joining/leaving.
- [ ] Automatic taint edge node on joining.
- [ ] Cloud workloads (such as daemonset) with wide tolerations should not be scheduled to edge node by adding a node selector.
- [ ] Scheduling workloads to edge nodes.
-
-### Observability
-
- [x] Utilizing existing Promethues stack setup. [#3068](https://github.com/kubesphere/kubesphere/issues/3068) [#1164](https://github.com/kubesphere/ks-installer/pull/1164) [Guide](https://kubesphere.io/docs/faq/observability/byop/)
-
-#### Custom monitoring [#3067](https://github.com/kubesphere/kubesphere/issues/3067)
-
- [x] Configure ServiceMonitor via UI. [#1031](https://github.com/kubesphere/console/pull/1301) 
- [x] PromQL auto-completion and syntax highlighting. [#1307](https://github.com/kubesphere/console/pull/1307)
- [x] Support cluster-level custom monitoring. [#3193](https://github.com/kubesphere/kubesphere/pull/3193)
- [x] Tools to convert Grafana dashboard to KubeSphere Dashboard [#9](https://github.com/kubesphere/monitoring-dashboard/pull/9)
-
-#### Custom Alerting [#3065](https://github.com/kubesphere/kubesphere/issues/3065)
-
- [x] Prometheus alert rule management. [#3181](https://github.com/kubesphere/kubesphere/pull/3181)
- [x] Alert rule tenant control: global/namespace level alert rules. [#3181](https://github.com/kubesphere/kubesphere/pull/3181)
- [x] List alerts for a specific alert rule. [#3181](https://github.com/kubesphere/kubesphere/pull/3181)
-
-#### Multi-tenant Notification support including Email/DingTalk/Slack/Wechat works/Webhook [#3066](https://github.com/kubesphere/kubesphere/issues/3066)
-
- [ ] More notification channels including Email, DingTalk, Slack, WeChat works, Webhook
- [ ] Multi-tenant control of notification
-
-#### Logging
-
- [x] Support collecting kubelet/docker/containerd logs. [38](https://github.com/kubesphere/fluentbit-operator/pull/38)
- [x] Support output logs to Loki. [#39](https://github.com/kubesphere/fluentbit-operator/pull/39)
- [ ] Support containerd log format
-
-### Application Lifecycle Management (OpenPitrix)
-
- [ ] Refactoring OpenPitrix with CRD, while fix bugs caused by legacy architecture [#3036](https://github.com/kubesphere/kubesphere/issues/3036) [#3001](https://github.com/kubesphere/kubesphere/issues/3001) [#2995](https://github.com/kubesphere/kubesphere/issues/2995) [#2981](https://github.com/kubesphere/kubesphere/issues/2981) [#2954](https://github.com/kubesphere/kubesphere/issues/2954) [#2951](https://github.com/kubesphere/kubesphere/issues/2951) [#2783](https://github.com/kubesphere/kubesphere/issues/2783) [#2713](https://github.com/kubesphere/kubesphere/issues/2713) [#2700](https://github.com/kubesphere/kubesphere/issues/2700) [#1903](https://github.com/kubesphere/kubesphere/issues/1903) 
-
-### Network
-
- [x] IPPool for Calico and VMs [#3057](https://github.com/kubesphere/kubesphere/issues/3057)
- [x] Support for deployment using static IPs [#3058](https://github.com/kubesphere/kubesphere/issues/3058)
- [x] Support network visualization [#3061](https://github.com/kubesphere/kubesphere/issues/3061) [#583](https://github.com/kubesphere/kubesphere/issues/583)
-
-### Metering
-
- [ ] Support for viewing resource consumption at the cluster, workspace, and application template levels [#3062](https://github.com/kubesphere/kubesphere/issues/3062)
-
-### MultiCluster:
-
- [x] Validate member cluster configuration if the member cluster config(e.g. jwtSecret) is not same as host cluster, which can reduce the complexity of joining clusters. [#3232](https://github.com/kubesphere/kubesphere/issues/3232)
- [x] Support configuring cluster controller resync period, some users need to update the cluster component more quickly. [#3213](https://github.com/kubesphere/kubesphere/issues/3213)
- [x] Lightweight member cluster installation, support running kubepshere without redis and ldap and so on. [#3056](https://github.com/kubesphere/kubesphere/issues/3056)
- [x] Refactor cluster controller(it should updates the cluster.staus field instead of the tower server when using proxy connection).  [#3234](https://github.com/kubesphere/kubesphere/issues/3234)
- [ ] Support the tower server and agent running highly available(server with leader election option and running more than one agent). [#31](https://github.com/kubesphere/tower/issues/31)
-
-## **Upgrade:**
-
- [x] Upgrade isito version from 1.4.8 => 1.6.10[#3326](https://github.com/kubesphere/kubesphere/issues/3236)
- [x] Upgrade prometheus client_golang to v1.5.1, upgrade prometheus to v1.8.2 [3097](https://github.com/kubesphere/kubesphere/pull/3097)
- [x] Upgrade Kubectl version for the Toolbox, and the Kubectl Verion will be matched with the Kubernetes Server version. [#3103](https://github.com/kubesphere/kubesphere/issues/3103)
- [x] Upgrade dependencies of K8s version from v1.17 to V1.18. [#3274](https://github.com/kubesphere/kubesphere/issues/3274)
-
-### DevOps:
-
- [x] Upgrade Jenkins Version to 2.249.1. [#2618](https://github.com/kubesphere/kubesphere/issues/2618)
- [x] Using Jenkins distribution solution to deploy Jenkins, [#2182](https://github.com/kubesphere/kubesphere/issues/2182)
- [x] Using human-readable error message for pipeline cron text , [#2919](https://github.com/kubesphere/kubesphere/issues/2919)
- [ ] Using human-readable error message for S2I, [#140](https://github.com/kubesphere/s2ioperator/issues/140)
-
- [ ] https://github.com/kubesphere/tower/issues/31)
-### Observability
-
- [ ] Upgrade to Prometheus v2.25.0
- [x] Upgrade Notification Manager to v0.7.0+ [Releases](https://github.com/kubesphere/notification-manager/releases)
- [x] Upgrade FluentBit Operator to v0.3.0+ [Releases](https://github.com/kubesphere/fluentbit-operator/releases)
- [ ] Upgrade FluentBit to v1.6.9+
-
-## **BugFix:**
-
- [x] Fix unable to get service mesh graph when in a namespace whose name starts with kube[#3126](https://github.com/kubesphere/kubesphere/issues/3162)
- [x] Fix deploy workloads to kubernetes encountered bad_certificate error. [#3112](https://github.com/kubesphere/kubesphere/issues/3112)
- [x] Fix DevOps project admin cannot download artifacts. [#3088](https://github.com/kubesphere/kubesphere/issues/3083)
- [x] Fix cannot create pipeline caused by user admin not found in directory. [#3105](https://github.com/kubesphere/kubesphere/issues/3105)
- [x] Fix the security risk caused by pod viewer can connect to the container terminal. [#3041](https://github.com/kubesphere/kubesphere/issues/3041)
- [ ] Fix some resources cannot be deleted in cascade. [#2912](https://github.com/kubesphere/kubesphere/issues/2912)
- [x] Fix self-signed certificate for admission webhook relies on legacy Common Name field. [#2928](https://github.com/kubesphere/kubesphere/issues/2928)
- [x] Fix application monitor view [#1394](https://github.com/kubesphere/console/issues/1394)
- [x] Fix Grayscale Release when service app label is not the same with the servicename [#3128](https://github.com/kubesphere/kubesphere/issues/3128)
- [x] Fix application status which is always updating [#3241](https://github.com/kubesphere/kubesphere/issues/3241)
- [x] Fix workspaces on member cluster would be deleted when joining to host if there are workspaces with same name on the host. [#3169](https://github.com/kubesphere/kubesphere/issues/3169)
- [x] Fix cluster unjoin federation plane when using proxy connection. [#3202](https://github.com/kubesphere/kubesphere/pull/3203)
- [x] Fix multi-cluster-status-condition. [#3135](https://github.com/kubesphere/kubesphere/issues/3135)
- [x] Fix edge cluster can't join to the host cluster. [#3198](https://github.com/kubesphere/kubesphere/issues/3198)
- [x] Fix can not rollout restart tower agent. [#29](https://github.com/kubesphere/tower/issues/29)
-
-### DevOps:
-
- [x] Fix error webhook link under multicluster. [forum #2626](https://kubesphere.com.cn/forum/d/2626-webhook-jenkins)
- [x] Fix some data lost in jenkinsfile when users edit pipeline. [#1270](https://github.com/kubesphere/console/issues/1270)
- [x] Fix error when clicking "Docker Container Registry Credentials". [console #1269](https://github.com/kubesphere/console/issues/1269)
- [x] Fix chinese show in code quality under English language. [consoel #1278](https://github.com/kubesphere/console/issues/1278)
- [x] Fix get the error when it displays a bool parameter from InSCM Jenkinsfile. [#3043](https://github.com/kubesphere/kubesphere/issues/3043)
-
-# v3.0
-
-## Multi-cluster
-
-## DevOps
+#### DevOps

 - [ ] Create / Edit Pipeline Process Optimization.
 - [ ] S2I/B2I supports webhook.
@@ -174,7 +38,7 @@ KubeSphere Roadmap demonstrates a list of open source product development plans
 - [ ] Pipeline integrates the notification of KubeSphere.
 - [ ] Pipeline integrates KubeSphere custom monitoring.

-## Observability
+#### Observability

 - [ ] Logging console enhancement
 - [ ] Monitoring stack upgrade including Prometheus, Prometheus Operator, Node exporter, kube-state-metrics etc.
@@ -184,9 +48,9 @@ KubeSphere Roadmap demonstrates a list of open source product development plans
 - [ ] K8s Audit Support
 - [ ] Notification Enhancement

-## Network
+#### Network

-## Storage
+#### Storage

 - [ ] Snapshot management
 - [ ] Volume cloning
@@ -195,18 +59,10 @@ KubeSphere Roadmap demonstrates a list of open source product development plans
 - [ ] Restore volume to available status
 - [ ] Unified integrate third-party storage plugin

-## Security & Multitenancy
+#### Security & Multitenancy

 - [ ] Support the OAuth2 SSO plugin.
 - [ ] Workspace resource quota.
 - [ ] Refactor access management framework to adapt to multi-cluster design.

-## Application Lifecycle Management (OpenPitrix)
-
-
-# v2.1
-
- [ ] Most of the work will be bugfix
- [ ] Refactor RBAC in order to support future versions regarding third-party plugins with custom access control.
- [ ] Refactor installer
- [ ] FluentBit Operator upgrade
+#### Application Lifecycle Management (OpenPitrix)
--- a/go.mod
+++ b/go.mod
@@ -4,602 +4,365 @@

 module kubesphere.io/kubesphere

-go 1.13
+go 1.12

 require (
 	code.cloudfoundry.org/bytefmt v0.0.0-20190710193110-1eb035ffe2b6
-	github.com/MakeNowJust/heredoc v0.0.0-20171113091838-e9091a26100e // indirect
-	github.com/Masterminds/semver/v3 v3.1.0
-	github.com/Microsoft/go-winio v0.4.12 // indirect
+	github.com/NYTimes/gziphandler v1.1.1 // indirect
 	github.com/PuerkitoBio/goquery v1.5.0
-	github.com/asaskevich/govalidator v0.0.0-20200428143746-21a406dcc535
-	github.com/aws/aws-sdk-go v1.33.12
+	github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a
+	github.com/aws/aws-sdk-go v1.25.21
 	github.com/beevik/etree v1.1.0
 	github.com/container-storage-interface/spec v1.2.0
-	github.com/containernetworking/cni v0.8.0
-	github.com/coreos/go-oidc v2.1.0+incompatible
-	github.com/davecgh/go-spew v1.1.1
-	github.com/deckarep/golang-set v1.7.1 // indirect
 	github.com/dgrijalva/jwt-go v3.2.0+incompatible
 	github.com/docker/distribution v2.7.1+incompatible
-	github.com/docker/docker v1.4.2-0.20200203170920-46ec8731fbce
-	github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c // indirect
+	github.com/docker/docker v1.4.2-0.20190822205725-ed20165a37b4
 	github.com/elastic/go-elasticsearch/v5 v5.6.1
 	github.com/elastic/go-elasticsearch/v6 v6.8.2
 	github.com/elastic/go-elasticsearch/v7 v7.3.0
-	github.com/elazarl/goproxy v0.0.0-20200315184450-1f3cb6622dad // indirect
-	github.com/emicklei/go-restful v2.14.3+incompatible
-	github.com/emicklei/go-restful-openapi v1.4.1
+	github.com/elazarl/goproxy/ext v0.0.0-20190711103511-473e67f1d7d2 // indirect
+	github.com/emicklei/go-restful v2.11.1+incompatible
+	github.com/emicklei/go-restful-openapi v1.0.0
 	github.com/emirpasic/gods v1.12.0 // indirect
 	github.com/fatih/structs v1.1.0
-	github.com/ghodss/yaml v1.0.0
 	github.com/go-ldap/ldap v3.0.3+incompatible
-	github.com/go-logr/logr v0.1.0
 	github.com/go-logr/zapr v0.1.1 // indirect
-	github.com/go-openapi/loads v0.19.5
-	github.com/go-openapi/spec v0.19.7
-	github.com/go-openapi/strfmt v0.19.5
-	github.com/go-openapi/validate v0.19.8
+	github.com/go-openapi/loads v0.19.2
+	github.com/go-openapi/spec v0.19.4
+	github.com/go-openapi/strfmt v0.19.3
+	github.com/go-openapi/validate v0.19.5
 	github.com/go-redis/redis v6.15.2+incompatible
-	github.com/go-sql-driver/mysql v1.5.0
+	github.com/go-sql-driver/mysql v1.4.1
 	github.com/gocraft/dbr v0.0.0-20180507214907-a0fd650918f6
 	github.com/golang/example v0.0.0-20170904185048-46695d81d1fa
-	github.com/google/go-cmp v0.5.0
+	github.com/golang/mock v1.2.0
+	github.com/golang/protobuf v1.3.2
+	github.com/google/go-cmp v0.3.1
+	github.com/google/go-querystring v1.0.0 // indirect
 	github.com/google/uuid v1.1.1
+	github.com/gorilla/mux v1.7.1 // indirect
 	github.com/gorilla/websocket v1.4.1
-	github.com/gregjones/httpcache v0.0.0-20181110185634-c63ab54fda8f // indirect
-	github.com/hashicorp/golang-lru v0.5.4
-	github.com/json-iterator/go v1.1.10
-	github.com/jszwec/csvutil v1.5.0
+	github.com/hashicorp/go-version v1.2.0 // indirect
+	github.com/json-iterator/go v1.1.9
 	github.com/kelseyhightower/envconfig v1.4.0 // indirect
-	github.com/kubernetes-csi/external-snapshotter/client/v3 v3.0.0
+	github.com/kiali/kiali v0.15.1-0.20200520152915-769a61d75460
+	github.com/kubernetes-csi/external-snapshotter/v2 v2.1.0
 	github.com/kubesphere/sonargo v0.0.2
-	github.com/mitchellh/mapstructure v1.2.2
-	github.com/onsi/ginkgo v1.14.0
-	github.com/onsi/gomega v1.10.1
+	github.com/lib/pq v1.2.0 // indirect
+	github.com/mattn/go-sqlite3 v1.11.0 // indirect
+	github.com/onsi/ginkgo v1.12.0
+	github.com/onsi/gomega v1.9.0
 	github.com/open-policy-agent/opa v0.18.0
-	github.com/opencontainers/go-digest v1.0.0
+	github.com/opencontainers/go-digest v1.0.0-rc1
+	github.com/openshift/api v0.0.0-20180801171038-322a19404e37 // indirect
+	github.com/opentracing/opentracing-go v1.1.0 // indirect
 	github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
 	github.com/pkg/errors v0.9.1
 	github.com/projectcalico/kube-controllers v3.8.8+incompatible
-	github.com/projectcalico/libcalico-go v1.7.2-0.20191014160346-2382c6cdd056
-	github.com/prometheus-community/prom-label-proxy v0.2.0
-	github.com/prometheus-operator/prometheus-operator v0.42.2-0.20200928114327-fbd01683839a
-	github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.42.1
-	github.com/prometheus/client_golang v1.7.1
-	github.com/prometheus/common v0.11.1
-	github.com/prometheus/prometheus v1.8.2-0.20200907175821-8219b442c864
-	github.com/sony/sonyflake v0.0.0-20181109022403-6d5bd6181009
+	github.com/projectcalico/libcalico-go v1.7.2-0.20191104213956-8f81e1e344ce
+	github.com/prometheus/client_golang v1.2.1
+	github.com/prometheus/common v0.4.1
+	github.com/prometheus/prometheus v1.8.2
+	github.com/sony/sonyflake v1.0.0
 	github.com/speps/go-hashids v2.0.0+incompatible
-	github.com/spf13/cobra v1.0.0
+	github.com/spf13/cobra v0.0.5
 	github.com/spf13/pflag v1.0.5
 	github.com/spf13/viper v1.4.0
-	github.com/stretchr/testify v1.6.1
+	github.com/stretchr/testify v1.4.0
+	github.com/syndtr/goleveldb v1.0.0 // indirect
 	github.com/xanzy/ssh-agent v0.2.1 // indirect
-	golang.org/x/crypto v0.0.0-20200728195943-123391ffb6de
-	golang.org/x/net v0.0.0-20201224014010-6772e930b67b
-	golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
-	google.golang.org/grpc v1.30.0
+	golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975
+	golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553
+	golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
+	google.golang.org/grpc v1.26.0
 	gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d // indirect
-	gopkg.in/cas.v2 v2.2.0
-	gopkg.in/square/go-jose.v2 v2.4.0
 	gopkg.in/src-d/go-billy.v4 v4.3.0 // indirect
 	gopkg.in/src-d/go-git.v4 v4.11.0
-	gopkg.in/yaml.v2 v2.3.0
-	gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776
-	gotest.tools v2.2.0+incompatible
-	helm.sh/helm/v3 v3.3.0
-	istio.io/api v0.0.0-20201113182140-d4b7e3fc2b44
-	istio.io/client-go v0.0.0-20201113183938-0734e976e785
-	istio.io/gogo-genproto v0.0.0-20201113182723-5b8563d8a012 // indirect
-	k8s.io/api v0.19.0
-	k8s.io/apiextensions-apiserver v0.18.6
-	k8s.io/apimachinery v0.19.0
-	k8s.io/apiserver v0.18.6
-	k8s.io/cli-runtime v0.18.6
-	k8s.io/client-go v12.0.0+incompatible
-	k8s.io/code-generator v0.19.0
-	k8s.io/component-base v0.18.6
+	gopkg.in/yaml.v2 v2.2.8
+	gopkg.in/yaml.v3 v3.0.0-20190905181640-827449938966
+	istio.io/api v0.0.0-20191111210003-35e06ef8d838
+	istio.io/client-go v0.0.0-20191113122552-9bd0ba57c3d2
+	k8s.io/api v0.17.3
+	k8s.io/apiextensions-apiserver v0.17.3
+	k8s.io/apimachinery v0.17.3
+	k8s.io/apiserver v0.17.3
+	k8s.io/cli-runtime v0.17.3
+	k8s.io/client-go v0.17.3
+	k8s.io/code-generator v0.17.3
+	k8s.io/component-base v0.17.3
+	k8s.io/gengo v0.0.0-20191120174120-e74f70b9b27e // indirect
 	k8s.io/klog v1.0.0
-	k8s.io/klog/v2 v2.0.0
-	k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6
-	k8s.io/kubectl v0.18.6
-	k8s.io/metrics v0.18.6
-	k8s.io/utils v0.0.0-20200603063816-c1c6865ac451
-	kubesphere.io/client-go v0.0.0
-	kubesphere.io/monitoring-dashboard v0.1.2
-	sigs.k8s.io/application v0.8.4-0.20201016185654-c8e2959e57a0
-	sigs.k8s.io/controller-runtime v0.6.4
-	sigs.k8s.io/controller-tools v0.4.0
-	sigs.k8s.io/kubefed v0.7.0
-	sigs.k8s.io/kustomize v2.0.3+incompatible
-	sigs.k8s.io/yaml v1.2.0
+	k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
+	k8s.io/kubectl v0.17.3
+	openpitrix.io/openpitrix v0.4.9-0.20200611125425-ae07f141e797
+	sigs.k8s.io/application v1.0.0
+	sigs.k8s.io/controller-runtime v0.5.0
+	sigs.k8s.io/controller-tools v0.2.4
+	sigs.k8s.io/kubefed v0.2.0-alpha.1
+	sigs.k8s.io/testing_frameworks v0.1.2
 )

 replace (
-	cloud.google.com/go => cloud.google.com/go v0.56.0
-	cloud.google.com/go/bigquery => cloud.google.com/go/bigquery v1.4.0
-	cloud.google.com/go/bigtable => cloud.google.com/go/bigtable v1.2.0
-	cloud.google.com/go/datastore => cloud.google.com/go/datastore v1.1.0
-	cloud.google.com/go/pubsub => cloud.google.com/go/pubsub v1.2.0
-	cloud.google.com/go/storage => cloud.google.com/go/storage v1.6.0
+	cloud.google.com/go => cloud.google.com/go v0.38.0
 	code.cloudfoundry.org/bytefmt => code.cloudfoundry.org/bytefmt v0.0.0-20190710193110-1eb035ffe2b6
-	collectd.org => collectd.org v0.3.0
-	github.com/Azure/azure-pipeline-go => github.com/Azure/azure-pipeline-go v0.2.2
-	github.com/Azure/azure-sdk-for-go => github.com/Azure/azure-sdk-for-go v41.3.0+incompatible
-	github.com/Azure/azure-storage-blob-go => github.com/Azure/azure-storage-blob-go v0.8.0
 	github.com/Azure/go-ansiterm => github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
-	github.com/Azure/go-autorest/autorest => github.com/Azure/go-autorest/autorest v0.10.0
-	github.com/Azure/go-autorest/autorest/adal => github.com/Azure/go-autorest/autorest/adal v0.8.3
-	github.com/Azure/go-autorest/autorest/date => github.com/Azure/go-autorest/autorest/date v0.2.0
-	github.com/Azure/go-autorest/autorest/mocks => github.com/Azure/go-autorest/autorest/mocks v0.3.0
-	github.com/Azure/go-autorest/autorest/to => github.com/Azure/go-autorest/autorest/to v0.3.0
-	github.com/Azure/go-autorest/autorest/validation => github.com/Azure/go-autorest/autorest/validation v0.2.0
+	github.com/Azure/go-autorest/autorest => github.com/Azure/go-autorest/autorest v0.9.0
+	github.com/Azure/go-autorest/autorest/adal => github.com/Azure/go-autorest/autorest/adal v0.5.0
+	github.com/Azure/go-autorest/autorest/date => github.com/Azure/go-autorest/autorest/date v0.1.0
+	github.com/Azure/go-autorest/autorest/mocks => github.com/Azure/go-autorest/autorest/mocks v0.2.0
 	github.com/Azure/go-autorest/logger => github.com/Azure/go-autorest/logger v0.1.0
 	github.com/Azure/go-autorest/tracing => github.com/Azure/go-autorest/tracing v0.5.0
 	github.com/BurntSushi/toml => github.com/BurntSushi/toml v0.3.1
-	github.com/DATA-DOG/go-sqlmock => github.com/DATA-DOG/go-sqlmock v1.3.3
-	github.com/DataDog/datadog-go => github.com/DataDog/datadog-go v3.2.0+incompatible
-	github.com/Knetic/govaluate => github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible
-	github.com/MakeNowJust/heredoc => github.com/MakeNowJust/heredoc v0.0.0-20171113091838-e9091a26100e
+	github.com/MakeNowJust/heredoc => github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
 	github.com/Masterminds/goutils => github.com/Masterminds/goutils v1.1.0
+	github.com/Masterminds/semver => github.com/Masterminds/semver v1.5.0
 	github.com/Masterminds/semver/v3 => github.com/Masterminds/semver/v3 v3.0.1
 	github.com/Masterminds/sprig/v3 => github.com/Masterminds/sprig/v3 v3.0.0
-	github.com/Masterminds/squirrel => github.com/Masterminds/squirrel v0.0.0-20161115235646-20f192218cf5
 	github.com/Masterminds/vcs => github.com/Masterminds/vcs v1.13.0
 	github.com/Microsoft/go-winio => github.com/Microsoft/go-winio v0.4.12
 	github.com/Microsoft/hcsshim => github.com/Microsoft/hcsshim v0.8.6
 	github.com/NYTimes/gziphandler => github.com/NYTimes/gziphandler v1.1.1
 	github.com/Nvveen/Gotty => github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
-	github.com/OneOfOne/xxhash => github.com/OneOfOne/xxhash v1.2.7
+	github.com/OneOfOne/xxhash => github.com/OneOfOne/xxhash v1.2.2
 	github.com/PuerkitoBio/goquery => github.com/PuerkitoBio/goquery v1.5.0
 	github.com/PuerkitoBio/purell => github.com/PuerkitoBio/purell v1.1.1
 	github.com/PuerkitoBio/urlesc => github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
 	github.com/Shopify/logrus-bugsnag => github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d
-	github.com/Shopify/sarama => github.com/Shopify/sarama v1.19.0
-	github.com/Shopify/toxiproxy => github.com/Shopify/toxiproxy v2.1.4+incompatible
-	github.com/VividCortex/gohistogram => github.com/VividCortex/gohistogram v1.0.0
-	github.com/afex/hystrix-go => github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5
-	github.com/agnivade/levenshtein => github.com/agnivade/levenshtein v1.0.1
-	github.com/ajstarks/svgo => github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af
+	github.com/StackExchange/wmi => github.com/StackExchange/wmi v0.0.0-20170410192909-ea383cf3ba6e
 	github.com/alcortesm/tgz => github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
-	github.com/alecthomas/template => github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
-	github.com/alecthomas/units => github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
-	github.com/alessio/shellescape => github.com/alessio/shellescape v1.2.2
-	github.com/aliyun/aliyun-oss-go-sdk => github.com/aliyun/aliyun-oss-go-sdk v2.0.4+incompatible
-	github.com/andreyvit/diff => github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883
+	github.com/alecthomas/template => github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc
+	github.com/alecthomas/units => github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf
 	github.com/andybalholm/cascadia => github.com/andybalholm/cascadia v1.0.0
 	github.com/anmitsu/go-shlex => github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
-	github.com/antihax/optional => github.com/antihax/optional v0.0.0-20180407024304-ca021399b1a6
-	github.com/apache/arrow/go/arrow => github.com/apache/arrow/go/arrow v0.0.0-20191024131854-af6fa24be0db
-	github.com/apache/thrift => github.com/apache/thrift v0.13.0
-	github.com/armon/circbuf => github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e
+	github.com/appscode/jsonpatch => github.com/appscode/jsonpatch v0.0.0-20190108182946-7c0e3b262f30
 	github.com/armon/consul-api => github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6
-	github.com/armon/go-metrics => github.com/armon/go-metrics v0.3.3
-	github.com/armon/go-radix => github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310
-	github.com/armon/go-socks5 => github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
-	github.com/aryann/difflib => github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a
-	github.com/asaskevich/govalidator => github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496
-	github.com/aws/aws-lambda-go => github.com/aws/aws-lambda-go v1.13.3
-	github.com/aws/aws-sdk-go => github.com/aws/aws-sdk-go v1.30.12
-	github.com/aws/aws-sdk-go-v2 => github.com/aws/aws-sdk-go-v2 v0.18.0
+	github.com/asaskevich/govalidator => github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a
+	github.com/aws/aws-sdk-go => github.com/aws/aws-sdk-go v1.22.2
 	github.com/beevik/etree => github.com/beevik/etree v1.1.0
-	github.com/beevik/ntp => github.com/beevik/ntp v0.2.0
-	github.com/beorn7/perks => github.com/beorn7/perks v1.0.1
+	github.com/beorn7/perks => github.com/beorn7/perks v1.0.0
 	github.com/bgentry/speakeasy => github.com/bgentry/speakeasy v0.1.0
-	github.com/bitly/go-hostpool => github.com/bitly/go-hostpool v0.0.0-20171023180738-a3a6125de932
 	github.com/bitly/go-simplejson => github.com/bitly/go-simplejson v0.5.0
 	github.com/blang/semver => github.com/blang/semver v3.5.0+incompatible
 	github.com/bmizerany/assert => github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869
-	github.com/bmizerany/pat => github.com/bmizerany/pat v0.0.0-20170815010413-6226ea591a40
-	github.com/boltdb/bolt => github.com/boltdb/bolt v1.3.1
-	github.com/bradfitz/gomemcache => github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b
-	github.com/brancz/kube-rbac-proxy => github.com/brancz/kube-rbac-proxy v0.5.0
 	github.com/bshuster-repo/logrus-logstash-hook => github.com/bshuster-repo/logrus-logstash-hook v0.4.1
 	github.com/bugsnag/bugsnag-go => github.com/bugsnag/bugsnag-go v1.5.0
 	github.com/bugsnag/panicwrap => github.com/bugsnag/panicwrap v1.2.0
-	github.com/c-bata/go-prompt => github.com/c-bata/go-prompt v0.2.2
-	github.com/campoy/embedmd => github.com/campoy/embedmd v1.0.0
-	github.com/casbin/casbin/v2 => github.com/casbin/casbin/v2 v2.1.2
-	github.com/cenkalti/backoff => github.com/cenkalti/backoff v2.2.1+incompatible
-	github.com/census-instrumentation/opencensus-proto => github.com/census-instrumentation/opencensus-proto v0.2.1
 	github.com/cespare/xxhash => github.com/cespare/xxhash v1.1.0
-	github.com/cespare/xxhash/v2 => github.com/cespare/xxhash/v2 v2.1.1
 	github.com/chai2010/gettext-go => github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5
-	github.com/chromedp/cdproto => github.com/chromedp/cdproto v0.0.0-20200424080200-0de008e41fa0
-	github.com/chromedp/chromedp => github.com/chromedp/chromedp v0.5.3
-	github.com/chzyer/logex => github.com/chzyer/logex v1.1.10
-	github.com/chzyer/readline => github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
-	github.com/chzyer/test => github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1
-	github.com/circonus-labs/circonus-gometrics => github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible
-	github.com/circonus-labs/circonusllhist => github.com/circonus-labs/circonusllhist v0.1.3
-	github.com/clbanning/x2j => github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec
-	github.com/cockroachdb/apd => github.com/cockroachdb/apd v1.1.0
-	github.com/cockroachdb/cockroach-go => github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c
+	github.com/chai2010/jsonmap => github.com/chai2010/jsonmap v1.0.0
+	github.com/client9/misspell => github.com/client9/misspell v0.3.4
 	github.com/cockroachdb/datadriven => github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
-	github.com/codahale/hdrhistogram => github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd
 	github.com/container-storage-interface/spec => github.com/container-storage-interface/spec v1.2.0
 	github.com/containerd/containerd => github.com/containerd/containerd v1.3.0
 	github.com/containerd/continuity => github.com/containerd/continuity v0.0.0-20181203112020-004b46473808
-	github.com/containernetworking/cni => github.com/containernetworking/cni v0.8.0
 	github.com/coreos/bbolt => github.com/coreos/bbolt v1.3.3
 	github.com/coreos/etcd => github.com/coreos/etcd v3.3.17+incompatible
 	github.com/coreos/go-oidc => github.com/coreos/go-oidc v2.1.0+incompatible
 	github.com/coreos/go-semver => github.com/coreos/go-semver v0.3.0
 	github.com/coreos/go-systemd => github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f
 	github.com/coreos/pkg => github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f
-	github.com/cortexproject/cortex => github.com/cortexproject/cortex v1.3.1-0.20200901115931-255ff3306960
 	github.com/cpuguy83/go-md2man => github.com/cpuguy83/go-md2man v1.0.10
 	github.com/creack/pty => github.com/creack/pty v1.1.7
 	github.com/cyphar/filepath-securejoin => github.com/cyphar/filepath-securejoin v0.2.2
-	github.com/cznic/b => github.com/cznic/b v0.0.0-20180115125044-35e9bbe41f07
-	github.com/cznic/fileutil => github.com/cznic/fileutil v0.0.0-20180108211300-6a051e75936f
-	github.com/cznic/golex => github.com/cznic/golex v0.0.0-20170803123110-4ab7c5e190e4
-	github.com/cznic/internal => github.com/cznic/internal v0.0.0-20180608152220-f44710a21d00
-	github.com/cznic/lldb => github.com/cznic/lldb v1.1.0
-	github.com/cznic/mathutil => github.com/cznic/mathutil v0.0.0-20180504122225-ca4c9f2c1369
-	github.com/cznic/ql => github.com/cznic/ql v1.2.0
-	github.com/cznic/sortutil => github.com/cznic/sortutil v0.0.0-20150617083342-4c7342852e65
-	github.com/cznic/strutil => github.com/cznic/strutil v0.0.0-20171016134553-529a34b1c186
-	github.com/cznic/zappy => github.com/cznic/zappy v0.0.0-20160723133515-2533cb5b45cc
-	github.com/dave/jennifer => github.com/dave/jennifer v1.2.0
 	github.com/davecgh/go-spew => github.com/davecgh/go-spew v1.1.1
 	github.com/daviddengcn/go-colortext => github.com/daviddengcn/go-colortext v0.0.0-20160507010035-511bcaf42ccd
 	github.com/deckarep/golang-set => github.com/deckarep/golang-set v1.7.1
 	github.com/deislabs/oras => github.com/deislabs/oras v0.7.0
 	github.com/denisenkom/go-mssqldb => github.com/denisenkom/go-mssqldb v0.0.0-20190204142019-df6d76eb9289
 	github.com/dgrijalva/jwt-go => github.com/dgrijalva/jwt-go v3.2.0+incompatible
-	github.com/dgryski/go-bitstream => github.com/dgryski/go-bitstream v0.0.0-20180413035011-3522498ce2c8
-	github.com/dgryski/go-sip13 => github.com/dgryski/go-sip13 v0.0.0-20190329191031-25c5027a8c7b
-	github.com/dhui/dktest => github.com/dhui/dktest v0.3.0
+	github.com/disintegration/imaging => github.com/disintegration/imaging v1.6.1
 	github.com/docker/cli => github.com/docker/cli v0.0.0-20190506213505-d88565df0c2d
 	github.com/docker/distribution => github.com/docker/distribution v2.7.1+incompatible
-	github.com/docker/docker => github.com/docker/engine v1.4.2-0.20200203170920-46ec8731fbce
+	github.com/docker/docker => github.com/docker/engine v1.4.2-0.20190822205725-ed20165a37b4
 	github.com/docker/docker-credential-helpers => github.com/docker/docker-credential-helpers v0.6.1
-	github.com/docker/go-connections => github.com/docker/go-connections v0.4.0
+	github.com/docker/go-connections => github.com/docker/go-connections v0.3.0
 	github.com/docker/go-metrics => github.com/docker/go-metrics v0.0.0-20181218153428-b84716841b82
-	github.com/docker/go-units => github.com/docker/go-units v0.4.0
+	github.com/docker/go-units => github.com/docker/go-units v0.3.3
 	github.com/docker/libtrust => github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7
 	github.com/docker/spdystream => github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c
 	github.com/docopt/docopt-go => github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815
 	github.com/dustin/go-humanize => github.com/dustin/go-humanize v1.0.0
-	github.com/eapache/go-resiliency => github.com/eapache/go-resiliency v1.1.0
-	github.com/eapache/go-xerial-snappy => github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21
-	github.com/eapache/queue => github.com/eapache/queue v1.1.0
-	github.com/eclipse/paho.mqtt.golang => github.com/eclipse/paho.mqtt.golang v1.2.0
-	github.com/edsrzf/mmap-go => github.com/edsrzf/mmap-go v1.0.0
 	github.com/elastic/go-elasticsearch/v5 => github.com/elastic/go-elasticsearch/v5 v5.6.1
 	github.com/elastic/go-elasticsearch/v6 => github.com/elastic/go-elasticsearch/v6 v6.8.2
 	github.com/elastic/go-elasticsearch/v7 => github.com/elastic/go-elasticsearch/v7 v7.3.0
-	github.com/elastic/go-sysinfo => github.com/elastic/go-sysinfo v1.1.1
-	github.com/elastic/go-windows => github.com/elastic/go-windows v1.0.1
-	github.com/elazarl/goproxy => github.com/elazarl/goproxy v0.0.0-20200315184450-1f3cb6622dad
+	github.com/elazarl/goproxy => github.com/elazarl/goproxy v0.0.0-20190711103511-473e67f1d7d2
 	github.com/elazarl/goproxy/ext => github.com/elazarl/goproxy/ext v0.0.0-20190711103511-473e67f1d7d2
-	github.com/ema/qdisc => github.com/ema/qdisc v0.0.0-20190904071900-b82c76788043
-	github.com/emicklei/go-restful => github.com/emicklei/go-restful v2.14.3+incompatible
-	github.com/emicklei/go-restful-openapi => github.com/emicklei/go-restful-openapi v1.4.1
+	github.com/emicklei/go-restful => github.com/emicklei/go-restful v2.9.5+incompatible
+	github.com/emicklei/go-restful-openapi => github.com/emicklei/go-restful-openapi v1.0.0
 	github.com/emirpasic/gods => github.com/emirpasic/gods v1.12.0
-	github.com/envoyproxy/go-control-plane => github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473
-	github.com/envoyproxy/protoc-gen-validate => github.com/envoyproxy/protoc-gen-validate v0.1.0
-	github.com/evanphx/json-patch => github.com/evanphx/json-patch v4.9.0+incompatible
-	github.com/evanphx/json-patch/v5 => github.com/evanphx/json-patch/v5 v5.0.0
+	github.com/erikstmartin/go-testdb => github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5
+	github.com/evanphx/json-patch => github.com/evanphx/json-patch v4.5.0+incompatible
 	github.com/exponent-io/jsonpath => github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d
-	github.com/facette/natsort => github.com/facette/natsort v0.0.0-20181210072756-2cd4dd1e2dcb
 	github.com/fatih/camelcase => github.com/fatih/camelcase v1.0.0
-	github.com/fatih/color => github.com/fatih/color v1.9.0
+	github.com/fatih/color => github.com/fatih/color v1.7.0
 	github.com/fatih/structs => github.com/fatih/structs v1.1.0
-	github.com/fatih/structtag => github.com/fatih/structtag v1.1.0
-	github.com/felixge/fgprof => github.com/felixge/fgprof v0.9.1
-	github.com/felixge/httpsnoop => github.com/felixge/httpsnoop v1.0.1
 	github.com/flynn/go-shlex => github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
-	github.com/fogleman/gg => github.com/fogleman/gg v1.2.1-0.20190220221249-0403632d5b90
-	github.com/franela/goblin => github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db
-	github.com/franela/goreq => github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8
-	github.com/fsnotify/fsnotify => github.com/fsnotify/fsnotify v1.4.9
-	github.com/fsouza/fake-gcs-server => github.com/fsouza/fake-gcs-server v1.7.0
+	github.com/fsnotify/fsnotify => github.com/fsnotify/fsnotify v1.4.7
 	github.com/garyburd/redigo => github.com/garyburd/redigo v1.6.0
 	github.com/ghodss/yaml => github.com/ghodss/yaml v1.0.0
+	github.com/gin-contrib/sse => github.com/gin-contrib/sse v0.0.0-20190301062529-5545eab6dad3
+	github.com/gin-gonic/gin => github.com/gin-gonic/gin v1.4.0
 	github.com/gliderlabs/ssh => github.com/gliderlabs/ssh v0.1.1
-	github.com/glycerine/go-unsnap-stream => github.com/glycerine/go-unsnap-stream v0.0.0-20180323001048-9f0cb55181dd
-	github.com/glycerine/goconvey => github.com/glycerine/goconvey v0.0.0-20190410193231-58a59202ab31
-	github.com/go-kit/kit => github.com/go-kit/kit v0.10.0
+	github.com/globalsign/mgo => github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
+	github.com/go-kit/kit => github.com/go-kit/kit v0.8.0
 	github.com/go-ldap/ldap => github.com/go-ldap/ldap v3.0.3+incompatible
-	github.com/go-logfmt/logfmt => github.com/go-logfmt/logfmt v0.5.0
+	github.com/go-logfmt/logfmt => github.com/go-logfmt/logfmt v0.4.0
 	github.com/go-logr/logr => github.com/go-logr/logr v0.1.0
 	github.com/go-logr/zapr => github.com/go-logr/zapr v0.1.1
-	github.com/go-openapi/analysis => github.com/go-openapi/analysis v0.19.10
-	github.com/go-openapi/errors => github.com/go-openapi/errors v0.19.4
+	github.com/go-ole/go-ole => github.com/go-ole/go-ole v1.2.1
+	github.com/go-openapi/analysis => github.com/go-openapi/analysis v0.19.2
+	github.com/go-openapi/errors => github.com/go-openapi/errors v0.19.2
 	github.com/go-openapi/jsonpointer => github.com/go-openapi/jsonpointer v0.19.3
 	github.com/go-openapi/jsonreference => github.com/go-openapi/jsonreference v0.19.3
-	github.com/go-openapi/loads => github.com/go-openapi/loads v0.19.5
-	github.com/go-openapi/runtime => github.com/go-openapi/runtime v0.19.15
-	github.com/go-openapi/spec => github.com/go-openapi/spec v0.19.7
-	github.com/go-openapi/strfmt => github.com/go-openapi/strfmt v0.19.5
-	github.com/go-openapi/swag => github.com/go-openapi/swag v0.19.9
-	github.com/go-openapi/validate => github.com/go-openapi/validate v0.19.8
+	github.com/go-openapi/loads => github.com/go-openapi/loads v0.19.2
+	github.com/go-openapi/runtime => github.com/go-openapi/runtime v0.19.0
+	github.com/go-openapi/spec => github.com/go-openapi/spec v0.19.3
+	github.com/go-openapi/strfmt => github.com/go-openapi/strfmt v0.19.0
+	github.com/go-openapi/swag => github.com/go-openapi/swag v0.19.5
+	github.com/go-openapi/validate => github.com/go-openapi/validate v0.19.2
 	github.com/go-playground/locales => github.com/go-playground/locales v0.12.1
-	github.com/go-playground/universal-translator => github.com/go-playground/universal-translator v0.0.0-20170327191703-71201497bace
+	github.com/go-playground/universal-translator => github.com/go-playground/universal-translator v0.16.0
 	github.com/go-redis/redis => github.com/go-redis/redis v6.15.2+incompatible
-	github.com/go-resty/resty/v2 => github.com/go-resty/resty/v2 v2.5.0
-	github.com/go-sql-driver/mysql => github.com/go-sql-driver/mysql v1.5.0
+	github.com/go-sql-driver/mysql => github.com/go-sql-driver/mysql v1.4.1
 	github.com/go-stack/stack => github.com/go-stack/stack v1.8.0
-	github.com/gobuffalo/attrs => github.com/gobuffalo/attrs v0.0.0-20190224210810-a9411de4debd
-	github.com/gobuffalo/depgen => github.com/gobuffalo/depgen v0.1.0
-	github.com/gobuffalo/envy => github.com/gobuffalo/envy v1.7.0
-	github.com/gobuffalo/flect => github.com/gobuffalo/flect v0.2.0
-	github.com/gobuffalo/genny => github.com/gobuffalo/genny v0.1.1
-	github.com/gobuffalo/gitgen => github.com/gobuffalo/gitgen v0.0.0-20190315122116-cc086187d211
-	github.com/gobuffalo/gogen => github.com/gobuffalo/gogen v0.1.1
-	github.com/gobuffalo/logger => github.com/gobuffalo/logger v0.0.0-20190315122211-86e12af44bc2
-	github.com/gobuffalo/mapi => github.com/gobuffalo/mapi v1.0.2
-	github.com/gobuffalo/packd => github.com/gobuffalo/packd v0.1.0
-	github.com/gobuffalo/packr/v2 => github.com/gobuffalo/packr/v2 v2.2.0
-	github.com/gobuffalo/syncx => github.com/gobuffalo/syncx v0.0.0-20190224160051-33c29581e754
+	github.com/gobuffalo/flect => github.com/gobuffalo/flect v0.1.5
 	github.com/gobwas/glob => github.com/gobwas/glob v0.2.3
-	github.com/gobwas/httphead => github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee
-	github.com/gobwas/pool => github.com/gobwas/pool v0.2.0
-	github.com/gobwas/ws => github.com/gobwas/ws v1.0.2
-	github.com/gocql/gocql => github.com/gocql/gocql v0.0.0-20200526081602-cd04bd7f22a7
 	github.com/gocraft/dbr => github.com/gocraft/dbr v0.0.0-20180507214907-a0fd650918f6
-	github.com/godbus/dbus => github.com/godbus/dbus v0.0.0-20190402143921-271e53dc4968
-	github.com/godror/godror => github.com/godror/godror v0.13.3
 	github.com/gofrs/flock => github.com/gofrs/flock v0.7.1
 	github.com/gofrs/uuid => github.com/gofrs/uuid v3.2.0+incompatible
-	github.com/gogo/googleapis => github.com/gogo/googleapis v1.1.0
-	github.com/gogo/protobuf => github.com/gogo/protobuf v1.3.1
-	github.com/gogo/status => github.com/gogo/status v1.0.3
-	github.com/golang-migrate/migrate/v4 => github.com/golang-migrate/migrate/v4 v4.7.0
+	github.com/gogo/protobuf => github.com/gogo/protobuf v1.3.0
 	github.com/golang/example => github.com/golang/example v0.0.0-20170904185048-46695d81d1fa
-	github.com/golang/freetype => github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0
-	github.com/golang/geo => github.com/golang/geo v0.0.0-20190916061304-5b978397cfec
 	github.com/golang/glog => github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
-	github.com/golang/groupcache => github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
-	github.com/golang/lint => github.com/golang/lint v0.0.0-20180702182130-06c8688daad7
-	github.com/golang/mock => github.com/golang/mock v1.4.3
-	github.com/golang/protobuf => github.com/golang/protobuf v1.4.2
-	github.com/golang/snappy => github.com/golang/snappy v0.0.1
+	github.com/golang/groupcache => github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6
+	github.com/golang/mock => github.com/golang/mock v1.2.0
+	github.com/golang/protobuf => github.com/golang/protobuf v1.3.2
+	github.com/golang/snappy => github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
 	github.com/golangplus/bytes => github.com/golangplus/bytes v0.0.0-20160111154220-45c989fe5450
 	github.com/golangplus/fmt => github.com/golangplus/fmt v0.0.0-20150411045040-2a5d6d7d2995
 	github.com/golangplus/testing => github.com/golangplus/testing v0.0.0-20180327235837-af21d9c3145e
-	github.com/gomodule/redigo => github.com/gomodule/redigo v2.0.0+incompatible
-	github.com/google/addlicense => github.com/google/addlicense v0.0.0-20200906110928-a0294312aa76
 	github.com/google/btree => github.com/google/btree v1.0.0
-	github.com/google/flatbuffers => github.com/google/flatbuffers v1.11.0
-	github.com/google/go-cmp => github.com/google/go-cmp v0.4.0
-	github.com/google/go-github => github.com/google/go-github v17.0.0+incompatible
+	github.com/google/go-cmp => github.com/google/go-cmp v0.3.0
 	github.com/google/go-querystring => github.com/google/go-querystring v1.0.0
-	github.com/google/gofuzz => github.com/google/gofuzz v1.1.0
+	github.com/google/gofuzz => github.com/google/gofuzz v1.0.0
+	github.com/google/gops => github.com/google/gops v0.3.6
 	github.com/google/martian => github.com/google/martian v2.1.0+incompatible
-	github.com/google/pprof => github.com/google/pprof v0.0.0-20200417002340-c6e0a841f49a
-	github.com/google/renameio => github.com/google/renameio v0.1.0
+	github.com/google/pprof => github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
 	github.com/google/uuid => github.com/google/uuid v1.1.1
-	github.com/googleapis/gax-go => github.com/googleapis/gax-go v2.0.2+incompatible
-	github.com/googleapis/gax-go/v2 => github.com/googleapis/gax-go/v2 v2.0.5
-	github.com/googleapis/gnostic => github.com/googleapis/gnostic v0.4.0
-	github.com/gophercloud/gophercloud => github.com/gophercloud/gophercloud v0.10.0
-	github.com/gopherjs/gopherjs => github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1
-	github.com/gorilla/context => github.com/gorilla/context v1.1.1
+	github.com/googleapis/gax-go/v2 => github.com/googleapis/gax-go/v2 v2.0.4
+	github.com/googleapis/gnostic => github.com/googleapis/gnostic v0.3.1
+	github.com/gophercloud/gophercloud => github.com/gophercloud/gophercloud v0.3.0
 	github.com/gorilla/handlers => github.com/gorilla/handlers v1.4.0
-	github.com/gorilla/mux => github.com/gorilla/mux v1.7.3
-	github.com/gorilla/websocket => github.com/gorilla/websocket v1.4.1
+	github.com/gorilla/mux => github.com/gorilla/mux v1.7.1
+	github.com/gorilla/websocket => github.com/gorilla/websocket v1.4.0
 	github.com/gosuri/uitable => github.com/gosuri/uitable v0.0.1
-	github.com/gregjones/httpcache => github.com/gregjones/httpcache v0.0.0-20181110185634-c63ab54fda8f
-	github.com/grpc-ecosystem/go-grpc-middleware => github.com/grpc-ecosystem/go-grpc-middleware v1.1.0
+	github.com/gregjones/httpcache => github.com/gregjones/httpcache v0.0.0-20170728041850-787624de3eb7
+	github.com/grpc-ecosystem/go-grpc-middleware => github.com/grpc-ecosystem/go-grpc-middleware v1.0.0
 	github.com/grpc-ecosystem/go-grpc-prometheus => github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
-	github.com/grpc-ecosystem/grpc-gateway => github.com/grpc-ecosystem/grpc-gateway v1.14.4
-	github.com/hailocab/go-hostpool => github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed
-	github.com/hashicorp/consul/api => github.com/hashicorp/consul/api v1.4.0
-	github.com/hashicorp/consul/sdk => github.com/hashicorp/consul/sdk v0.4.0
-	github.com/hashicorp/errwrap => github.com/hashicorp/errwrap v1.0.0
-	github.com/hashicorp/go-cleanhttp => github.com/hashicorp/go-cleanhttp v0.5.1
-	github.com/hashicorp/go-hclog => github.com/hashicorp/go-hclog v0.12.2
-	github.com/hashicorp/go-immutable-radix => github.com/hashicorp/go-immutable-radix v1.2.0
-	github.com/hashicorp/go-msgpack => github.com/hashicorp/go-msgpack v0.5.3
-	github.com/hashicorp/go-multierror => github.com/hashicorp/go-multierror v1.0.0
-	github.com/hashicorp/go-retryablehttp => github.com/hashicorp/go-retryablehttp v0.5.3
-	github.com/hashicorp/go-rootcerts => github.com/hashicorp/go-rootcerts v1.0.2
-	github.com/hashicorp/go-sockaddr => github.com/hashicorp/go-sockaddr v1.0.2
-	github.com/hashicorp/go-syslog => github.com/hashicorp/go-syslog v1.0.0
-	github.com/hashicorp/go-uuid => github.com/hashicorp/go-uuid v1.0.1
+	github.com/grpc-ecosystem/grpc-gateway => github.com/grpc-ecosystem/grpc-gateway v1.11.3
 	github.com/hashicorp/go-version => github.com/hashicorp/go-version v1.2.0
-	github.com/hashicorp/golang-lru => github.com/hashicorp/golang-lru v0.5.4
+	github.com/hashicorp/golang-lru => github.com/hashicorp/golang-lru v0.5.3
 	github.com/hashicorp/hcl => github.com/hashicorp/hcl v1.0.0
-	github.com/hashicorp/logutils => github.com/hashicorp/logutils v1.0.0
-	github.com/hashicorp/mdns => github.com/hashicorp/mdns v1.0.1
-	github.com/hashicorp/memberlist => github.com/hashicorp/memberlist v0.2.0
-	github.com/hashicorp/serf => github.com/hashicorp/serf v0.9.0
-	github.com/hodgesds/perf-utils => github.com/hodgesds/perf-utils v0.0.8
+	github.com/hpcloud/tail => github.com/hpcloud/tail v1.0.0
 	github.com/huandu/xstrings => github.com/huandu/xstrings v1.2.0
-	github.com/hudl/fargo => github.com/hudl/fargo v1.3.0
-	github.com/ianlancetaylor/demangle => github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6
-	github.com/imdario/mergo => github.com/imdario/mergo v0.3.9
+	github.com/imdario/mergo => github.com/imdario/mergo v0.3.7
 	github.com/inconshreveable/mousetrap => github.com/inconshreveable/mousetrap v1.0.0
-	github.com/influxdata/flux => github.com/influxdata/flux v0.65.0
-	github.com/influxdata/influxdb => github.com/influxdata/influxdb v1.8.0
-	github.com/influxdata/influxdb1-client => github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d
-	github.com/influxdata/influxql => github.com/influxdata/influxql v1.1.0
-	github.com/influxdata/line-protocol => github.com/influxdata/line-protocol v0.0.0-20180522152040-32c6aa80de5e
-	github.com/influxdata/promql/v2 => github.com/influxdata/promql/v2 v2.12.0
-	github.com/influxdata/roaring => github.com/influxdata/roaring v0.4.13-0.20180809181101-fc520f41fab6
-	github.com/influxdata/tdigest => github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9
-	github.com/influxdata/usage-client => github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368
-	github.com/jackc/fake => github.com/jackc/fake v0.0.0-20150926172116-812a484cc733
-	github.com/jackc/pgx => github.com/jackc/pgx v3.2.0+incompatible
 	github.com/jbenet/go-context => github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
 	github.com/jessevdk/go-flags => github.com/jessevdk/go-flags v1.4.0
-	github.com/jmespath/go-jmespath => github.com/jmespath/go-jmespath v0.3.0
-	github.com/jmoiron/sqlx => github.com/jmoiron/sqlx v1.2.0
-	github.com/joeshaw/multierror => github.com/joeshaw/multierror v0.0.0-20140124173710-69b34d4ec901
-	github.com/joho/godotenv => github.com/joho/godotenv v1.3.0
+	github.com/jinzhu/gorm => github.com/jinzhu/gorm v1.9.2
+	github.com/jinzhu/inflection => github.com/jinzhu/inflection v0.0.0-20180308033659-04140366298a
+	github.com/jinzhu/now => github.com/jinzhu/now v1.0.0
+	github.com/jmespath/go-jmespath => github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
 	github.com/jonboulle/clockwork => github.com/jonboulle/clockwork v0.1.0
-	github.com/jpillora/backoff => github.com/jpillora/backoff v1.0.0
-	github.com/jsimonetti/rtnetlink => github.com/jsimonetti/rtnetlink v0.0.0-20200117123717-f846d4f6c1f4
-	github.com/json-iterator/go => github.com/json-iterator/go v1.1.10
-	github.com/jstemmer/go-junit-report => github.com/jstemmer/go-junit-report v0.9.1
-	github.com/jsternberg/zap-logfmt => github.com/jsternberg/zap-logfmt v1.0.0
-	github.com/jszwec/csvutil => github.com/jszwec/csvutil v1.5.0
-	github.com/jtolds/gls => github.com/jtolds/gls v4.20.0+incompatible
-	github.com/julienschmidt/httprouter => github.com/julienschmidt/httprouter v1.3.0
-	github.com/jung-kurt/gofpdf => github.com/jung-kurt/gofpdf v1.0.3-0.20190309125859-24315acbbda5
-	github.com/jwilder/encoding => github.com/jwilder/encoding v0.0.0-20170811194829-b4e1701a28ef
+	github.com/json-iterator/go => github.com/json-iterator/go v1.1.8
+	github.com/jstemmer/go-junit-report => github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
+	github.com/julienschmidt/httprouter => github.com/julienschmidt/httprouter v1.2.0
 	github.com/kardianos/osext => github.com/kardianos/osext v0.0.0-20170510131534-ae77be60afb1
-	github.com/karrick/godirwalk => github.com/karrick/godirwalk v1.10.3
 	github.com/kelseyhightower/envconfig => github.com/kelseyhightower/envconfig v1.4.0
 	github.com/kevinburke/ssh_config => github.com/kevinburke/ssh_config v0.0.0-20180830205328-81db2a75821e
+	github.com/keybase/go-ps => github.com/keybase/go-ps v0.0.0-20161005175911-668c8856d999
+	github.com/kiali/kiali => github.com/kubesphere/kiali v0.15.1-0.20200520152915-769a61d75460
 	github.com/kisielk/errcheck => github.com/kisielk/errcheck v1.2.0
 	github.com/kisielk/gotool => github.com/kisielk/gotool v1.0.0
-	github.com/klauspost/compress => github.com/klauspost/compress v1.9.5
-	github.com/klauspost/cpuid => github.com/klauspost/cpuid v0.0.0-20170728055534-ae7887de9fa5
-	github.com/klauspost/crc32 => github.com/klauspost/crc32 v0.0.0-20161016154125-cb6bfca970f6
-	github.com/klauspost/pgzip => github.com/klauspost/pgzip v1.0.2-0.20170402124221-0bf5dcad4ada
-	github.com/knq/sysutil => github.com/knq/sysutil v0.0.0-20191005231841-15668db23d08
+	github.com/koding/multiconfig => github.com/koding/multiconfig v0.0.0-20171124222453-69c27309b2d7
 	github.com/konsorten/go-windows-terminal-sequences => github.com/konsorten/go-windows-terminal-sequences v1.0.2
-	github.com/kr/pretty => github.com/kr/pretty v0.2.0
+	github.com/kr/logfmt => github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
+	github.com/kr/pretty => github.com/kr/pretty v0.1.0
 	github.com/kr/pty => github.com/kr/pty v1.1.5
 	github.com/kr/text => github.com/kr/text v0.1.0
-	github.com/kshvakov/clickhouse => github.com/kshvakov/clickhouse v1.3.5
-	github.com/kubernetes-csi/external-snapshotter/client/v3 => github.com/kubernetes-csi/external-snapshotter/client/v3 v3.0.0
+	github.com/kubernetes-csi/csi-lib-utils => github.com/kubernetes-csi/csi-lib-utils v0.7.0
+	github.com/kubernetes-csi/csi-test => github.com/kubernetes-csi/csi-test v2.0.0+incompatible
+	github.com/kubernetes-csi/external-snapshotter/v2 => github.com/kubernetes-csi/external-snapshotter/v2 v2.1.0
 	github.com/kubesphere/sonargo => github.com/kubesphere/sonargo v0.0.2
-	github.com/kylelemons/godebug => github.com/kylelemons/godebug v0.0.0-20160406211939-eadb3ce320cb
-	github.com/leanovate/gopter => github.com/leanovate/gopter v0.2.4
-	github.com/leodido/go-urn => github.com/leodido/go-urn v0.0.0-20181204092800-a67a23e1c1af
+	github.com/leodido/go-urn => github.com/leodido/go-urn v1.1.0
 	github.com/lib/pq => github.com/lib/pq v1.2.0
 	github.com/liggitt/tabwriter => github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
-	github.com/lightstep/lightstep-tracer-common/golang/gogo => github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743
-	github.com/lightstep/lightstep-tracer-go => github.com/lightstep/lightstep-tracer-go v0.18.1
 	github.com/lithammer/dedent => github.com/lithammer/dedent v1.1.0
-	github.com/lovoo/gcloud-opentracing => github.com/lovoo/gcloud-opentracing v0.3.0
-	github.com/lufia/iostat => github.com/lufia/iostat v1.1.0
 	github.com/magiconair/properties => github.com/magiconair/properties v1.8.0
-	github.com/mailru/easyjson => github.com/mailru/easyjson v0.7.1
-	github.com/markbates/oncer => github.com/markbates/oncer v0.0.0-20181203154359-bf2de49a0be2
-	github.com/markbates/safe => github.com/markbates/safe v1.0.1
-	github.com/mattn/go-colorable => github.com/mattn/go-colorable v0.1.6
-	github.com/mattn/go-ieproxy => github.com/mattn/go-ieproxy v0.0.0-20191113090002-7c0f6868bffe
-	github.com/mattn/go-isatty => github.com/mattn/go-isatty v0.0.12
-	github.com/mattn/go-oci8 => github.com/mattn/go-oci8 v0.0.7
-	github.com/mattn/go-runewidth => github.com/mattn/go-runewidth v0.0.4
+	github.com/mailru/easyjson => github.com/mailru/easyjson v0.7.0
+	github.com/mattn/go-colorable => github.com/mattn/go-colorable v0.1.2
+	github.com/mattn/go-isatty => github.com/mattn/go-isatty v0.0.8
+	github.com/mattn/go-runewidth => github.com/mattn/go-runewidth v0.0.0-20181025052659-b20a3daf6a39
 	github.com/mattn/go-shellwords => github.com/mattn/go-shellwords v1.0.5
 	github.com/mattn/go-sqlite3 => github.com/mattn/go-sqlite3 v1.11.0
-	github.com/mattn/go-tty => github.com/mattn/go-tty v0.0.0-20180907095812-13ff1204f104
-	github.com/mattn/go-xmlrpc => github.com/mattn/go-xmlrpc v0.0.3
 	github.com/matttproud/golang_protobuf_extensions => github.com/matttproud/golang_protobuf_extensions v1.0.1
-	github.com/mdlayher/genetlink => github.com/mdlayher/genetlink v1.0.0
-	github.com/mdlayher/netlink => github.com/mdlayher/netlink v1.1.0
-	github.com/mdlayher/wifi => github.com/mdlayher/wifi v0.0.0-20190303161829-b1436901ddee
-	github.com/mgutz/ansi => github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b
-	github.com/miekg/dns => github.com/miekg/dns v1.1.29
-	github.com/minio/md5-simd => github.com/minio/md5-simd v1.1.0
-	github.com/minio/minio-go/v7 => github.com/minio/minio-go/v7 v7.0.2
-	github.com/minio/sha256-simd => github.com/minio/sha256-simd v0.1.1
-	github.com/mitchellh/cli => github.com/mitchellh/cli v1.0.0
+	github.com/miekg/dns => github.com/miekg/dns v0.0.0-20181005163659-0d29b283ac0f
 	github.com/mitchellh/copystructure => github.com/mitchellh/copystructure v1.0.0
 	github.com/mitchellh/go-homedir => github.com/mitchellh/go-homedir v1.1.0
-	github.com/mitchellh/go-testing-interface => github.com/mitchellh/go-testing-interface v1.0.0
 	github.com/mitchellh/go-wordwrap => github.com/mitchellh/go-wordwrap v1.0.0
-	github.com/mitchellh/hashstructure => github.com/mitchellh/hashstructure v0.0.0-20170609045927-2bca23e0e452
-	github.com/mitchellh/mapstructure => github.com/mitchellh/mapstructure v1.2.2
+	github.com/mitchellh/mapstructure => github.com/mitchellh/mapstructure v1.1.2
 	github.com/mitchellh/reflectwalk => github.com/mitchellh/reflectwalk v1.0.0
 	github.com/mna/pigeon => github.com/mna/pigeon v0.0.0-20180808201053-bb0192cfc2ae
 	github.com/modern-go/concurrent => github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
 	github.com/modern-go/reflect2 => github.com/modern-go/reflect2 v1.0.1
-	github.com/montanaflynn/stats => github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe
 	github.com/morikuni/aec => github.com/morikuni/aec v0.0.0-20170113033406-39771216ff4c
-	github.com/mozillazg/go-cos => github.com/mozillazg/go-cos v0.13.0
-	github.com/mozillazg/go-httpheader => github.com/mozillazg/go-httpheader v0.2.1
-	github.com/mschoch/smat => github.com/mschoch/smat v0.0.0-20160514031455-90eadee771ae
-	github.com/munnerz/goautoneg => github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
-	github.com/mwitkow/go-conntrack => github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
+	github.com/munnerz/goautoneg => github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d
+	github.com/mwitkow/go-conntrack => github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223
 	github.com/mxk/go-flowrate => github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f
-	github.com/nakagami/firebirdsql => github.com/nakagami/firebirdsql v0.0.0-20190310045651-3c02a58cfed8
-	github.com/nats-io/jwt => github.com/nats-io/jwt v0.3.2
-	github.com/nats-io/nats-server/v2 => github.com/nats-io/nats-server/v2 v2.1.2
-	github.com/nats-io/nats.go => github.com/nats-io/nats.go v1.9.1
-	github.com/nats-io/nkeys => github.com/nats-io/nkeys v0.1.3
-	github.com/nats-io/nuid => github.com/nats-io/nuid v1.0.1
-	github.com/ncw/swift => github.com/ncw/swift v1.0.50
-	github.com/nxadm/tail => github.com/nxadm/tail v1.4.4
-	github.com/oklog/oklog => github.com/oklog/oklog v0.3.2
-	github.com/oklog/run => github.com/oklog/run v1.1.0
-	github.com/oklog/ulid => github.com/oklog/ulid v1.3.1
 	github.com/olekukonko/tablewriter => github.com/olekukonko/tablewriter v0.0.1
-	github.com/onsi/ginkgo => github.com/onsi/ginkgo v1.14.0
-	github.com/onsi/gomega => github.com/onsi/gomega v1.10.1
-	github.com/op/go-logging => github.com/op/go-logging v0.0.0-20160315200505-970db520ece7
+	github.com/onsi/ginkgo => github.com/onsi/ginkgo v1.8.0
+	github.com/onsi/gomega => github.com/onsi/gomega v1.5.0
 	github.com/open-policy-agent/opa => github.com/open-policy-agent/opa v0.18.0
-	github.com/opencontainers/go-digest => github.com/opencontainers/go-digest v1.0.0
+	github.com/opencontainers/go-digest => github.com/opencontainers/go-digest v1.0.0-rc1
 	github.com/opencontainers/image-spec => github.com/opencontainers/image-spec v1.0.1
 	github.com/opencontainers/runc => github.com/opencontainers/runc v0.1.1
-	github.com/opentracing-contrib/go-grpc => github.com/opentracing-contrib/go-grpc v0.0.0-20180928155321-4b5a12d3ff02
-	github.com/opentracing-contrib/go-observer => github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492
-	github.com/opentracing-contrib/go-stdlib => github.com/opentracing-contrib/go-stdlib v0.0.0-20190519235532-cf7a6c988dc9
-	github.com/opentracing/basictracer-go => github.com/opentracing/basictracer-go v1.0.0
+	github.com/openshift/api => github.com/openshift/api v0.0.0-20180801171038-322a19404e37
+	github.com/openshift/generic-admission-server => github.com/openshift/generic-admission-server v1.14.0
 	github.com/opentracing/opentracing-go => github.com/opentracing/opentracing-go v1.1.0
-	github.com/openzipkin-contrib/zipkin-go-opentracing => github.com/openzipkin-contrib/zipkin-go-opentracing v0.4.5
-	github.com/openzipkin/zipkin-go => github.com/openzipkin/zipkin-go v0.2.2
-	github.com/pact-foundation/pact-go => github.com/pact-foundation/pact-go v1.0.4
-	github.com/pascaldekloe/goe => github.com/pascaldekloe/goe v0.1.0
 	github.com/patrickmn/go-cache => github.com/patrickmn/go-cache v2.1.0+incompatible
-	github.com/paulbellamy/ratecounter => github.com/paulbellamy/ratecounter v0.2.0
 	github.com/pborman/uuid => github.com/pborman/uuid v1.2.0
 	github.com/pelletier/go-buffruneio => github.com/pelletier/go-buffruneio v0.2.0
-	github.com/pelletier/go-toml => github.com/pelletier/go-toml v1.7.0
-	github.com/performancecopilot/speed => github.com/performancecopilot/speed v3.0.0+incompatible
+	github.com/pelletier/go-toml => github.com/pelletier/go-toml v1.2.0
 	github.com/peterbourgon/diskv => github.com/peterbourgon/diskv v2.0.1+incompatible
-	github.com/peterh/liner => github.com/peterh/liner v1.0.1-0.20180619022028-8c1271fcf47f
+	github.com/peterh/liner => github.com/peterh/liner v0.0.0-20170211195444-bf27d3ba8e1d
 	github.com/phayes/freeport => github.com/phayes/freeport v0.0.0-20171002181615-b8543db493a5
 	github.com/philhofer/fwd => github.com/philhofer/fwd v1.0.0
-	github.com/pierrec/lz4 => github.com/pierrec/lz4 v2.0.5+incompatible
-	github.com/pkg/errors => github.com/pkg/errors v0.9.1
-	github.com/pkg/term => github.com/pkg/term v0.0.0-20180730021639-bffc007b7fd5
+	github.com/pkg/errors => github.com/pkg/errors v0.8.1
 	github.com/pmezard/go-difflib => github.com/pmezard/go-difflib v1.0.0
-	github.com/posener/complete => github.com/posener/complete v1.1.1
 	github.com/pquerna/cachecontrol => github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021
 	github.com/pquerna/ffjson => github.com/pquerna/ffjson v0.0.0-20190813045741-dac163c6c0a9
 	github.com/projectcalico/go-json => github.com/projectcalico/go-json v0.0.0-20161128004156-6219dc7339ba
 	github.com/projectcalico/go-yaml => github.com/projectcalico/go-yaml v0.0.0-20161201183616-955bc3e451ef
 	github.com/projectcalico/go-yaml-wrapper => github.com/projectcalico/go-yaml-wrapper v0.0.0-20161127220527-598e54215bee
 	github.com/projectcalico/kube-controllers => github.com/projectcalico/kube-controllers v3.8.8+incompatible
-	github.com/projectcalico/libcalico-go => github.com/projectcalico/libcalico-go v1.7.2-0.20191014160346-2382c6cdd056
-	github.com/prometheus-community/prom-label-proxy => github.com/prometheus-community/prom-label-proxy v0.2.0
-	github.com/prometheus-operator/prometheus-operator => github.com/prometheus-operator/prometheus-operator v0.42.2-0.20200928114327-fbd01683839a
-	github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring => github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.42.1
-	github.com/prometheus/alertmanager => github.com/prometheus/alertmanager v0.20.0
-	github.com/prometheus/client_golang => github.com/prometheus/client_golang v1.7.1
-	github.com/prometheus/client_model => github.com/prometheus/client_model v0.2.0
-	github.com/prometheus/common => github.com/prometheus/common v0.10.0
-	github.com/prometheus/node_exporter => github.com/prometheus/node_exporter v1.0.0-rc.0.0.20200428091818-01054558c289
-	github.com/prometheus/procfs => github.com/prometheus/procfs v0.1.3
-	github.com/prometheus/prometheus => github.com/prometheus/prometheus v1.8.2-0.20200507164740-ecee9c8abfd1
-	github.com/rafaeljusto/redigomock => github.com/rafaeljusto/redigomock v0.0.0-20190202135759-257e089e14a1
+	github.com/projectcalico/libcalico-go => github.com/projectcalico/libcalico-go v1.7.2-0.20191104213956-8f81e1e344ce
+	github.com/prometheus/client_golang => github.com/prometheus/client_golang v0.9.4
+	github.com/prometheus/client_model => github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90
+	github.com/prometheus/common => github.com/prometheus/common v0.4.0
+	github.com/prometheus/procfs => github.com/prometheus/procfs v0.0.2
+	github.com/prometheus/prometheus => github.com/prometheus/prometheus v1.8.2
 	github.com/rcrowley/go-metrics => github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a
-	github.com/retailnext/hllpp => github.com/retailnext/hllpp v1.0.1-0.20180308014038-101a6d2f8b52
+	github.com/remyoudompheng/bigfft => github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446
+	github.com/robfig/cron => github.com/robfig/cron v1.2.0
 	github.com/rogpeppe/fastuuid => github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af
 	github.com/rogpeppe/go-charset => github.com/rogpeppe/go-charset v0.0.0-20180617210344-2471d30d28b4
-	github.com/rogpeppe/go-internal => github.com/rogpeppe/go-internal v1.3.0
-	github.com/rs/cors => github.com/rs/cors v1.6.0
-	github.com/rubenv/sql-migrate => github.com/rubenv/sql-migrate v0.0.0-20200616145509-8d140a17f351
 	github.com/russross/blackfriday => github.com/russross/blackfriday v1.5.2
-	github.com/ryanuber/columnize => github.com/ryanuber/columnize v2.1.0+incompatible
-	github.com/samuel/go-zookeeper => github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da
-	github.com/santhosh-tekuri/jsonschema => github.com/santhosh-tekuri/jsonschema v1.2.4
 	github.com/satori/go.uuid => github.com/satori/go.uuid v1.2.0
-	github.com/sean-/seed => github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529
-	github.com/segmentio/fasthash => github.com/segmentio/fasthash v0.0.0-20180216231524-a72b379d632e
-	github.com/segmentio/kafka-go => github.com/segmentio/kafka-go v0.2.0
-	github.com/sercand/kuberesolver => github.com/sercand/kuberesolver v2.4.0+incompatible
 	github.com/sergi/go-diff => github.com/sergi/go-diff v1.0.0
-	github.com/shopspring/decimal => github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24
-	github.com/shurcooL/httpfs => github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749
-	github.com/shurcooL/vfsgen => github.com/shurcooL/vfsgen v0.0.0-20181202132449-6a9ea43bcacd
-	github.com/siebenmann/go-kstat => github.com/siebenmann/go-kstat v0.0.0-20160321171754-d34789b79745
+	github.com/shirou/gopsutil => github.com/shirou/gopsutil v0.0.0-20180427012116-c95755e4bcd7
+	github.com/shirou/w32 => github.com/shirou/w32 v0.0.0-20160930032740-bb4de0191aa4
 	github.com/sirupsen/logrus => github.com/sirupsen/logrus v1.4.2
-	github.com/smartystreets/assertions => github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d
-	github.com/smartystreets/goconvey => github.com/smartystreets/goconvey v1.6.4
 	github.com/soheilhy/cmux => github.com/soheilhy/cmux v0.1.4
-	github.com/sony/gobreaker => github.com/sony/gobreaker v0.4.1
 	github.com/sony/sonyflake => github.com/sony/sonyflake v0.0.0-20181109022403-6d5bd6181009
-	github.com/soundcloud/go-runit => github.com/soundcloud/go-runit v0.0.0-20150630195641-06ad41a06c4a
 	github.com/spaolacci/murmur3 => github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72
 	github.com/speps/go-hashids => github.com/speps/go-hashids v2.0.0+incompatible
 	github.com/spf13/afero => github.com/spf13/afero v1.2.2
@@ -609,27 +372,15 @@ replace (
 	github.com/spf13/pflag => github.com/spf13/pflag v1.0.5
 	github.com/spf13/viper => github.com/spf13/viper v1.4.0
 	github.com/src-d/gcfg => github.com/src-d/gcfg v1.4.0
-	github.com/streadway/amqp => github.com/streadway/amqp v0.0.0-20190827072141-edfb9018d271
-	github.com/streadway/handy => github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a
 	github.com/stretchr/objx => github.com/stretchr/objx v0.2.0
 	github.com/stretchr/testify => github.com/stretchr/testify v1.4.0
-	github.com/thanos-io/thanos => github.com/thanos-io/thanos v0.13.1-0.20200910143741-e0b7f7b32e9c
-	github.com/tidwall/pretty => github.com/tidwall/pretty v1.0.0
+	github.com/syndtr/goleveldb => github.com/syndtr/goleveldb v1.0.0
 	github.com/tinylib/msgp => github.com/tinylib/msgp v1.1.0
 	github.com/tmc/grpc-websocket-proxy => github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5
-	github.com/tv42/httpunix => github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926
-	github.com/uber/jaeger-client-go => github.com/uber/jaeger-client-go v2.23.0+incompatible
-	github.com/uber/jaeger-lib => github.com/uber/jaeger-lib v2.2.0+incompatible
 	github.com/ugorji/go => github.com/ugorji/go v1.1.4
+	github.com/ugorji/go/codec => github.com/ugorji/go/codec v0.0.0-20190128213124-ee1426cffec0
 	github.com/urfave/cli => github.com/urfave/cli v1.20.0
-	github.com/vektah/gqlparser => github.com/vektah/gqlparser v1.1.2
-	github.com/weaveworks/common => github.com/weaveworks/common v0.0.0-20200820123129-280614068c5e
-	github.com/weaveworks/promrus => github.com/weaveworks/promrus v1.2.0
-	github.com/willf/bitset => github.com/willf/bitset v1.1.3
-	github.com/xanzy/go-gitlab => github.com/xanzy/go-gitlab v0.15.0
 	github.com/xanzy/ssh-agent => github.com/xanzy/ssh-agent v0.2.1
-	github.com/xdg/scram => github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
-	github.com/xdg/stringprep => github.com/xdg/stringprep v0.0.0-20180714160509-73f8eece6fdc
 	github.com/xeipuuv/gojsonpointer => github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
 	github.com/xeipuuv/gojsonreference => github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
 	github.com/xeipuuv/gojsonschema => github.com/xeipuuv/gojsonschema v1.2.0
@@ -642,27 +393,16 @@ replace (
 	github.com/yvasiyarov/go-metrics => github.com/yvasiyarov/go-metrics v0.0.0-20150112132944-c25f46c4b940
 	github.com/yvasiyarov/gorelic => github.com/yvasiyarov/gorelic v0.0.6
 	github.com/yvasiyarov/newrelic_platform_go => github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f
-	github.com/ziutek/mymysql => github.com/ziutek/mymysql v1.5.4
-	gitlab.com/nyarla/go-crypt => gitlab.com/nyarla/go-crypt v0.0.0-20160106005555-d9a5dc2b789b
-	go.elastic.co/apm => go.elastic.co/apm v1.5.0
-	go.elastic.co/apm/module/apmhttp => go.elastic.co/apm/module/apmhttp v1.5.0
-	go.elastic.co/apm/module/apmot => go.elastic.co/apm/module/apmot v1.5.0
-	go.elastic.co/fastjson => go.elastic.co/fastjson v1.0.0
 	go.etcd.io/bbolt => go.etcd.io/bbolt v1.3.3
 	go.etcd.io/etcd => go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738
-	go.mongodb.org/mongo-driver => go.mongodb.org/mongo-driver v1.3.2
-	go.opencensus.io => go.opencensus.io v0.22.3
-	go.uber.org/atomic => go.uber.org/atomic v1.6.0
-	go.uber.org/automaxprocs => go.uber.org/automaxprocs v1.2.0
-	go.uber.org/goleak => go.uber.org/goleak v1.1.0
-	go.uber.org/multierr => go.uber.org/multierr v1.3.0
-	go.uber.org/tools => go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee
-	go.uber.org/zap => go.uber.org/zap v1.13.0
-	golang.org/x/crypto => golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
+	go.opencensus.io => go.opencensus.io v0.21.0
+	go.uber.org/atomic => go.uber.org/atomic v1.4.0
+	go.uber.org/multierr => go.uber.org/multierr v1.1.0
+	go.uber.org/zap => go.uber.org/zap v1.10.0
+	golang.org/x/crypto => golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
 	golang.org/x/exp => golang.org/x/exp v0.0.0-20190121172915-509febef88a4
 	golang.org/x/image => golang.org/x/image v0.0.0-20180708004352-c73c2afc3b81
 	golang.org/x/lint => golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f
-	golang.org/x/mod => golang.org/x/mod v0.2.0
 	golang.org/x/net => golang.org/x/net v0.0.0-20190620200207-3b0461eec859
 	golang.org/x/oauth2 => golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a
 	golang.org/x/sync => golang.org/x/sync v0.0.0-20190423024810-112230192c58
@@ -672,80 +412,80 @@ replace (
 	golang.org/x/tools => golang.org/x/tools v0.0.0-20190710153321-831012c29e42
 	golang.org/x/xerrors => golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7
 	gomodules.xyz/jsonpatch/v2 => gomodules.xyz/jsonpatch/v2 v2.0.1
-	gonum.org/v1/gonum => gonum.org/v1/gonum v0.6.0
-	gonum.org/v1/netlib => gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0
-	gonum.org/v1/plot => gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b
-	google.golang.org/api => google.golang.org/api v0.22.0
-	google.golang.org/appengine => google.golang.org/appengine v1.6.6
-	google.golang.org/genproto => google.golang.org/genproto v0.0.0-20200420144010-e5e8543f8aeb
-	google.golang.org/grpc => google.golang.org/grpc v1.26.0
-	google.golang.org/protobuf => google.golang.org/protobuf v1.23.0
+	gonum.org/v1/gonum => gonum.org/v1/gonum v0.0.0-20190331200053-3d26580ed485
+	gonum.org/v1/netlib => gonum.org/v1/netlib v0.0.0-20190331212654-76723241ea4e
+	google.golang.org/api => google.golang.org/api v0.4.0
+	google.golang.org/appengine => google.golang.org/appengine v1.6.5
+	google.golang.org/genproto => google.golang.org/genproto v0.0.0-20190916214212-f660b8655731
+	google.golang.org/grpc => google.golang.org/grpc v1.23.1
 	gopkg.in/airbrake/gobrake.v2 => gopkg.in/airbrake/gobrake.v2 v2.0.9
 	gopkg.in/alecthomas/kingpin.v2 => gopkg.in/alecthomas/kingpin.v2 v2.2.6
+	gopkg.in/alexcesaro/quotedprintable.v3 => gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc
 	gopkg.in/asn1-ber.v1 => gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d
-	gopkg.in/cas.v2 => gopkg.in/cas.v2 v2.2.0
 	gopkg.in/check.v1 => gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15
 	gopkg.in/cheggaaa/pb.v1 => gopkg.in/cheggaaa/pb.v1 v1.0.25
-	gopkg.in/errgo.v2 => gopkg.in/errgo.v2 v2.1.0
 	gopkg.in/fsnotify.v1 => gopkg.in/fsnotify.v1 v1.4.7
-	gopkg.in/fsnotify/fsnotify.v1 => gopkg.in/fsnotify/fsnotify.v1 v1.4.7
-	gopkg.in/gcfg.v1 => gopkg.in/gcfg.v1 v1.2.3
 	gopkg.in/gemnasium/logrus-airbrake-hook.v2 => gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2
 	gopkg.in/go-playground/assert.v1 => gopkg.in/go-playground/assert.v1 v1.2.1
-	gopkg.in/go-playground/validator.v9 => gopkg.in/go-playground/validator.v9 v9.27.0
-	gopkg.in/gorp.v1 => gopkg.in/gorp.v1 v1.7.2
+	gopkg.in/go-playground/validator.v8 => gopkg.in/go-playground/validator.v8 v8.18.2
+	gopkg.in/go-playground/validator.v9 => gopkg.in/go-playground/validator.v9 v9.29.1
+	gopkg.in/gomail.v2 => gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df
 	gopkg.in/inf.v0 => gopkg.in/inf.v0 v0.9.1
-	gopkg.in/ini.v1 => gopkg.in/ini.v1 v1.57.0
+	gopkg.in/mail.v2 => gopkg.in/mail.v2 v2.3.1
 	gopkg.in/natefinch/lumberjack.v2 => gopkg.in/natefinch/lumberjack.v2 v2.0.0
+	gopkg.in/resty.v1 => gopkg.in/resty.v1 v1.12.0
 	gopkg.in/square/go-jose.v1 => gopkg.in/square/go-jose.v1 v1.1.2
-	gopkg.in/square/go-jose.v2 => gopkg.in/square/go-jose.v2 v2.4.0
+	gopkg.in/square/go-jose.v2 => gopkg.in/square/go-jose.v2 v2.3.1
 	gopkg.in/src-d/go-billy.v4 => gopkg.in/src-d/go-billy.v4 v4.3.0
 	gopkg.in/src-d/go-git-fixtures.v3 => gopkg.in/src-d/go-git-fixtures.v3 v3.1.1
 	gopkg.in/src-d/go-git.v4 => gopkg.in/src-d/go-git.v4 v4.11.0
 	gopkg.in/tchap/go-patricia.v2 => gopkg.in/tchap/go-patricia.v2 v2.2.6
 	gopkg.in/tomb.v1 => gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
 	gopkg.in/warnings.v0 => gopkg.in/warnings.v0 v0.1.2
-	gopkg.in/yaml.v2 => gopkg.in/yaml.v2 v2.3.0
-	gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
+	gopkg.in/yaml.v1 => gopkg.in/yaml.v1 v1.0.0-20140924161607-9f9df34309c0
+	gopkg.in/yaml.v2 => gopkg.in/yaml.v2 v2.2.8
+	gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.0-20190905181640-827449938966
 	gotest.tools => gotest.tools v2.2.0+incompatible
-	helm.sh/helm/v3 => helm.sh/helm/v3 v3.3.0
-	honnef.co/go/tools => honnef.co/go/tools v0.0.1-2020.1.3
-	howett.net/plist => howett.net/plist v0.0.0-20181124034731-591f970eefbb
-	istio.io/api => istio.io/api v0.0.0-20201113182140-d4b7e3fc2b44
-	istio.io/client-go => istio.io/client-go v0.0.0-20201113183938-0734e976e785
-
-	istio.io/gogo-genproto => istio.io/gogo-genproto v0.0.0-20201113182723-5b8563d8a012
-	k8s.io/api => k8s.io/api v0.18.6
-	k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.18.6
-	k8s.io/apimachinery => k8s.io/apimachinery v0.18.6
-	k8s.io/apiserver => k8s.io/apiserver v0.18.6
-	k8s.io/cli-runtime => k8s.io/cli-runtime v0.18.6
-	k8s.io/client-go => k8s.io/client-go v0.18.6
-	k8s.io/code-generator => k8s.io/code-generator v0.18.6
-	k8s.io/component-base => k8s.io/component-base v0.18.6
-	k8s.io/gengo => k8s.io/gengo v0.0.0-20200114144118-36b2048a9120
+	helm.sh/helm/v3 => helm.sh/helm/v3 v3.0.1
+	honnef.co/go/tools => honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc
+	istio.io/api => istio.io/api v0.0.0-20191111210003-35e06ef8d838
+	istio.io/client-go => istio.io/client-go v0.0.0-20191113122552-9bd0ba57c3d2
+	istio.io/gogo-genproto => istio.io/gogo-genproto v0.0.0-20190930162913-45029607206a
+	k8s.io/api => k8s.io/api v0.17.3
+	k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.17.3
+	k8s.io/apimachinery => k8s.io/apimachinery v0.17.3
+	k8s.io/apiserver => k8s.io/apiserver v0.17.3
+	k8s.io/cli-runtime => k8s.io/cli-runtime v0.17.3
+	k8s.io/client-go => k8s.io/client-go v0.17.3
+	k8s.io/code-generator => k8s.io/code-generator v0.17.3
+	k8s.io/component-base => k8s.io/component-base v0.17.3
+	k8s.io/gengo => k8s.io/gengo v0.0.0-20191120174120-e74f70b9b27e
 	k8s.io/klog => k8s.io/klog v1.0.0
-	k8s.io/klog/v2 => k8s.io/klog/v2 v2.0.0
-	k8s.io/kube-openapi => k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6
-	k8s.io/kubectl => k8s.io/kubectl v0.18.6
-	k8s.io/metrics => k8s.io/metrics v0.18.6
-	k8s.io/utils => k8s.io/utils v0.0.0-20200603063816-c1c6865ac451
-	kubesphere.io/client-go => ./staging/src/kubesphere.io/client-go
-	kubesphere.io/monitoring-dashboard => kubesphere.io/monitoring-dashboard v0.1.2
-	rsc.io/binaryregexp => rsc.io/binaryregexp v0.2.0
+	k8s.io/kube-openapi => k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
+	k8s.io/kubectl => k8s.io/kubectl v0.17.3
+	k8s.io/kubernetes => k8s.io/kubernetes v1.14.0
+	k8s.io/metrics => k8s.io/metrics v0.17.3
+	k8s.io/utils => k8s.io/utils v0.0.0-20191114184206-e782cd3c129f
+	kubesphere.io/im => kubesphere.io/im v0.1.0
+	modernc.org/cc => modernc.org/cc v1.0.0
+	modernc.org/golex => modernc.org/golex v1.0.0
+	modernc.org/mathutil => modernc.org/mathutil v1.0.0
+	modernc.org/strutil => modernc.org/strutil v1.0.0
+	modernc.org/xc => modernc.org/xc v1.0.0
+	openpitrix.io/iam => openpitrix.io/iam v0.1.0
+	openpitrix.io/libqueue => openpitrix.io/libqueue v0.4.1
+	openpitrix.io/logger => openpitrix.io/logger v0.1.0
+	openpitrix.io/notification => openpitrix.io/notification v0.2.2
+	openpitrix.io/openpitrix => openpitrix.io/openpitrix v0.4.9-0.20200611125425-ae07f141e797
+	rsc.io/goversion => rsc.io/goversion v1.0.0
 	rsc.io/letsencrypt => rsc.io/letsencrypt v0.0.1
-	rsc.io/pdf => rsc.io/pdf v0.1.1
-	rsc.io/quote/v3 => rsc.io/quote/v3 v3.1.0
-	rsc.io/sampler => rsc.io/sampler v1.3.0
-	sigs.k8s.io/apiserver-network-proxy/konnectivity-client => sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.7
-	sigs.k8s.io/application => sigs.k8s.io/application v0.8.4-0.20201016185654-c8e2959e57a0
-	sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.6.4
-	sigs.k8s.io/controller-tools => sigs.k8s.io/controller-tools v0.4.0
-	sigs.k8s.io/kind => sigs.k8s.io/kind v0.8.1
-	sigs.k8s.io/kubefed => sigs.k8s.io/kubefed v0.4.0
+	sigs.k8s.io/application => kubesphere.io/application v1.0.0
+	sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.4.0
+	sigs.k8s.io/controller-tools => sigs.k8s.io/controller-tools v0.2.4
+	sigs.k8s.io/kubefed => sigs.k8s.io/kubefed v0.2.0-alpha.1
 	sigs.k8s.io/kustomize => sigs.k8s.io/kustomize v2.0.3+incompatible
-	sigs.k8s.io/structured-merge-diff/v3 => sigs.k8s.io/structured-merge-diff/v3 v3.0.0
-	sigs.k8s.io/yaml => sigs.k8s.io/yaml v1.2.0
-	sourcegraph.com/sourcegraph/appdash => sourcegraph.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0
+	sigs.k8s.io/structured-merge-diff => sigs.k8s.io/structured-merge-diff v1.0.1-0.20191108220359-b1b620dd3f06
+	sigs.k8s.io/testing_frameworks => sigs.k8s.io/testing_frameworks v0.1.2
+	sigs.k8s.io/yaml => sigs.k8s.io/yaml v1.1.0
 	vbom.ml/util => vbom.ml/util v0.0.0-20160121211510-db5cfe13f5cc
 )
--- a/go.sum
+++ b/go.sum
--- a/hack/build_e2e.sh
+++ b/hack/build_e2e.sh
@@ -1,48 +0,0 @@
-#!/usr/bin/env bash
-
-# Copyright 2020 KubeSphere Authors
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# This script builds and link stamps the output
-
-set -o errexit
-set -o nounset
-set -o pipefail
-
-KUBE_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
-source "${KUBE_ROOT}/hack/lib/init.sh"
-
-VERBOSE=${VERBOSE:-"0"}
-V=""
-if [[ "${VERBOSE}" == "1" ]];then
-    V="-x"
-    set -x
-fi
-
-ROOTDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
-
-OUTPUT_DIR=bin
-BUILDPATH=./${1:?"path to build"}
-OUT=${OUTPUT_DIR}/${1:?"output path"}
-
-BUILD_GOOS=${GOOS:-$(go env GOOS)}
-BUILD_GOARCH=${GOARCH:-$(go env GOARCH)}
-GOBINARY=${GOBINARY:-go}
-LDFLAGS=$(kube::version::ldflags)
-
-time GOOS=${BUILD_GOOS} GOARCH=${BUILD_GOARCH} ${GOBINARY} test \
-        -c \
-        -ldflags "${LDFLAGS}" \
-        -o ${OUT} \
-        ${BUILDPATH}
--- a/hack/deploy-kubesphere.sh
+++ b/hack/deploy-kubesphere.sh
@@ -1,56 +0,0 @@
-#!/usr/bin/env bash
-
-# Copyright 2020 The KubeSphere Authors.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-set -o errexit
-set -o nounset
-set -o pipefail
-
-function wait_for_installation_finish() {
-    echo "waiting for ks-installer pod ready"
-    kubectl -n kubesphere-system wait --timeout=180s --for=condition=Ready $(kubectl -n kubesphere-system get pod -l app=ks-install -oname)
-    echo "waiting for KubeSphere ready"
-    while IFS= read -r line; do
-        if [[ $line =~ "Welcome to KubeSphere" ]]
-            then
-                break
-        fi
-    done < <(timeout 900 kubectl logs -n kubesphere-system deploy/ks-installer -f)
-}
-
-# Use kubespheredev and latest tag as default image
-TAG="${TAG:-latest}"
-REPO="${REPO:-kubespheredev}"
-
-# Use KIND_LOAD_IMAGE=y .hack/deploy-kubesphere.sh to load
-# the built docker image into kind before deploying.
-if [[ "${KIND_LOAD_IMAGE:-}" == "y" ]]; then
-    kind load docker-image "$REPO/ks-apiserver:$TAG" --name="${KIND_CLUSTER_NAME:-kind}"
-    kind load docker-image "$REPO/ks-controller-manager:$TAG" --name="${KIND_CLUSTER_NAME:-kind}"
-fi
-
-# Download the latest ks-install to deploy KubeSphere
-wget --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 --tries 3 https://raw.githubusercontent.com/kubesphere/ks-installer/master/deploy/kubesphere-installer.yaml
-wget --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 --tries 3 https://raw.githubusercontent.com/kubesphere/ks-installer/master/deploy/cluster-configuration.yaml
-
-#TODO: override ks-apiserver and ks-controller-manager images with specific tag 
-
-kubectl apply -f kubesphere-installer.yaml
-kubectl apply -f cluster-configuration.yaml
-
-wait_for_installation_finish
-
-# Expose KubeSphere API Server
-kubectl -n kubesphere-system patch svc ks-apiserver -p '{"spec":{"type":"NodePort","ports":[{"name":"ks-apiserver","port":80,"protocal":"TCP","targetPort":9090,"nodePort":30881}]}}'
--- a/hack/docker_build.sh
+++ b/hack/docker_build.sh
@@ -5,47 +5,20 @@ set -o pipefail

 tag_for_branch() {
    local tag=$1
-    if [[ "${tag}" == "" ]]; then
-        tag=$(git branch --show-current)
-        tag=${tag/\//-}
-    fi
-
    if [[ "${tag}" == "master" ]]; then
        tag="latest"
    fi
    echo ${tag}
 }

-get_repo() {
-    local repo=${REPO} # read from env
-    repo=${repo:-kubespheredev}
-    if [[ "$1" != "" ]]; then
-      repo="$1"
-    fi
-
-    # set the default value if there's no user defined
-    if [[ "${repo}" == "" ]]; then
-      repo="kubespheredev"
-    fi
-    echo "$repo"
-}
-
 # push to kubespheredev with default latest tag
-TAG=$(tag_for_branch "$1")
-REPO=$(get_repo "$2")
-
-# Push image to dockerhub, need to support multiple push
-cat ~/.docker/config.json | grep index.docker.io
-if [[ $? != 0 ]]; then
-  echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
-fi
+REPO=${REPO:-kubespheredev}
+TAG=$(tag_for_branch $1)

 docker build -f build/ks-apiserver/Dockerfile -t $REPO/ks-apiserver:$TAG .
-docker push $REPO/ks-apiserver:$TAG
-# print the full docker image path for your convience
-docker images --digests | grep $REPO/ks-apiserver | grep $TAG | awk '{print $1":"$2"@"$3}'
-
 docker build -f build/ks-controller-manager/Dockerfile -t $REPO/ks-controller-manager:$TAG .
+
+# Push image to dockerhub, need to support multiple push
+echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
+docker push $REPO/ks-apiserver:$TAG
 docker push $REPO/ks-controller-manager:$TAG
-# print the full docker image path for your convience
-docker images --digests | grep $REPO/ks-controller-manager | grep $TAG | awk '{print $1":"$2"@"$3}'
--- a/hack/generate_client.sh
+++ b/hack/generate_client.sh
@@ -2,7 +2,7 @@

 set -e

-GV="network:v1alpha1 servicemesh:v1alpha2 tenant:v1alpha1 tenant:v1alpha2 devops:v1alpha1 iam:v1alpha2 devops:v1alpha3 cluster:v1alpha1 storage:v1alpha1 auditing:v1alpha1 types:v1beta1 quota:v1alpha2 application:v1alpha1 notification:v2beta1"
+GV="network:v1alpha1 servicemesh:v1alpha2 tenant:v1alpha1 tenant:v1alpha2 devops:v1alpha1 iam:v1alpha2 devops:v1alpha3 cluster:v1alpha1 storage:v1alpha1 auditing:v1alpha1 types:v1beta1"

 rm -rf ./pkg/client
 ./hack/generate_group.sh "client,lister,informer" kubesphere.io/kubesphere/pkg/client kubesphere.io/kubesphere/pkg/apis "$GV" --output-base=./  -h "$PWD/hack/boilerplate.go.txt"
--- a/hack/gobuild.sh
+++ b/hack/gobuild.sh
@@ -36,8 +36,8 @@ OUTPUT_DIR=bin
 BUILDPATH=./${1:?"path to build"}
 OUT=${OUTPUT_DIR}/${1:?"output path"}

-BUILD_GOOS=${GOOS:-$(go env GOOS)}
-BUILD_GOARCH=${GOARCH:-$(go env GOARCH)}
+BUILD_GOOS=${GOOS:-linux}
+BUILD_GOARCH=${GOARCH:-amd64}
 GOBINARY=${GOBINARY:-go}
 LDFLAGS=$(kube::version::ldflags)

--- a/hack/lib/util.sh
+++ b/hack/lib/util.sh
@@ -540,7 +540,7 @@ EOF
 # $KUBE_ROOT must be set.
 function kube::util::list_staging_repos() {
  (
-    cd "${KUBE_ROOT}/staging/src/kubesphere.io" && \
+    cd "${KUBE_ROOT}/staging/src/k8s.io" && \
    find . -mindepth 1 -maxdepth 1 -type d | cut -c 3- | sort
  )
 }
--- a/hack/update-goimports.sh
+++ b/hack/update-goimports.sh
@@ -1,35 +0,0 @@
-#!/usr/bin/env bash
-
-# Copyright 2020 The Kubernetes Authors.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-set -o errexit
-set -o nounset
-set -o pipefail
-
-. "$(dirname "${BASH_SOURCE[0]}")/lib/init.sh"
-
-cd "${KUBE_ROOT}/hack" || exit 1
-
-if ! command -v goimports &> /dev/null
-then
-    echo "goimports could not be found on your machine, please install it first"
-    exit
-fi
-
-cd "${KUBE_ROOT}" || exit 1
-
-IFS=$'\n' read -r -d '' -a files < <( find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./pkg/client/*" && printf '\0' )
-
-"goimports" -w -local kubesphere.io/kubesphere "${files[@]}"
--- a/hack/update-vendor.sh
+++ b/hack/update-vendor.sh
@@ -122,17 +122,6 @@ function add_generated_comments() {


 # Phase 1: ensure go.mod files for staging modules and main module
-for repo in $(kube::util::list_staging_repos); do
-  pushd "staging/src/kubesphere.io/${repo}" >/dev/null 2>&1
-    if [[ ! -f go.mod ]]; then
-      kube::log::status "go.mod: initialize ${repo}"
-      rm -f Godeps/Godeps.json # remove before initializing, staging Godeps are not authoritative
-      go mod init "kubesphere.io/${repo}"
-      go mod edit -fmt
-    fi
-  popd >/dev/null 2>&1
-done
-
 if [[ ! -f go.mod ]]; then
  kube::log::status "go.mod: initialize kubesphere.io/kubesphere"
  go mod init "kubesphere.io/kubesphere"
@@ -145,9 +134,6 @@ kube::log::status "go.mod: update references"
 # Prune
 go mod edit -json | jq -r '.Require[]? | select(.Version == "v0.0.0")                 | "-droprequire \(.Path)"'     | xargs -L 100 go mod edit -fmt
 go mod edit -json | jq -r '.Replace[]? | select(.New.Path | startswith("./staging/")) | "-dropreplace \(.Old.Path)"' | xargs -L 100 go mod edit -fmt
-# Readd
-kube::util::list_staging_repos | xargs -n 1 -I {} echo "-require kubesphere.io/{}@v0.0.0"                  | xargs -L 100 go mod edit -fmt
-kube::util::list_staging_repos | xargs -n 1 -I {} echo "-replace kubesphere.io/{}=./staging/src/kubesphere.io/{}" | xargs -L 100 go mod edit -fmt

 # Phase 3: capture required (minimum) versions from all modules, and replaced (pinned) versions from the root module

@@ -184,12 +170,5 @@ go mod vendor >>"${LOG_FILE}" 2>&1
 awk '{if($1=="#") print $2 " " $0; else print}' < vendor/modules.txt | sort -k1,1 -s | sed 's/.*#/#/' > "${TMP_DIR}/modules.txt.tmp"
 mv "${TMP_DIR}/modules.txt.tmp" vendor/modules.txt

-# create a symlink in vendor directory pointing to the staging components.
-# This lets other packages and tools use the local staging components as if they were vendored.
-for repo in $(kube::util::list_staging_repos); do
-  rm -fr "${KUBE_ROOT}/vendor/kubesphere.io/${repo}"
-  ln -s "../../staging/src/kubesphere.io/${repo}" "${KUBE_ROOT}/vendor/kubesphere.io/${repo}"
-done
-
 #kube::log::status "vendor: updating LICENSES file"
 #hack/update-vendor-licenses.sh >>"${LOG_FILE}" 2>&1
--- a/hack/verify-goimports.sh
+++ b/hack/verify-goimports.sh
@@ -1,43 +0,0 @@
-#!/usr/bin/env bash
-
-# Copyright 2020 The Kubernetes Authors.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-set -o errexit
-set -o nounset
-set -o pipefail
-
-. "$(dirname "${BASH_SOURCE[0]}")/lib/init.sh"
-
-cd "${KUBE_ROOT}/hack" || exit 1
-
-if ! command -v goimports &> /dev/null
-then
-    echo "goimports could not be found on your machine, please install it first"
-    exit 1
-fi
-
-cd "${KUBE_ROOT}" || exit 1
-
-IFS=$'\n' read -r -d '' -a files < <( find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./pkg/apis/*" -not -path "./pkg/client/*" && printf '\0' )
-
-output=$(goimports -local kubesphere.io/kubesphere -l "${files[@]}")
-
-if [ "${output}" != "" ]; then
-    echo "The following files are not import formatted "
-    printf '%s\n' "${output[@]}"
-    echo "Please run the following command:"
-    echo "  make goimports"
-    exit 1
-fi
--- a/install/scripts/docker_push
+++ b/install/scripts/docker_push
@@ -1,7 +1,4 @@
 #!/bin/bash

-cat ~/.docker/config.json | grep index.docker.io
-if [[ $? != 0 ]]; then
-  echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
-fi
+echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
 docker push kubespheredev/ks-apiserver:latest
--- a/kube/pkg/api/v1/pod/util.go
+++ b/kube/pkg/api/v1/pod/util.go
@@ -1,337 +0,0 @@
-/*
-Copyright 2015 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package pod
-
-import (
-	"fmt"
-	"time"
-
-	v1 "k8s.io/api/core/v1"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/util/intstr"
-	utilfeature "k8s.io/apiserver/pkg/util/feature"
-
-	"kubesphere.io/kubesphere/kube/pkg/features"
-)
-
-// FindPort locates the container port for the given pod and portName.  If the
-// targetPort is a number, use that.  If the targetPort is a string, look that
-// string up in all named ports in all containers in the target pod.  If no
-// match is found, fail.
-func FindPort(pod *v1.Pod, svcPort *v1.ServicePort) (int, error) {
-	portName := svcPort.TargetPort
-	switch portName.Type {
-	case intstr.String:
-		name := portName.StrVal
-		for _, container := range pod.Spec.Containers {
-			for _, port := range container.Ports {
-				if port.Name == name && port.Protocol == svcPort.Protocol {
-					return int(port.ContainerPort), nil
-				}
-			}
-		}
-	case intstr.Int:
-		return portName.IntValue(), nil
-	}
-
-	return 0, fmt.Errorf("no suitable port for manifest: %s", pod.UID)
-}
-
-// ContainerVisitor is called with each container spec, and returns true
-// if visiting should continue.
-type ContainerVisitor func(container *v1.Container) (shouldContinue bool)
-
-// VisitContainers invokes the visitor function with a pointer to the container
-// spec of every container in the given pod spec. If visitor returns false,
-// visiting is short-circuited. VisitContainers returns true if visiting completes,
-// false if visiting was short-circuited.
-func VisitContainers(podSpec *v1.PodSpec, visitor ContainerVisitor) bool {
-	for i := range podSpec.InitContainers {
-		if !visitor(&podSpec.InitContainers[i]) {
-			return false
-		}
-	}
-	for i := range podSpec.Containers {
-		if !visitor(&podSpec.Containers[i]) {
-			return false
-		}
-	}
-	if utilfeature.DefaultFeatureGate.Enabled(features.EphemeralContainers) {
-		for i := range podSpec.EphemeralContainers {
-			if !visitor((*v1.Container)(&podSpec.EphemeralContainers[i].EphemeralContainerCommon)) {
-				return false
-			}
-		}
-	}
-	return true
-}
-
-// Visitor is called with each object name, and returns true if visiting should continue
-type Visitor func(name string) (shouldContinue bool)
-
-// VisitPodSecretNames invokes the visitor function with the name of every secret
-// referenced by the pod spec. If visitor returns false, visiting is short-circuited.
-// Transitive references (e.g. pod -> pvc -> pv -> secret) are not visited.
-// Returns true if visiting completed, false if visiting was short-circuited.
-func VisitPodSecretNames(pod *v1.Pod, visitor Visitor) bool {
-	for _, reference := range pod.Spec.ImagePullSecrets {
-		if !visitor(reference.Name) {
-			return false
-		}
-	}
-	VisitContainers(&pod.Spec, func(c *v1.Container) bool {
-		return visitContainerSecretNames(c, visitor)
-	})
-	var source *v1.VolumeSource
-
-	for i := range pod.Spec.Volumes {
-		source = &pod.Spec.Volumes[i].VolumeSource
-		switch {
-		case source.AzureFile != nil:
-			if len(source.AzureFile.SecretName) > 0 && !visitor(source.AzureFile.SecretName) {
-				return false
-			}
-		case source.CephFS != nil:
-			if source.CephFS.SecretRef != nil && !visitor(source.CephFS.SecretRef.Name) {
-				return false
-			}
-		case source.Cinder != nil:
-			if source.Cinder.SecretRef != nil && !visitor(source.Cinder.SecretRef.Name) {
-				return false
-			}
-		case source.FlexVolume != nil:
-			if source.FlexVolume.SecretRef != nil && !visitor(source.FlexVolume.SecretRef.Name) {
-				return false
-			}
-		case source.Projected != nil:
-			for j := range source.Projected.Sources {
-				if source.Projected.Sources[j].Secret != nil {
-					if !visitor(source.Projected.Sources[j].Secret.Name) {
-						return false
-					}
-				}
-			}
-		case source.RBD != nil:
-			if source.RBD.SecretRef != nil && !visitor(source.RBD.SecretRef.Name) {
-				return false
-			}
-		case source.Secret != nil:
-			if !visitor(source.Secret.SecretName) {
-				return false
-			}
-		case source.ScaleIO != nil:
-			if source.ScaleIO.SecretRef != nil && !visitor(source.ScaleIO.SecretRef.Name) {
-				return false
-			}
-		case source.ISCSI != nil:
-			if source.ISCSI.SecretRef != nil && !visitor(source.ISCSI.SecretRef.Name) {
-				return false
-			}
-		case source.StorageOS != nil:
-			if source.StorageOS.SecretRef != nil && !visitor(source.StorageOS.SecretRef.Name) {
-				return false
-			}
-		case source.CSI != nil:
-			if source.CSI.NodePublishSecretRef != nil && !visitor(source.CSI.NodePublishSecretRef.Name) {
-				return false
-			}
-		}
-	}
-	return true
-}
-
-func visitContainerSecretNames(container *v1.Container, visitor Visitor) bool {
-	for _, env := range container.EnvFrom {
-		if env.SecretRef != nil {
-			if !visitor(env.SecretRef.Name) {
-				return false
-			}
-		}
-	}
-	for _, envVar := range container.Env {
-		if envVar.ValueFrom != nil && envVar.ValueFrom.SecretKeyRef != nil {
-			if !visitor(envVar.ValueFrom.SecretKeyRef.Name) {
-				return false
-			}
-		}
-	}
-	return true
-}
-
-// VisitPodConfigmapNames invokes the visitor function with the name of every configmap
-// referenced by the pod spec. If visitor returns false, visiting is short-circuited.
-// Transitive references (e.g. pod -> pvc -> pv -> secret) are not visited.
-// Returns true if visiting completed, false if visiting was short-circuited.
-func VisitPodConfigmapNames(pod *v1.Pod, visitor Visitor) bool {
-	VisitContainers(&pod.Spec, func(c *v1.Container) bool {
-		return visitContainerConfigmapNames(c, visitor)
-	})
-	var source *v1.VolumeSource
-	for i := range pod.Spec.Volumes {
-		source = &pod.Spec.Volumes[i].VolumeSource
-		switch {
-		case source.Projected != nil:
-			for j := range source.Projected.Sources {
-				if source.Projected.Sources[j].ConfigMap != nil {
-					if !visitor(source.Projected.Sources[j].ConfigMap.Name) {
-						return false
-					}
-				}
-			}
-		case source.ConfigMap != nil:
-			if !visitor(source.ConfigMap.Name) {
-				return false
-			}
-		}
-	}
-	return true
-}
-
-func visitContainerConfigmapNames(container *v1.Container, visitor Visitor) bool {
-	for _, env := range container.EnvFrom {
-		if env.ConfigMapRef != nil {
-			if !visitor(env.ConfigMapRef.Name) {
-				return false
-			}
-		}
-	}
-	for _, envVar := range container.Env {
-		if envVar.ValueFrom != nil && envVar.ValueFrom.ConfigMapKeyRef != nil {
-			if !visitor(envVar.ValueFrom.ConfigMapKeyRef.Name) {
-				return false
-			}
-		}
-	}
-	return true
-}
-
-// GetContainerStatus extracts the status of container "name" from "statuses".
-// It also returns if "name" exists.
-func GetContainerStatus(statuses []v1.ContainerStatus, name string) (v1.ContainerStatus, bool) {
-	for i := range statuses {
-		if statuses[i].Name == name {
-			return statuses[i], true
-		}
-	}
-	return v1.ContainerStatus{}, false
-}
-
-// GetExistingContainerStatus extracts the status of container "name" from "statuses",
-// It also returns if "name" exists.
-func GetExistingContainerStatus(statuses []v1.ContainerStatus, name string) v1.ContainerStatus {
-	status, _ := GetContainerStatus(statuses, name)
-	return status
-}
-
-// IsPodAvailable returns true if a pod is available; false otherwise.
-// Precondition for an available pod is that it must be ready. On top
-// of that, there are two cases when a pod can be considered available:
-// 1. minReadySeconds == 0, or
-// 2. LastTransitionTime (is set) + minReadySeconds < current time
-func IsPodAvailable(pod *v1.Pod, minReadySeconds int32, now metav1.Time) bool {
-	if !IsPodReady(pod) {
-		return false
-	}
-
-	c := GetPodReadyCondition(pod.Status)
-	minReadySecondsDuration := time.Duration(minReadySeconds) * time.Second
-	if minReadySeconds == 0 || !c.LastTransitionTime.IsZero() && c.LastTransitionTime.Add(minReadySecondsDuration).Before(now.Time) {
-		return true
-	}
-	return false
-}
-
-// IsPodReady returns true if a pod is ready; false otherwise.
-func IsPodReady(pod *v1.Pod) bool {
-	return IsPodReadyConditionTrue(pod.Status)
-}
-
-// IsPodReadyConditionTrue returns true if a pod is ready; false otherwise.
-func IsPodReadyConditionTrue(status v1.PodStatus) bool {
-	condition := GetPodReadyCondition(status)
-	return condition != nil && condition.Status == v1.ConditionTrue
-}
-
-// GetPodReadyCondition extracts the pod ready condition from the given status and returns that.
-// Returns nil if the condition is not present.
-func GetPodReadyCondition(status v1.PodStatus) *v1.PodCondition {
-	_, condition := GetPodCondition(&status, v1.PodReady)
-	return condition
-}
-
-// GetPodCondition extracts the provided condition from the given status and returns that.
-// Returns nil and -1 if the condition is not present, and the index of the located condition.
-func GetPodCondition(status *v1.PodStatus, conditionType v1.PodConditionType) (int, *v1.PodCondition) {
-	if status == nil {
-		return -1, nil
-	}
-	return GetPodConditionFromList(status.Conditions, conditionType)
-}
-
-// GetPodConditionFromList extracts the provided condition from the given list of condition and
-// returns the index of the condition and the condition. Returns -1 and nil if the condition is not present.
-func GetPodConditionFromList(conditions []v1.PodCondition, conditionType v1.PodConditionType) (int, *v1.PodCondition) {
-	if conditions == nil {
-		return -1, nil
-	}
-	for i := range conditions {
-		if conditions[i].Type == conditionType {
-			return i, &conditions[i]
-		}
-	}
-	return -1, nil
-}
-
-// UpdatePodCondition updates existing pod condition or creates a new one. Sets LastTransitionTime to now if the
-// status has changed.
-// Returns true if pod condition has changed or has been added.
-func UpdatePodCondition(status *v1.PodStatus, condition *v1.PodCondition) bool {
-	condition.LastTransitionTime = metav1.Now()
-	// Try to find this pod condition.
-	conditionIndex, oldCondition := GetPodCondition(status, condition.Type)
-
-	if oldCondition == nil {
-		// We are adding new pod condition.
-		status.Conditions = append(status.Conditions, *condition)
-		return true
-	}
-	// We are updating an existing condition, so we need to check if it has changed.
-	if condition.Status == oldCondition.Status {
-		condition.LastTransitionTime = oldCondition.LastTransitionTime
-	}
-
-	isEqual := condition.Status == oldCondition.Status &&
-		condition.Reason == oldCondition.Reason &&
-		condition.Message == oldCondition.Message &&
-		condition.LastProbeTime.Equal(&oldCondition.LastProbeTime) &&
-		condition.LastTransitionTime.Equal(&oldCondition.LastTransitionTime)
-
-	status.Conditions[conditionIndex] = *condition
-	// Return true if one of the fields have changed.
-	return !isEqual
-}
-
-// GetPodPriority returns priority of the given pod.
-func GetPodPriority(pod *v1.Pod) int32 {
-	if pod.Spec.Priority != nil {
-		return *pod.Spec.Priority
-	}
-	// When priority of a running pod is nil, it means it was created at a time
-	// that there was no global default priority class and the priority class
-	// name of the pod was empty. So, we resolve to the static default priority.
-	return 0
-}
--- a/kube/pkg/apis/core/helper/helpers.go
+++ b/kube/pkg/apis/core/helper/helpers.go
@@ -1,51 +0,0 @@
-/*
-
-Copyright 2021 The KubeSphere Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-
-*/
-
-package helper
-
-import (
-	"k8s.io/apimachinery/pkg/api/resource"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/conversion"
-	"k8s.io/apimachinery/pkg/fields"
-	"k8s.io/apimachinery/pkg/labels"
-)
-
-// Semantic can do semantic deep equality checks for core objects.
-// Example: apiequality.Semantic.DeepEqual(aPod, aPodWithNonNilButEmptyMaps) == true
-var Semantic = conversion.EqualitiesOrDie(
-	func(a, b resource.Quantity) bool {
-		// Ignore formatting, only care that numeric value stayed the same.
-		// TODO: if we decide it's important, it should be safe to start comparing the format.
-		//
-		// Uninitialized quantities are equivalent to 0 quantities.
-		return a.Cmp(b) == 0
-	},
-	func(a, b metav1.MicroTime) bool {
-		return a.UTC() == b.UTC()
-	},
-	func(a, b metav1.Time) bool {
-		return a.UTC() == b.UTC()
-	},
-	func(a, b labels.Selector) bool {
-		return a.String() == b.String()
-	},
-	func(a, b fields.Selector) bool {
-		return a.String() == b.String()
-	},
-)
--- a/kube/pkg/apis/core/v1/helper/helpers.go
+++ b/kube/pkg/apis/core/v1/helper/helpers.go
@@ -1,501 +0,0 @@
-/*
-Copyright 2014 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package helper
-
-import (
-	"encoding/json"
-	"fmt"
-	"strings"
-
-	v1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/apimachinery/pkg/fields"
-	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apimachinery/pkg/selection"
-	"k8s.io/apimachinery/pkg/util/validation"
-
-	"kubesphere.io/kubesphere/kube/pkg/apis/core/helper"
-)
-
-// IsExtendedResourceName returns true if:
-// 1. the resource name is not in the default namespace;
-// 2. resource name does not have "requests." prefix,
-// to avoid confusion with the convention in quota
-// 3. it satisfies the rules in IsQualifiedName() after converted into quota resource name
-func IsExtendedResourceName(name v1.ResourceName) bool {
-	if IsNativeResource(name) || strings.HasPrefix(string(name), v1.DefaultResourceRequestsPrefix) {
-		return false
-	}
-	// Ensure it satisfies the rules in IsQualifiedName() after converted into quota resource name
-	nameForQuota := fmt.Sprintf("%s%s", v1.DefaultResourceRequestsPrefix, string(name))
-	if errs := validation.IsQualifiedName(string(nameForQuota)); len(errs) != 0 {
-		return false
-	}
-	return true
-}
-
-// IsPrefixedNativeResource returns true if the resource name is in the
-// *kubernetes.io/ namespace.
-func IsPrefixedNativeResource(name v1.ResourceName) bool {
-	return strings.Contains(string(name), v1.ResourceDefaultNamespacePrefix)
-}
-
-// IsNativeResource returns true if the resource name is in the
-// *kubernetes.io/ namespace. Partially-qualified (unprefixed) names are
-// implicitly in the kubernetes.io/ namespace.
-func IsNativeResource(name v1.ResourceName) bool {
-	return !strings.Contains(string(name), "/") ||
-		IsPrefixedNativeResource(name)
-}
-
-// IsHugePageResourceName returns true if the resource name has the huge page
-// resource prefix.
-func IsHugePageResourceName(name v1.ResourceName) bool {
-	return strings.HasPrefix(string(name), v1.ResourceHugePagesPrefix)
-}
-
-// HugePageResourceName returns a ResourceName with the canonical hugepage
-// prefix prepended for the specified page size.  The page size is converted
-// to its canonical representation.
-func HugePageResourceName(pageSize resource.Quantity) v1.ResourceName {
-	return v1.ResourceName(fmt.Sprintf("%s%s", v1.ResourceHugePagesPrefix, pageSize.String()))
-}
-
-// HugePageSizeFromResourceName returns the page size for the specified huge page
-// resource name.  If the specified input is not a valid huge page resource name
-// an error is returned.
-func HugePageSizeFromResourceName(name v1.ResourceName) (resource.Quantity, error) {
-	if !IsHugePageResourceName(name) {
-		return resource.Quantity{}, fmt.Errorf("resource name: %s is an invalid hugepage name", name)
-	}
-	pageSize := strings.TrimPrefix(string(name), v1.ResourceHugePagesPrefix)
-	return resource.ParseQuantity(pageSize)
-}
-
-// IsOvercommitAllowed returns true if the resource is in the default
-// namespace and is not hugepages.
-func IsOvercommitAllowed(name v1.ResourceName) bool {
-	return IsNativeResource(name) &&
-		!IsHugePageResourceName(name)
-}
-
-func IsAttachableVolumeResourceName(name v1.ResourceName) bool {
-	return strings.HasPrefix(string(name), v1.ResourceAttachableVolumesPrefix)
-}
-
-// Extended and Hugepages resources
-func IsScalarResourceName(name v1.ResourceName) bool {
-	return IsExtendedResourceName(name) || IsHugePageResourceName(name) ||
-		IsPrefixedNativeResource(name) || IsAttachableVolumeResourceName(name)
-}
-
-// this function aims to check if the service's ClusterIP is set or not
-// the objective is not to perform validation here
-func IsServiceIPSet(service *v1.Service) bool {
-	return service.Spec.ClusterIP != v1.ClusterIPNone && service.Spec.ClusterIP != ""
-}
-
-// TODO: make method on LoadBalancerStatus?
-func LoadBalancerStatusEqual(l, r *v1.LoadBalancerStatus) bool {
-	return ingressSliceEqual(l.Ingress, r.Ingress)
-}
-
-func ingressSliceEqual(lhs, rhs []v1.LoadBalancerIngress) bool {
-	if len(lhs) != len(rhs) {
-		return false
-	}
-	for i := range lhs {
-		if !ingressEqual(&lhs[i], &rhs[i]) {
-			return false
-		}
-	}
-	return true
-}
-
-func ingressEqual(lhs, rhs *v1.LoadBalancerIngress) bool {
-	if lhs.IP != rhs.IP {
-		return false
-	}
-	if lhs.Hostname != rhs.Hostname {
-		return false
-	}
-	return true
-}
-
-// GetAccessModesAsString returns a string representation of an array of access modes.
-// modes, when present, are always in the same order: RWO,ROX,RWX.
-func GetAccessModesAsString(modes []v1.PersistentVolumeAccessMode) string {
-	modes = removeDuplicateAccessModes(modes)
-	modesStr := []string{}
-	if containsAccessMode(modes, v1.ReadWriteOnce) {
-		modesStr = append(modesStr, "RWO")
-	}
-	if containsAccessMode(modes, v1.ReadOnlyMany) {
-		modesStr = append(modesStr, "ROX")
-	}
-	if containsAccessMode(modes, v1.ReadWriteMany) {
-		modesStr = append(modesStr, "RWX")
-	}
-	return strings.Join(modesStr, ",")
-}
-
-// GetAccessModesAsString returns an array of AccessModes from a string created by GetAccessModesAsString
-func GetAccessModesFromString(modes string) []v1.PersistentVolumeAccessMode {
-	strmodes := strings.Split(modes, ",")
-	accessModes := []v1.PersistentVolumeAccessMode{}
-	for _, s := range strmodes {
-		s = strings.Trim(s, " ")
-		switch {
-		case s == "RWO":
-			accessModes = append(accessModes, v1.ReadWriteOnce)
-		case s == "ROX":
-			accessModes = append(accessModes, v1.ReadOnlyMany)
-		case s == "RWX":
-			accessModes = append(accessModes, v1.ReadWriteMany)
-		}
-	}
-	return accessModes
-}
-
-// removeDuplicateAccessModes returns an array of access modes without any duplicates
-func removeDuplicateAccessModes(modes []v1.PersistentVolumeAccessMode) []v1.PersistentVolumeAccessMode {
-	accessModes := []v1.PersistentVolumeAccessMode{}
-	for _, m := range modes {
-		if !containsAccessMode(accessModes, m) {
-			accessModes = append(accessModes, m)
-		}
-	}
-	return accessModes
-}
-
-func containsAccessMode(modes []v1.PersistentVolumeAccessMode, mode v1.PersistentVolumeAccessMode) bool {
-	for _, m := range modes {
-		if m == mode {
-			return true
-		}
-	}
-	return false
-}
-
-// NodeSelectorRequirementsAsSelector converts the []NodeSelectorRequirement api type into a struct that implements
-// labels.Selector.
-func NodeSelectorRequirementsAsSelector(nsm []v1.NodeSelectorRequirement) (labels.Selector, error) {
-	if len(nsm) == 0 {
-		return labels.Nothing(), nil
-	}
-	selector := labels.NewSelector()
-	for _, expr := range nsm {
-		var op selection.Operator
-		switch expr.Operator {
-		case v1.NodeSelectorOpIn:
-			op = selection.In
-		case v1.NodeSelectorOpNotIn:
-			op = selection.NotIn
-		case v1.NodeSelectorOpExists:
-			op = selection.Exists
-		case v1.NodeSelectorOpDoesNotExist:
-			op = selection.DoesNotExist
-		case v1.NodeSelectorOpGt:
-			op = selection.GreaterThan
-		case v1.NodeSelectorOpLt:
-			op = selection.LessThan
-		default:
-			return nil, fmt.Errorf("%q is not a valid node selector operator", expr.Operator)
-		}
-		r, err := labels.NewRequirement(expr.Key, op, expr.Values)
-		if err != nil {
-			return nil, err
-		}
-		selector = selector.Add(*r)
-	}
-	return selector, nil
-}
-
-// NodeSelectorRequirementsAsFieldSelector converts the []NodeSelectorRequirement core type into a struct that implements
-// fields.Selector.
-func NodeSelectorRequirementsAsFieldSelector(nsm []v1.NodeSelectorRequirement) (fields.Selector, error) {
-	if len(nsm) == 0 {
-		return fields.Nothing(), nil
-	}
-
-	selectors := []fields.Selector{}
-	for _, expr := range nsm {
-		switch expr.Operator {
-		case v1.NodeSelectorOpIn:
-			if len(expr.Values) != 1 {
-				return nil, fmt.Errorf("unexpected number of value (%d) for node field selector operator %q",
-					len(expr.Values), expr.Operator)
-			}
-			selectors = append(selectors, fields.OneTermEqualSelector(expr.Key, expr.Values[0]))
-
-		case v1.NodeSelectorOpNotIn:
-			if len(expr.Values) != 1 {
-				return nil, fmt.Errorf("unexpected number of value (%d) for node field selector operator %q",
-					len(expr.Values), expr.Operator)
-			}
-			selectors = append(selectors, fields.OneTermNotEqualSelector(expr.Key, expr.Values[0]))
-
-		default:
-			return nil, fmt.Errorf("%q is not a valid node field selector operator", expr.Operator)
-		}
-	}
-
-	return fields.AndSelectors(selectors...), nil
-}
-
-// NodeSelectorRequirementKeysExistInNodeSelectorTerms checks if a NodeSelectorTerm with key is already specified in terms
-func NodeSelectorRequirementKeysExistInNodeSelectorTerms(reqs []v1.NodeSelectorRequirement, terms []v1.NodeSelectorTerm) bool {
-	for _, req := range reqs {
-		for _, term := range terms {
-			for _, r := range term.MatchExpressions {
-				if r.Key == req.Key {
-					return true
-				}
-			}
-		}
-	}
-	return false
-}
-
-// MatchNodeSelectorTerms checks whether the node labels and fields match node selector terms in ORed;
-// nil or empty term matches no objects.
-func MatchNodeSelectorTerms(
-	nodeSelectorTerms []v1.NodeSelectorTerm,
-	nodeLabels labels.Set,
-	nodeFields fields.Set,
-) bool {
-	for _, req := range nodeSelectorTerms {
-		// nil or empty term selects no objects
-		if len(req.MatchExpressions) == 0 && len(req.MatchFields) == 0 {
-			continue
-		}
-
-		if len(req.MatchExpressions) != 0 {
-			labelSelector, err := NodeSelectorRequirementsAsSelector(req.MatchExpressions)
-			if err != nil || !labelSelector.Matches(nodeLabels) {
-				continue
-			}
-		}
-
-		if len(req.MatchFields) != 0 {
-			fieldSelector, err := NodeSelectorRequirementsAsFieldSelector(req.MatchFields)
-			if err != nil || !fieldSelector.Matches(nodeFields) {
-				continue
-			}
-		}
-
-		return true
-	}
-
-	return false
-}
-
-// TopologySelectorRequirementsAsSelector converts the []TopologySelectorLabelRequirement api type into a struct
-// that implements labels.Selector.
-func TopologySelectorRequirementsAsSelector(tsm []v1.TopologySelectorLabelRequirement) (labels.Selector, error) {
-	if len(tsm) == 0 {
-		return labels.Nothing(), nil
-	}
-
-	selector := labels.NewSelector()
-	for _, expr := range tsm {
-		r, err := labels.NewRequirement(expr.Key, selection.In, expr.Values)
-		if err != nil {
-			return nil, err
-		}
-		selector = selector.Add(*r)
-	}
-
-	return selector, nil
-}
-
-// MatchTopologySelectorTerms checks whether given labels match topology selector terms in ORed;
-// nil or empty term matches no objects; while empty term list matches all objects.
-func MatchTopologySelectorTerms(topologySelectorTerms []v1.TopologySelectorTerm, lbls labels.Set) bool {
-	if len(topologySelectorTerms) == 0 {
-		// empty term list matches all objects
-		return true
-	}
-
-	for _, req := range topologySelectorTerms {
-		// nil or empty term selects no objects
-		if len(req.MatchLabelExpressions) == 0 {
-			continue
-		}
-
-		labelSelector, err := TopologySelectorRequirementsAsSelector(req.MatchLabelExpressions)
-		if err != nil || !labelSelector.Matches(lbls) {
-			continue
-		}
-
-		return true
-	}
-
-	return false
-}
-
-// AddOrUpdateTolerationInPodSpec tries to add a toleration to the toleration list in PodSpec.
-// Returns true if something was updated, false otherwise.
-func AddOrUpdateTolerationInPodSpec(spec *v1.PodSpec, toleration *v1.Toleration) bool {
-	podTolerations := spec.Tolerations
-
-	var newTolerations []v1.Toleration
-	updated := false
-	for i := range podTolerations {
-		if toleration.MatchToleration(&podTolerations[i]) {
-			if helper.Semantic.DeepEqual(toleration, podTolerations[i]) {
-				return false
-			}
-			newTolerations = append(newTolerations, *toleration)
-			updated = true
-			continue
-		}
-
-		newTolerations = append(newTolerations, podTolerations[i])
-	}
-
-	if !updated {
-		newTolerations = append(newTolerations, *toleration)
-	}
-
-	spec.Tolerations = newTolerations
-	return true
-}
-
-// AddOrUpdateTolerationInPod tries to add a toleration to the pod's toleration list.
-// Returns true if something was updated, false otherwise.
-func AddOrUpdateTolerationInPod(pod *v1.Pod, toleration *v1.Toleration) bool {
-	return AddOrUpdateTolerationInPodSpec(&pod.Spec, toleration)
-}
-
-// TolerationsTolerateTaint checks if taint is tolerated by any of the tolerations.
-func TolerationsTolerateTaint(tolerations []v1.Toleration, taint *v1.Taint) bool {
-	for i := range tolerations {
-		if tolerations[i].ToleratesTaint(taint) {
-			return true
-		}
-	}
-	return false
-}
-
-type taintsFilterFunc func(*v1.Taint) bool
-
-// TolerationsTolerateTaintsWithFilter checks if given tolerations tolerates
-// all the taints that apply to the filter in given taint list.
-func TolerationsTolerateTaintsWithFilter(tolerations []v1.Toleration, taints []v1.Taint, applyFilter taintsFilterFunc) bool {
-	if len(taints) == 0 {
-		return true
-	}
-
-	for i := range taints {
-		if applyFilter != nil && !applyFilter(&taints[i]) {
-			continue
-		}
-
-		if !TolerationsTolerateTaint(tolerations, &taints[i]) {
-			return false
-		}
-	}
-
-	return true
-}
-
-// Returns true and list of Tolerations matching all Taints if all are tolerated, or false otherwise.
-func GetMatchingTolerations(taints []v1.Taint, tolerations []v1.Toleration) (bool, []v1.Toleration) {
-	if len(taints) == 0 {
-		return true, []v1.Toleration{}
-	}
-	if len(tolerations) == 0 && len(taints) > 0 {
-		return false, []v1.Toleration{}
-	}
-	result := []v1.Toleration{}
-	for i := range taints {
-		tolerated := false
-		for j := range tolerations {
-			if tolerations[j].ToleratesTaint(&taints[i]) {
-				result = append(result, tolerations[j])
-				tolerated = true
-				break
-			}
-		}
-		if !tolerated {
-			return false, []v1.Toleration{}
-		}
-	}
-	return true, result
-}
-
-func GetAvoidPodsFromNodeAnnotations(annotations map[string]string) (v1.AvoidPods, error) {
-	var avoidPods v1.AvoidPods
-	if len(annotations) > 0 && annotations[v1.PreferAvoidPodsAnnotationKey] != "" {
-		err := json.Unmarshal([]byte(annotations[v1.PreferAvoidPodsAnnotationKey]), &avoidPods)
-		if err != nil {
-			return avoidPods, err
-		}
-	}
-	return avoidPods, nil
-}
-
-// GetPersistentVolumeClass returns StorageClassName.
-func GetPersistentVolumeClass(volume *v1.PersistentVolume) string {
-	// Use beta annotation first
-	if class, found := volume.Annotations[v1.BetaStorageClassAnnotation]; found {
-		return class
-	}
-
-	return volume.Spec.StorageClassName
-}
-
-// GetPersistentVolumeClaimClass returns StorageClassName. If no storage class was
-// requested, it returns "".
-func GetPersistentVolumeClaimClass(claim *v1.PersistentVolumeClaim) string {
-	// Use beta annotation first
-	if class, found := claim.Annotations[v1.BetaStorageClassAnnotation]; found {
-		return class
-	}
-
-	if claim.Spec.StorageClassName != nil {
-		return *claim.Spec.StorageClassName
-	}
-
-	return ""
-}
-
-// ScopedResourceSelectorRequirementsAsSelector converts the ScopedResourceSelectorRequirement api type into a struct that implements
-// labels.Selector.
-func ScopedResourceSelectorRequirementsAsSelector(ssr v1.ScopedResourceSelectorRequirement) (labels.Selector, error) {
-	selector := labels.NewSelector()
-	var op selection.Operator
-	switch ssr.Operator {
-	case v1.ScopeSelectorOpIn:
-		op = selection.In
-	case v1.ScopeSelectorOpNotIn:
-		op = selection.NotIn
-	case v1.ScopeSelectorOpExists:
-		op = selection.Exists
-	case v1.ScopeSelectorOpDoesNotExist:
-		op = selection.DoesNotExist
-	default:
-		return nil, fmt.Errorf("%q is not a valid scope selector operator", ssr.Operator)
-	}
-	r, err := labels.NewRequirement(string(ssr.ScopeName), op, ssr.Values)
-	if err != nil {
-		return nil, err
-	}
-	selector = selector.Add(*r)
-	return selector, nil
-}
--- a/kube/pkg/apis/core/v1/helper/qos/qos.go
+++ b/kube/pkg/apis/core/v1/helper/qos/qos.go
@@ -1,103 +0,0 @@
-/*
-
- Copyright 2021 The KubeSphere Authors.
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
-*/
-
-package qos
-
-import (
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/apimachinery/pkg/util/sets"
-)
-
-var supportedQoSComputeResources = sets.NewString(string(corev1.ResourceCPU), string(corev1.ResourceMemory))
-
-// QOSList is a set of (resource name, QoS class) pairs.
-type QOSList map[corev1.ResourceName]corev1.PodQOSClass
-
-func isSupportedQoSComputeResource(name corev1.ResourceName) bool {
-	return supportedQoSComputeResources.Has(string(name))
-}
-
-// GetPodQOS returns the QoS class of a pod.
-// A pod is besteffort if none of its containers have specified any requests or limits.
-// A pod is guaranteed only when requests and limits are specified for all the containers and they are equal.
-// A pod is burstable if limits and requests do not match across all containers.
-func GetPodQOS(pod *corev1.Pod) corev1.PodQOSClass {
-	requests := corev1.ResourceList{}
-	limits := corev1.ResourceList{}
-	zeroQuantity := resource.MustParse("0")
-	isGuaranteed := true
-	allContainers := []corev1.Container{}
-	allContainers = append(allContainers, pod.Spec.Containers...)
-	allContainers = append(allContainers, pod.Spec.InitContainers...)
-	for _, container := range allContainers {
-		// process requests
-		for name, quantity := range container.Resources.Requests {
-			if !isSupportedQoSComputeResource(name) {
-				continue
-			}
-			if quantity.Cmp(zeroQuantity) == 1 {
-				delta := quantity.DeepCopy()
-				if _, exists := requests[name]; !exists {
-					requests[name] = delta
-				} else {
-					delta.Add(requests[name])
-					requests[name] = delta
-				}
-			}
-		}
-		// process limits
-		qosLimitsFound := sets.NewString()
-		for name, quantity := range container.Resources.Limits {
-			if !isSupportedQoSComputeResource(name) {
-				continue
-			}
-			if quantity.Cmp(zeroQuantity) == 1 {
-				qosLimitsFound.Insert(string(name))
-				delta := quantity.DeepCopy()
-				if _, exists := limits[name]; !exists {
-					limits[name] = delta
-				} else {
-					delta.Add(limits[name])
-					limits[name] = delta
-				}
-			}
-		}
-
-		if !qosLimitsFound.HasAll(string(corev1.ResourceMemory), string(corev1.ResourceCPU)) {
-			isGuaranteed = false
-		}
-	}
-	if len(requests) == 0 && len(limits) == 0 {
-		return corev1.PodQOSBestEffort
-	}
-	// Check is requests match limits for all resources.
-	if isGuaranteed {
-		for name, req := range requests {
-			if lim, exists := limits[name]; !exists || lim.Cmp(req) != 0 {
-				isGuaranteed = false
-				break
-			}
-		}
-	}
-	if isGuaranteed &&
-		len(requests) == len(limits) {
-		return corev1.PodQOSGuaranteed
-	}
-	return corev1.PodQOSBurstable
-}
--- a/kube/pkg/features/features.go
+++ b/kube/pkg/features/features.go
@@ -1,680 +0,0 @@
-/*
-Copyright 2017 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package features
-
-import (
-	"k8s.io/component-base/featuregate"
-)
-
-const (
-	// Every feature gate should add method here following this template:
-	//
-	// // owner: @username
-	// // alpha: v1.X
-	// MyFeature featuregate.Feature = "MyFeature"
-
-	// owner: @tallclair
-	// beta: v1.4
-	AppArmor featuregate.Feature = "AppArmor"
-
-	// owner: @mtaufen
-	// alpha: v1.4
-	// beta: v1.11
-	DynamicKubeletConfig featuregate.Feature = "DynamicKubeletConfig"
-
-	// owner: @pweil-
-	// alpha: v1.5
-	//
-	// Default userns=host for containers that are using other host namespaces, host mounts, the pod
-	// contains a privileged container, or specific non-namespaced capabilities (MKNOD, SYS_MODULE,
-	// SYS_TIME). This should only be enabled if user namespace remapping is enabled in the docker daemon.
-	ExperimentalHostUserNamespaceDefaultingGate featuregate.Feature = "ExperimentalHostUserNamespaceDefaulting"
-
-	// owner: @jiayingz
-	// beta: v1.10
-	//
-	// Enables support for Device Plugins
-	DevicePlugins featuregate.Feature = "DevicePlugins"
-
-	// owner: @dxist
-	// alpha: v1.16
-	//
-	// Enables support of HPA scaling to zero pods when an object or custom metric is configured.
-	HPAScaleToZero featuregate.Feature = "HPAScaleToZero"
-
-	// owner: @mikedanese
-	// alpha: v1.7
-	// beta: v1.12
-	//
-	// Gets a server certificate for the kubelet from the Certificate Signing
-	// Request API instead of generating one self signed and auto rotates the
-	// certificate as expiration approaches.
-	RotateKubeletServerCertificate featuregate.Feature = "RotateKubeletServerCertificate"
-
-	// owner: @jinxu
-	// beta: v1.10
-	//
-	// New local storage types to support local storage capacity isolation
-	LocalStorageCapacityIsolation featuregate.Feature = "LocalStorageCapacityIsolation"
-
-	// owner: @gnufied
-	// beta: v1.11
-	// Ability to Expand persistent volumes
-	ExpandPersistentVolumes featuregate.Feature = "ExpandPersistentVolumes"
-
-	// owner: @mlmhl
-	// beta: v1.15
-	// Ability to expand persistent volumes' file system without unmounting volumes.
-	ExpandInUsePersistentVolumes featuregate.Feature = "ExpandInUsePersistentVolumes"
-
-	// owner: @gnufied
-	// alpha: v1.14
-	// beta: v1.16
-	// Ability to expand CSI volumes
-	ExpandCSIVolumes featuregate.Feature = "ExpandCSIVolumes"
-
-	// owner: @verb
-	// alpha: v1.16
-	//
-	// Allows running an ephemeral container in pod namespaces to troubleshoot a running pod.
-	EphemeralContainers featuregate.Feature = "EphemeralContainers"
-
-	// owner: @sjenning
-	// alpha: v1.11
-	//
-	// Allows resource reservations at the QoS level preventing pods at lower QoS levels from
-	// bursting into resources requested at higher QoS levels (memory only for now)
-	QOSReserved featuregate.Feature = "QOSReserved"
-
-	// owner: @ConnorDoyle
-	// alpha: v1.8
-	// beta: v1.10
-	//
-	// Alternative container-level CPU affinity policies.
-	CPUManager featuregate.Feature = "CPUManager"
-
-	// owner: @szuecs
-	// alpha: v1.12
-	//
-	// Enable nodes to change CPUCFSQuotaPeriod
-	CPUCFSQuotaPeriod featuregate.Feature = "CustomCPUCFSQuotaPeriod"
-
-	// owner: @lmdaly
-	// alpha: v1.16
-	// beta: v1.18
-	//
-	// Enable resource managers to make NUMA aligned decisions
-	TopologyManager featuregate.Feature = "TopologyManager"
-
-	// owner: @sjenning
-	// beta: v1.11
-	//
-	// Enable pods to set sysctls on a pod
-	Sysctls featuregate.Feature = "Sysctls"
-
-	// owner @smarterclayton
-	// alpha: v1.16
-	// beta:  v1.19
-	// ga:  v1.21
-	//
-	// Enable legacy behavior to vary cluster functionality on the node-role.kubernetes.io labels. On by default (legacy), will be turned off in 1.18.
-	// Lock to false in v1.21 and remove in v1.22.
-	LegacyNodeRoleBehavior featuregate.Feature = "LegacyNodeRoleBehavior"
-
-	// owner @brendandburns
-	// alpha: v1.9
-	// beta:  v1.19
-	// ga:  v1.21
-	//
-	// Enable nodes to exclude themselves from service load balancers
-	ServiceNodeExclusion featuregate.Feature = "ServiceNodeExclusion"
-
-	// owner @smarterclayton
-	// alpha: v1.16
-	// beta:  v1.19
-	// ga:  v1.21
-	//
-	// Enable nodes to exclude themselves from network disruption checks
-	NodeDisruptionExclusion featuregate.Feature = "NodeDisruptionExclusion"
-
-	// owner: @saad-ali
-	// alpha: v1.12
-	// beta:  v1.14
-	// GA:    v1.18
-	// Enable all logic related to the CSIDriver API object in storage.k8s.io
-	CSIDriverRegistry featuregate.Feature = "CSIDriverRegistry"
-
-	// owner: @screeley44
-	// alpha: v1.9
-	// beta:  v1.13
-	// ga: 	  v1.18
-	//
-	// Enable Block volume support in containers.
-	BlockVolume featuregate.Feature = "BlockVolume"
-
-	// owner: @pospispa
-	// GA: v1.11
-	//
-	// Postpone deletion of a PV or a PVC when they are being used
-	StorageObjectInUseProtection featuregate.Feature = "StorageObjectInUseProtection"
-
-	// owner: @dims, @derekwaynecarr
-	// alpha: v1.10
-	// beta: v1.14
-	// GA: v1.20
-	//
-	// Implement support for limiting pids in pods
-	SupportPodPidsLimit featuregate.Feature = "SupportPodPidsLimit"
-
-	// owner: @mikedanese
-	// alpha: v1.13
-	//
-	// Migrate ServiceAccount volumes to use a projected volume consisting of a
-	// ServiceAccountTokenVolumeProjection. This feature adds new required flags
-	// to the API server.
-	BoundServiceAccountTokenVolume featuregate.Feature = "BoundServiceAccountTokenVolume"
-
-	// owner: @mtaufen
-	// alpha: v1.18
-	// beta: v1.20
-	//
-	// Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service
-	// account issuer in the API server.
-	// Note these endpoints serve minimally-compliant discovery docs that are
-	// intended to be used for service account token verification.
-	ServiceAccountIssuerDiscovery featuregate.Feature = "ServiceAccountIssuerDiscovery"
-
-	// owner: @Random-Liu
-	// beta: v1.11
-	//
-	// Enable container log rotation for cri container runtime
-	CRIContainerLogRotation featuregate.Feature = "CRIContainerLogRotation"
-
-	// owner: @krmayankk
-	// beta: v1.14
-	//
-	// Enables control over the primary group ID of containers' init processes.
-	RunAsGroup featuregate.Feature = "RunAsGroup"
-
-	// owner: @saad-ali
-	// ga
-	//
-	// Allow mounting a subpath of a volume in a container
-	// Do not remove this feature gate even though it's GA
-	VolumeSubpath featuregate.Feature = "VolumeSubpath"
-
-	// owner: @ravig
-	// alpha: v1.11
-	//
-	// Include volume count on node to be considered for balanced resource allocation while scheduling.
-	// A node which has closer cpu,memory utilization and volume count is favoured by scheduler
-	// while making decisions.
-	BalanceAttachedNodeVolumes featuregate.Feature = "BalanceAttachedNodeVolumes"
-
-	// owner: @vladimirvivien
-	// alpha: v1.11
-	// beta:  v1.14
-	// ga: 	  v1.18
-	//
-	// Enables CSI to use raw block storage volumes
-	CSIBlockVolume featuregate.Feature = "CSIBlockVolume"
-
-	// owner: @pohly
-	// alpha: v1.14
-	// beta: v1.16
-	//
-	// Enables CSI Inline volumes support for pods
-	CSIInlineVolume featuregate.Feature = "CSIInlineVolume"
-
-	// owner: @pohly
-	// alpha: v1.19
-	//
-	// Enables tracking of available storage capacity that CSI drivers provide.
-	CSIStorageCapacity featuregate.Feature = "CSIStorageCapacity"
-
-	// owner: @alculquicondor
-	// beta: v1.20
-	//
-	// Enables the use of PodTopologySpread scheduling plugin to do default
-	// spreading and disables legacy SelectorSpread plugin.
-	DefaultPodTopologySpread featuregate.Feature = "DefaultPodTopologySpread"
-
-	// owner: @pohly
-	// alpha: v1.19
-	//
-	// Enables generic ephemeral inline volume support for pods
-	GenericEphemeralVolume featuregate.Feature = "GenericEphemeralVolume"
-
-	// owner: @chendave
-	// alpha: v1.21
-	//
-	// PreferNominatedNode tells scheduler whether the nominated node will be checked first before looping
-	// all the rest of nodes in the cluster.
-	// Enabling this feature also implies the preemptor pod might not be dispatched to the best candidate in
-	// some corner case, e.g. another node releases enough resources after the nominated node has been set
-	// and hence is the best candidate instead.
-	PreferNominatedNode featuregate.Feature = "PreferNominatedNode"
-
-	// owner: @tallclair
-	// alpha: v1.12
-	// beta:  v1.14
-	// GA: v1.20
-	//
-	// Enables RuntimeClass, for selecting between multiple runtimes to run a pod.
-	RuntimeClass featuregate.Feature = "RuntimeClass"
-
-	// owner: @mtaufen
-	// alpha: v1.12
-	// beta:  v1.14
-	// GA: v1.17
-	//
-	// Kubelet uses the new Lease API to report node heartbeats,
-	// (Kube) Node Lifecycle Controller uses these heartbeats as a node health signal.
-	NodeLease featuregate.Feature = "NodeLease"
-
-	// owner: @janosi
-	// alpha: v1.12
-	// beta:  v1.18
-	// GA:    v1.20
-	//
-	// Enables SCTP as new protocol for Service ports, NetworkPolicy, and ContainerPort in Pod/Containers definition
-	SCTPSupport featuregate.Feature = "SCTPSupport"
-
-	// owner: @xing-yang
-	// alpha: v1.12
-	// beta: v1.17
-	// GA: v1.20
-	//
-	// Enable volume snapshot data source support.
-	VolumeSnapshotDataSource featuregate.Feature = "VolumeSnapshotDataSource"
-
-	// owner: @jessfraz
-	// alpha: v1.12
-	//
-	// Enables control over ProcMountType for containers.
-	ProcMountType featuregate.Feature = "ProcMountType"
-
-	// owner: @janetkuo
-	// alpha: v1.12
-	//
-	// Allow TTL controller to clean up Pods and Jobs after they finish.
-	TTLAfterFinished featuregate.Feature = "TTLAfterFinished"
-
-	// owner: @dashpole
-	// alpha: v1.13
-	// beta: v1.15
-	//
-	// Enables the kubelet's pod resources grpc endpoint
-	KubeletPodResources featuregate.Feature = "KubeletPodResources"
-
-	// owner: @davidz627
-	// alpha: v1.14
-	// beta: v1.17
-	//
-	// Enables the in-tree storage to CSI Plugin migration feature.
-	CSIMigration featuregate.Feature = "CSIMigration"
-
-	// owner: @davidz627
-	// alpha: v1.14
-	// beta: v1.17
-	//
-	// Enables the GCE PD in-tree driver to GCE CSI Driver migration feature.
-	CSIMigrationGCE featuregate.Feature = "CSIMigrationGCE"
-
-	// owner: @davidz627
-	// alpha: v1.17
-	//
-	// Disables the GCE PD in-tree driver.
-	// Expects GCE PD CSI Driver to be installed and configured on all nodes.
-	CSIMigrationGCEComplete featuregate.Feature = "CSIMigrationGCEComplete"
-
-	// owner: @leakingtapan
-	// alpha: v1.14
-	// beta: v1.17
-	//
-	// Enables the AWS EBS in-tree driver to AWS EBS CSI Driver migration feature.
-	CSIMigrationAWS featuregate.Feature = "CSIMigrationAWS"
-
-	// owner: @leakingtapan
-	// alpha: v1.17
-	//
-	// Disables the AWS EBS in-tree driver.
-	// Expects AWS EBS CSI Driver to be installed and configured on all nodes.
-	CSIMigrationAWSComplete featuregate.Feature = "CSIMigrationAWSComplete"
-
-	// owner: @andyzhangx
-	// alpha: v1.15
-	// beta: v1.19
-	//
-	// Enables the Azure Disk in-tree driver to Azure Disk Driver migration feature.
-	CSIMigrationAzureDisk featuregate.Feature = "CSIMigrationAzureDisk"
-
-	// owner: @andyzhangx
-	// alpha: v1.17
-	//
-	// Disables the Azure Disk in-tree driver.
-	// Expects Azure Disk CSI Driver to be installed and configured on all nodes.
-	CSIMigrationAzureDiskComplete featuregate.Feature = "CSIMigrationAzureDiskComplete"
-
-	// owner: @andyzhangx
-	// alpha: v1.15
-	//
-	// Enables the Azure File in-tree driver to Azure File Driver migration feature.
-	CSIMigrationAzureFile featuregate.Feature = "CSIMigrationAzureFile"
-
-	// owner: @andyzhangx
-	// alpha: v1.17
-	//
-	// Disables the Azure File in-tree driver.
-	// Expects Azure File CSI Driver to be installed and configured on all nodes.
-	CSIMigrationAzureFileComplete featuregate.Feature = "CSIMigrationAzureFileComplete"
-
-	// owner: @divyenpatel
-	// beta: v1.19 (requires: vSphere vCenter/ESXi Version: 7.0u1, HW Version: VM version 15)
-	//
-	// Enables the vSphere in-tree driver to vSphere CSI Driver migration feature.
-	CSIMigrationvSphere featuregate.Feature = "CSIMigrationvSphere"
-
-	// owner: @divyenpatel
-	// beta: v1.19 (requires: vSphere vCenter/ESXi Version: 7.0u1, HW Version: VM version 15)
-	//
-	// Disables the vSphere in-tree driver.
-	// Expects vSphere CSI Driver to be installed and configured on all nodes.
-	CSIMigrationvSphereComplete featuregate.Feature = "CSIMigrationvSphereComplete"
-
-	// owner: @huffmanca
-	// alpha: v1.19
-	// beta: v1.20
-	//
-	// Determines if a CSI Driver supports applying fsGroup.
-	CSIVolumeFSGroupPolicy featuregate.Feature = "CSIVolumeFSGroupPolicy"
-
-	// owner: @gnufied
-	// alpha: v1.18
-	// beta: v1.20
-	// Allows user to configure volume permission change policy for fsGroups when mounting
-	// a volume in a Pod.
-	ConfigurableFSGroupPolicy featuregate.Feature = "ConfigurableFSGroupPolicy"
-
-	// owner: @RobertKrawitz, @derekwaynecarr
-	// beta: v1.15
-	// GA: v1.20
-	//
-	// Implement support for limiting pids in nodes
-	SupportNodePidsLimit featuregate.Feature = "SupportNodePidsLimit"
-
-	// owner: @wk8
-	// alpha: v1.14
-	// beta: v1.16
-	//
-	// Enables GMSA support for Windows workloads.
-	WindowsGMSA featuregate.Feature = "WindowsGMSA"
-
-	// owner: @bclau
-	// alpha: v1.16
-	// beta: v1.17
-	// GA: v1.18
-	//
-	// Enables support for running container entrypoints as different usernames than their default ones.
-	WindowsRunAsUserName featuregate.Feature = "WindowsRunAsUserName"
-
-	// owner: @adisky
-	// alpha: v1.14
-	// beta: v1.18
-	//
-	// Enables the OpenStack Cinder in-tree driver to OpenStack Cinder CSI Driver migration feature.
-	CSIMigrationOpenStack featuregate.Feature = "CSIMigrationOpenStack"
-
-	// owner: @adisky
-	// alpha: v1.17
-	//
-	// Disables the OpenStack Cinder in-tree driver.
-	// Expects the OpenStack Cinder CSI Driver to be installed and configured on all nodes.
-	CSIMigrationOpenStackComplete featuregate.Feature = "CSIMigrationOpenStackComplete"
-
-	// owner: @RobertKrawitz
-	// alpha: v1.15
-	//
-	// Allow use of filesystems for ephemeral storage monitoring.
-	// Only applies if LocalStorageCapacityIsolation is set.
-	LocalStorageCapacityIsolationFSQuotaMonitoring featuregate.Feature = "LocalStorageCapacityIsolationFSQuotaMonitoring"
-
-	// owner: @denkensk
-	// alpha: v1.15
-	// beta: v1.19
-	//
-	// Enables NonPreempting option for priorityClass and pod.
-	NonPreemptingPriority featuregate.Feature = "NonPreemptingPriority"
-
-	// owner: @egernst
-	// alpha: v1.16
-	// beta: v1.18
-	//
-	// Enables PodOverhead, for accounting pod overheads which are specific to a given RuntimeClass
-	PodOverhead featuregate.Feature = "PodOverhead"
-
-	// owner: @khenidak
-	// alpha: v1.15
-	//
-	// Enables ipv6 dual stack
-	IPv6DualStack featuregate.Feature = "IPv6DualStack"
-
-	// owner: @robscott @freehan
-	// alpha: v1.16
-	//
-	// Enable Endpoint Slices for more scalable Service endpoints.
-	EndpointSlice featuregate.Feature = "EndpointSlice"
-
-	// owner: @robscott @freehan
-	// alpha: v1.18
-	// beta: v1.19
-	//
-	// Enable Endpoint Slice consumption by kube-proxy for improved scalability.
-	EndpointSliceProxying featuregate.Feature = "EndpointSliceProxying"
-
-	// owner: @robscott @kumarvin123
-	// alpha: v1.19
-	//
-	// Enable Endpoint Slice consumption by kube-proxy in Windows for improved scalability.
-	WindowsEndpointSliceProxying featuregate.Feature = "WindowsEndpointSliceProxying"
-
-	// owner: @matthyx
-	// alpha: v1.16
-	// beta: v1.18
-	// GA: v1.20
-	//
-	// Enables the startupProbe in kubelet worker.
-	StartupProbe featuregate.Feature = "StartupProbe"
-
-	// owner: @deads2k
-	// beta: v1.17
-	//
-	// Enables the users to skip TLS verification of kubelets on pod logs requests
-	AllowInsecureBackendProxy featuregate.Feature = "AllowInsecureBackendProxy"
-
-	// owner: @mortent
-	// alpha: v1.3
-	// beta:  v1.5
-	//
-	// Enable all logic related to the PodDisruptionBudget API object in policy
-	PodDisruptionBudget featuregate.Feature = "PodDisruptionBudget"
-
-	// owner: @alaypatel07, @soltysh
-	// alpha: v1.20
-	// beta: v1.21
-	//
-	// CronJobControllerV2 controls whether the controller manager starts old cronjob
-	// controller or new one which is implemented with informers and delaying queue
-	//
-	// This feature is deprecated, and will be removed in v1.22.
-	CronJobControllerV2 featuregate.Feature = "CronJobControllerV2"
-
-	// owner: @smarterclayton
-	// alpha: v1.21
-	//
-	// DaemonSets allow workloads to maintain availability during update per node
-	DaemonSetUpdateSurge featuregate.Feature = "DaemonSetUpdateSurge"
-
-	// owner: @m1093782566
-	// alpha: v1.17
-	//
-	// Enables topology aware service routing
-	ServiceTopology featuregate.Feature = "ServiceTopology"
-
-	// owner: @robscott
-	// alpha: v1.18
-	// beta:  v1.19
-	// ga:    v1.20
-	//
-	// Enables AppProtocol field for Services and Endpoints.
-	ServiceAppProtocol featuregate.Feature = "ServiceAppProtocol"
-
-	// owner: @wojtek-t
-	// alpha: v1.18
-	// beta:  v1.19
-	// ga:    v1.21
-	//
-	// Enables a feature to make secrets and configmaps data immutable.
-	ImmutableEphemeralVolumes featuregate.Feature = "ImmutableEphemeralVolumes"
-
-	// owner: @bart0sh
-	// alpha: v1.18
-	// beta: v1.19
-	//
-	// Enables usage of HugePages-<size> in a volume medium,
-	// e.g. emptyDir:
-	//        medium: HugePages-1Gi
-	HugePageStorageMediumSize featuregate.Feature = "HugePageStorageMediumSize"
-
-	// owner: @derekwaynecarr
-	// alpha: v1.20
-	//
-	// Enables usage of hugepages-<size> in downward API.
-	DownwardAPIHugePages featuregate.Feature = "DownwardAPIHugePages"
-
-	// owner: @freehan
-	// GA: v1.18
-	//
-	// Enable ExternalTrafficPolicy for Service ExternalIPs.
-	// This is for bug fix #69811
-	ExternalPolicyForExternalIP featuregate.Feature = "ExternalPolicyForExternalIP"
-
-	// owner: @bswartz
-	// alpha: v1.18
-	//
-	// Enables usage of any object for volume data source in PVCs
-	AnyVolumeDataSource featuregate.Feature = "AnyVolumeDataSource"
-
-	// owner: @javidiaz
-	// alpha: v1.19
-	// beta: v1.20
-	//
-	// Allow setting the Fully Qualified Domain Name (FQDN) in the hostname of a Pod. If a Pod does not
-	// have FQDN, this feature has no effect.
-	SetHostnameAsFQDN featuregate.Feature = "SetHostnameAsFQDN"
-
-	// owner: @ksubrmnn
-	// alpha: v1.14
-	// beta: v1.20
-	//
-	// Allows kube-proxy to run in Overlay mode for Windows
-	WinOverlay featuregate.Feature = "WinOverlay"
-
-	// owner: @ksubrmnn
-	// alpha: v1.14
-	//
-	// Allows kube-proxy to create DSR loadbalancers for Windows
-	WinDSR featuregate.Feature = "WinDSR"
-
-	// owner: @RenaudWasTaken @dashpole
-	// alpha: v1.19
-	// beta: v1.20
-	//
-	// Disables Accelerator Metrics Collected by Kubelet
-	DisableAcceleratorUsageMetrics featuregate.Feature = "DisableAcceleratorUsageMetrics"
-
-	// owner: @arjunrn @mwielgus @josephburnett
-	// alpha: v1.20
-	//
-	// Add support for the HPA to scale based on metrics from individual containers
-	// in target pods
-	HPAContainerMetrics featuregate.Feature = "HPAContainerMetrics"
-
-	// owner: @zshihang
-	// alpha: v1.13
-	// beta: v1.20
-	//
-	// Allows kube-controller-manager to publish kube-root-ca.crt configmap to
-	// every namespace. This feature is a prerequisite of BoundServiceAccountTokenVolume.
-	RootCAConfigMap featuregate.Feature = "RootCAConfigMap"
-
-	// owner: @andrewsykim
-	// alpha: v1.20
-	//
-	// Enable Terminating condition in Endpoint Slices.
-	EndpointSliceTerminatingCondition featuregate.Feature = "EndpointSliceTerminatingCondition"
-
-	// owner: @robscott
-	// alpha: v1.20
-	//
-	// Enable NodeName field on Endpoint Slices.
-	EndpointSliceNodeName featuregate.Feature = "EndpointSliceNodeName"
-
-	// owner: @derekwaynecarr
-	// alpha: v1.20
-	//
-	// Enables kubelet support to size memory backed volumes
-	SizeMemoryBackedVolumes featuregate.Feature = "SizeMemoryBackedVolumes"
-
-	// owner: @andrewsykim @SergeyKanzhelev
-	// GA: v1.20
-	//
-	// Ensure kubelet respects exec probe timeouts. Feature gate exists in-case existing workloads
-	// may depend on old behavior where exec probe timeouts were ignored.
-	// Lock to default in v1.21 and remove in v1.22.
-	ExecProbeTimeout featuregate.Feature = "ExecProbeTimeout"
-
-	// owner: @andrewsykim
-	// alpha: v1.20
-	//
-	// Enable kubelet exec plugins for image pull credentials.
-	KubeletCredentialProviders featuregate.Feature = "KubeletCredentialProviders"
-
-	// owner: @zshihang
-	// alpha: v1.20
-	//
-	// Enable kubelet to pass pod's service account token to NodePublishVolume
-	// call of CSI driver which is mounting volumes for that pod.
-	CSIServiceAccountToken featuregate.Feature = "CSIServiceAccountToken"
-
-	// owner: @bobbypage
-	// alpha: v1.20
-	// Adds support for kubelet to detect node shutdown and gracefully terminate pods prior to the node being shutdown.
-	GracefulNodeShutdown featuregate.Feature = "GracefulNodeShutdown"
-
-	// owner: @andrewsykim @uablrek
-	// alpha: v1.20
-	//
-	// Allows control if NodePorts shall be created for services with "type: LoadBalancer" by defining the spec.AllocateLoadBalancerNodePorts field (bool)
-	ServiceLBNodePortControl featuregate.Feature = "ServiceLBNodePortControl"
-
-	// owner: @janosi
-	// alpha: v1.20
-	//
-	// Enables the usage of different protocols in the same Service with type=LoadBalancer
-	MixedProtocolLBService featuregate.Feature = "MixedProtocolLBService"
-)
--- a/kube/pkg/quota/v1/evaluator/core/persistent_volume_claims.go
+++ b/kube/pkg/quota/v1/evaluator/core/persistent_volume_claims.go
@@ -1,190 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package core
-
-import (
-	"fmt"
-	"strings"
-
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-	"k8s.io/apiserver/pkg/admission"
-	utilfeature "k8s.io/apiserver/pkg/util/feature"
-
-	"kubesphere.io/kubesphere/kube/pkg/apis/core/v1/helper"
-	k8sfeatures "kubesphere.io/kubesphere/kube/pkg/features"
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-)
-
-// the name used for object count quota
-var pvcObjectCountName = generic.ObjectCountQuotaResourceNameFor(corev1.SchemeGroupVersion.WithResource("persistentvolumeclaims").GroupResource())
-
-// pvcResources are the set of static resources managed by quota associated with pvcs.
-// for each resource in this list, it may be refined dynamically based on storage class.
-var pvcResources = []corev1.ResourceName{
-	corev1.ResourcePersistentVolumeClaims,
-	corev1.ResourceRequestsStorage,
-}
-
-// storageClassSuffix is the suffix to the qualified portion of storage class resource name.
-// For example, if you want to quota storage by storage class, you would have a declaration
-// that follows <storage-class>.storageclass.storage.k8s.io/<resource>.
-// For example:
-// * gold.storageclass.storage.k8s.io/: 500Gi
-// * bronze.storageclass.storage.k8s.io/requests.storage: 500Gi
-const storageClassSuffix string = ".storageclass.storage.k8s.io/"
-
-/* TODO: prune?
-// ResourceByStorageClass returns a quota resource name by storage class.
-func ResourceByStorageClass(storageClass string, resourceName corev1.ResourceName) corev1.ResourceName {
-	return corev1.ResourceName(string(storageClass + storageClassSuffix + string(resourceName)))
-}
-*/
-
-// V1ResourceByStorageClass returns a quota resource name by storage class.
-func V1ResourceByStorageClass(storageClass string, resourceName corev1.ResourceName) corev1.ResourceName {
-	return corev1.ResourceName(string(storageClass + storageClassSuffix + string(resourceName)))
-}
-
-// NewPersistentVolumeClaimEvaluator returns an evaluator that can evaluate persistent volume claims
-func NewPersistentVolumeClaimEvaluator(f quota.ListerForResourceFunc) quota.Evaluator {
-	listFuncByNamespace := generic.ListResourceUsingListerFunc(f, corev1.SchemeGroupVersion.WithResource("persistentvolumeclaims"))
-	pvcEvaluator := &pvcEvaluator{listFuncByNamespace: listFuncByNamespace}
-	return pvcEvaluator
-}
-
-// pvcEvaluator knows how to evaluate quota usage for persistent volume claims
-type pvcEvaluator struct {
-	// listFuncByNamespace knows how to list pvc claims
-	listFuncByNamespace generic.ListFuncByNamespace
-}
-
-// Constraints verifies that all required resources are present on the item.
-func (p *pvcEvaluator) Constraints(required []corev1.ResourceName, item runtime.Object) error {
-	// no-op for persistent volume claims
-	return nil
-}
-
-// GroupResource that this evaluator tracks
-func (p *pvcEvaluator) GroupResource() schema.GroupResource {
-	return corev1.SchemeGroupVersion.WithResource("persistentvolumeclaims").GroupResource()
-}
-
-// Handles returns true if the evaluator should handle the specified operation.
-func (p *pvcEvaluator) Handles(a admission.Attributes) bool {
-	op := a.GetOperation()
-	if op == admission.Create {
-		return true
-	}
-	if op == admission.Update && utilfeature.DefaultFeatureGate.Enabled(k8sfeatures.ExpandPersistentVolumes) {
-		return true
-	}
-	return false
-}
-
-// Matches returns true if the evaluator matches the specified quota with the provided input item
-func (p *pvcEvaluator) Matches(resourceQuota *corev1.ResourceQuota, item runtime.Object) (bool, error) {
-	return generic.Matches(resourceQuota, item, p.MatchingResources, generic.MatchesNoScopeFunc)
-}
-
-// MatchingScopes takes the input specified list of scopes and input object. Returns the set of scopes resource matches.
-func (p *pvcEvaluator) MatchingScopes(item runtime.Object, scopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	return []corev1.ScopedResourceSelectorRequirement{}, nil
-}
-
-// UncoveredQuotaScopes takes the input matched scopes which are limited by configuration and the matched quota scopes.
-// It returns the scopes which are in limited scopes but dont have a corresponding covering quota scope
-func (p *pvcEvaluator) UncoveredQuotaScopes(limitedScopes []corev1.ScopedResourceSelectorRequirement, matchedQuotaScopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	return []corev1.ScopedResourceSelectorRequirement{}, nil
-}
-
-// MatchingResources takes the input specified list of resources and returns the set of resources it matches.
-func (p *pvcEvaluator) MatchingResources(items []corev1.ResourceName) []corev1.ResourceName {
-	result := []corev1.ResourceName{}
-	for _, item := range items {
-		// match object count quota fields
-		if quota.Contains([]corev1.ResourceName{pvcObjectCountName}, item) {
-			result = append(result, item)
-			continue
-		}
-		// match pvc resources
-		if quota.Contains(pvcResources, item) {
-			result = append(result, item)
-			continue
-		}
-		// match pvc resources scoped by storage class (<storage-class-name>.storage-class.kubernetes.io/<resource>)
-		for _, resource := range pvcResources {
-			byStorageClass := storageClassSuffix + string(resource)
-			if strings.HasSuffix(string(item), byStorageClass) {
-				result = append(result, item)
-				break
-			}
-		}
-	}
-	return result
-}
-
-// Usage knows how to measure usage associated with item.
-func (p *pvcEvaluator) Usage(item runtime.Object) (corev1.ResourceList, error) {
-	result := corev1.ResourceList{}
-	pvc, err := toExternalPersistentVolumeClaimOrError(item)
-	if err != nil {
-		return result, err
-	}
-
-	// charge for claim
-	result[corev1.ResourcePersistentVolumeClaims] = *(resource.NewQuantity(1, resource.DecimalSI))
-	result[pvcObjectCountName] = *(resource.NewQuantity(1, resource.DecimalSI))
-	storageClassRef := helper.GetPersistentVolumeClaimClass(pvc)
-	if len(storageClassRef) > 0 {
-		storageClassClaim := corev1.ResourceName(storageClassRef + storageClassSuffix + string(corev1.ResourcePersistentVolumeClaims))
-		result[storageClassClaim] = *(resource.NewQuantity(1, resource.DecimalSI))
-	}
-
-	// charge for storage
-	if request, found := pvc.Spec.Resources.Requests[corev1.ResourceStorage]; found {
-		result[corev1.ResourceRequestsStorage] = request
-		// charge usage to the storage class (if present)
-		if len(storageClassRef) > 0 {
-			storageClassStorage := corev1.ResourceName(storageClassRef + storageClassSuffix + string(corev1.ResourceRequestsStorage))
-			result[storageClassStorage] = request
-		}
-	}
-	return result, nil
-}
-
-// UsageStats calculates aggregate usage for the object.
-func (p *pvcEvaluator) UsageStats(options quota.UsageStatsOptions) (quota.UsageStats, error) {
-	return generic.CalculateUsageStats(options, p.listFuncByNamespace, generic.MatchesNoScopeFunc, p.Usage)
-}
-
-// ensure we implement required interface
-var _ quota.Evaluator = &pvcEvaluator{}
-
-func toExternalPersistentVolumeClaimOrError(obj runtime.Object) (*corev1.PersistentVolumeClaim, error) {
-	pvc := &corev1.PersistentVolumeClaim{}
-	switch t := obj.(type) {
-	case *corev1.PersistentVolumeClaim:
-		pvc = t
-	default:
-		return nil, fmt.Errorf("expect *v1.PersistentVolumeClaim, got %v", t)
-	}
-	return pvc, nil
-}
--- a/kube/pkg/quota/v1/evaluator/core/pods.go
+++ b/kube/pkg/quota/v1/evaluator/core/pods.go
@@ -1,398 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package core
-
-import (
-	"fmt"
-	"strings"
-	"time"
-
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-
-	"k8s.io/apimachinery/pkg/util/clock"
-	"k8s.io/apimachinery/pkg/util/sets"
-	"k8s.io/apiserver/pkg/admission"
-
-	"kubesphere.io/kubesphere/kube/pkg/apis/core/v1/helper"
-	"kubesphere.io/kubesphere/kube/pkg/apis/core/v1/helper/qos"
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-)
-
-// the name used for object count quota
-var podObjectCountName = generic.ObjectCountQuotaResourceNameFor(corev1.SchemeGroupVersion.WithResource("pods").GroupResource())
-
-// podResources are the set of resources managed by quota associated with pods.
-var podResources = []corev1.ResourceName{
-	podObjectCountName,
-	corev1.ResourceCPU,
-	corev1.ResourceMemory,
-	corev1.ResourceEphemeralStorage,
-	corev1.ResourceRequestsCPU,
-	corev1.ResourceRequestsMemory,
-	corev1.ResourceRequestsEphemeralStorage,
-	corev1.ResourceLimitsCPU,
-	corev1.ResourceLimitsMemory,
-	corev1.ResourceLimitsEphemeralStorage,
-	corev1.ResourcePods,
-}
-
-// podResourcePrefixes are the set of prefixes for resources (Hugepages, and other
-// potential extended reources with specific prefix) managed by quota associated with pods.
-var podResourcePrefixes = []string{
-	corev1.ResourceHugePagesPrefix,
-	corev1.ResourceRequestsHugePagesPrefix,
-}
-
-// requestedResourcePrefixes are the set of prefixes for resources
-// that might be declared in pod's Resources.Requests/Limits
-var requestedResourcePrefixes = []string{
-	corev1.ResourceHugePagesPrefix,
-}
-
-// maskResourceWithPrefix mask resource with certain prefix
-// e.g. hugepages-XXX -> requests.hugepages-XXX
-func maskResourceWithPrefix(resource corev1.ResourceName, prefix string) corev1.ResourceName {
-	return corev1.ResourceName(fmt.Sprintf("%s%s", prefix, string(resource)))
-}
-
-// isExtendedResourceNameForQuota returns true if the extended resource name
-// has the quota related resource prefix.
-func isExtendedResourceNameForQuota(name corev1.ResourceName) bool {
-	// As overcommit is not supported by extended resources for now,
-	// only quota objects in format of "requests.resourceName" is allowed.
-	return !helper.IsNativeResource(name) && strings.HasPrefix(string(name), corev1.DefaultResourceRequestsPrefix)
-}
-
-// NOTE: it was a mistake, but if a quota tracks cpu or memory related resources,
-// the incoming pod is required to have those values set.  we should not repeat
-// this mistake for other future resources (gpus, ephemeral-storage,etc).
-// do not add more resources to this list!
-var validationSet = sets.NewString(
-	string(corev1.ResourceCPU),
-	string(corev1.ResourceMemory),
-	string(corev1.ResourceRequestsCPU),
-	string(corev1.ResourceRequestsMemory),
-	string(corev1.ResourceLimitsCPU),
-	string(corev1.ResourceLimitsMemory),
-)
-
-// NewPodEvaluator returns an evaluator that can evaluate pods
-func NewPodEvaluator(f quota.ListerForResourceFunc, clock clock.Clock) quota.Evaluator {
-	listFuncByNamespace := generic.ListResourceUsingListerFunc(f, corev1.SchemeGroupVersion.WithResource("pods"))
-	podEvaluator := &podEvaluator{listFuncByNamespace: listFuncByNamespace, clock: clock}
-	return podEvaluator
-}
-
-// podEvaluator knows how to measure usage of pods.
-type podEvaluator struct {
-	// knows how to list pods
-	listFuncByNamespace generic.ListFuncByNamespace
-	// used to track time
-	clock clock.Clock
-}
-
-// Constraints verifies that all required resources are present on the pod
-// In addition, it validates that the resources are valid (i.e. requests < limits)
-func (p *podEvaluator) Constraints(required []corev1.ResourceName, item runtime.Object) error {
-	pod, err := toExternalPodOrError(item)
-	if err != nil {
-		return err
-	}
-
-	// BACKWARD COMPATIBILITY REQUIREMENT: if we quota cpu or memory, then each container
-	// must make an explicit request for the resource.  this was a mistake.  it coupled
-	// validation with resource counting, but we did this before QoS was even defined.
-	// let's not make that mistake again with other resources now that QoS is defined.
-	requiredSet := quota.ToSet(required).Intersection(validationSet)
-	missingSet := sets.NewString()
-	for i := range pod.Spec.Containers {
-		enforcePodContainerConstraints(&pod.Spec.Containers[i], requiredSet, missingSet)
-	}
-	for i := range pod.Spec.InitContainers {
-		enforcePodContainerConstraints(&pod.Spec.InitContainers[i], requiredSet, missingSet)
-	}
-	if len(missingSet) == 0 {
-		return nil
-	}
-	return fmt.Errorf("must specify %s", strings.Join(missingSet.List(), ","))
-}
-
-// GroupResource that this evaluator tracks
-func (p *podEvaluator) GroupResource() schema.GroupResource {
-	return corev1.SchemeGroupVersion.WithResource("pods").GroupResource()
-}
-
-// Handles returns true if the evaluator should handle the specified attributes.
-func (p *podEvaluator) Handles(a admission.Attributes) bool {
-	op := a.GetOperation()
-	if op == admission.Create {
-		return true
-	}
-	return false
-}
-
-// Matches returns true if the evaluator matches the specified quota with the provided input item
-func (p *podEvaluator) Matches(resourceQuota *corev1.ResourceQuota, item runtime.Object) (bool, error) {
-	return generic.Matches(resourceQuota, item, p.MatchingResources, podMatchesScopeFunc)
-}
-
-// MatchingResources takes the input specified list of resources and returns the set of resources it matches.
-func (p *podEvaluator) MatchingResources(input []corev1.ResourceName) []corev1.ResourceName {
-	result := quota.Intersection(input, podResources)
-	for _, resource := range input {
-		// for resources with certain prefix, e.g. hugepages
-		if quota.ContainsPrefix(podResourcePrefixes, resource) {
-			result = append(result, resource)
-		}
-		// for extended resources
-		if isExtendedResourceNameForQuota(resource) {
-			result = append(result, resource)
-		}
-	}
-
-	return result
-}
-
-// MatchingScopes takes the input specified list of scopes and pod object. Returns the set of scope selectors pod matches.
-func (p *podEvaluator) MatchingScopes(item runtime.Object, scopeSelectors []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	matchedScopes := []corev1.ScopedResourceSelectorRequirement{}
-	for _, selector := range scopeSelectors {
-		match, err := podMatchesScopeFunc(selector, item)
-		if err != nil {
-			return []corev1.ScopedResourceSelectorRequirement{}, fmt.Errorf("error on matching scope %v: %v", selector, err)
-		}
-		if match {
-			matchedScopes = append(matchedScopes, selector)
-		}
-	}
-	return matchedScopes, nil
-}
-
-// UncoveredQuotaScopes takes the input matched scopes which are limited by configuration and the matched quota scopes.
-// It returns the scopes which are in limited scopes but dont have a corresponding covering quota scope
-func (p *podEvaluator) UncoveredQuotaScopes(limitedScopes []corev1.ScopedResourceSelectorRequirement, matchedQuotaScopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	uncoveredScopes := []corev1.ScopedResourceSelectorRequirement{}
-	for _, selector := range limitedScopes {
-		isCovered := false
-		for _, matchedScopeSelector := range matchedQuotaScopes {
-			if matchedScopeSelector.ScopeName == selector.ScopeName {
-				isCovered = true
-				break
-			}
-		}
-
-		if !isCovered {
-			uncoveredScopes = append(uncoveredScopes, selector)
-		}
-	}
-	return uncoveredScopes, nil
-}
-
-// Usage knows how to measure usage associated with pods
-func (p *podEvaluator) Usage(item runtime.Object) (corev1.ResourceList, error) {
-	// delegate to normal usage
-	return PodUsageFunc(item, p.clock)
-}
-
-// UsageStats calculates aggregate usage for the object.
-func (p *podEvaluator) UsageStats(options quota.UsageStatsOptions) (quota.UsageStats, error) {
-	return generic.CalculateUsageStats(options, p.listFuncByNamespace, podMatchesScopeFunc, p.Usage)
-}
-
-// verifies we implement the required interface.
-var _ quota.Evaluator = &podEvaluator{}
-
-// enforcePodContainerConstraints checks for required resources that are not set on this container and
-// adds them to missingSet.
-func enforcePodContainerConstraints(container *corev1.Container, requiredSet, missingSet sets.String) {
-	requests := container.Resources.Requests
-	limits := container.Resources.Limits
-	containerUsage := podComputeUsageHelper(requests, limits)
-	containerSet := quota.ToSet(quota.ResourceNames(containerUsage))
-	if !containerSet.Equal(requiredSet) {
-		difference := requiredSet.Difference(containerSet)
-		missingSet.Insert(difference.List()...)
-	}
-}
-
-// podComputeUsageHelper can summarize the pod compute quota usage based on requests and limits
-func podComputeUsageHelper(requests corev1.ResourceList, limits corev1.ResourceList) corev1.ResourceList {
-	result := corev1.ResourceList{}
-	result[corev1.ResourcePods] = resource.MustParse("1")
-	if request, found := requests[corev1.ResourceCPU]; found {
-		result[corev1.ResourceCPU] = request
-		result[corev1.ResourceRequestsCPU] = request
-	}
-	if limit, found := limits[corev1.ResourceCPU]; found {
-		result[corev1.ResourceLimitsCPU] = limit
-	}
-	if request, found := requests[corev1.ResourceMemory]; found {
-		result[corev1.ResourceMemory] = request
-		result[corev1.ResourceRequestsMemory] = request
-	}
-	if limit, found := limits[corev1.ResourceMemory]; found {
-		result[corev1.ResourceLimitsMemory] = limit
-	}
-	if request, found := requests[corev1.ResourceEphemeralStorage]; found {
-		result[corev1.ResourceEphemeralStorage] = request
-		result[corev1.ResourceRequestsEphemeralStorage] = request
-	}
-	if limit, found := limits[corev1.ResourceEphemeralStorage]; found {
-		result[corev1.ResourceLimitsEphemeralStorage] = limit
-	}
-	for resource, request := range requests {
-		// for resources with certain prefix, e.g. hugepages
-		if quota.ContainsPrefix(requestedResourcePrefixes, resource) {
-			result[resource] = request
-			result[maskResourceWithPrefix(resource, corev1.DefaultResourceRequestsPrefix)] = request
-		}
-		// for extended resources
-		if helper.IsExtendedResourceName(resource) {
-			// only quota objects in format of "requests.resourceName" is allowed for extended resource.
-			result[maskResourceWithPrefix(resource, corev1.DefaultResourceRequestsPrefix)] = request
-		}
-	}
-
-	return result
-}
-
-func toExternalPodOrError(obj runtime.Object) (*corev1.Pod, error) {
-	pod := &corev1.Pod{}
-	switch t := obj.(type) {
-	case *corev1.Pod:
-		pod = t
-	default:
-		return nil, fmt.Errorf("expect *v1.Pod, got %v", t)
-	}
-	return pod, nil
-}
-
-// podMatchesScopeFunc is a function that knows how to evaluate if a pod matches a scope
-func podMatchesScopeFunc(selector corev1.ScopedResourceSelectorRequirement, object runtime.Object) (bool, error) {
-	pod, err := toExternalPodOrError(object)
-	if err != nil {
-		return false, err
-	}
-	switch selector.ScopeName {
-	case corev1.ResourceQuotaScopeTerminating:
-		return isTerminating(pod), nil
-	case corev1.ResourceQuotaScopeNotTerminating:
-		return !isTerminating(pod), nil
-	case corev1.ResourceQuotaScopeBestEffort:
-		return isBestEffort(pod), nil
-	case corev1.ResourceQuotaScopeNotBestEffort:
-		return !isBestEffort(pod), nil
-	case corev1.ResourceQuotaScopePriorityClass:
-		return podMatchesSelector(pod, selector)
-	}
-	return false, nil
-}
-
-// PodUsageFunc returns the quota usage for a pod.
-// A pod is charged for quota if the following are not true.
-//  - pod has a terminal phase (failed or succeeded)
-//  - pod has been marked for deletion and grace period has expired
-func PodUsageFunc(obj runtime.Object, clock clock.Clock) (corev1.ResourceList, error) {
-	pod, err := toExternalPodOrError(obj)
-	if err != nil {
-		return corev1.ResourceList{}, err
-	}
-
-	// always quota the object count (even if the pod is end of life)
-	// object count quotas track all objects that are in storage.
-	// where "pods" tracks all pods that have not reached a terminal state,
-	// count/pods tracks all pods independent of state.
-	result := corev1.ResourceList{
-		podObjectCountName: *(resource.NewQuantity(1, resource.DecimalSI)),
-	}
-
-	// by convention, we do not quota compute resources that have reached end-of life
-	// note: the "pods" resource is considered a compute resource since it is tied to life-cycle.
-	if !QuotaV1Pod(pod, clock) {
-		return result, nil
-	}
-
-	requests := corev1.ResourceList{}
-	limits := corev1.ResourceList{}
-	// TODO: ideally, we have pod level requests and limits in the future.
-	for i := range pod.Spec.Containers {
-		requests = quota.Add(requests, pod.Spec.Containers[i].Resources.Requests)
-		limits = quota.Add(limits, pod.Spec.Containers[i].Resources.Limits)
-	}
-	// InitContainers are run sequentially before other containers start, so the highest
-	// init container resource is compared against the sum of app containers to determine
-	// the effective usage for both requests and limits.
-	for i := range pod.Spec.InitContainers {
-		requests = quota.Max(requests, pod.Spec.InitContainers[i].Resources.Requests)
-		limits = quota.Max(limits, pod.Spec.InitContainers[i].Resources.Limits)
-	}
-
-	result = quota.Add(result, podComputeUsageHelper(requests, limits))
-	return result, nil
-}
-
-func isBestEffort(pod *corev1.Pod) bool {
-	return qos.GetPodQOS(pod) == corev1.PodQOSBestEffort
-}
-
-func isTerminating(pod *corev1.Pod) bool {
-	if pod.Spec.ActiveDeadlineSeconds != nil && *pod.Spec.ActiveDeadlineSeconds >= int64(0) {
-		return true
-	}
-	return false
-}
-
-func podMatchesSelector(pod *corev1.Pod, selector corev1.ScopedResourceSelectorRequirement) (bool, error) {
-	labelSelector, err := helper.ScopedResourceSelectorRequirementsAsSelector(selector)
-	if err != nil {
-		return false, fmt.Errorf("failed to parse and convert selector: %v", err)
-	}
-	var m map[string]string
-	if len(pod.Spec.PriorityClassName) != 0 {
-		m = map[string]string{string(corev1.ResourceQuotaScopePriorityClass): pod.Spec.PriorityClassName}
-	}
-	if labelSelector.Matches(labels.Set(m)) {
-		return true, nil
-	}
-	return false, nil
-}
-
-// QuotaV1Pod returns true if the pod is eligible to track against a quota
-// if it's not in a terminal state according to its phase.
-func QuotaV1Pod(pod *corev1.Pod, clock clock.Clock) bool {
-	// if pod is terminal, ignore it for quota
-	if corev1.PodFailed == pod.Status.Phase || corev1.PodSucceeded == pod.Status.Phase {
-		return false
-	}
-	// if pods are stuck terminating (for example, a node is lost), we do not want
-	// to charge the user for that pod in quota because it could prevent them from
-	// scaling up new pods to service their application.
-	if pod.DeletionTimestamp != nil && pod.DeletionGracePeriodSeconds != nil {
-		now := clock.Now()
-		deletionTime := pod.DeletionTimestamp.Time
-		gracePeriod := time.Duration(*pod.DeletionGracePeriodSeconds) * time.Second
-		if now.After(deletionTime.Add(gracePeriod)) {
-			return false
-		}
-	}
-	return true
-}
--- a/kube/pkg/quota/v1/evaluator/core/registry.go
+++ b/kube/pkg/quota/v1/evaluator/core/registry.go
@@ -1,50 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package core
-
-import (
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-	"k8s.io/apimachinery/pkg/util/clock"
-
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-)
-
-// legacyObjectCountAliases are what we used to do simple object counting quota with mapped to alias
-var legacyObjectCountAliases = map[schema.GroupVersionResource]corev1.ResourceName{
-	corev1.SchemeGroupVersion.WithResource("configmaps"):             corev1.ResourceConfigMaps,
-	corev1.SchemeGroupVersion.WithResource("resourcequotas"):         corev1.ResourceQuotas,
-	corev1.SchemeGroupVersion.WithResource("replicationcontrollers"): corev1.ResourceReplicationControllers,
-	corev1.SchemeGroupVersion.WithResource("secrets"):                corev1.ResourceSecrets,
-}
-
-// NewEvaluators returns the list of static evaluators that manage more than counts
-func NewEvaluators(f quota.ListerForResourceFunc) []quota.Evaluator {
-	// these evaluators have special logic
-	result := []quota.Evaluator{
-		NewPodEvaluator(f, clock.RealClock{}),
-		NewServiceEvaluator(f),
-		NewPersistentVolumeClaimEvaluator(f),
-	}
-	// these evaluators require an alias for backwards compatibility
-	for gvr, alias := range legacyObjectCountAliases {
-		result = append(result,
-			generic.NewObjectCountEvaluator(gvr.GroupResource(), generic.ListResourceUsingListerFunc(f, gvr), alias))
-	}
-	return result
-}
--- a/kube/pkg/quota/v1/evaluator/core/services.go
+++ b/kube/pkg/quota/v1/evaluator/core/services.go
@@ -1,150 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package core
-
-import (
-	"fmt"
-
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-	"k8s.io/apiserver/pkg/admission"
-
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-)
-
-// the name used for object count quota
-var serviceObjectCountName = generic.ObjectCountQuotaResourceNameFor(corev1.SchemeGroupVersion.WithResource("services").GroupResource())
-
-// serviceResources are the set of resources managed by quota associated with services.
-var serviceResources = []corev1.ResourceName{
-	serviceObjectCountName,
-	corev1.ResourceServices,
-	corev1.ResourceServicesNodePorts,
-	corev1.ResourceServicesLoadBalancers,
-}
-
-// NewServiceEvaluator returns an evaluator that can evaluate services.
-func NewServiceEvaluator(f quota.ListerForResourceFunc) quota.Evaluator {
-	listFuncByNamespace := generic.ListResourceUsingListerFunc(f, corev1.SchemeGroupVersion.WithResource("services"))
-	serviceEvaluator := &serviceEvaluator{listFuncByNamespace: listFuncByNamespace}
-	return serviceEvaluator
-}
-
-// serviceEvaluator knows how to measure usage for services.
-type serviceEvaluator struct {
-	// knows how to list items by namespace
-	listFuncByNamespace generic.ListFuncByNamespace
-}
-
-// Constraints verifies that all required resources are present on the item
-func (p *serviceEvaluator) Constraints(required []corev1.ResourceName, item runtime.Object) error {
-	// this is a no-op for services
-	return nil
-}
-
-// GroupResource that this evaluator tracks
-func (p *serviceEvaluator) GroupResource() schema.GroupResource {
-	return corev1.SchemeGroupVersion.WithResource("services").GroupResource()
-}
-
-// Handles returns true of the evaluator should handle the specified operation.
-func (p *serviceEvaluator) Handles(a admission.Attributes) bool {
-	operation := a.GetOperation()
-	// We handle create and update because a service type can change.
-	return admission.Create == operation || admission.Update == operation
-}
-
-// Matches returns true if the evaluator matches the specified quota with the provided input item
-func (p *serviceEvaluator) Matches(resourceQuota *corev1.ResourceQuota, item runtime.Object) (bool, error) {
-	return generic.Matches(resourceQuota, item, p.MatchingResources, generic.MatchesNoScopeFunc)
-}
-
-// MatchingResources takes the input specified list of resources and returns the set of resources it matches.
-func (p *serviceEvaluator) MatchingResources(input []corev1.ResourceName) []corev1.ResourceName {
-	return quota.Intersection(input, serviceResources)
-}
-
-// MatchingScopes takes the input specified list of scopes and input object. Returns the set of scopes resource matches.
-func (p *serviceEvaluator) MatchingScopes(item runtime.Object, scopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	return []corev1.ScopedResourceSelectorRequirement{}, nil
-}
-
-// UncoveredQuotaScopes takes the input matched scopes which are limited by configuration and the matched quota scopes.
-// It returns the scopes which are in limited scopes but dont have a corresponding covering quota scope
-func (p *serviceEvaluator) UncoveredQuotaScopes(limitedScopes []corev1.ScopedResourceSelectorRequirement, matchedQuotaScopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	return []corev1.ScopedResourceSelectorRequirement{}, nil
-}
-
-// convert the input object to an internal service object or error.
-func toExternalServiceOrError(obj runtime.Object) (*corev1.Service, error) {
-	svc := &corev1.Service{}
-	switch t := obj.(type) {
-	case *corev1.Service:
-		svc = t
-	default:
-		return nil, fmt.Errorf("expect *v1.Service, got %v", t)
-	}
-	return svc, nil
-}
-
-// Usage knows how to measure usage associated with services
-func (p *serviceEvaluator) Usage(item runtime.Object) (corev1.ResourceList, error) {
-	result := corev1.ResourceList{}
-	svc, err := toExternalServiceOrError(item)
-	if err != nil {
-		return result, err
-	}
-	ports := len(svc.Spec.Ports)
-	// default service usage
-	result[serviceObjectCountName] = *(resource.NewQuantity(1, resource.DecimalSI))
-	result[corev1.ResourceServices] = *(resource.NewQuantity(1, resource.DecimalSI))
-	result[corev1.ResourceServicesLoadBalancers] = resource.Quantity{Format: resource.DecimalSI}
-	result[corev1.ResourceServicesNodePorts] = resource.Quantity{Format: resource.DecimalSI}
-	switch svc.Spec.Type {
-	case corev1.ServiceTypeNodePort:
-		// node port services need to count node ports
-		value := resource.NewQuantity(int64(ports), resource.DecimalSI)
-		result[corev1.ResourceServicesNodePorts] = *value
-	case corev1.ServiceTypeLoadBalancer:
-		// load balancer services need to count node ports and load balancers
-		value := resource.NewQuantity(int64(ports), resource.DecimalSI)
-		result[corev1.ResourceServicesNodePorts] = *value
-		result[corev1.ResourceServicesLoadBalancers] = *(resource.NewQuantity(1, resource.DecimalSI))
-	}
-	return result, nil
-}
-
-// UsageStats calculates aggregate usage for the object.
-func (p *serviceEvaluator) UsageStats(options quota.UsageStatsOptions) (quota.UsageStats, error) {
-	return generic.CalculateUsageStats(options, p.listFuncByNamespace, generic.MatchesNoScopeFunc, p.Usage)
-}
-
-var _ quota.Evaluator = &serviceEvaluator{}
-
-//GetQuotaServiceType returns ServiceType if the service type is eligible to track against a quota, nor return ""
-func GetQuotaServiceType(service *corev1.Service) corev1.ServiceType {
-	switch service.Spec.Type {
-	case corev1.ServiceTypeNodePort:
-		return corev1.ServiceTypeNodePort
-	case corev1.ServiceTypeLoadBalancer:
-		return corev1.ServiceTypeLoadBalancer
-	}
-	return corev1.ServiceType("")
-}
--- a/kube/pkg/quota/v1/generic/configuration.go
+++ b/kube/pkg/quota/v1/generic/configuration.go
@@ -1,45 +0,0 @@
-/*
-Copyright 2017 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package generic
-
-import (
-	"k8s.io/apimachinery/pkg/runtime/schema"
-
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1"
-)
-
-// implements a basic configuration
-type simpleConfiguration struct {
-	evaluators       []quota.Evaluator
-	ignoredResources map[schema.GroupResource]struct{}
-}
-
-// NewConfiguration creates a quota configuration
-func NewConfiguration(evaluators []quota.Evaluator, ignoredResources map[schema.GroupResource]struct{}) quota.Configuration {
-	return &simpleConfiguration{
-		evaluators:       evaluators,
-		ignoredResources: ignoredResources,
-	}
-}
-
-func (c *simpleConfiguration) IgnoredResources() map[schema.GroupResource]struct{} {
-	return c.ignoredResources
-}
-
-func (c *simpleConfiguration) Evaluators() []quota.Evaluator {
-	return c.evaluators
-}
--- a/kube/pkg/quota/v1/generic/evaluator.go
+++ b/kube/pkg/quota/v1/generic/evaluator.go
@@ -1,320 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package generic
-
-import (
-	"fmt"
-	"sync/atomic"
-
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-	"k8s.io/apiserver/pkg/admission"
-	"k8s.io/client-go/informers"
-	"k8s.io/client-go/tools/cache"
-
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-)
-
-// InformerForResourceFunc knows how to provision an informer
-type InformerForResourceFunc func(schema.GroupVersionResource) (informers.GenericInformer, error)
-
-// ListerFuncForResourceFunc knows how to provision a lister from an informer func.
-// The lister returns errors until the informer has synced.
-func ListerFuncForResourceFunc(f InformerForResourceFunc) quota.ListerForResourceFunc {
-	return func(gvr schema.GroupVersionResource) (cache.GenericLister, error) {
-		informer, err := f(gvr)
-		if err != nil {
-			return nil, err
-		}
-		return &protectedLister{
-			hasSynced:   cachedHasSynced(informer.Informer().HasSynced),
-			notReadyErr: fmt.Errorf("%v not yet synced", gvr),
-			delegate:    informer.Lister(),
-		}, nil
-	}
-}
-
-// cachedHasSynced returns a function that calls hasSynced() until it returns true once, then returns true
-func cachedHasSynced(hasSynced func() bool) func() bool {
-	cache := &atomic.Value{}
-	cache.Store(false)
-	return func() bool {
-		if cache.Load().(bool) {
-			// short-circuit if already synced
-			return true
-		}
-		if hasSynced() {
-			// remember we synced
-			cache.Store(true)
-			return true
-		}
-		return false
-	}
-}
-
-// protectedLister returns notReadyError if hasSynced returns false, otherwise delegates to delegate
-type protectedLister struct {
-	hasSynced   func() bool
-	notReadyErr error
-	delegate    cache.GenericLister
-}
-
-func (p *protectedLister) List(selector labels.Selector) (ret []runtime.Object, err error) {
-	if !p.hasSynced() {
-		return nil, p.notReadyErr
-	}
-	return p.delegate.List(selector)
-}
-func (p *protectedLister) Get(name string) (runtime.Object, error) {
-	if !p.hasSynced() {
-		return nil, p.notReadyErr
-	}
-	return p.delegate.Get(name)
-}
-func (p *protectedLister) ByNamespace(namespace string) cache.GenericNamespaceLister {
-	return &protectedNamespaceLister{p.hasSynced, p.notReadyErr, p.delegate.ByNamespace(namespace)}
-}
-
-// protectedNamespaceLister returns notReadyError if hasSynced returns false, otherwise delegates to delegate
-type protectedNamespaceLister struct {
-	hasSynced   func() bool
-	notReadyErr error
-	delegate    cache.GenericNamespaceLister
-}
-
-func (p *protectedNamespaceLister) List(selector labels.Selector) (ret []runtime.Object, err error) {
-	if !p.hasSynced() {
-		return nil, p.notReadyErr
-	}
-	return p.delegate.List(selector)
-}
-func (p *protectedNamespaceLister) Get(name string) (runtime.Object, error) {
-	if !p.hasSynced() {
-		return nil, p.notReadyErr
-	}
-	return p.delegate.Get(name)
-}
-
-// ListResourceUsingListerFunc returns a listing function based on the shared informer factory for the specified resource.
-func ListResourceUsingListerFunc(l quota.ListerForResourceFunc, resource schema.GroupVersionResource) ListFuncByNamespace {
-	return func(namespace string) ([]runtime.Object, error) {
-		lister, err := l(resource)
-		if err != nil {
-			return nil, err
-		}
-		return lister.ByNamespace(namespace).List(labels.Everything())
-	}
-}
-
-// ObjectCountQuotaResourceNameFor returns the object count quota name for specified groupResource
-func ObjectCountQuotaResourceNameFor(groupResource schema.GroupResource) corev1.ResourceName {
-	if len(groupResource.Group) == 0 {
-		return corev1.ResourceName("count/" + groupResource.Resource)
-	}
-	return corev1.ResourceName("count/" + groupResource.Resource + "." + groupResource.Group)
-}
-
-// ListFuncByNamespace knows how to list resources in a namespace
-type ListFuncByNamespace func(namespace string) ([]runtime.Object, error)
-
-// MatchesScopeFunc knows how to evaluate if an object matches a scope
-type MatchesScopeFunc func(scope corev1.ScopedResourceSelectorRequirement, object runtime.Object) (bool, error)
-
-// UsageFunc knows how to measure usage associated with an object
-type UsageFunc func(object runtime.Object) (corev1.ResourceList, error)
-
-// MatchingResourceNamesFunc is a function that returns the list of resources matched
-type MatchingResourceNamesFunc func(input []corev1.ResourceName) []corev1.ResourceName
-
-// MatchesNoScopeFunc returns false on all match checks
-func MatchesNoScopeFunc(scope corev1.ScopedResourceSelectorRequirement, object runtime.Object) (bool, error) {
-	return false, nil
-}
-
-// Matches returns true if the quota matches the specified item.
-func Matches(
-	resourceQuota *corev1.ResourceQuota, item runtime.Object,
-	matchFunc MatchingResourceNamesFunc, scopeFunc MatchesScopeFunc) (bool, error) {
-	if resourceQuota == nil {
-		return false, fmt.Errorf("expected non-nil quota")
-	}
-	// verify the quota matches on at least one resource
-	matchResource := len(matchFunc(quota.ResourceNames(resourceQuota.Status.Hard))) > 0
-	// by default, no scopes matches all
-	matchScope := true
-	for _, scope := range getScopeSelectorsFromQuota(resourceQuota) {
-		innerMatch, err := scopeFunc(scope, item)
-		if err != nil {
-			return false, err
-		}
-		matchScope = matchScope && innerMatch
-	}
-	return matchResource && matchScope, nil
-}
-
-func getScopeSelectorsFromQuota(quota *corev1.ResourceQuota) []corev1.ScopedResourceSelectorRequirement {
-	selectors := []corev1.ScopedResourceSelectorRequirement{}
-	for _, scope := range quota.Spec.Scopes {
-		selectors = append(selectors, corev1.ScopedResourceSelectorRequirement{
-			ScopeName: scope,
-			Operator:  corev1.ScopeSelectorOpExists})
-	}
-	if quota.Spec.ScopeSelector != nil {
-		selectors = append(selectors, quota.Spec.ScopeSelector.MatchExpressions...)
-	}
-	return selectors
-}
-
-// CalculateUsageStats is a utility function that knows how to calculate aggregate usage.
-func CalculateUsageStats(options quota.UsageStatsOptions,
-	listFunc ListFuncByNamespace,
-	scopeFunc MatchesScopeFunc,
-	usageFunc UsageFunc) (quota.UsageStats, error) {
-	// default each tracked resource to zero
-	result := quota.UsageStats{Used: corev1.ResourceList{}}
-	for _, resourceName := range options.Resources {
-		result.Used[resourceName] = resource.Quantity{Format: resource.DecimalSI}
-	}
-	items, err := listFunc(options.Namespace)
-	if err != nil {
-		return result, fmt.Errorf("failed to list content: %v", err)
-	}
-	for _, item := range items {
-		// need to verify that the item matches the set of scopes
-		matchesScopes := true
-		for _, scope := range options.Scopes {
-			innerMatch, err := scopeFunc(corev1.ScopedResourceSelectorRequirement{ScopeName: scope}, item)
-			if err != nil {
-				return result, nil
-			}
-			if !innerMatch {
-				matchesScopes = false
-			}
-		}
-		if options.ScopeSelector != nil {
-			for _, selector := range options.ScopeSelector.MatchExpressions {
-				innerMatch, err := scopeFunc(selector, item)
-				if err != nil {
-					return result, nil
-				}
-				matchesScopes = matchesScopes && innerMatch
-			}
-		}
-		// only count usage if there was a match
-		if matchesScopes {
-			usage, err := usageFunc(item)
-			if err != nil {
-				return result, err
-			}
-			result.Used = quota.Add(result.Used, usage)
-		}
-	}
-	return result, nil
-}
-
-// objectCountEvaluator provides an implementation for quota.Evaluator
-// that associates usage of the specified resource based on the number of items
-// returned by the specified listing function.
-type objectCountEvaluator struct {
-	// GroupResource that this evaluator tracks.
-	// It is used to construct a generic object count quota name
-	groupResource schema.GroupResource
-	// A function that knows how to list resources by namespace.
-	// TODO move to dynamic client in future
-	listFuncByNamespace ListFuncByNamespace
-	// Names associated with this resource in the quota for generic counting.
-	resourceNames []corev1.ResourceName
-}
-
-// Constraints returns an error if the configured resource name is not in the required set.
-func (o *objectCountEvaluator) Constraints(required []corev1.ResourceName, item runtime.Object) error {
-	// no-op for object counting
-	return nil
-}
-
-// Handles returns true if the object count evaluator needs to track this attributes.
-func (o *objectCountEvaluator) Handles(a admission.Attributes) bool {
-	operation := a.GetOperation()
-	return operation == admission.Create
-}
-
-// Matches returns true if the evaluator matches the specified quota with the provided input item
-func (o *objectCountEvaluator) Matches(resourceQuota *corev1.ResourceQuota, item runtime.Object) (bool, error) {
-	return Matches(resourceQuota, item, o.MatchingResources, MatchesNoScopeFunc)
-}
-
-// MatchingResources takes the input specified list of resources and returns the set of resources it matches.
-func (o *objectCountEvaluator) MatchingResources(input []corev1.ResourceName) []corev1.ResourceName {
-	return quota.Intersection(input, o.resourceNames)
-}
-
-// MatchingScopes takes the input specified list of scopes and input object. Returns the set of scopes resource matches.
-func (o *objectCountEvaluator) MatchingScopes(item runtime.Object, scopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	return []corev1.ScopedResourceSelectorRequirement{}, nil
-}
-
-// UncoveredQuotaScopes takes the input matched scopes which are limited by configuration and the matched quota scopes.
-// It returns the scopes which are in limited scopes but dont have a corresponding covering quota scope
-func (o *objectCountEvaluator) UncoveredQuotaScopes(limitedScopes []corev1.ScopedResourceSelectorRequirement, matchedQuotaScopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	return []corev1.ScopedResourceSelectorRequirement{}, nil
-}
-
-// Usage returns the resource usage for the specified object
-func (o *objectCountEvaluator) Usage(object runtime.Object) (corev1.ResourceList, error) {
-	quantity := resource.NewQuantity(1, resource.DecimalSI)
-	resourceList := corev1.ResourceList{}
-	for _, resourceName := range o.resourceNames {
-		resourceList[resourceName] = *quantity
-	}
-	return resourceList, nil
-}
-
-// GroupResource tracked by this evaluator
-func (o *objectCountEvaluator) GroupResource() schema.GroupResource {
-	return o.groupResource
-}
-
-// UsageStats calculates aggregate usage for the object.
-func (o *objectCountEvaluator) UsageStats(options quota.UsageStatsOptions) (quota.UsageStats, error) {
-	return CalculateUsageStats(options, o.listFuncByNamespace, MatchesNoScopeFunc, o.Usage)
-}
-
-// Verify implementation of interface at compile time.
-var _ quota.Evaluator = &objectCountEvaluator{}
-
-// NewObjectCountEvaluator returns an evaluator that can perform generic
-// object quota counting.  It allows an optional alias for backwards compatibility
-// purposes for the legacy object counting names in quota.  Unless its supporting
-// backward compatibility, alias should not be used.
-func NewObjectCountEvaluator(
-	groupResource schema.GroupResource, listFuncByNamespace ListFuncByNamespace,
-	alias corev1.ResourceName) quota.Evaluator {
-
-	resourceNames := []corev1.ResourceName{ObjectCountQuotaResourceNameFor(groupResource)}
-	if len(alias) > 0 {
-		resourceNames = append(resourceNames, alias)
-	}
-
-	return &objectCountEvaluator{
-		groupResource:       groupResource,
-		listFuncByNamespace: listFuncByNamespace,
-		resourceNames:       resourceNames,
-	}
-}
--- a/kube/pkg/quota/v1/generic/registry.go
+++ b/kube/pkg/quota/v1/generic/registry.go
@@ -1,82 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package generic
-
-import (
-	"sync"
-
-	"k8s.io/apimachinery/pkg/runtime/schema"
-
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-)
-
-// implements a basic registry
-type simpleRegistry struct {
-	lock sync.RWMutex
-	// evaluators tracked by the registry
-	evaluators map[schema.GroupResource]quota.Evaluator
-}
-
-// NewRegistry creates a simple registry with initial list of evaluators
-func NewRegistry(evaluators []quota.Evaluator) quota.Registry {
-	return &simpleRegistry{
-		evaluators: evaluatorsByGroupResource(evaluators),
-	}
-}
-
-func (r *simpleRegistry) Add(e quota.Evaluator) {
-	r.lock.Lock()
-	defer r.lock.Unlock()
-	r.evaluators[e.GroupResource()] = e
-}
-
-func (r *simpleRegistry) Remove(e quota.Evaluator) {
-	r.lock.Lock()
-	defer r.lock.Unlock()
-	delete(r.evaluators, e.GroupResource())
-}
-
-func (r *simpleRegistry) Get(gr schema.GroupResource) quota.Evaluator {
-	r.lock.RLock()
-	defer r.lock.RUnlock()
-	return r.evaluators[gr]
-}
-
-func (r *simpleRegistry) List() []quota.Evaluator {
-	r.lock.RLock()
-	defer r.lock.RUnlock()
-
-	return evaluatorsList(r.evaluators)
-}
-
-// evaluatorsByGroupResource converts a list of evaluators to a map by group resource.
-func evaluatorsByGroupResource(items []quota.Evaluator) map[schema.GroupResource]quota.Evaluator {
-	result := map[schema.GroupResource]quota.Evaluator{}
-	for _, item := range items {
-		result[item.GroupResource()] = item
-	}
-	return result
-}
-
-// evaluatorsList converts a map of evaluators to list
-func evaluatorsList(input map[schema.GroupResource]quota.Evaluator) []quota.Evaluator {
-	var result []quota.Evaluator
-	for _, item := range input {
-		result = append(result, item)
-	}
-	return result
-}
--- a/kube/pkg/quota/v1/install/registry.go
+++ b/kube/pkg/quota/v1/install/registry.go
@@ -1,48 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package install
-
-import (
-	"k8s.io/apimachinery/pkg/runtime/schema"
-
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	core "kubesphere.io/kubesphere/kube/pkg/quota/v1/evaluator/core"
-	generic "kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-)
-
-// NewQuotaConfigurationForAdmission returns a quota configuration for admission control.
-func NewQuotaConfigurationForAdmission() quota.Configuration {
-	evaluators := core.NewEvaluators(nil)
-	return generic.NewConfiguration(evaluators, DefaultIgnoredResources())
-}
-
-// NewQuotaConfigurationForControllers returns a quota configuration for controllers.
-func NewQuotaConfigurationForControllers(f quota.ListerForResourceFunc) quota.Configuration {
-	evaluators := core.NewEvaluators(f)
-	return generic.NewConfiguration(evaluators, DefaultIgnoredResources())
-}
-
-// ignoredResources are ignored by quota by default
-var ignoredResources = map[schema.GroupResource]struct{}{
-	{Group: "", Resource: "events"}: {},
-}
-
-// DefaultIgnoredResources returns the default set of resources that quota system
-// should ignore. This is exposed so downstream integrators can have access to them.
-func DefaultIgnoredResources() map[schema.GroupResource]struct{} {
-	return ignoredResources
-}
--- a/kube/pkg/quota/v1/interfaces.go
+++ b/kube/pkg/quota/v1/interfaces.go
@@ -1,88 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package quota
-
-import (
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-	"k8s.io/apiserver/pkg/admission"
-	"k8s.io/client-go/tools/cache"
-)
-
-// UsageStatsOptions is an options structs that describes how stats should be calculated
-type UsageStatsOptions struct {
-	// Namespace where stats should be calculate
-	Namespace string
-	// Scopes that must match counted objects
-	Scopes []corev1.ResourceQuotaScope
-	// Resources are the set of resources to include in the measurement
-	Resources     []corev1.ResourceName
-	ScopeSelector *corev1.ScopeSelector
-}
-
-// UsageStats is result of measuring observed resource use in the system
-type UsageStats struct {
-	// Used maps resource to quantity used
-	Used corev1.ResourceList
-}
-
-// Evaluator knows how to evaluate quota usage for a particular group resource
-type Evaluator interface {
-	// Constraints ensures that each required resource is present on item
-	Constraints(required []corev1.ResourceName, item runtime.Object) error
-	// GroupResource returns the groupResource that this object knows how to evaluate
-	GroupResource() schema.GroupResource
-	// Handles determines if quota could be impacted by the specified attribute.
-	// If true, admission control must perform quota processing for the operation, otherwise it is safe to ignore quota.
-	Handles(operation admission.Attributes) bool
-	// Matches returns true if the specified quota matches the input item
-	Matches(resourceQuota *corev1.ResourceQuota, item runtime.Object) (bool, error)
-	// MatchingScopes takes the input specified list of scopes and input object and returns the set of scopes that matches input object.
-	MatchingScopes(item runtime.Object, scopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error)
-	// UncoveredQuotaScopes takes the input matched scopes which are limited by configuration and the matched quota scopes. It returns the scopes which are in limited scopes but dont have a corresponding covering quota scope
-	UncoveredQuotaScopes(limitedScopes []corev1.ScopedResourceSelectorRequirement, matchedQuotaScopes []corev1.ScopedResourceSelectorRequirement) ([]corev1.ScopedResourceSelectorRequirement, error)
-	// MatchingResources takes the input specified list of resources and returns the set of resources evaluator matches.
-	MatchingResources(input []corev1.ResourceName) []corev1.ResourceName
-	// Usage returns the resource usage for the specified object
-	Usage(item runtime.Object) (corev1.ResourceList, error)
-	// UsageStats calculates latest observed usage stats for all objects
-	UsageStats(options UsageStatsOptions) (UsageStats, error)
-}
-
-// Configuration defines how the quota system is configured.
-type Configuration interface {
-	// IgnoredResources are ignored by quota.
-	IgnoredResources() map[schema.GroupResource]struct{}
-	// Evaluators for quota evaluation.
-	Evaluators() []Evaluator
-}
-
-// Registry maintains a list of evaluators
-type Registry interface {
-	// Add to registry
-	Add(e Evaluator)
-	// Remove from registry
-	Remove(e Evaluator)
-	// Get by group resource
-	Get(gr schema.GroupResource) Evaluator
-	// List from registry
-	List() []Evaluator
-}
-
-// ListerForResourceFunc knows how to get a lister for a specific resource
-type ListerForResourceFunc func(schema.GroupVersionResource) (cache.GenericLister, error)
--- a/kube/pkg/quota/v1/resources.go
+++ b/kube/pkg/quota/v1/resources.go
@@ -1,293 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package quota
-
-import (
-	"sort"
-	"strings"
-
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-	utilerrors "k8s.io/apimachinery/pkg/util/errors"
-	"k8s.io/apimachinery/pkg/util/sets"
-)
-
-// Equals returns true if the two lists are equivalent
-func Equals(a corev1.ResourceList, b corev1.ResourceList) bool {
-	if len(a) != len(b) {
-		return false
-	}
-
-	for key, value1 := range a {
-		value2, found := b[key]
-		if !found {
-			return false
-		}
-		if value1.Cmp(value2) != 0 {
-			return false
-		}
-	}
-
-	return true
-}
-
-// LessThanOrEqual returns true if a < b for each key in b
-// If false, it returns the keys in a that exceeded b
-func LessThanOrEqual(a corev1.ResourceList, b corev1.ResourceList) (bool, []corev1.ResourceName) {
-	result := true
-	resourceNames := []corev1.ResourceName{}
-	for key, value := range b {
-		if other, found := a[key]; found {
-			if other.Cmp(value) > 0 {
-				result = false
-				resourceNames = append(resourceNames, key)
-			}
-		}
-	}
-	return result, resourceNames
-}
-
-// Max returns the result of Max(a, b) for each named resource
-func Max(a corev1.ResourceList, b corev1.ResourceList) corev1.ResourceList {
-	result := corev1.ResourceList{}
-	for key, value := range a {
-		if other, found := b[key]; found {
-			if value.Cmp(other) <= 0 {
-				result[key] = other.DeepCopy()
-				continue
-			}
-		}
-		result[key] = value.DeepCopy()
-	}
-	for key, value := range b {
-		if _, found := result[key]; !found {
-			result[key] = value.DeepCopy()
-		}
-	}
-	return result
-}
-
-// Add returns the result of a + b for each named resource
-func Add(a corev1.ResourceList, b corev1.ResourceList) corev1.ResourceList {
-	result := corev1.ResourceList{}
-	for key, value := range a {
-		quantity := value.DeepCopy()
-		if other, found := b[key]; found {
-			quantity.Add(other)
-		}
-		result[key] = quantity
-	}
-	for key, value := range b {
-		if _, found := result[key]; !found {
-			result[key] = value.DeepCopy()
-		}
-	}
-	return result
-}
-
-// SubtractWithNonNegativeResult - subtracts and returns result of a - b but
-// makes sure we don't return negative values to prevent negative resource usage.
-func SubtractWithNonNegativeResult(a corev1.ResourceList, b corev1.ResourceList) corev1.ResourceList {
-	zero := resource.MustParse("0")
-
-	result := corev1.ResourceList{}
-	for key, value := range a {
-		quantity := value.DeepCopy()
-		if other, found := b[key]; found {
-			quantity.Sub(other)
-		}
-		if quantity.Cmp(zero) > 0 {
-			result[key] = quantity
-		} else {
-			result[key] = zero
-		}
-	}
-
-	for key := range b {
-		if _, found := result[key]; !found {
-			result[key] = zero
-		}
-	}
-	return result
-}
-
-// Subtract returns the result of a - b for each named resource
-func Subtract(a corev1.ResourceList, b corev1.ResourceList) corev1.ResourceList {
-	result := corev1.ResourceList{}
-	for key, value := range a {
-		quantity := value.DeepCopy()
-		if other, found := b[key]; found {
-			quantity.Sub(other)
-		}
-		result[key] = quantity
-	}
-	for key, value := range b {
-		if _, found := result[key]; !found {
-			quantity := value.DeepCopy()
-			quantity.Neg()
-			result[key] = quantity
-		}
-	}
-	return result
-}
-
-// Mask returns a new resource list that only has the values with the specified names
-func Mask(resources corev1.ResourceList, names []corev1.ResourceName) corev1.ResourceList {
-	nameSet := ToSet(names)
-	result := corev1.ResourceList{}
-	for key, value := range resources {
-		if nameSet.Has(string(key)) {
-			result[key] = value.DeepCopy()
-		}
-	}
-	return result
-}
-
-// ResourceNames returns a list of all resource names in the ResourceList
-func ResourceNames(resources corev1.ResourceList) []corev1.ResourceName {
-	result := []corev1.ResourceName{}
-	for resourceName := range resources {
-		result = append(result, resourceName)
-	}
-	return result
-}
-
-// Contains returns true if the specified item is in the list of items
-func Contains(items []corev1.ResourceName, item corev1.ResourceName) bool {
-	for _, i := range items {
-		if i == item {
-			return true
-		}
-	}
-	return false
-}
-
-// ContainsPrefix returns true if the specified item has a prefix that contained in given prefix Set
-func ContainsPrefix(prefixSet []string, item corev1.ResourceName) bool {
-	for _, prefix := range prefixSet {
-		if strings.HasPrefix(string(item), prefix) {
-			return true
-		}
-	}
-	return false
-}
-
-// Intersection returns the intersection of both list of resources, deduped and sorted
-func Intersection(a []corev1.ResourceName, b []corev1.ResourceName) []corev1.ResourceName {
-	result := make([]corev1.ResourceName, 0, len(a))
-	for _, item := range a {
-		if Contains(result, item) {
-			continue
-		}
-		if !Contains(b, item) {
-			continue
-		}
-		result = append(result, item)
-	}
-	sort.Slice(result, func(i, j int) bool { return result[i] < result[j] })
-	return result
-}
-
-// Difference returns the list of resources resulting from a-b, deduped and sorted
-func Difference(a []corev1.ResourceName, b []corev1.ResourceName) []corev1.ResourceName {
-	result := make([]corev1.ResourceName, 0, len(a))
-	for _, item := range a {
-		if Contains(b, item) || Contains(result, item) {
-			continue
-		}
-		result = append(result, item)
-	}
-	sort.Slice(result, func(i, j int) bool { return result[i] < result[j] })
-	return result
-}
-
-// IsZero returns true if each key maps to the quantity value 0
-func IsZero(a corev1.ResourceList) bool {
-	zero := resource.MustParse("0")
-	for _, v := range a {
-		if v.Cmp(zero) != 0 {
-			return false
-		}
-	}
-	return true
-}
-
-// IsNegative returns the set of resource names that have a negative value.
-func IsNegative(a corev1.ResourceList) []corev1.ResourceName {
-	results := []corev1.ResourceName{}
-	zero := resource.MustParse("0")
-	for k, v := range a {
-		if v.Cmp(zero) < 0 {
-			results = append(results, k)
-		}
-	}
-	return results
-}
-
-// ToSet takes a list of resource names and converts to a string set
-func ToSet(resourceNames []corev1.ResourceName) sets.String {
-	result := sets.NewString()
-	for _, resourceName := range resourceNames {
-		result.Insert(string(resourceName))
-	}
-	return result
-}
-
-// CalculateUsage calculates and returns the requested ResourceList usage.
-// If an error is returned, usage only contains the resources which encountered no calculation errors.
-func CalculateUsage(namespaceName string, scopes []corev1.ResourceQuotaScope, hardLimits corev1.ResourceList, registry Registry, scopeSelector *corev1.ScopeSelector) (corev1.ResourceList, error) {
-	// find the intersection between the hard resources on the quota
-	// and the resources this controller can track to know what we can
-	// look to measure updated usage stats for
-	hardResources := ResourceNames(hardLimits)
-	potentialResources := []corev1.ResourceName{}
-	evaluators := registry.List()
-	for _, evaluator := range evaluators {
-		potentialResources = append(potentialResources, evaluator.MatchingResources(hardResources)...)
-	}
-	// NOTE: the intersection just removes duplicates since the evaluator match intersects with hard
-	matchedResources := Intersection(hardResources, potentialResources)
-
-	errors := []error{}
-
-	// sum the observed usage from each evaluator
-	newUsage := corev1.ResourceList{}
-	for _, evaluator := range evaluators {
-		// only trigger the evaluator if it matches a resource in the quota, otherwise, skip calculating anything
-		intersection := evaluator.MatchingResources(matchedResources)
-		if len(intersection) == 0 {
-			continue
-		}
-
-		usageStatsOptions := UsageStatsOptions{Namespace: namespaceName, Scopes: scopes, Resources: intersection, ScopeSelector: scopeSelector}
-		stats, err := evaluator.UsageStats(usageStatsOptions)
-		if err != nil {
-			// remember the error
-			errors = append(errors, err)
-			// exclude resources which encountered calculation errors
-			matchedResources = Difference(matchedResources, intersection)
-			continue
-		}
-		newUsage = Add(newUsage, stats.Used)
-	}
-
-	// mask the observed usage to only the set of resources tracked by this quota
-	// merge our observed usage with the quota usage status
-	// if the new usage is different than the last usage, we will need to do an update
-	newUsage = Mask(newUsage, matchedResources)
-	return newUsage, utilerrors.NewAggregate(errors)
-}
--- a/kube/plugin/pkg/admission/resourcequota/admission.go
+++ b/kube/plugin/pkg/admission/resourcequota/admission.go
@@ -1,127 +0,0 @@
-/*
-Copyright 2014 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package resourcequota
-
-import (
-	"context"
-	"fmt"
-	"time"
-
-	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apiserver/pkg/admission"
-	genericadmissioninitializer "k8s.io/apiserver/pkg/admission/initializer"
-	"k8s.io/client-go/informers"
-	"k8s.io/client-go/kubernetes"
-
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-	resourcequotaapi "kubesphere.io/kubesphere/kube/plugin/pkg/admission/resourcequota/apis/resourcequota"
-)
-
-// QuotaAdmission implements an admission controller that can enforce quota constraints
-type QuotaAdmission struct {
-	*admission.Handler
-	config             *resourcequotaapi.Configuration
-	stopCh             <-chan struct{}
-	quotaConfiguration quota.Configuration
-	numEvaluators      int
-	quotaAccessor      *quotaAccessor
-	evaluator          Evaluator
-}
-
-// WantsQuotaConfiguration defines a function which sets quota configuration for admission plugins that need it.
-type WantsQuotaConfiguration interface {
-	SetQuotaConfiguration(quota.Configuration)
-	admission.InitializationValidator
-}
-
-var _ admission.ValidationInterface = &QuotaAdmission{}
-var _ = genericadmissioninitializer.WantsExternalKubeInformerFactory(&QuotaAdmission{})
-var _ = genericadmissioninitializer.WantsExternalKubeClientSet(&QuotaAdmission{})
-var _ = WantsQuotaConfiguration(&QuotaAdmission{})
-
-type liveLookupEntry struct {
-	expiry time.Time
-	items  []*corev1.ResourceQuota
-}
-
-// NewResourceQuota configures an admission controller that can enforce quota constraints
-// using the provided registry.  The registry must have the capability to handle group/kinds that
-// are persisted by the server this admission controller is intercepting
-func NewResourceQuota(config *resourcequotaapi.Configuration, numEvaluators int, stopCh <-chan struct{}) (*QuotaAdmission, error) {
-	quotaAccessor, err := newQuotaAccessor()
-	if err != nil {
-		return nil, err
-	}
-
-	return &QuotaAdmission{
-		Handler:       admission.NewHandler(admission.Create, admission.Update),
-		stopCh:        stopCh,
-		numEvaluators: numEvaluators,
-		config:        config,
-		quotaAccessor: quotaAccessor,
-	}, nil
-}
-
-// SetExternalKubeClientSet registers the client into QuotaAdmission
-func (a *QuotaAdmission) SetExternalKubeClientSet(client kubernetes.Interface) {
-	a.quotaAccessor.client = client
-}
-
-// SetExternalKubeInformerFactory registers an informer factory into QuotaAdmission
-func (a *QuotaAdmission) SetExternalKubeInformerFactory(f informers.SharedInformerFactory) {
-	a.quotaAccessor.lister = f.Core().V1().ResourceQuotas().Lister()
-}
-
-// SetQuotaConfiguration assigns and initializes configuration and evaluator for QuotaAdmission
-func (a *QuotaAdmission) SetQuotaConfiguration(c quota.Configuration) {
-	a.quotaConfiguration = c
-	a.evaluator = NewQuotaEvaluator(a.quotaAccessor, a.quotaConfiguration.IgnoredResources(), generic.NewRegistry(a.quotaConfiguration.Evaluators()), nil, a.config, a.numEvaluators, a.stopCh)
-}
-
-// ValidateInitialization ensures an authorizer is set.
-func (a *QuotaAdmission) ValidateInitialization() error {
-	if a.quotaAccessor == nil {
-		return fmt.Errorf("missing quotaAccessor")
-	}
-	if a.quotaAccessor.client == nil {
-		return fmt.Errorf("missing quotaAccessor.client")
-	}
-	if a.quotaAccessor.lister == nil {
-		return fmt.Errorf("missing quotaAccessor.lister")
-	}
-	if a.quotaConfiguration == nil {
-		return fmt.Errorf("missing quotaConfiguration")
-	}
-	if a.evaluator == nil {
-		return fmt.Errorf("missing evaluator")
-	}
-	return nil
-}
-
-// Validate makes admission decisions while enforcing quota
-func (a *QuotaAdmission) Validate(ctx context.Context, attr admission.Attributes, o admission.ObjectInterfaces) (err error) {
-	// ignore all operations that correspond to sub-resource actions
-	if attr.GetSubresource() != "" {
-		return nil
-	}
-	// ignore all operations that are not namespaced
-	if attr.GetNamespace() == "" {
-		return nil
-	}
-	return a.evaluator.Evaluate(attr)
-}
--- a/kube/plugin/pkg/admission/resourcequota/apis/resourcequota/types.go
+++ b/kube/plugin/pkg/admission/resourcequota/apis/resourcequota/types.go
@@ -1,74 +0,0 @@
-/*
-
- Copyright 2021 The KubeSphere Authors.
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
-*/
-
-package resourcequota
-
-import (
-	corev1 "k8s.io/api/core/v1"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-)
-
-// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
-
-// Configuration provides configuration for the ResourceQuota admission controller.
-type Configuration struct {
-	metav1.TypeMeta
-
-	// LimitedResources whose consumption is limited by default.
-	// +optional
-	LimitedResources []LimitedResource
-}
-
-// LimitedResource matches a resource whose consumption is limited by default.
-// To consume the resource, there must exist an associated quota that limits
-// its consumption.
-type LimitedResource struct {
-
-	// APIGroup is the name of the APIGroup that contains the limited resource.
-	// +optional
-	APIGroup string `json:"apiGroup,omitempty"`
-
-	// Resource is the name of the resource this rule applies to.
-	// For example, if the administrator wants to limit consumption
-	// of a storage resource associated with persistent volume claims,
-	// the value would be "persistentvolumeclaims".
-	Resource string `json:"resource"`
-
-	// For each intercepted request, the quota system will evaluate
-	// its resource usage.  It will iterate through each resource consumed
-	// and if the resource contains any substring in this listing, the
-	// quota system will ensure that there is a covering quota.  In the
-	// absence of a covering quota, the quota system will deny the request.
-	// For example, if an administrator wants to globally enforce that
-	// that a quota must exist to consume persistent volume claims associated
-	// with any storage class, the list would include
-	// ".storageclass.storage.k8s.io/requests.storage"
-	MatchContains []string
-
-	// For each intercepted request, the quota system will figure out if the input object
-	// satisfies a scope which is present in this listing, then
-	// quota system will ensure that there is a covering quota.  In the
-	// absence of a covering quota, the quota system will deny the request.
-	// For example, if an administrator wants to globally enforce that
-	// a quota must exist to create a pod with "cluster-services" priorityclass
-	// the list would include
-	// "PriorityClassNameIn=cluster-services"
-	// +optional
-	//	MatchScopes []string `json:"matchScopes,omitempty"`
-	MatchScopes []corev1.ScopedResourceSelectorRequirement `json:"matchScopes,omitempty"`
-}
--- a/kube/plugin/pkg/admission/resourcequota/controller.go
+++ b/kube/plugin/pkg/admission/resourcequota/controller.go
@@ -1,718 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package resourcequota
-
-import (
-	"fmt"
-	"sort"
-	"strings"
-	"sync"
-	"time"
-
-	"k8s.io/klog"
-
-	corev1 "k8s.io/api/core/v1"
-	apierrors "k8s.io/apimachinery/pkg/api/errors"
-	"k8s.io/apimachinery/pkg/api/meta"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/runtime/schema"
-	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
-	"k8s.io/apimachinery/pkg/util/sets"
-	"k8s.io/apimachinery/pkg/util/wait"
-	"k8s.io/apiserver/pkg/admission"
-	"k8s.io/client-go/util/workqueue"
-
-	quota "kubesphere.io/kubesphere/kube/pkg/quota/v1"
-	"kubesphere.io/kubesphere/kube/pkg/quota/v1/generic"
-	resourcequotaapi "kubesphere.io/kubesphere/kube/plugin/pkg/admission/resourcequota/apis/resourcequota"
-)
-
-// Evaluator is used to see if quota constraints are satisfied.
-type Evaluator interface {
-	// Evaluate takes an operation and checks to see if quota constraints are satisfied.  It returns an error if they are not.
-	// The default implementation process related operations in chunks when possible.
-	Evaluate(a admission.Attributes) error
-}
-
-type quotaEvaluator struct {
-	quotaAccessor QuotaAccessor
-	// lockAcquisitionFunc acquires any required locks and returns a cleanup method to defer
-	lockAcquisitionFunc func([]corev1.ResourceQuota) func()
-
-	ignoredResources map[schema.GroupResource]struct{}
-
-	// registry that knows how to measure usage for objects
-	registry quota.Registry
-
-	// TODO these are used together to bucket items by namespace and then batch them up for processing.
-	// The technique is valuable for rollup activities to avoid fanout and reduce resource contention.
-	// We could move this into a library if another component needed it.
-	// queue is indexed by namespace, so that we bundle up on a per-namespace basis
-	queue      *workqueue.Type
-	workLock   sync.Mutex
-	work       map[string][]*admissionWaiter
-	dirtyWork  map[string][]*admissionWaiter
-	inProgress sets.String
-
-	// controls the run method so that we can cleanly conform to the Evaluator interface
-	workers int
-	stopCh  <-chan struct{}
-	init    sync.Once
-
-	// lets us know what resources are limited by default
-	config *resourcequotaapi.Configuration
-}
-
-type admissionWaiter struct {
-	attributes admission.Attributes
-	finished   chan struct{}
-	result     error
-}
-
-type defaultDeny struct{}
-
-func (defaultDeny) Error() string {
-	return "DEFAULT DENY"
-}
-
-// IsDefaultDeny returns true if the error is defaultDeny
-func IsDefaultDeny(err error) bool {
-	if err == nil {
-		return false
-	}
-
-	_, ok := err.(defaultDeny)
-	return ok
-}
-
-func newAdmissionWaiter(a admission.Attributes) *admissionWaiter {
-	return &admissionWaiter{
-		attributes: a,
-		finished:   make(chan struct{}),
-		result:     defaultDeny{},
-	}
-}
-
-// NewQuotaEvaluator configures an admission controller that can enforce quota constraints
-// using the provided registry.  The registry must have the capability to handle group/kinds that
-// are persisted by the server this admission controller is intercepting
-func NewQuotaEvaluator(quotaAccessor QuotaAccessor, ignoredResources map[schema.GroupResource]struct{}, quotaRegistry quota.Registry, lockAcquisitionFunc func([]corev1.ResourceQuota) func(), config *resourcequotaapi.Configuration, workers int, stopCh <-chan struct{}) Evaluator {
-	// if we get a nil config, just create an empty default.
-	if config == nil {
-		config = &resourcequotaapi.Configuration{}
-	}
-
-	return &quotaEvaluator{
-		quotaAccessor:       quotaAccessor,
-		lockAcquisitionFunc: lockAcquisitionFunc,
-
-		ignoredResources: ignoredResources,
-		registry:         quotaRegistry,
-
-		queue:      workqueue.NewNamed("admission_quota_controller"),
-		work:       map[string][]*admissionWaiter{},
-		dirtyWork:  map[string][]*admissionWaiter{},
-		inProgress: sets.String{},
-
-		workers: workers,
-		stopCh:  stopCh,
-		config:  config,
-	}
-}
-
-// Run begins watching and syncing.
-func (e *quotaEvaluator) run() {
-	defer utilruntime.HandleCrash()
-
-	for i := 0; i < e.workers; i++ {
-		go wait.Until(e.doWork, time.Second, e.stopCh)
-	}
-	<-e.stopCh
-	klog.Infof("Shutting down quota evaluator")
-	e.queue.ShutDown()
-}
-
-func (e *quotaEvaluator) doWork() {
-	workFunc := func() bool {
-		ns, admissionAttributes, quit := e.getWork()
-		if quit {
-			return true
-		}
-		defer e.completeWork(ns)
-		if len(admissionAttributes) == 0 {
-			return false
-		}
-		e.checkAttributes(ns, admissionAttributes)
-		return false
-	}
-	for {
-		if quit := workFunc(); quit {
-			klog.Infof("quota evaluator worker shutdown")
-			return
-		}
-	}
-}
-
-// checkAttributes iterates evaluates all the waiting admissionAttributes.  It will always notify all waiters
-// before returning.  The default is to deny.
-func (e *quotaEvaluator) checkAttributes(ns string, admissionAttributes []*admissionWaiter) {
-	// notify all on exit
-	defer func() {
-		for _, admissionAttribute := range admissionAttributes {
-			close(admissionAttribute.finished)
-		}
-	}()
-
-	quotas, err := e.quotaAccessor.GetQuotas(ns)
-	if err != nil {
-		for _, admissionAttribute := range admissionAttributes {
-			admissionAttribute.result = err
-		}
-		return
-	}
-	// if limited resources are disabled, we can just return safely when there are no quotas.
-	limitedResourcesDisabled := len(e.config.LimitedResources) == 0
-	if len(quotas) == 0 && limitedResourcesDisabled {
-		for _, admissionAttribute := range admissionAttributes {
-			admissionAttribute.result = nil
-		}
-		return
-	}
-
-	if e.lockAcquisitionFunc != nil {
-		releaseLocks := e.lockAcquisitionFunc(quotas)
-		defer releaseLocks()
-	}
-
-	e.checkQuotas(quotas, admissionAttributes, 3)
-}
-
-// checkQuotas checks the admission attributes against the passed quotas.  If a quota applies, it will attempt to update it
-// AFTER it has checked all the admissionAttributes.  The method breaks down into phase like this:
-// 0. make a copy of the quotas to act as a "running" quota so we know what we need to update and can still compare against the
-//    originals
-// 1. check each admission attribute to see if it fits within *all* the quotas.  If it doesn't fit, mark the waiter as failed
-//    and the running quota don't change.  If it did fit, check to see if any quota was changed.  It there was no quota change
-//    mark the waiter as succeeded.  If some quota did change, update the running quotas
-// 2. If no running quota was changed, return now since no updates are needed.
-// 3. for each quota that has changed, attempt an update.  If all updates succeeded, update all unset waiters to success status and return.  If the some
-//    updates failed on conflict errors and we have retries left, re-get the failed quota from our cache for the latest version
-//    and recurse into this method with the subset.  It's safe for us to evaluate ONLY the subset, because the other quota
-//    documents for these waiters have already been evaluated.  Step 1, will mark all the ones that should already have succeeded.
-func (e *quotaEvaluator) checkQuotas(quotas []corev1.ResourceQuota, admissionAttributes []*admissionWaiter, remainingRetries int) {
-	// yet another copy to compare against originals to see if we actually have deltas
-	originalQuotas, err := copyQuotas(quotas)
-	if err != nil {
-		utilruntime.HandleError(err)
-		return
-	}
-
-	atLeastOneChanged := false
-	for i := range admissionAttributes {
-		admissionAttribute := admissionAttributes[i]
-		newQuotas, err := e.checkRequest(quotas, admissionAttribute.attributes)
-		if err != nil {
-			admissionAttribute.result = err
-			continue
-		}
-
-		// Don't update quota for admissionAttributes that correspond to dry-run requests
-		if admissionAttribute.attributes.IsDryRun() {
-			admissionAttribute.result = nil
-			continue
-		}
-
-		// if the new quotas are the same as the old quotas, then this particular one doesn't issue any updates
-		// that means that no quota docs applied, so it can get a pass
-		atLeastOneChangeForThisWaiter := false
-		for j := range newQuotas {
-			if !quota.Equals(quotas[j].Status.Used, newQuotas[j].Status.Used) {
-				atLeastOneChanged = true
-				atLeastOneChangeForThisWaiter = true
-				break
-			}
-		}
-
-		if !atLeastOneChangeForThisWaiter {
-			admissionAttribute.result = nil
-		}
-
-		quotas = newQuotas
-	}
-
-	// if none of the requests changed anything, there's no reason to issue an update, just fail them all now
-	if !atLeastOneChanged {
-		return
-	}
-
-	// now go through and try to issue updates.  Things get a little weird here:
-	// 1. check to see if the quota changed.  If not, skip.
-	// 2. if the quota changed and the update passes, be happy
-	// 3. if the quota changed and the update fails, add the original to a retry list
-	var updatedFailedQuotas []corev1.ResourceQuota
-	var lastErr error
-	for i := range quotas {
-		newQuota := quotas[i]
-
-		// if this quota didn't have its status changed, skip it
-		if quota.Equals(originalQuotas[i].Status.Used, newQuota.Status.Used) {
-			continue
-		}
-
-		if err := e.quotaAccessor.UpdateQuotaStatus(&newQuota); err != nil {
-			updatedFailedQuotas = append(updatedFailedQuotas, newQuota)
-			lastErr = err
-		}
-	}
-
-	if len(updatedFailedQuotas) == 0 {
-		// all the updates succeeded.  At this point, anything with the default deny error was just waiting to
-		// get a successful update, so we can mark and notify
-		for _, admissionAttribute := range admissionAttributes {
-			if IsDefaultDeny(admissionAttribute.result) {
-				admissionAttribute.result = nil
-			}
-		}
-		return
-	}
-
-	// at this point, errors are fatal.  Update all waiters without status to failed and return
-	if remainingRetries <= 0 {
-		for _, admissionAttribute := range admissionAttributes {
-			if IsDefaultDeny(admissionAttribute.result) {
-				admissionAttribute.result = lastErr
-			}
-		}
-		return
-	}
-
-	// this retry logic has the same bug that its possible to be checking against quota in a state that never actually exists where
-	// you've added a new documented, then updated an old one, your resource matches both and you're only checking one
-	// updates for these quota names failed.  Get the current quotas in the namespace, compare by name, check to see if the
-	// resource versions have changed.  If not, we're going to fall through an fail everything.  If they all have, then we can try again
-	newQuotas, err := e.quotaAccessor.GetQuotas(quotas[0].Namespace)
-	if err != nil {
-		// this means that updates failed.  Anything with a default deny error has failed and we need to let them know
-		for _, admissionAttribute := range admissionAttributes {
-			if IsDefaultDeny(admissionAttribute.result) {
-				admissionAttribute.result = lastErr
-			}
-		}
-		return
-	}
-
-	// this logic goes through our cache to find the new version of all quotas that failed update.  If something has been removed
-	// it is skipped on this retry.  After all, you removed it.
-	quotasToCheck := []corev1.ResourceQuota{}
-	for _, newQuota := range newQuotas {
-		for _, oldQuota := range updatedFailedQuotas {
-			if newQuota.Name == oldQuota.Name {
-				quotasToCheck = append(quotasToCheck, newQuota)
-				break
-			}
-		}
-	}
-	e.checkQuotas(quotasToCheck, admissionAttributes, remainingRetries-1)
-}
-
-func copyQuotas(in []corev1.ResourceQuota) ([]corev1.ResourceQuota, error) {
-	out := make([]corev1.ResourceQuota, 0, len(in))
-	for _, quota := range in {
-		out = append(out, *quota.DeepCopy())
-	}
-
-	return out, nil
-}
-
-// filterLimitedResourcesByGroupResource filters the input that match the specified groupResource
-func filterLimitedResourcesByGroupResource(input []resourcequotaapi.LimitedResource, groupResource schema.GroupResource) []resourcequotaapi.LimitedResource {
-	result := []resourcequotaapi.LimitedResource{}
-	for i := range input {
-		limitedResource := input[i]
-		limitedGroupResource := schema.GroupResource{Group: limitedResource.APIGroup, Resource: limitedResource.Resource}
-		if limitedGroupResource == groupResource {
-			result = append(result, limitedResource)
-		}
-	}
-	return result
-}
-
-// limitedByDefault determines from the specified usage and limitedResources the set of resources names
-// that must be present in a covering quota.  It returns empty set if it was unable to determine if
-// a resource was not limited by default.
-func limitedByDefault(usage corev1.ResourceList, limitedResources []resourcequotaapi.LimitedResource) []corev1.ResourceName {
-	result := []corev1.ResourceName{}
-	for _, limitedResource := range limitedResources {
-		for k, v := range usage {
-			// if a resource is consumed, we need to check if it matches on the limited resource list.
-			if v.Sign() == 1 {
-				// if we get a match, we add it to limited set
-				for _, matchContain := range limitedResource.MatchContains {
-					if strings.Contains(string(k), matchContain) {
-						result = append(result, k)
-						break
-					}
-				}
-			}
-		}
-	}
-	return result
-}
-
-func getMatchedLimitedScopes(evaluator quota.Evaluator, inputObject runtime.Object, limitedResources []resourcequotaapi.LimitedResource) ([]corev1.ScopedResourceSelectorRequirement, error) {
-	scopes := []corev1.ScopedResourceSelectorRequirement{}
-	for _, limitedResource := range limitedResources {
-		matched, err := evaluator.MatchingScopes(inputObject, limitedResource.MatchScopes)
-		if err != nil {
-			klog.Errorf("Error while matching limited Scopes: %v", err)
-			return []corev1.ScopedResourceSelectorRequirement{}, err
-		}
-		for _, scope := range matched {
-			scopes = append(scopes, scope)
-		}
-	}
-	return scopes, nil
-}
-
-// checkRequest verifies that the request does not exceed any quota constraint. it returns a copy of quotas not yet persisted
-// that capture what the usage would be if the request succeeded.  It return an error if there is insufficient quota to satisfy the request
-func (e *quotaEvaluator) checkRequest(quotas []corev1.ResourceQuota, a admission.Attributes) ([]corev1.ResourceQuota, error) {
-	evaluator := e.registry.Get(a.GetResource().GroupResource())
-	if evaluator == nil {
-		return quotas, nil
-	}
-	return CheckRequest(quotas, a, evaluator, e.config.LimitedResources)
-}
-
-// CheckRequest is a static version of quotaEvaluator.checkRequest, possible to be called from outside.
-func CheckRequest(quotas []corev1.ResourceQuota, a admission.Attributes, evaluator quota.Evaluator,
-	limited []resourcequotaapi.LimitedResource) ([]corev1.ResourceQuota, error) {
-	if !evaluator.Handles(a) {
-		return quotas, nil
-	}
-
-	// if we have limited resources enabled for this resource, always calculate usage
-	inputObject := a.GetObject()
-
-	// Check if object matches AdmissionConfiguration matchScopes
-	limitedScopes, err := getMatchedLimitedScopes(evaluator, inputObject, limited)
-	if err != nil {
-		return quotas, nil
-	}
-
-	// determine the set of resource names that must exist in a covering quota
-	limitedResourceNames := []corev1.ResourceName{}
-	limitedResources := filterLimitedResourcesByGroupResource(limited, a.GetResource().GroupResource())
-	if len(limitedResources) > 0 {
-		deltaUsage, err := evaluator.Usage(inputObject)
-		if err != nil {
-			return quotas, err
-		}
-		limitedResourceNames = limitedByDefault(deltaUsage, limitedResources)
-	}
-	limitedResourceNamesSet := quota.ToSet(limitedResourceNames)
-
-	// find the set of quotas that are pertinent to this request
-	// reject if we match the quota, but usage is not calculated yet
-	// reject if the input object does not satisfy quota constraints
-	// if there are no pertinent quotas, we can just return
-	interestingQuotaIndexes := []int{}
-	// track the cumulative set of resources that were required across all quotas
-	// this is needed to know if we have satisfied any constraints where consumption
-	// was limited by default.
-	restrictedResourcesSet := sets.String{}
-	restrictedScopes := []corev1.ScopedResourceSelectorRequirement{}
-	for i := range quotas {
-		resourceQuota := quotas[i]
-		scopeSelectors := getScopeSelectorsFromQuota(resourceQuota)
-		localRestrictedScopes, err := evaluator.MatchingScopes(inputObject, scopeSelectors)
-		if err != nil {
-			return nil, fmt.Errorf("error matching scopes of quota %s, err: %v", resourceQuota.Name, err)
-		}
-		for _, scope := range localRestrictedScopes {
-			restrictedScopes = append(restrictedScopes, scope)
-		}
-
-		match, err := evaluator.Matches(&resourceQuota, inputObject)
-		if err != nil {
-			klog.Errorf("Error occurred while matching resource quota, %v, against input object. Err: %v", resourceQuota, err)
-			return quotas, err
-		}
-		if !match {
-			continue
-		}
-
-		hardResources := quota.ResourceNames(resourceQuota.Status.Hard)
-		restrictedResources := evaluator.MatchingResources(hardResources)
-		if err := evaluator.Constraints(restrictedResources, inputObject); err != nil {
-			return nil, admission.NewForbidden(a, fmt.Errorf("failed quota: %s: %v", resourceQuota.Name, err))
-		}
-		if !hasUsageStats(&resourceQuota, restrictedResources) {
-			return nil, admission.NewForbidden(a, fmt.Errorf("status unknown for quota: %s, resources: %s", resourceQuota.Name, prettyPrintResourceNames(restrictedResources)))
-		}
-		interestingQuotaIndexes = append(interestingQuotaIndexes, i)
-		localRestrictedResourcesSet := quota.ToSet(restrictedResources)
-		restrictedResourcesSet.Insert(localRestrictedResourcesSet.List()...)
-	}
-
-	// Usage of some resources cannot be counted in isolation. For example, when
-	// the resource represents a number of unique references to external
-	// resource. In such a case an evaluator needs to process other objects in
-	// the same namespace which needs to be known.
-	namespace := a.GetNamespace()
-	if accessor, err := meta.Accessor(inputObject); namespace != "" && err == nil {
-		if accessor.GetNamespace() == "" {
-			accessor.SetNamespace(namespace)
-		}
-	}
-	// there is at least one quota that definitely matches our object
-	// as a result, we need to measure the usage of this object for quota
-	// on updates, we need to subtract the previous measured usage
-	// if usage shows no change, just return since it has no impact on quota
-	deltaUsage, err := evaluator.Usage(inputObject)
-	if err != nil {
-		return quotas, err
-	}
-
-	// ensure that usage for input object is never negative (this would mean a resource made a negative resource requirement)
-	if negativeUsage := quota.IsNegative(deltaUsage); len(negativeUsage) > 0 {
-		return nil, admission.NewForbidden(a, fmt.Errorf("quota usage is negative for resource(s): %s", prettyPrintResourceNames(negativeUsage)))
-	}
-
-	if admission.Update == a.GetOperation() {
-		prevItem := a.GetOldObject()
-		if prevItem == nil {
-			return nil, admission.NewForbidden(a, fmt.Errorf("unable to get previous usage since prior version of object was not found"))
-		}
-
-		// if we can definitively determine that this is not a case of "create on update",
-		// then charge based on the delta.  Otherwise, bill the maximum
-		metadata, err := meta.Accessor(prevItem)
-		if err == nil && len(metadata.GetResourceVersion()) > 0 {
-			prevUsage, innerErr := evaluator.Usage(prevItem)
-			if innerErr != nil {
-				return quotas, innerErr
-			}
-			deltaUsage = quota.SubtractWithNonNegativeResult(deltaUsage, prevUsage)
-		}
-	}
-
-	if quota.IsZero(deltaUsage) {
-		return quotas, nil
-	}
-
-	// verify that for every resource that had limited by default consumption
-	// enabled that there was a corresponding quota that covered its use.
-	// if not, we reject the request.
-	hasNoCoveringQuota := limitedResourceNamesSet.Difference(restrictedResourcesSet)
-	if len(hasNoCoveringQuota) > 0 {
-		return quotas, admission.NewForbidden(a, fmt.Errorf("insufficient quota to consume: %v", strings.Join(hasNoCoveringQuota.List(), ",")))
-	}
-
-	// verify that for every scope that had limited access enabled
-	// that there was a corresponding quota that covered it.
-	// if not, we reject the request.
-	scopesHasNoCoveringQuota, err := evaluator.UncoveredQuotaScopes(limitedScopes, restrictedScopes)
-	if err != nil {
-		return quotas, err
-	}
-	if len(scopesHasNoCoveringQuota) > 0 {
-		return quotas, fmt.Errorf("insufficient quota to match these scopes: %v", scopesHasNoCoveringQuota)
-	}
-
-	if len(interestingQuotaIndexes) == 0 {
-		return quotas, nil
-	}
-
-	outQuotas, err := copyQuotas(quotas)
-	if err != nil {
-		return nil, err
-	}
-
-	for _, index := range interestingQuotaIndexes {
-		resourceQuota := outQuotas[index]
-
-		hardResources := quota.ResourceNames(resourceQuota.Status.Hard)
-		requestedUsage := quota.Mask(deltaUsage, hardResources)
-		newUsage := quota.Add(resourceQuota.Status.Used, requestedUsage)
-		maskedNewUsage := quota.Mask(newUsage, quota.ResourceNames(requestedUsage))
-
-		if allowed, exceeded := quota.LessThanOrEqual(maskedNewUsage, resourceQuota.Status.Hard); !allowed {
-			failedRequestedUsage := quota.Mask(requestedUsage, exceeded)
-			failedUsed := quota.Mask(resourceQuota.Status.Used, exceeded)
-			failedHard := quota.Mask(resourceQuota.Status.Hard, exceeded)
-			return nil, admission.NewForbidden(a,
-				fmt.Errorf("exceeded quota: %s, requested: %s, used: %s, limited: %s",
-					resourceQuota.Name,
-					prettyPrint(failedRequestedUsage),
-					prettyPrint(failedUsed),
-					prettyPrint(failedHard)))
-		}
-
-		// update to the new usage number
-		outQuotas[index].Status.Used = newUsage
-	}
-
-	return outQuotas, nil
-}
-
-func getScopeSelectorsFromQuota(quota corev1.ResourceQuota) []corev1.ScopedResourceSelectorRequirement {
-	selectors := []corev1.ScopedResourceSelectorRequirement{}
-	for _, scope := range quota.Spec.Scopes {
-		selectors = append(selectors, corev1.ScopedResourceSelectorRequirement{
-			ScopeName: scope,
-			Operator:  corev1.ScopeSelectorOpExists})
-	}
-	if quota.Spec.ScopeSelector != nil {
-		for _, scopeSelector := range quota.Spec.ScopeSelector.MatchExpressions {
-			selectors = append(selectors, scopeSelector)
-		}
-	}
-	return selectors
-}
-
-func (e *quotaEvaluator) Evaluate(a admission.Attributes) error {
-	e.init.Do(func() {
-		go e.run()
-	})
-
-	// is this resource ignored?
-	gvr := a.GetResource()
-	gr := gvr.GroupResource()
-	if _, ok := e.ignoredResources[gr]; ok {
-		return nil
-	}
-
-	// if we do not know how to evaluate use for this resource, create an evaluator
-	evaluator := e.registry.Get(gr)
-	if evaluator == nil {
-		// create an object count evaluator if no evaluator previously registered
-		// note, we do not need aggregate usage here, so we pass a nil informer func
-		evaluator = generic.NewObjectCountEvaluator(gr, nil, "")
-		e.registry.Add(evaluator)
-		klog.Infof("quota admission added evaluator for: %s", gr)
-	}
-	// for this kind, check if the operation could mutate any quota resources
-	// if no resources tracked by quota are impacted, then just return
-	if !evaluator.Handles(a) {
-		return nil
-	}
-	waiter := newAdmissionWaiter(a)
-
-	e.addWork(waiter)
-
-	// wait for completion or timeout
-	select {
-	case <-waiter.finished:
-	case <-time.After(10 * time.Second):
-		return apierrors.NewInternalError(fmt.Errorf("resource quota evaluates timeout"))
-	}
-
-	return waiter.result
-}
-
-func (e *quotaEvaluator) addWork(a *admissionWaiter) {
-	e.workLock.Lock()
-	defer e.workLock.Unlock()
-
-	ns := a.attributes.GetNamespace()
-	// this Add can trigger a Get BEFORE the work is added to a list, but this is ok because the getWork routine
-	// waits the worklock before retrieving the work to do, so the writes in this method will be observed
-	e.queue.Add(ns)
-
-	if e.inProgress.Has(ns) {
-		e.dirtyWork[ns] = append(e.dirtyWork[ns], a)
-		return
-	}
-
-	e.work[ns] = append(e.work[ns], a)
-}
-
-func (e *quotaEvaluator) completeWork(ns string) {
-	e.workLock.Lock()
-	defer e.workLock.Unlock()
-
-	e.queue.Done(ns)
-	e.work[ns] = e.dirtyWork[ns]
-	delete(e.dirtyWork, ns)
-	e.inProgress.Delete(ns)
-}
-
-// getWork returns a namespace, a list of work items in that
-// namespace, and a shutdown boolean.  If not shutdown then the return
-// must eventually be followed by a call on completeWork for the
-// returned namespace (regardless of whether the work item list is
-// empty).
-func (e *quotaEvaluator) getWork() (string, []*admissionWaiter, bool) {
-	uncastNS, shutdown := e.queue.Get()
-	if shutdown {
-		return "", []*admissionWaiter{}, shutdown
-	}
-	ns := uncastNS.(string)
-
-	e.workLock.Lock()
-	defer e.workLock.Unlock()
-	// at this point, we know we have a coherent view of e.work.  It is entirely possible
-	// that our workqueue has another item requeued to it, but we'll pick it up early.  This ok
-	// because the next time will go into our dirty list
-
-	work := e.work[ns]
-	delete(e.work, ns)
-	delete(e.dirtyWork, ns)
-	e.inProgress.Insert(ns)
-	return ns, work, false
-}
-
-// prettyPrint formats a resource list for usage in errors
-// it outputs resources sorted in increasing order
-func prettyPrint(item corev1.ResourceList) string {
-	parts := []string{}
-	keys := []string{}
-	for key := range item {
-		keys = append(keys, string(key))
-	}
-	sort.Strings(keys)
-	for _, key := range keys {
-		value := item[corev1.ResourceName(key)]
-		constraint := key + "=" + value.String()
-		parts = append(parts, constraint)
-	}
-	return strings.Join(parts, ",")
-}
-
-func prettyPrintResourceNames(a []corev1.ResourceName) string {
-	values := []string{}
-	for _, value := range a {
-		values = append(values, string(value))
-	}
-	sort.Strings(values)
-	return strings.Join(values, ",")
-}
-
-// hasUsageStats returns true if for each hard constraint in interestingResources there is a value for its current usage
-func hasUsageStats(resourceQuota *corev1.ResourceQuota, interestingResources []corev1.ResourceName) bool {
-	interestingSet := quota.ToSet(interestingResources)
-	for resourceName := range resourceQuota.Status.Hard {
-		if !interestingSet.Has(string(resourceName)) {
-			continue
-		}
-		if _, found := resourceQuota.Status.Used[resourceName]; !found {
-			return false
-		}
-	}
-	return true
-}
--- a/kube/plugin/pkg/admission/resourcequota/resource_access.go
+++ b/kube/plugin/pkg/admission/resourcequota/resource_access.go
@@ -1,155 +0,0 @@
-/*
-Copyright 2016 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package resourcequota
-
-import (
-	"context"
-	"fmt"
-	"time"
-
-	lru "github.com/hashicorp/golang-lru"
-
-	corev1 "k8s.io/api/core/v1"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apiserver/pkg/storage/etcd3"
-	"k8s.io/client-go/kubernetes"
-	corev1listers "k8s.io/client-go/listers/core/v1"
-)
-
-// QuotaAccessor abstracts the get/set logic from the rest of the Evaluator.  This could be a test stub, a straight passthrough,
-// or most commonly a series of deconflicting caches.
-type QuotaAccessor interface {
-	// UpdateQuotaStatus is called to persist final status.  This method should write to persistent storage.
-	// An error indicates that write didn't complete successfully.
-	UpdateQuotaStatus(newQuota *corev1.ResourceQuota) error
-
-	// GetQuotas gets all possible quotas for a given namespace
-	GetQuotas(namespace string) ([]corev1.ResourceQuota, error)
-}
-
-type quotaAccessor struct {
-	client kubernetes.Interface
-
-	// lister can list/get quota objects from a shared informer's cache
-	lister corev1listers.ResourceQuotaLister
-
-	// liveLookups holds the last few live lookups we've done to help ammortize cost on repeated lookup failures.
-	// This lets us handle the case of latent caches, by looking up actual results for a namespace on cache miss/no results.
-	// We track the lookup result here so that for repeated requests, we don't look it up very often.
-	liveLookupCache *lru.Cache
-	liveTTL         time.Duration
-	// updatedQuotas holds a cache of quotas that we've updated.  This is used to pull the "really latest" during back to
-	// back quota evaluations that touch the same quota doc.  This only works because we can compare etcd resourceVersions
-	// for the same resource as integers.  Before this change: 22 updates with 12 conflicts.  after this change: 15 updates with 0 conflicts
-	updatedQuotas *lru.Cache
-}
-
-// newQuotaAccessor creates an object that conforms to the QuotaAccessor interface to be used to retrieve quota objects.
-func newQuotaAccessor() (*quotaAccessor, error) {
-	liveLookupCache, err := lru.New(100)
-	if err != nil {
-		return nil, err
-	}
-	updatedCache, err := lru.New(100)
-	if err != nil {
-		return nil, err
-	}
-
-	// client and lister will be set when SetInternalKubeClientSet and SetInternalKubeInformerFactory are invoked
-	return &quotaAccessor{
-		liveLookupCache: liveLookupCache,
-		liveTTL:         time.Duration(30 * time.Second),
-		updatedQuotas:   updatedCache,
-	}, nil
-}
-
-func (e *quotaAccessor) UpdateQuotaStatus(newQuota *corev1.ResourceQuota) error {
-	updatedQuota, err := e.client.CoreV1().ResourceQuotas(newQuota.Namespace).UpdateStatus(context.TODO(), newQuota, metav1.UpdateOptions{})
-	if err != nil {
-		return err
-	}
-
-	key := newQuota.Namespace + "/" + newQuota.Name
-	e.updatedQuotas.Add(key, updatedQuota)
-	return nil
-}
-
-var etcdVersioner = etcd3.APIObjectVersioner{}
-
-// checkCache compares the passed quota against the value in the look-aside cache and returns the newer
-// if the cache is out of date, it deletes the stale entry.  This only works because of etcd resourceVersions
-// being monotonically increasing integers
-func (e *quotaAccessor) checkCache(quota *corev1.ResourceQuota) *corev1.ResourceQuota {
-	key := quota.Namespace + "/" + quota.Name
-	uncastCachedQuota, ok := e.updatedQuotas.Get(key)
-	if !ok {
-		return quota
-	}
-	cachedQuota := uncastCachedQuota.(*corev1.ResourceQuota)
-
-	if etcdVersioner.CompareResourceVersion(quota, cachedQuota) >= 0 {
-		e.updatedQuotas.Remove(key)
-		return quota
-	}
-	return cachedQuota
-}
-
-func (e *quotaAccessor) GetQuotas(namespace string) ([]corev1.ResourceQuota, error) {
-	// determine if there are any quotas in this namespace
-	// if there are no quotas, we don't need to do anything
-	items, err := e.lister.ResourceQuotas(namespace).List(labels.Everything())
-	if err != nil {
-		return nil, fmt.Errorf("error resolving quota: %v", err)
-	}
-
-	// if there are no items held in our indexer, check our live-lookup LRU, if that misses, do the live lookup to prime it.
-	if len(items) == 0 {
-		lruItemObj, ok := e.liveLookupCache.Get(namespace)
-		if !ok || lruItemObj.(liveLookupEntry).expiry.Before(time.Now()) {
-			// TODO: If there are multiple operations at the same time and cache has just expired,
-			// this may cause multiple List operations being issued at the same time.
-			// If there is already in-flight List() for a given namespace, we should wait until
-			// it is finished and cache is updated instead of doing the same, also to avoid
-			// throttling - see #22422 for details.
-			liveList, err := e.client.CoreV1().ResourceQuotas(namespace).List(context.TODO(), metav1.ListOptions{})
-			if err != nil {
-				return nil, err
-			}
-			newEntry := liveLookupEntry{expiry: time.Now().Add(e.liveTTL)}
-			for i := range liveList.Items {
-				newEntry.items = append(newEntry.items, &liveList.Items[i])
-			}
-			e.liveLookupCache.Add(namespace, newEntry)
-			lruItemObj = newEntry
-		}
-		lruEntry := lruItemObj.(liveLookupEntry)
-		for i := range lruEntry.items {
-			items = append(items, lruEntry.items[i])
-		}
-	}
-
-	resourceQuotas := []corev1.ResourceQuota{}
-	for i := range items {
-		quota := items[i]
-		quota = e.checkCache(quota)
-		// always make a copy.  We're going to muck around with this and we should never mutate the originals
-		resourceQuotas = append(resourceQuotas, *quota)
-	}
-
-	return resourceQuotas, nil
-}
--- a/pkg/api/alerting/v2alpha1/types.go
+++ b/pkg/api/alerting/v2alpha1/types.go
@@ -1,535 +0,0 @@
-/*
-Copyright 2020 KubeSphere Authors
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package v2alpha1
-
-import (
-	"context"
-	"regexp"
-	"sort"
-	"strconv"
-	"strings"
-	"time"
-
-	"github.com/emicklei/go-restful"
-	"github.com/pkg/errors"
-	prommodel "github.com/prometheus/common/model"
-	"github.com/prometheus/prometheus/pkg/timestamp"
-	"github.com/prometheus/prometheus/promql/parser"
-	"github.com/prometheus/prometheus/template"
-	utilerrors "k8s.io/apimachinery/pkg/util/errors"
-)
-
-const (
-	RuleLevelCluster   RuleLevel = "cluster"
-	RuleLevelNamespace RuleLevel = "namespace"
-
-	AnnotationKeyRuleUpdateTime = "rule_update_time"
-)
-
-var (
-	ErrThanosRulerNotEnabled     = errors.New("The request operation to custom alerting rule could not be done because thanos ruler is not enabled")
-	ErrAlertingRuleNotFound      = errors.New("The alerting rule was not found")
-	ErrAlertingRuleAlreadyExists = errors.New("The alerting rule already exists")
-	ErrAlertingAPIV2NotEnabled   = errors.New("The alerting v2 API is not enabled")
-
-	templateTestData       = template.AlertTemplateData(map[string]string{}, map[string]string{}, 0)
-	templateTestTextPrefix = "{{$labels := .Labels}}{{$externalLabels := .ExternalLabels}}{{$value := .Value}}"
-
-	ruleNameMatcher = regexp.MustCompile(`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`)
-)
-
-type RuleLevel string
-
-type AlertingRule struct {
-	Id   string `json:"id,omitempty" description:"rule id is only used by built-in alerting rules"`
-	Name string `json:"name,omitempty" description:"rule name should be unique in one namespace for custom alerting rules"`
-
-	Query       string            `json:"query,omitempty" description:"prometheus query expression, grammars of which may be referred to https://prometheus.io/docs/prometheus/latest/querying/basics/"`
-	Duration    string            `json:"duration,omitempty" description:"duration an alert transitions from Pending to Firing state, which must match ^([0-9]+)(y|w|d|h|m|s|ms)$"`
-	Labels      map[string]string `json:"labels,omitempty" description:"extra labels to attach to the resulting alert sample vectors (the key string has to match [a-zA-Z_][a-zA-Z0-9_]*). eg: a typical label called severity, whose value may be info, warning, error, critical, is usually used to indicate the severity of an alert"`
-	Annotations map[string]string `json:"annotations,omitempty" description:"non-identifying key/value pairs. summary, message, description are the commonly used annotation names"`
-}
-
-type PostableAlertingRule struct {
-	AlertingRule `json:",omitempty"`
-}
-
-func (r *PostableAlertingRule) Validate() error {
-	errs := []error{}
-
-	if r.Name == "" {
-		errs = append(errs, errors.New("name can not be empty"))
-	} else {
-		if !ruleNameMatcher.MatchString(r.Name) {
-			errs = append(errs, errors.New("rule name must match regular expression ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$"))
-		}
-	}
-
-	if r.Query == "" {
-		errs = append(errs, errors.New("query can not be empty"))
-	} else if _, err := parser.ParseExpr(r.Query); err != nil {
-		errs = append(errs, errors.Wrapf(err, "query is invalid: %s", r.Query))
-	}
-	if r.Duration != "" {
-		if _, err := prommodel.ParseDuration(r.Duration); err != nil {
-			errs = append(errs, errors.Wrapf(err, "duration is invalid: %s", r.Duration))
-		}
-	}
-
-	parseTest := func(text string) error {
-		tmpl := template.NewTemplateExpander(
-			context.TODO(),
-			templateTestTextPrefix+text,
-			"__alert_"+r.Name,
-			templateTestData,
-			prommodel.Time(timestamp.FromTime(time.Now())),
-			nil,
-			nil,
-		)
-		return tmpl.ParseTest()
-	}
-
-	if len(r.Labels) > 0 {
-		for name, v := range r.Labels {
-			if !prommodel.LabelName(name).IsValid() || strings.HasPrefix(name, "__") {
-				errs = append(errs, errors.Errorf(
-					"label name (%s) is not valid. The name must match [a-zA-Z_][a-zA-Z0-9_]* and has not the __ prefix (label names with this prefix are for internal use)", name))
-			}
-			if !prommodel.LabelValue(v).IsValid() {
-				errs = append(errs, errors.Errorf("invalid label value: %s", v))
-			}
-			if err := parseTest(v); err != nil {
-				errs = append(errs, errors.Errorf("invalid label value: %s", v))
-			}
-		}
-	}
-
-	if len(r.Annotations) > 0 {
-		for name, v := range r.Annotations {
-			if !prommodel.LabelName(name).IsValid() {
-				errs = append(errs, errors.Errorf("invalid annotation name: %s", v))
-			}
-			if err := parseTest(v); err != nil {
-				errs = append(errs, errors.Errorf("invalid annotation value: %s", v))
-			}
-		}
-	}
-
-	return utilerrors.NewAggregate(errs)
-}
-
-type GettableAlertingRule struct {
-	AlertingRule `json:",omitempty"`
-
-	State                     string     `json:"state,omitempty" description:"state of a rule based on its alerts, one of firing, pending, inactive"`
-	Health                    string     `json:"health,omitempty" description:"health state of a rule based on the last execution, one of ok, err, unknown"`
-	LastError                 string     `json:"lastError,omitempty" description:"error for the last execution"`
-	EvaluationDurationSeconds float64    `json:"evaluationTime,omitempty" description:"taken seconds for evaluation of query expression"`
-	LastEvaluation            *time.Time `json:"lastEvaluation,omitempty" description:"time for last evaluation of query expression"`
-
-	Alerts []*Alert `json:"alerts,omitempty" description:"alerts"`
-}
-
-type GettableAlertingRuleList struct {
-	Items []*GettableAlertingRule `json:"items"`
-	Total int                     `json:"total"`
-}
-
-type Alert struct {
-	ActiveAt    *time.Time        `json:"activeAt,omitempty" description:"time when alert is active"`
-	Annotations map[string]string `json:"annotations,omitempty" description:"annotations"`
-	Labels      map[string]string `json:"labels,omitempty" description:"labels"`
-	State       string            `json:"state,omitempty" description:"state"`
-	Value       string            `json:"value,omitempty" description:"the value at the last evaluation of the query expression"`
-
-	RuleId   string `json:"ruleId,omitempty" description:"rule id triggering the alert"`
-	RuleName string `json:"ruleName,omitempty" description:"rule name triggering the alert"`
-}
-
-type AlertList struct {
-	Items []*Alert `json:"items"`
-	Total int      `json:"total"`
-}
-
-type AlertingRuleQueryParams struct {
-	NameContainFilter   string
-	State               string
-	Health              string
-	LabelEqualFilters   map[string]string
-	LabelContainFilters map[string]string
-
-	PageNum   int
-	Limit     int
-	SortField string
-	SortType  string
-}
-
-func (q *AlertingRuleQueryParams) Filter(rules []*GettableAlertingRule) []*GettableAlertingRule {
-	var ret []*GettableAlertingRule
-	for _, rule := range rules {
-		if rule == nil {
-			continue
-		}
-		if q == nil || q.matches(rule) {
-			ret = append(ret, rule)
-		}
-	}
-	return ret
-}
-
-func (q *AlertingRuleQueryParams) matches(rule *GettableAlertingRule) bool {
-	if q.NameContainFilter != "" && !containsCaseInsensitive(rule.Name, q.NameContainFilter) {
-		return false
-	}
-	if q.State != "" && q.State != rule.State {
-		return false
-	}
-	if q.Health != "" && q.Health != rule.Health {
-		return false
-	}
-	if len(rule.Labels) == 0 {
-		return len(q.LabelEqualFilters) == 0 && len(q.LabelContainFilters) == 0
-	}
-	for k, v := range q.LabelEqualFilters {
-		if fv, ok := rule.Labels[k]; !ok || fv != v {
-			return false
-		}
-	}
-	for k, v := range q.LabelContainFilters {
-		if fv, ok := rule.Labels[k]; !ok || !containsCaseInsensitive(fv, v) {
-			return false
-		}
-	}
-	return true
-}
-
-// AlertingRuleIdCompare defines the default order for the alerting rules.
-// For the alerting rule list, it guarantees a stable sort. For the custom alerting rules with possible same names
-// and the builtin alerting rules with possible same ids, it guarantees the stability of get operations.
-func AlertingRuleIdCompare(leftId, rightId string) bool {
-	// default to ascending order of id
-	return leftId <= rightId
-}
-
-func (q *AlertingRuleQueryParams) Sort(rules []*GettableAlertingRule) {
-	baseCompare := func(left, right *GettableAlertingRule) bool {
-		var leftUpdateTime, rightUpdateTime string
-		if len(left.Annotations) > 0 {
-			leftUpdateTime = left.Annotations[AnnotationKeyRuleUpdateTime]
-		}
-		if len(right.Annotations) > 0 {
-			rightUpdateTime = right.Annotations[AnnotationKeyRuleUpdateTime]
-		}
-
-		if leftUpdateTime != rightUpdateTime {
-			return leftUpdateTime > rightUpdateTime
-		}
-
-		return AlertingRuleIdCompare(left.Id, right.Id)
-	}
-	var compare = baseCompare
-	if q != nil {
-		reverse := q.SortType == "desc"
-		switch q.SortField {
-		case "name":
-			compare = func(left, right *GettableAlertingRule) bool {
-				if c := strings.Compare(left.Name, right.Name); c != 0 {
-					if reverse {
-						return c > 0
-					}
-					return c < 0
-				}
-				return baseCompare(left, right)
-			}
-		case "lastEvaluation":
-			compare = func(left, right *GettableAlertingRule) bool {
-				if left.LastEvaluation == nil {
-					if right.LastEvaluation != nil {
-						return false
-					}
-				} else {
-					if right.LastEvaluation == nil {
-						return true
-					} else if left.LastEvaluation.Equal(*right.LastEvaluation) {
-						if reverse {
-							return left.LastEvaluation.After(*right.LastEvaluation)
-						}
-						return left.LastEvaluation.Before(*right.LastEvaluation)
-					}
-				}
-				return baseCompare(left, right)
-			}
-		case "evaluationTime":
-			compare = func(left, right *GettableAlertingRule) bool {
-				if left.EvaluationDurationSeconds != right.EvaluationDurationSeconds {
-					if reverse {
-						return left.EvaluationDurationSeconds > right.EvaluationDurationSeconds
-					}
-					return left.EvaluationDurationSeconds < right.EvaluationDurationSeconds
-				}
-				return baseCompare(left, right)
-			}
-		}
-	}
-	sort.Slice(rules, func(i, j int) bool {
-		return compare(rules[i], rules[j])
-	})
-}
-
-func (q *AlertingRuleQueryParams) Sub(rules []*GettableAlertingRule) []*GettableAlertingRule {
-	start, stop := 0, 10
-	if q != nil {
-		start, stop = (q.PageNum-1)*q.Limit, q.PageNum*q.Limit
-	}
-	total := len(rules)
-	if start < total {
-		if stop > total {
-			stop = total
-		}
-		return rules[start:stop]
-	}
-	return nil
-}
-
-type AlertQueryParams struct {
-	State               string
-	LabelEqualFilters   map[string]string
-	LabelContainFilters map[string]string
-
-	PageNum int
-	Limit   int
-}
-
-func (q *AlertQueryParams) Filter(alerts []*Alert) []*Alert {
-	var ret []*Alert
-	for _, alert := range alerts {
-		if alert == nil {
-			continue
-		}
-		if q == nil || q.matches(alert) {
-			ret = append(ret, alert)
-		}
-	}
-	return ret
-}
-
-func (q *AlertQueryParams) matches(alert *Alert) bool {
-	if q.State != "" && q.State != alert.State {
-		return false
-	}
-	if len(alert.Labels) == 0 {
-		return len(q.LabelEqualFilters) == 0 && len(q.LabelContainFilters) == 0
-	}
-	for k, v := range q.LabelEqualFilters {
-		if fv, ok := alert.Labels[k]; !ok || fv != v {
-			return false
-		}
-	}
-	for k, v := range q.LabelContainFilters {
-		if fv, ok := alert.Labels[k]; !ok || !containsCaseInsensitive(fv, v) {
-			return false
-		}
-	}
-	return true
-}
-
-func (q *AlertQueryParams) Sort(alerts []*Alert) {
-	compare := func(left, right *Alert) bool {
-		if left.ActiveAt == nil {
-			if right.ActiveAt != nil {
-				return false
-			}
-		} else {
-			if right.ActiveAt == nil {
-				return true
-			} else if !left.ActiveAt.Equal(*right.ActiveAt) {
-				return left.ActiveAt.After(*right.ActiveAt)
-			}
-		}
-		return prommodel.LabelsToSignature(left.Labels) <= prommodel.LabelsToSignature(right.Labels)
-	}
-	sort.Slice(alerts, func(i, j int) bool {
-		return compare(alerts[i], alerts[j])
-	})
-}
-
-func (q *AlertQueryParams) Sub(alerts []*Alert) []*Alert {
-	start, stop := 0, 10
-	if q != nil {
-		start, stop = (q.PageNum-1)*q.Limit, q.PageNum*q.Limit
-	}
-	total := len(alerts)
-	if start < total {
-		if stop > total {
-			stop = total
-		}
-		return alerts[start:stop]
-	}
-	return nil
-}
-
-func ParseAlertingRuleQueryParams(req *restful.Request) (*AlertingRuleQueryParams, error) {
-	var (
-		q   = &AlertingRuleQueryParams{}
-		err error
-	)
-
-	q.NameContainFilter = req.QueryParameter("name")
-	q.State = req.QueryParameter("state")
-	q.Health = req.QueryParameter("health")
-	q.PageNum, err = strconv.Atoi(req.QueryParameter("page"))
-	if err != nil {
-		q.PageNum = 1
-		err = nil
-	}
-	if q.PageNum <= 0 {
-		q.PageNum = 1
-	}
-	q.Limit, err = strconv.Atoi(req.QueryParameter("limit"))
-	if err != nil {
-		q.Limit = 10
-		err = nil
-	}
-	q.LabelEqualFilters, q.LabelContainFilters = parseLabelFilters(req)
-	q.SortField = req.QueryParameter("sort_field")
-	q.SortType = req.QueryParameter("sort_type")
-	return q, err
-}
-
-func ParseAlertQueryParams(req *restful.Request) (*AlertQueryParams, error) {
-	var (
-		q   = &AlertQueryParams{}
-		err error
-	)
-
-	q.State = req.QueryParameter("state")
-	q.PageNum, err = strconv.Atoi(req.QueryParameter("page"))
-	if err != nil {
-		q.PageNum = 1
-		err = nil
-	}
-	if q.PageNum <= 0 {
-		q.PageNum = 1
-	}
-	q.Limit, err = strconv.Atoi(req.QueryParameter("limit"))
-	if err != nil {
-		q.Limit = 10
-		err = nil
-	}
-	q.LabelEqualFilters, q.LabelContainFilters = parseLabelFilters(req)
-	return q, err
-}
-
-func parseLabelFilters(req *restful.Request) (map[string]string, map[string]string) {
-	var (
-		labelEqualFilters   = make(map[string]string)
-		labelContainFilters = make(map[string]string)
-		labelFiltersString  = req.QueryParameter("label_filters")
-	)
-	for _, filter := range strings.Split(labelFiltersString, ",") {
-		if i := strings.Index(filter, "="); i > 0 && len(filter) > i+1 {
-			labelEqualFilters[filter[:i]] = filter[i+1:]
-		} else if i := strings.Index(filter, "~"); i > 0 && len(filter) > i+1 {
-			labelContainFilters[filter[:i]] = filter[i+1:]
-		}
-	}
-	return labelEqualFilters, labelContainFilters
-}
-
-const (
-	ErrBadData       ErrorType = "bad_data"
-	ErrDuplicateName ErrorType = "duplicate_name"
-	ErrNotFound      ErrorType = "not_found"
-	ErrServer        ErrorType = "server_error"
-
-	StatusSuccess Status = "success"
-	StatusError   Status = "error"
-
-	ResultCreated Result = "created"
-	ResultUpdated Result = "updated"
-	ResultDeleted Result = "deleted"
-)
-
-type Status string
-
-type ErrorType string
-
-type Result string
-
-type BulkResponse struct {
-	Errors bool                `json:"errors" description:"If true, one or more operations in the bulk request don't complete successfully"`
-	Items  []*BulkItemResponse `json:"items" description:"It contains the result of each operation in the bulk request"`
-}
-
-// MakeBulkResponse tidies the internal items and sets the errors
-func (br *BulkResponse) MakeBulkResponse() *BulkResponse {
-	var (
-		items   []*BulkItemResponse
-		itemMap = make(map[string]*BulkItemResponse)
-	)
-	for i, item := range br.Items {
-		if item.Status == StatusError {
-			br.Errors = true
-		}
-		pitem, ok := itemMap[item.RuleName]
-		if !ok || (pitem.Status == StatusSuccess || item.Status == StatusError) {
-			itemMap[item.RuleName] = br.Items[i]
-		}
-	}
-	for k := range itemMap {
-		item := itemMap[k]
-		if item.Error != nil {
-			item.ErrorStr = item.Error.Error()
-		}
-		items = append(items, itemMap[k])
-	}
-	br.Items = items
-	return br
-}
-
-type BulkItemResponse struct {
-	RuleName  string    `json:"ruleName,omitempty"`
-	Status    Status    `json:"status,omitempty" description:"It may be success or error"`
-	Result    Result    `json:"result,omitempty" description:"It may be created, updated or deleted, and only for successful operations"`
-	ErrorType ErrorType `json:"errorType,omitempty" description:"It may be bad_data, duplicate_name, not_found or server_error, and only for failed operations"`
-	Error     error     `json:"-"`
-	ErrorStr  string    `json:"error,omitempty" description:"It is only returned for failed operations"`
-}
-
-func NewBulkItemSuccessResponse(ruleName string, result Result) *BulkItemResponse {
-	return &BulkItemResponse{
-		RuleName: ruleName,
-		Status:   StatusSuccess,
-		Result:   result,
-	}
-}
-
-func NewBulkItemErrorServerResponse(ruleName string, err error) *BulkItemResponse {
-	return &BulkItemResponse{
-		RuleName:  ruleName,
-		Status:    StatusError,
-		ErrorType: ErrServer,
-		Error:     err,
-	}
-}
-
-// containsCaseInsensitive reports whether substr is case-insensitive within s.
-func containsCaseInsensitive(s, substr string) bool {
-	return strings.Contains(
-		strings.ToLower(s),
-		strings.ToLower(substr))
-}
--- a/pkg/api/auditing/v1alpha1/types.go
+++ b/pkg/api/auditing/v1alpha1/types.go
@@ -17,12 +17,10 @@ limitations under the License.
 package v1alpha1

 import (
+	"github.com/emicklei/go-restful"
+	"kubesphere.io/kubesphere/pkg/simple/client/auditing"
 	"strconv"
 	"time"
-
-	"github.com/emicklei/go-restful"
-
-	"kubesphere.io/kubesphere/pkg/simple/client/auditing"
 )

 type APIResponse struct {
@@ -50,8 +48,8 @@ type Query struct {
 	ResponseCodeFilter         string `json:"response_code_filter,omitempty"`
 	ResponseStatusFilter       string `json:"response_status_filter,omitempty"`

-	StartTime time.Time `json:"start_time,omitempty"`
-	EndTime   time.Time `json:"end_time,omitempty"`
+	StartTime *time.Time `json:"start_time,omitempty"`
+	EndTime   *time.Time `json:"end_time,omitempty"`

 	Interval string `json:"interval,omitempty"`
 	Sort     string `json:"sort,omitempty"`
@@ -86,7 +84,7 @@ func ParseQueryParameter(req *restful.Request) (*Query, error) {
 			return nil, err
 		}
 		t := time.Unix(sec, 0)
-		q.StartTime = t
+		q.StartTime = &t
 	}
 	if tstr := req.QueryParameter("end_time"); tstr != "" {
 		sec, err := strconv.ParseInt(tstr, 10, 64)
@@ -94,7 +92,7 @@ func ParseQueryParameter(req *restful.Request) (*Query, error) {
 			return nil, err
 		}
 		t := time.Unix(sec, 0)
-		q.EndTime = t
+		q.EndTime = &t
 	}
 	if q.Interval = req.QueryParameter("interval"); q.Interval == "" {
 		q.Interval = "15m"
--- a/pkg/api/auth/types.go
+++ b/pkg/api/auth/types.go
@@ -0,0 +1,51 @@
+/*
+Copyright 2020 The KubeSphere Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package auth
+
+import "fmt"
+
+const (
+	KindTokenReview = "TokenReview"
+)
+
+type Spec struct {
+	Token string `json:"token" description:"access token"`
+}
+
+type Status struct {
+	Authenticated bool                   `json:"authenticated" description:"is authenticated"`
+	User          map[string]interface{} `json:"user,omitempty" description:"user info"`
+}
+
+type TokenReview struct {
+	APIVersion string  `json:"apiVersion" description:"Kubernetes API version"`
+	Kind       string  `json:"kind" description:"kind of the API object"`
+	Spec       *Spec   `json:"spec,omitempty"`
+	Status     *Status `json:"status,omitempty" description:"token review status"`
+}
+
+type LoginRequest struct {
+	Username string `json:"username" description:"username"`
+	Password string `json:"password" description:"password"`
+}
+
+func (request *TokenReview) Validate() error {
+	if request.Spec == nil || request.Spec.Token == "" {
+		return fmt.Errorf("token must not be null")
+	}
+	return nil
+}
--- a/pkg/api/events/v1alpha1/types.go
+++ b/pkg/api/events/v1alpha1/types.go
@@ -17,12 +17,10 @@ limitations under the License.
 package v1alpha1

 import (
+	"github.com/emicklei/go-restful"
+	"kubesphere.io/kubesphere/pkg/simple/client/events"
 	"strconv"
 	"time"
-
-	"github.com/emicklei/go-restful"
-
-	"kubesphere.io/kubesphere/pkg/simple/client/events"
 )

 type APIResponse struct {
@@ -45,8 +43,8 @@ type Query struct {
 	MessageSearch                 string `json:"message_search,omitempty"`
 	TypeFilter                    string `json:"type_filter,omitempty"`

-	StartTime time.Time `json:"start_time,omitempty"`
-	EndTime   time.Time `json:"end_time,omitempty"`
+	StartTime *time.Time `json:"start_time,omitempty"`
+	EndTime   *time.Time `json:"end_time,omitempty"`

 	Interval string `json:"interval,omitempty"`
 	Sort     string `json:"sort,omitempty"`
@@ -76,7 +74,7 @@ func ParseQueryParameter(req *restful.Request) (*Query, error) {
 			return nil, err
 		}
 		t := time.Unix(sec, 0)
-		q.StartTime = t
+		q.StartTime = &t
 	}
 	if tstr := req.QueryParameter("end_time"); tstr != "" {
 		sec, err := strconv.ParseInt(tstr, 10, 64)
@@ -84,7 +82,7 @@ func ParseQueryParameter(req *restful.Request) (*Query, error) {
 			return nil, err
 		}
 		t := time.Unix(sec, 0)
-		q.EndTime = t
+		q.EndTime = &t
 	}
 	if q.Interval = req.QueryParameter("interval"); q.Interval == "" {
 		q.Interval = "15m"
--- a/pkg/api/iam/types.go
+++ b/pkg/api/iam/types.go
@@ -0,0 +1,22 @@
+/*
+Copyright 2020 The KubeSphere Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package iam
+
+type PasswordReset struct {
+	CurrentPassword string `json:"currentPassword"`
+	Password        string `json:"password"`
+}
--- a/pkg/api/logging/v1alpha2/types.go
+++ b/pkg/api/logging/v1alpha2/types.go
@@ -17,12 +17,10 @@ limitations under the License.
 package v1alpha2

 import (
+	"github.com/emicklei/go-restful"
+	"kubesphere.io/kubesphere/pkg/simple/client/logging"
 	"strconv"
 	"time"
-
-	"github.com/emicklei/go-restful"
-
-	"kubesphere.io/kubesphere/pkg/simple/client/logging"
 )

 const (
--- a/pkg/api/metering/v1alpha1/types.go
+++ b/pkg/api/metering/v1alpha1/types.go
@@ -1,90 +0,0 @@
-package v1alpha1
-
-import (
-	"time"
-
-	"github.com/emicklei/go-restful"
-
-	"kubesphere.io/kubesphere/pkg/apiserver/query"
-	model "kubesphere.io/kubesphere/pkg/models/monitoring"
-	"kubesphere.io/kubesphere/pkg/simple/client/monitoring"
-)
-
-const (
-	DefaultStep   = 10 * time.Minute
-	DefaultFilter = ".*"
-	DefaultOrder  = model.OrderDescending
-	DefaultPage   = 1
-	DefaultLimit  = 5
-
-	ErrNoHit             = "'end' or 'time' must be after the namespace creation time."
-	ErrParamConflict     = "'time' and the combination of 'start' and 'end' are mutually exclusive."
-	ErrInvalidStartEnd   = "'start' must be before 'end'."
-	ErrInvalidPage       = "Invalid parameter 'page'."
-	ErrInvalidLimit      = "Invalid parameter 'limit'."
-	ErrParameterNotfound = "Parmameter [%s] not found"
-	ErrResourceNotfound  = "resource not found"
-	ErrScopeNotAllowed   = "scope [%s] not allowed"
-)
-
-type Query struct {
-	Level            monitoring.Level
-	Operation        string
-	LabelSelector    string
-	Time             string
-	Start            string
-	End              string
-	Step             string
-	Target           string
-	Order            string
-	Page             string
-	Limit            string
-	MetricFilter     string
-	ResourceFilter   string
-	NodeName         string
-	WorkspaceName    string
-	NamespaceName    string
-	WorkloadKind     string
-	WorkloadName     string
-	PodName          string
-	Applications     string
-	Services         string
-	StorageClassName string
-	PVCFilter        string
-}
-
-func ParseQueryParameter(req *restful.Request) *Query {
-	var q Query
-
-	q.LabelSelector = req.QueryParameter(query.ParameterLabelSelector)
-
-	q.Level = monitoring.Level(monitoring.MeteringLevelMap[req.QueryParameter("level")])
-	q.Operation = req.QueryParameter("operation")
-	q.Time = req.QueryParameter("time")
-	q.Start = req.QueryParameter("start")
-	q.End = req.QueryParameter("end")
-	q.Step = req.QueryParameter("step")
-	q.Target = req.QueryParameter("sort_metric")
-	q.Order = req.QueryParameter("sort_type")
-	q.Page = req.QueryParameter("page")
-	q.Limit = req.QueryParameter("limit")
-	q.MetricFilter = req.QueryParameter("metrics_filter")
-	q.ResourceFilter = req.QueryParameter("resources_filter")
-	q.WorkspaceName = req.QueryParameter("workspace")
-
-	q.NamespaceName = req.QueryParameter("namespace")
-	if q.NamespaceName == "" {
-		q.NamespaceName = req.PathParameter("namespace")
-	}
-
-	q.NodeName = req.QueryParameter("node")
-	q.WorkloadKind = req.QueryParameter("kind")
-	q.WorkloadName = req.QueryParameter("workload")
-	q.PodName = req.QueryParameter("pod")
-	q.Applications = req.QueryParameter("applications")
-	q.Services = req.QueryParameter("services")
-	q.StorageClassName = req.QueryParameter("storageclass")
-	q.PVCFilter = req.QueryParameter("pvc_filter")
-
-	return &q
-}
--- a/pkg/apiserver/authorization/rbac/evaluation_helpers.go
+++ b/pkg/apiserver/authorization/rbac/evaluation_helpers.go
@@ -14,9 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */

-// Following code copied from k8s.io/kubernetes/pkg/apis/rbac/v1  to avoid import collision
-
-package rbac
+package v1

 import (
 	"fmt"
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
KubeSphere CI Bot	60e58964a6	Merge pull request #3350 from benyp/release-3.0 fix duplicate args error	2021-02-18 10:47:26 +08:00
zhouyue	ccd7a306de	fix duplicate args error Signed-off-by: Zhou Yue <zhouyue@fin-shine.com>	2021-02-07 15:43:37 +08:00
KubeSphere CI Bot	e8505c12c6	Merge pull request #2967 from imjoey/release-3.0/backport/2931+2934 [release-3.0] Backport 2931 2934 from master branch	2020-09-10 10:46:12 +08:00
zryfish	d06c55a221	Update build.yml Add release branch to pull request ci	2020-09-08 17:32:26 +08:00
shaowenchen	56ebafcfd5	fix: scm choices Signed-off-by: shaowenchen <mail@chenshaowen.com> (cherry picked from commit `018a18e784`)	2020-09-08 16:59:30 +08:00
shaowenchen	1adb9b4aba	fix crendential sync Signed-off-by: shaowenchen <mail@chenshaowen.com> (cherry picked from commit `2dcabb0bdb`)	2020-09-08 16:59:30 +08:00