add monitoring dev guides

Signed-off-by: huanggze <loganhuang@yunify.com>

docs/en/concepts-and-designs/README.md

@@ -44,7 +44,8 @@ TODO(@soulseen)
 
 ### KubeSphere Monitoring 
 
-TODO(@huanggze)
+- [Overview](kubesphere-monitoring.md#Overview): Explains the architecture and key components behind the monitoring system.
+- [Setup](kubesphere-monitoring.md#Setup): Introduces manifests related to Prometheus setup.
 
 ### KubeSphere Logging 
 

docs/en/concepts-and-designs/kubesphere-monitoring.md

@@ -0,0 +1,36 @@
+# KubeSphere Monitoring
+
+## Overview
+
+The KubeSphere monitoring system comprises many components that work together to achieve overall monitoring functionality. KubeSphere defaults to installing the monitoring module. It will deploy  or create a few components within the namespace `kubesphere-monitoring-system`:
+
+- Prometheus Operator: automates tasks for operating Prometheus instances.
+- Prometheus: scrapes metrics and provides monitoring service.
+- Kube-state-metrics: exposes metrics for kubernetes objects like deployment, pod, namespace, etc.
+- Node-exporter: exposes node metrics.
+
+Besides, the monitoring stack includes CRD resources:
+
+- ServiceMonitor: declaratively specifies how groups of services should be monitored.
+- Prometheus: defines a desired Prometheus deployment.
+- PrometheusRule: defines a desired Prometheus rule file.
+
+In KubeSphere, Prometheus will monitor the following services by default: 
+
+- kube-state-metrics
+- node-exporter
+- kubelet
+- s2i-operator
+- etcd
+- coredns
+- kube-apiserver
+- kube-scheduler
+- kube-controller-manager
+
+## Setup
+
+The [contrib/kube-prometheus](https://github.com/kubesphere/prometheus-operator/tree/ks-v0.27.0/contrib/kube-prometheus) folder provides manifests for setting up the monitoring stack. The [kubernetes-mixin/rules](https://github.com/kubesphere/kubernetes-mixin/blob/ks-v0.27.0/rules/rules.libsonnet) project provides recording rule templates. KubeSphere monitoring backend serves as Prometheus client performing metrics query.
+
+The diagram below presents the overall monitoring architecture:
+
+![](../../images/kubesphere-monitoring-architecture.png)

docs/en/guides/README.md

@@ -64,7 +64,10 @@ TODO(@soulseen)
 
 ### KubeSphere Monitoring developer
 
-TODO(@huanggze)
+1. Read kubesphere's [Concepts And Designs for Monitoring](../concepts-and-designs/kubesphere-monitoring.md). Understand KubeSphere's monitoring stack.
+2. For Prometheus and its wider eco-system setup, go to [kube-prometheus](https://github.com/kubesphere/prometheus-operator/tree/ks-v0.27.0/contrib/kube-prometheus).
+3. For KubeSphere builtin metric rules, see [metrics_rules.go](https://github.com/kubesphere/kubesphere/blob/master/pkg/models/metrics/metrics_rules.go) and [kubernetes-mixin](https://github.com/kubesphere/kubernetes-mixin/blob/ks-v0.27.0/rules/rules.libsonnet).
+4. For developers who are interested in KubeSphere monitoring backend, read [Development Guide for Monitoring](kubesphere-monitoring-development-guide.md) and [API doc](https://kubesphere.com.cn/docs/v2.1/api/kubesphere#tag/Cluster-Metrics).
 
 ### KubeSphere Logging developer
 

docs/en/guides/kubesphere-monitoring-development-guide.md

@@ -0,0 +1,49 @@
+# Monitoring
+
+This documentation contains backend development guides for interaction with Prometheus. The monitoring backend provides the capabilities of:
+
+ - Metrics query
+ - Metrics sorting
+ - Multi-tenant isolation
+
+## File Tree
+
+The listing below covers all folders related to the monitoring backend.
+
+```
+/pkg
+  ├─api
+  │  └─monitoring         # declares structs for api responses
+  │     └─v1alpha2
+  ├─apiserver             # implements handler for http requests
+  │  └─monitoring
+  ├─kapis                 # registers APIs and routing
+  │  └─monitoring
+  │     ├─install
+  │     └─v1alpha2
+  ├─models
+  │  └─metrics
+  │     ├─constants.go
+  │     ├─metrics.go       # proxies prometheus metrics query
+  │     ├─metrics_rules.go # promql expressions for builtin metrics
+  │     ├─namespaces.go    # appends metric info to namespace resource request
+  │     ├─types.go
+  │     └─util.go          # metrics sorting
+  └─simple
+     ├─factory.go          # factory functions for prometheus client options
+     └─client
+        └─prometheus
+           ├─options.go    # prometheus client options
+           └─prometheus.go # prometheus client code
+```
+
+## API Design
+
+To support multi-tenant isolation, the monitoring backend proxies Prometheus query requests. KubeSphere's APIs have the format like below:
+
+```
+GET /namespaces/{namespace}/pods
+GET /namespaces/{namespace}/pods/{pod}
+```
+
+KubeSphere API gateway will decode the URL and conduct authorization. A person who doesn't belong to a namespace will be rejected to make a request. Besides, note that the two examples above have slightly different meanings. The first is to retrieve all pod-level metrics in the namespace, while the latter is for a specific pod.