admin/kubesphere
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add description

Browse Source 
Signed-off-by: soulseen <sunzhu@yunify.com>
This commit is contained in:
soulseen
2019-06-05 17:21:58 +08:00
parent 2b427b8e68 dd87197de3
commit 24ec90376c
23 changed files with 1029 additions and 806 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
Gopkg.lock generated
View File

@@ -568,7 +568,6 @@
"pkg/kbcontroller",
"pkg/resource"
]
revision = "1be8f5eada07fe5b17804e4b91fc2f4c4fc4ecb9"
source = "https://github.com/kubesphere/application"
[[projects]]

60
README.md
View File

@@ -8,23 +8,56 @@
[KubeSphere](https://kubesphere.io/) is an enterprise-grade multi-tenant container management platform that built on [Kubernetes](https://kubernetes.io). It provides an easy-to-use UI enables creation of computing resources with a few clicks and one-click deployment, which reduces the learning curve and empower the DevOps teams. It greatly reduces the complexity of the daily work of development, testing, operation and maintenance, aiming to solve the pain spots of Kubernetes' storage, network, security and ease of use, etc.
> See this [document](https://docs.kubesphere.io/advanced-v1.0/en/introduction/intro/) that describes the KubeSphere landscape and details.
> See this [document](https://docs.kubesphere.io/advanced-v2.0/zh-CN/introduction/intro/) that describes the KubeSphere landscape and details.
## Features
KubeSphere provides an easy-to-use console with the awesome user experience that allows you to quickly get started with a container management platform. KubeSphere provides and integrates workload management, DevOps Delivery, multi-tenant management, multi-dimensional monitoring, service and network management, application scheduling, infrastructure management, image registry management, etc. It also supports multiple open source storage and high-performance cloud storage as the persistent storage services.
KubeSphere Advanced Edition 2.0.0 provides an easy-to-use console with the awesome user experience that allows you to quickly get started with a container management platform. KubeSphere provides and supports following core features:
> See this [document](https://docs.kubesphere.io/advanced-v1.0/en/introduction/features/) that elaborates on the KubeSphere features and services from a professional point of view.
- Workload management
- Service mesh (Istio-based)
- DevOps
- Source to Image
- Multi-tenant management
- Multi-dimensional and Multi-tenant Monitoring, Logging, Alerting, Notification
- Service and network management
- Application template and repository
- Infrastructure management, image registry management
- Integrate Harbor and GitLab
- LB controller for Kubernetes on bare metal ([Porter](https://github.com/kubesphere/porter)), [cloud LB plugin](https://github.com/yunify/qingcloud-cloud-controller-manager)
- Support GPU node
It also supports multiple open source storage and high-performance cloud storage as the persistent storage services, as well as supports multiple open source network plugins.
> See this [document](https://docs.kubesphere.io/advanced-v2.0/zh-CN/introduction/features/) that elaborates on the KubeSphere features and services from a professional point of view.
----
## Latest Release
KubeSphere Advanced Edition 2.0.0 was released on **May 18th, 2019**. See the [Release Notes For 2.0.0](https://docs.kubesphere.io/advanced-v2.0/release/release-v200/) to preview the updates.
## Installation
KubeSphere installation supports following 2 kinds of installation:
KubeSphere installation supports following 2 kinds of installation, please reference the following guides on how to get KubeSphere up and running.
### All-in-One
[All-in-One](https://docs.kubesphere.io/advanced-v2.0/zh-CN/installation/all-in-one/): For those who are new to KubeSphere and looking for the fastest way to install and experience the dashboard.
Just download the installer and execute the `install.sh` under `/scripts` folder, choose `"1) All-in-one"` to trigger the installation. Generally, you can install it directly without any modification, for details please reference [All-in-One](https://docs.kubesphere.io/advanced-v2.0/zh-CN/installation/all-in-one/).
```bash
$ curl -L https://kubesphere.io/download/stable/advanced-2.0.0 > advanced-2.0.0.tar.gz
$ tar -zxf advanced-2.0.0.tar.gz
```
### Multi-Node
[Multi-Node](https://docs.kubesphere.io/advanced-v2.0/zh-CN/installation/multi-node/) is used for installing KubeSphere on multiple instances, supports for installing a highly available cluster which is able to use in a formal environment.
- [All-in-One](https://docs.kubesphere.io/advanced-v1.0/en/installation/all-in-one/): For those who are new to KubeSphere and looking for the fastest way to install and experience the dashboard, the all-in-one installation must be your best choice since it supports one-click installation.
- [Multi-Node](../https://docs.kubesphere.io/advanced-v1.0/en/installation/multi-node/): Multi-node is used for installing KubeSphere on multiple instances, supports for installing a highly available master and etcd cluster which is able to use in a formal environment.
- For Chinese version, see [KubeSphere Installation Guide (安装指南) ](https://docs.kubesphere.io/advanced-v1.0/zh-CN/installation/intro/).
### Minimum Requirements
@@ -34,28 +67,25 @@ KubeSphere installation supports following 2 kinds of installation:
- Red Hat Enterprise Linux Server 7.4 (64 bit)
- Debian Stretch 9.5 (64 bit)
- Hardware
- CPU：4 Core, Memory：8 G, Disk Space：100 G
- CPU：8 Core, Memory：16 G, Disk Space：100 G
## Quick Start
The [Quick Start Guide](https://docs.kubesphere.io/advanced-v1.0/en/quick-start/admin-quick-start/) provides 7 quick-start examples to walk you through the process and common manipulation in KubeSphere, with a quick overview of the basic features of KubeSphere that helps you to get familiar with it.
The [Quick Start Guide](https://docs.kubesphere.io/advanced-v2.0/quick-start/admin-quick-start/) provides 12 quick-start examples to walk you through the process and common manipulation in KubeSphere, with a quick overview of the core features of KubeSphere that helps you to get familiar with it.
## Latest Release
KubeSphere Advanced Edition 1.0.1 was released on **January 28th, 2019**. See the [Release Notes For 1.0.1](https://docs.kubesphere.io/advanced-v1.0/en/release/release-v101/) to preview the updates.
## RoadMap
Currently, KubeSphere has released the following three major editions. Advanced Edition 2.0.0 will be released on April 18, 2019. The 2.0.0 release will include microservice governance, log query and collection, alerting, S2i, code continuous inspection (SonarQube), quota management for workspace, improve security performance, GPU support, as well as providing [Porter](https://github.com/kubesphere/porter), which is a load balancer for bare metal Kubernetes clusters.
Currently, KubeSphere has released the following 4 major editions. Advanced Edition 2.0.0 was released on May 18, 2019. The future releases will include Big data, AI, Multicluster, QingCloud SDN, etc.
**Community Edition** => **Express Edition** => **Advanced Edition**
**Community Edition** => **Express Edition** => **Advanced Edition 1.0.0** => **Advanced Edition 2.0.0**
![Roadmap](docs/images/roadmap-en.png)
## Documentation
- [KubeSphere Documentation (En/中) ](https://docs.kubesphere.io/)
- [KubeSphere Docementation (PDF)](https://docs.kubesphere.io/KubeSphere-advanced-v1.0.pdf)
- [KubeSphere Docementation (PDF)](https://docs.kubesphere.io/KubeSphere-advanced-v2.0.pdf)
## Support, Discussion, and Community

3
cmd/controller-manager/app/controllers.go
View File

@@ -88,7 +88,8 @@ func AddControllers(mgr manager.Manager, cfg *rest.Config, stopCh <-chan struct{
informerFactory.Core().V1().Services(),
servicemeshInformer.Servicemesh().V1alpha2().ServicePolicies(),
kubeClient,
istioclient)
istioclient,
servicemeshclient)
apController := application.NewApplicationController(informerFactory.Core().V1().Services(),
informerFactory.Apps().V1().Deployments(),

BIN
docs/images/roadmap-en.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 220 KiB

After

Width:  |  Height:  |  Size: 215 KiB

224
pkg/apis/iam/v1alpha2/register.go
View File

@@ -20,11 +20,14 @@ package v1alpha2
import (
"github.com/emicklei/go-restful"
"github.com/emicklei/go-restful-openapi"
rbacv1 "k8s.io/api/rbac/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
"kubesphere.io/kubesphere/pkg/apiserver/iam"
"kubesphere.io/kubesphere/pkg/apiserver/runtime"
"kubesphere.io/kubesphere/pkg/errors"
"kubesphere.io/kubesphere/pkg/models"
"kubesphere.io/kubesphere/pkg/models/iam/policy"
"net/http"
)
const GroupName = "iam.kubesphere.io"
@@ -40,181 +43,220 @@ func addWebService(c *restful.Container) error {
tags := []string{"IAM"}
ws := runtime.NewWebService(GroupVersion)
ok := "ok"
pageableUserList := struct {
Items []models.User `json:"items"`
TotalCount int `json:"total_count"`
}{}
ws.Route(ws.POST("/authenticate").
To(iam.TokenReviewHandler).
Doc("Token review").
Doc("TokenReview attempts to authenticate a token to a known user. Note: TokenReview requests may be cached by the webhook token authenticator plugin in the kube-apiserver.").
Reads(iam.TokenReview{}).
Writes(iam.TokenReview{}).
Doc("k8s token review").
Returns(http.StatusOK, ok, iam.TokenReview{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.POST("/login").
To(iam.LoginHandler).
Doc("User login").
Doc("KubeSphere APIs support token-based authentication via the Authtoken request header. The POST Login API is used to retrieve the authentication token. After the authentication token is obtained, it must be inserted into the Authtoken header for all requests.").
Reads(iam.LoginRequest{}).
Writes(models.Token{}).
Returns(http.StatusOK, ok, models.Token{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/users/{username}").
To(iam.DescribeUser).
Doc("User detail").
Doc("Describes the specified user.").
Param(ws.PathParameter("username", "username")).
Writes(models.User{}).
Returns(http.StatusOK, ok, models.User{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.POST("/users").
To(iam.CreateUser).
Doc("Create a user account.").
Reads(models.User{}).
Writes(errors.Error{}).
Doc("Create user").
Returns(http.StatusOK, ok, errors.Error{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.DELETE("/users/{name}").
To(iam.DeleteUser).
Doc("Remove a specified user.").
Param(ws.PathParameter("name", "username")).
Doc("Delete user").
Writes(errors.Error{}).
Returns(http.StatusOK, ok, errors.Error{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.PUT("/users/{name}").
To(iam.UpdateUser).
Doc("Updates information about the specified user.").
Param(ws.PathParameter("name", "username")).
Reads(models.User{}).
Writes(errors.Error{}).
Doc("Update user").
Returns(http.StatusOK, ok, errors.Error{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/users/{name}/log").
To(iam.UserLoginLog).
Doc("This method is used to retrieve the \"login logs\" for the specified user.").
Param(ws.PathParameter("name", "username")).
Doc("User login log").
Writes([]map[string]string{}).
Returns(http.StatusOK, ok, struct {
LoginTime string `json:"login_time"`
LoginIP string `json:"login_ip"`
}{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/users").
To(iam.ListUsers).
Doc("User list").
Writes(models.PageableResponse{}).
Doc("List all users.").
Returns(http.StatusOK, ok, pageableUserList).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/groups").
To(iam.ListGroups).
Writes([]models.Group{}).
Doc("User group list").
Doc("List all user groups.").
Returns(http.StatusOK, ok, []models.Group{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/groups/{path}").
To(iam.DescribeGroup).
Param(ws.PathParameter("path", "group path")).
Doc("User group detail").
Doc("Describes the specified user group.").
Param(ws.PathParameter("path", "user group path separated by colon.")).
Returns(http.StatusOK, ok, models.Group{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/groups/{path}/users").
To(iam.ListGroupUsers).
Param(ws.PathParameter("path", "group path")).
Doc("Group user list").
Doc("List all users in the specified user group.").
Param(ws.PathParameter("path", "user group path separated by colon.")).
Returns(http.StatusOK, ok, []models.User{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.POST("/groups").
To(iam.CreateGroup).
Doc("Create a user group.").
Reads(models.Group{}).
Doc("Create user group").
Returns(http.StatusOK, ok, models.Group{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.DELETE("/groups/{path}").
To(iam.DeleteGroup).
Param(ws.PathParameter("path", "group path")).
Doc("Delete user group").
Doc("Delete a user group.").
Param(ws.PathParameter("path", "user group path separated by colon.")).
Returns(http.StatusOK, ok, errors.Error{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.PUT("/groups/{path}").
To(iam.UpdateGroup).
Param(ws.PathParameter("path", "group path")).
Doc("Update user group").
Doc("Updates information about the user group.").
Param(ws.PathParameter("path", "user group path separated by colon.")).
Reads(models.Group{}).
Returns(http.StatusOK, ok, models.Group{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/users/{username}/roles").
To(iam.ListUserRoles).
Doc("This method is used to retrieve all the roles that are assigned to the user.").
Param(ws.PathParameter("username", "username")).
Doc("Get user role list").
Returns(http.StatusOK, ok, iam.RoleList{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/namespaces/{namespace}/roles").
To(iam.ListRoles).
Param(ws.PathParameter("namespace", "namespace")).
Doc("Get role list").
Doc("This method is used to retrieve the roles that are assigned to the user in the specified namespace.").
Param(ws.PathParameter("namespace", "kubernetes namespace")).
Returns(http.StatusOK, ok, struct {
Items []rbacv1.Role `json:"items"`
TotalCount int `json:"total_count"`
}{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/clusterroles").
To(iam.ListClusterRoles).
Doc("Get cluster role list").
Doc("List all cluster roles.").
Returns(http.StatusOK, ok, struct {
Items []rbacv1.ClusterRole `json:"items"`
TotalCount int `json:"total_count"`
}{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/namespaces/{namespace}/roles/{role}/users").
To(iam.ListRoleUsers).
Param(ws.PathParameter("namespace", "namespace")).
Doc("This method is used to retrieve the users that are bind the role in the specified namespace.").
Param(ws.PathParameter("namespace", "kubernetes namespace")).
Param(ws.PathParameter("role", "role name")).
Doc("Get user list by role").
Returns(http.StatusOK, ok, []models.User{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/namespaces/{namespace}/users").
To(iam.ListNamespaceUsers).
Param(ws.PathParameter("namespace", "namespace")).
Doc("Get user list by namespace").
Doc("List all users in the specified namespace").
Param(ws.PathParameter("namespace", "kubernetes namespace")).
Returns(http.StatusOK, ok, []models.User{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/clusterroles/{clusterrole}/users").
To(iam.ListClusterRoleUsers).
Doc("List all users that are bind the cluster role.").
Param(ws.PathParameter("clusterrole", "cluster role name")).
Doc("Get user list by cluster role").
Returns(http.StatusOK, ok, pageableUserList).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/clusterroles/{clusterrole}/rules").
To(iam.ListClusterRoleRules).
Doc("List all policy rules of the specified cluster role.").
Param(ws.PathParameter("clusterrole", "cluster role name")).
Doc("Get cluster role detail").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/rulesmapping/clusterroles").
To(iam.ClusterRulesMapping).
Doc("Get cluster role policy rules mapping").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/rulesmapping/roles").
To(iam.RulesMapping).
Doc("Get role policy rules mapping").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/roles").
To(iam.ListWorkspaceRoles).
Param(ws.PathParameter("workspace", "workspace name")).
Doc("List workspace role").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/roles/{role}").
To(iam.DescribeWorkspaceRole).
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("role", "workspace role name")).
Doc("Describe workspace role").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/roles/{role}/rules").
To(iam.ListWorkspaceRoleRules).
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("role", "workspace role name")).
Doc("Get workspace role policy rules").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/members").
To(iam.ListWorkspaceUsers).
Param(ws.PathParameter("workspace", "workspace name")).
Doc("Get workspace member list").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.POST("/workspaces/{workspace}/members").
To(iam.InviteUser).
Param(ws.PathParameter("workspace", "workspace name")).
Doc("Add user to workspace").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.DELETE("/workspaces/{workspace}/members/{username}").
To(iam.RemoveUser).
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("name", "username")).
Doc("Remove user from workspace").
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/members/{username}").
To(iam.DescribeWorkspaceUser).
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("username", "username")).
Doc("Describe user in workspace").
Returns(http.StatusOK, ok, []models.SimpleRule{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/namespaces/{namespace}/roles/{role}/rules").
To(iam.ListRoleRules).
Param(ws.PathParameter("namespace", "namespace")).
Doc("List all policy rules of the specified role.").
Param(ws.PathParameter("namespace", "kubernetes namespace")).
Param(ws.PathParameter("role", "role name")).
Doc("Get namespace role policy rules").
Returns(http.StatusOK, ok, []models.SimpleRule{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/devops/{devops}/roles/{role}/rules").
To(iam.ListDevopsRoleRules).
Doc("List all policy rules of the specified role.").
Param(ws.PathParameter("devops", "devops project id")).
Param(ws.PathParameter("role", "devops role name")).
Doc("Get devops role policy rules").
Returns(http.StatusOK, ok, []models.SimpleRule{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/rulesmapping/clusterroles").
To(iam.ClusterRulesMapping).
Doc("Get the mapping relationships between cluster roles and policy rules.").
Returns(http.StatusOK, ok, policy.ClusterRoleRuleMapping).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/rulesmapping/roles").
To(iam.RulesMapping).
Doc("Get the mapping relationships between namespaced roles and policy rules.").
Returns(http.StatusOK, ok, policy.RoleRuleMapping).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/roles").
To(iam.ListWorkspaceRoles).
Doc("List all workspace roles.").
Param(ws.PathParameter("workspace", "workspace name")).
Returns(http.StatusOK, ok, struct {
Items []rbacv1.ClusterRole `json:"items"`
TotalCount int `json:"total_count"`
}{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/roles/{role}").
To(iam.DescribeWorkspaceRole).
Doc("Describes the workspace role.").
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("role", "workspace role name")).
Returns(http.StatusOK, ok, rbacv1.ClusterRole{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/roles/{role}/rules").
To(iam.ListWorkspaceRoleRules).
Doc("List all policy rules of the specified workspace role.").
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("role", "workspace role name")).
Returns(http.StatusOK, ok, []models.SimpleRule{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/members").
To(iam.ListWorkspaceUsers).
Doc("List all members in the specified workspace.").
Param(ws.PathParameter("workspace", "workspace name")).
Returns(http.StatusOK, ok, pageableUserList).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.POST("/workspaces/{workspace}/members").
To(iam.InviteUser).
Doc("Invite members to a workspace.").
Param(ws.PathParameter("workspace", "workspace name")).
Reads(models.User{}).
Returns(http.StatusOK, ok, errors.Error{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.DELETE("/workspaces/{workspace}/members/{username}").
To(iam.RemoveUser).
Doc("Remove members from workspace.").
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("name", "username")).
Returns(http.StatusOK, ok, errors.Error{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
ws.Route(ws.GET("/workspaces/{workspace}/members/{username}").
To(iam.DescribeWorkspaceUser).
Doc("Describes the specified user.").
Param(ws.PathParameter("workspace", "workspace name")).
Param(ws.PathParameter("username", "username")).
Returns(http.StatusOK, ok, models.User{}).
Metadata(restfulspec.KeyOpenAPITags, tags))
c.Add(ws)
return nil

18
pkg/apiserver/devops/devops.go
View File

@@ -23,8 +23,11 @@ import (
log "github.com/golang/glog"
"kubesphere.io/kubesphere/pkg/models/devops"
"net/http"
"strings"
)
const jenkinsHeaderPre = "X-"
func GetPipeline(req *restful.Request, resp *restful.Response) {
projectName := req.PathParameter("projectName")
pipelineName := req.PathParameter("pipelineName")
@@ -104,11 +107,17 @@ func GetBranchStepLog(req *restful.Request, resp *restful.Response) {
nodeId := req.PathParameter("nodeId")
stepId := req.PathParameter("stepId")
res, err := devops.GetBranchStepLog(projectName, pipelineName, branchName, runId, nodeId, stepId, req.Request)
res, header, err := devops.GetBranchStepLog(projectName, pipelineName, branchName, runId, nodeId, stepId, req.Request)
if err != nil {
parseErr(err, resp)
return
}
for k, v := range header {
if strings.HasPrefix(k, jenkinsHeaderPre) {
resp.AddHeader(k, v[0])
}
}
resp.Write(res)
}
@@ -119,11 +128,16 @@ func GetStepLog(req *restful.Request, resp *restful.Response) {
nodeId := req.PathParameter("nodeId")
stepId := req.PathParameter("stepId")
res, err := devops.GetStepLog(projectName, pipelineName, runId, nodeId, stepId, req.Request)
res, header, err := devops.GetStepLog(projectName, pipelineName, runId, nodeId, stepId, req.Request)
if err != nil {
parseErr(err, resp)
return
}
for k, v := range header {
if strings.HasPrefix(k, jenkinsHeaderPre) {
resp.AddHeader(k, v[0])
}
}
resp.Write(res)
}

4
pkg/apiserver/iam/am.go
View File

@@ -30,7 +30,7 @@ import (
"kubesphere.io/kubesphere/pkg/models/iam/policy"
)
type roleList struct {
type RoleList struct {
ClusterRoles []*v1.ClusterRole `json:"clusterRole" protobuf:"bytes,2,rep,name=clusterRoles"`
Roles []*v1.Role `json:"roles" protobuf:"bytes,2,rep,name=roles"`
}
@@ -132,7 +132,7 @@ func ListUserRoles(req *restful.Request, resp *restful.Response) {
return
}
roleList := roleList{}
roleList := RoleList{}
roleList.Roles = roles
roleList.ClusterRoles = clusterRoles

4
pkg/controller/add_clusterrolebinding.go
View File

@@ -18,7 +18,9 @@
package controller
import "kubesphere.io/kubesphere/pkg/controller/clusterrolebinding"
func init() {
// AddToManagerFuncs is a list of functions to create controllers and add them to a manager.
//AddToManagerFuncs = append(AddToManagerFuncs, clusterrolebinding.Add)
AddToManagerFuncs = append(AddToManagerFuncs, clusterrolebinding.Add)
}

2
pkg/controller/application/application_controller.go
View File

@@ -149,7 +149,7 @@ func NewApplicationController(serviceInformer coreinformers.ServiceInformer,
}
func (v *ApplicationController) Start(stopCh <-chan struct{}) error {
v.Run(5, stopCh)
v.Run(2, stopCh)
return nil
}

23
pkg/controller/destinationrule/destinationrule_controller.go
View File

@@ -34,6 +34,7 @@ import (
"k8s.io/client-go/util/workqueue"
"time"
servicemeshclient "kubesphere.io/kubesphere/pkg/client/clientset/versioned"
servicemeshinformers "kubesphere.io/kubesphere/pkg/client/informers/externalversions/servicemesh/v1alpha2"
servicemeshlisters "kubesphere.io/kubesphere/pkg/client/listers/servicemesh/v1alpha2"
)
@@ -53,6 +54,7 @@ type DestinationRuleController struct {
client clientset.Interface
destinationRuleClient istioclientset.Interface
servicemeshClient servicemeshclient.Interface
eventBroadcaster record.EventBroadcaster
eventRecorder record.EventRecorder
@@ -79,7 +81,8 @@ func NewDestinationRuleController(deploymentInformer informersv1.DeploymentInfor
serviceInformer coreinformers.ServiceInformer,
servicePolicyInformer servicemeshinformers.ServicePolicyInformer,
client clientset.Interface,
destinationRuleClient istioclientset.Interface) *DestinationRuleController {
destinationRuleClient istioclientset.Interface,
servicemeshClient servicemeshclient.Interface) *DestinationRuleController {
broadcaster := record.NewBroadcaster()
broadcaster.StartLogging(func(format string, args ...interface{}) {
@@ -95,6 +98,7 @@ func NewDestinationRuleController(deploymentInformer informersv1.DeploymentInfor
v := &DestinationRuleController{
client: client,
destinationRuleClient: destinationRuleClient,
servicemeshClient: servicemeshClient,
queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "destinationrule"),
workerLoopPeriod: time.Second,
}
@@ -211,12 +215,20 @@ func (v *DestinationRuleController) syncService(key string) error {
service, err := v.serviceLister.Services(namespace).Get(name)
if err != nil {
// Delete the corresponding destinationrule, as the service has been deleted.
// delete the corresponding destinationrule if there is any, as the service has been deleted.
err = v.destinationRuleClient.NetworkingV1alpha3().DestinationRules(namespace).Delete(name, nil)
if !errors.IsNotFound(err) {
if err != nil && !errors.IsNotFound(err) {
log.Error(err, "delete destination rule failed", "namespace", namespace, "name", name)
return err
}
// delete orphan service policy if there is any
err = v.servicemeshClient.ServicemeshV1alpha2().ServicePolicies(namespace).Delete(name, nil)
if err != nil && !errors.IsNotFound(err) {
log.Error(err, "delete orphan service policy failed", "namespace", namespace, "name", name)
return err
}
return nil
}
@@ -293,6 +305,7 @@ func (v *DestinationRuleController) syncService(key string) error {
}
dr := currentDestinationRule.DeepCopy()
dr.Spec.TrafficPolicy = nil
dr.Spec.Subsets = subsets
//
if len(servicePolicies) > 0 {
@@ -332,9 +345,9 @@ func (v *DestinationRuleController) syncService(key string) error {
}
if createDestinationRule {
newDestinationRule, err = v.destinationRuleClient.NetworkingV1alpha3().DestinationRules(namespace).Create(newDestinationRule)
_, err = v.destinationRuleClient.NetworkingV1alpha3().DestinationRules(namespace).Create(newDestinationRule)
} else {
newDestinationRule, err = v.destinationRuleClient.NetworkingV1alpha3().DestinationRules(namespace).Update(newDestinationRule)
_, err = v.destinationRuleClient.NetworkingV1alpha3().DestinationRules(namespace).Update(newDestinationRule)
}
if err != nil {

57
pkg/controller/namespace/namespace_controller.go
View File

@@ -35,6 +35,8 @@ import (
"kubesphere.io/kubesphere/pkg/constants"
"kubesphere.io/kubesphere/pkg/simple/client/openpitrix"
"kubesphere.io/kubesphere/pkg/utils/k8sutil"
"kubesphere.io/kubesphere/pkg/utils/sliceutil"
"math"
"reflect"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
@@ -44,6 +46,7 @@ import (
"sigs.k8s.io/controller-runtime/pkg/reconcile"
logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
"sigs.k8s.io/controller-runtime/pkg/source"
"time"
)
const (
@@ -115,25 +118,47 @@ func (r *ReconcileNamespace) Reconcile(request reconcile.Request) (reconcile.Res
if errors.IsNotFound(err) {
// Object not found, return. Created objects are automatically garbage collected.
// For additional cleanup logic use finalizers.
// The object is being deleted
// our finalizer is present, so lets handle our external dependency
return reconcile.Result{}, nil
}
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
if !instance.ObjectMeta.DeletionTimestamp.IsZero() {
// name of your custom finalizer
finalizer := "finalizers.kubesphere.io/namespaces"
if instance.ObjectMeta.DeletionTimestamp.IsZero() {
// The object is not being deleted, so if it does not have our finalizer,
// then lets add the finalizer and update the object.
if !sliceutil.HasString(instance.ObjectMeta.Finalizers, finalizer) {
instance.ObjectMeta.Finalizers = append(instance.ObjectMeta.Finalizers, finalizer)
if err := r.Update(context.Background(), instance); err != nil {
return reconcile.Result{}, err
}
}
} else {
// The object is being deleted
if sliceutil.HasString(instance.ObjectMeta.Finalizers, finalizer) {
if err := r.deleteRouter(instance.Name); err != nil {
return reconcile.Result{}, err
}
if err := r.deleteRouter(instance.Name); err != nil {
return reconcile.Result{}, err
}
if err := r.deleteRuntime(instance); err != nil {
// if fail to delete the external dependency here, return with error
// so that it can be retried
return reconcile.Result{}, err
// delete runtime in the background, retry 3 times
go r.deleteRuntime(instance)
// remove our finalizer from the list and update it.
instance.ObjectMeta.Finalizers = sliceutil.RemoveString(instance.ObjectMeta.Finalizers, func(item string) bool {
return item == finalizer
})
if err := r.Update(context.Background(), instance); err != nil {
return reconcile.Result{}, err
}
}
// Our finalizer has finished, so the reconciler can do nothing.
return reconcile.Result{}, nil
}
@@ -347,9 +372,17 @@ func (r *ReconcileNamespace) checkAndCreateRuntime(namespace *corev1.Namespace)
func (r *ReconcileNamespace) deleteRuntime(namespace *corev1.Namespace) error {
if runtimeId := namespace.Annotations[constants.OpenPitrixRuntimeAnnotationKey]; runtimeId != "" {
log.Info("Deleting openpitrix runtime", "namespace", namespace.Name, "runtime", runtimeId)
if err := openpitrix.Client().DeleteRuntime(runtimeId); err != nil {
return err
maxRetries := float64(3)
for i := float64(0); i < maxRetries; i++ {
time.Sleep(time.Duration(i*math.Pow(2, i)) * time.Second)
log.Info("Deleting openpitrix runtime", "namespace", namespace.Name, "runtime", runtimeId)
err := openpitrix.Client().DeleteRuntime(runtimeId)
if err == nil || openpitrix.IsNotFound(err) || openpitrix.IsDeleted(err) {
return nil
}
log.Error(err, fmt.Sprintf("Deleteing openpitrix runtime failed: %v times left", maxRetries-i-1))
}
}

16
pkg/controller/virtualservice/virtualservice_controller.go
View File

@@ -231,6 +231,14 @@ func (v *VirtualServiceController) syncService(key string) error {
log.Error(err, "delete orphan virtualservice failed", "namespace", namespace, "name", service.Name)
return err
}
// delete the orphan strategy if there is any
err = v.servicemeshClient.ServicemeshV1alpha2().Strategies(namespace).Delete(name, nil)
if err != nil && !errors.IsNotFound(err) {
log.Error(err, "delete orphan strategy failed", "namespace", namespace, "name", service.Name)
return err
}
return nil
}
log.Error(err, "get service failed", "namespace", namespace, "name", name)
@@ -264,9 +272,7 @@ func (v *VirtualServiceController) syncService(key string) error {
subsets := destinationRule.Spec.Subsets
if len(subsets) == 0 {
// destination rule with no subsets, not possibly
err = fmt.Errorf("found destinationrule with no subsets for service %s", name)
log.Error(err, "found destinationrule with no subsets", "namespace", namespace, "name", appName)
return err
return nil
}
// fetch all strategies applied to service
@@ -387,9 +393,9 @@ func (v *VirtualServiceController) syncService(key string) error {
}
if createVirtualService {
newVirtualService, err = v.virtualServiceClient.NetworkingV1alpha3().VirtualServices(namespace).Create(newVirtualService)
_, err = v.virtualServiceClient.NetworkingV1alpha3().VirtualServices(namespace).Create(newVirtualService)
} else {
newVirtualService, err = v.virtualServiceClient.NetworkingV1alpha3().VirtualServices(namespace).Update(newVirtualService)
_, err = v.virtualServiceClient.NetworkingV1alpha3().VirtualServices(namespace).Update(newVirtualService)
}
if err != nil {

141
pkg/models/devops/devops.go
View File

@@ -68,10 +68,11 @@ func SearchPipelines(req *http.Request) ([]byte, error) {
}
func SearchPipelineRuns(projectName, pipelineName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+SearchPipelineRunUrl+req.URL.RawQuery, projectName, pipelineName)
log.Infof("Jenkins-url: " + baseUrl)
baseUrl := fmt.Sprintf(jenkins.Server+SearchPipelineRunUrl, projectName, pipelineName)
res, err := sendJenkinsRequest(baseUrl, req)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl+req.URL.RawQuery, req)
if err != nil {
log.Error(err)
return nil, err
@@ -82,7 +83,7 @@ func SearchPipelineRuns(projectName, pipelineName string, req *http.Request) ([]
func GetBranchPipelineRun(projectName, pipelineName, branchName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetPipeBranchRunUrl, projectName, pipelineName, branchName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -95,7 +96,7 @@ func GetBranchPipelineRun(projectName, pipelineName, branchName, runId string, r
func GetPipelineRunNodesbyBranch(projectName, pipelineName, branchName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchPipeRunNodesUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -106,35 +107,36 @@ func GetPipelineRunNodesbyBranch(projectName, pipelineName, branchName, runId st
return res, err
}
func GetBranchStepLog(projectName, pipelineName, branchName, runId, nodeId, stepId string, req *http.Request) ([]byte, error) {
func GetBranchStepLog(projectName, pipelineName, branchName, runId, nodeId, stepId string, req *http.Request) ([]byte, http.Header, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchStepLogUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId, nodeId, stepId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
resBody, header, err := jenkinsClient(baseUrl, req)
if err != nil {
log.Error(err)
return nil, err
return nil, nil, err
}
return resBody, err
return resBody, header, err
}
func GetStepLog(projectName, pipelineName, runId, nodeId, stepId string, req *http.Request) ([]byte, error) {
func GetStepLog(projectName, pipelineName, runId, nodeId, stepId string, req *http.Request) ([]byte, http.Header, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetStepLogUrl+req.URL.RawQuery, projectName, pipelineName, runId, nodeId, stepId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
resBody, header, err := jenkinsClient(baseUrl, req)
if err != nil {
log.Error(err)
return nil, err
return nil, nil, err
}
return resBody, err
return resBody, header, err
}
func Validate(scmId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+ValidateUrl, scmId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -147,7 +149,7 @@ func Validate(scmId string, req *http.Request) ([]byte, error) {
func GetSCMOrg(scmId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetSCMOrgUrl+req.URL.RawQuery, scmId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -160,7 +162,7 @@ func GetSCMOrg(scmId string, req *http.Request) ([]byte, error) {
func GetOrgRepo(scmId, organizationId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetOrgRepoUrl+req.URL.RawQuery, scmId, organizationId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -173,7 +175,7 @@ func GetOrgRepo(scmId, organizationId string, req *http.Request) ([]byte, error)
func StopBranchPipeline(projectName, pipelineName, branchName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+StopBranchPipelineUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -186,7 +188,7 @@ func StopBranchPipeline(projectName, pipelineName, branchName, runId string, req
func StopPipeline(projectName, pipelineName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+StopPipelineUrl+req.URL.RawQuery, projectName, pipelineName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -199,7 +201,7 @@ func StopPipeline(projectName, pipelineName, runId string, req *http.Request) ([
func ReplayBranchPipeline(projectName, pipelineName, branchName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+ReplayBranchPipelineUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -212,7 +214,7 @@ func ReplayBranchPipeline(projectName, pipelineName, branchName, runId string, r
func ReplayPipeline(projectName, pipelineName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+ReplayPipelineUrl+req.URL.RawQuery, projectName, pipelineName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -225,7 +227,7 @@ func ReplayPipeline(projectName, pipelineName, runId string, req *http.Request)
func GetBranchRunLog(projectName, pipelineName, branchName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchRunLogUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -238,7 +240,7 @@ func GetBranchRunLog(projectName, pipelineName, branchName, runId string, req *h
func GetRunLog(projectName, pipelineName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetRunLogUrl+req.URL.RawQuery, projectName, pipelineName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -251,7 +253,7 @@ func GetRunLog(projectName, pipelineName, runId string, req *http.Request) ([]by
func GetBranchArtifacts(projectName, pipelineName, branchName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchArtifactsUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -264,7 +266,7 @@ func GetBranchArtifacts(projectName, pipelineName, branchName, runId string, req
func GetArtifacts(projectName, pipelineName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetArtifactsUrl+req.URL.RawQuery, projectName, pipelineName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -276,10 +278,10 @@ func GetArtifacts(projectName, pipelineName, runId string, req *http.Request) ([
}
func GetPipeBranch(projectName, pipelineName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetPipeBranchUrl+req.URL.RawQuery, projectName, pipelineName)
log.Infof("Jenkins-url: " + baseUrl)
baseUrl := fmt.Sprintf(jenkins.Server+GetPipeBranchUrl, projectName, pipelineName)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
res, err := sendJenkinsRequest(baseUrl+req.URL.RawQuery, req)
if err != nil {
log.Error(err)
return nil, err
@@ -290,7 +292,7 @@ func GetPipeBranch(projectName, pipelineName string, req *http.Request) ([]byte,
func CheckBranchPipeline(projectName, pipelineName, branchName, runId, nodeId, stepId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+CheckBranchPipelineUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId, nodeId, stepId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -303,7 +305,7 @@ func CheckBranchPipeline(projectName, pipelineName, branchName, runId, nodeId, s
func CheckPipeline(projectName, pipelineName, runId, nodeId, stepId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+CheckPipelineUrl+req.URL.RawQuery, projectName, pipelineName, runId, nodeId, stepId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -316,7 +318,7 @@ func CheckPipeline(projectName, pipelineName, runId, nodeId, stepId string, req
func GetConsoleLog(projectName, pipelineName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetConsoleLogUrl+req.URL.RawQuery, projectName, pipelineName)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -329,7 +331,7 @@ func GetConsoleLog(projectName, pipelineName string, req *http.Request) ([]byte,
func ScanBranch(projectName, pipelineName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+ScanBranchUrl+req.URL.RawQuery, projectName, pipelineName)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
resBody, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -342,7 +344,7 @@ func ScanBranch(projectName, pipelineName string, req *http.Request) ([]byte, er
func RunBranchPipeline(projectName, pipelineName, branchName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+RunBranchPipelineUrl+req.URL.RawQuery, projectName, pipelineName, branchName)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -355,7 +357,7 @@ func RunBranchPipeline(projectName, pipelineName, branchName string, req *http.R
func RunPipeline(projectName, pipelineName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+RunPipelineUrl+req.URL.RawQuery, projectName, pipelineName)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -368,7 +370,7 @@ func RunPipeline(projectName, pipelineName string, req *http.Request) ([]byte, e
func GetBranchStepsStatus(projectName, pipelineName, branchName, runId, nodeId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchStepsStatusUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId, nodeId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -381,7 +383,7 @@ func GetBranchStepsStatus(projectName, pipelineName, branchName, runId, nodeId s
func GetStepsStatus(projectName, pipelineName, runId, nodeId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetStepsStatusUrl+req.URL.RawQuery, projectName, pipelineName, runId, nodeId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -394,7 +396,7 @@ func GetStepsStatus(projectName, pipelineName, runId, nodeId string, req *http.R
func GetCrumb(req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server + GetCrumbUrl)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -407,7 +409,7 @@ func GetCrumb(req *http.Request) ([]byte, error) {
func CheckScriptCompile(req *http.Request) ([]byte, error) {
baseUrl := jenkins.Server + CheckScriptCompileUrl
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
req.SetBasicAuth(jenkins.Requester.BasicAuth.Username, jenkins.Requester.BasicAuth.Password)
resBody, err := sendJenkinsRequest(baseUrl, req)
@@ -420,12 +422,19 @@ func CheckScriptCompile(req *http.Request) ([]byte, error) {
}
func CheckCron(req *http.Request) (*CheckCronRes, error) {
baseUrl := jenkins.Server + CheckCronUrl + req.URL.RawQuery
log.Infof("Jenkins-url: " + baseUrl)
req.SetBasicAuth(jenkins.Requester.BasicAuth.Username, jenkins.Requester.BasicAuth.Password)
var res = new(CheckCronRes)
newurl, err := url.Parse(jenkins.Server + CheckCronUrl + req.URL.RawQuery)
resp, err := http.Get(baseUrl)
reqJenkins := &http.Request{
Method: http.MethodGet,
URL: newurl,
Header: http.Header{},
}
var res = new(CheckCronRes)
client := &http.Client{Timeout: 30 * time.Second}
reqJenkins.SetBasicAuth(jenkins.Requester.BasicAuth.Username, jenkins.Requester.BasicAuth.Password)
resp, err := client.Do(reqJenkins)
if err != nil {
log.Error(err)
return res, err
@@ -446,7 +455,7 @@ func CheckCron(req *http.Request) (*CheckCronRes, error) {
func GetPipelineRun(projectName, pipelineName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetPipelineRunUrl, projectName, pipelineName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -459,7 +468,7 @@ func GetPipelineRun(projectName, pipelineName, runId string, req *http.Request)
func GetBranchPipeline(projectName, pipelineName, branchName string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchPipeUrl, projectName, pipelineName, branchName)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -472,7 +481,7 @@ func GetBranchPipeline(projectName, pipelineName, branchName string, req *http.R
func GetPipelineRunNodes(projectName, pipelineName, runId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetPipeRunNodesUrl+req.URL.RawQuery, projectName, pipelineName, runId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -485,7 +494,7 @@ func GetPipelineRunNodes(projectName, pipelineName, runId string, req *http.Requ
func GetBranchNodeSteps(projectName, pipelineName, branchName, runId, nodeId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetBranchNodeStepsUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId, nodeId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -498,7 +507,7 @@ func GetBranchNodeSteps(projectName, pipelineName, branchName, runId, nodeId str
func GetNodeSteps(projectName, pipelineName, runId, nodeId string, req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server+GetNodeStepsUrl+req.URL.RawQuery, projectName, pipelineName, runId, nodeId)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -511,7 +520,7 @@ func GetNodeSteps(projectName, pipelineName, runId, nodeId string, req *http.Req
func ToJenkinsfile(req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server + ToJenkinsfileUrl)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -524,7 +533,7 @@ func ToJenkinsfile(req *http.Request) ([]byte, error) {
func ToJson(req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server + ToJsonUrl)
log.Infof("Jenkins-url: " + baseUrl)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -536,8 +545,8 @@ func ToJson(req *http.Request) ([]byte, error) {
}
func GetNotifyCommit(req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server + GetNotifyCommitUrl + req.URL.RawQuery)
log.Infof("Jenkins-url: " + baseUrl)
baseUrl := fmt.Sprint(jenkins.Server, GetNotifyCommitUrl, req.URL.RawQuery)
log.Info("Jenkins-url: " + baseUrl)
req.Method = "GET"
res, err := sendJenkinsRequest(baseUrl, req)
@@ -550,8 +559,8 @@ func GetNotifyCommit(req *http.Request) ([]byte, error) {
}
func GithubWebhook(req *http.Request) ([]byte, error) {
baseUrl := fmt.Sprintf(jenkins.Server + GithubWebhookUrl + req.URL.RawQuery)
log.Infof("Jenkins-url: " + baseUrl)
baseUrl := fmt.Sprint(jenkins.Server, GithubWebhookUrl, req.URL.RawQuery)
log.Info("Jenkins-url: " + baseUrl)
res, err := sendJenkinsRequest(baseUrl, req)
if err != nil {
@@ -564,7 +573,7 @@ func GithubWebhook(req *http.Request) ([]byte, error) {
func GetBranchNodesDetail(projectName, pipelineName, branchName, runId string, req *http.Request) ([]NodesDetail, error) {
getNodesUrl := fmt.Sprintf(jenkins.Server+GetBranchPipeRunNodesUrl+req.URL.RawQuery, projectName, pipelineName, branchName, runId)
log.Infof("getNodesUrl: " + getNodesUrl)
log.Info("getNodesUrl: " + getNodesUrl)
var wg sync.WaitGroup
var nodesDetails []NodesDetail
stepChan := make(chan *NodesStepsIndex, channelMaxCapacity)
@@ -611,7 +620,7 @@ func GetBranchNodesDetail(projectName, pipelineName, branchName, runId string, r
func GetNodesDetail(projectName, pipelineName, runId string, req *http.Request) ([]NodesDetail, error) {
getNodesUrl := fmt.Sprintf(jenkins.Server+GetPipeRunNodesUrl+req.URL.RawQuery, projectName, pipelineName, runId)
log.Infof("getNodesUrl: " + getNodesUrl)
log.Info("getNodesUrl: " + getNodesUrl)
var wg sync.WaitGroup
var nodesDetails []NodesDetail
stepChan := make(chan *NodesStepsIndex, channelMaxCapacity)
@@ -658,10 +667,15 @@ func GetNodesDetail(projectName, pipelineName, runId string, req *http.Request)
// create jenkins request
func sendJenkinsRequest(baseUrl string, req *http.Request) ([]byte, error) {
resBody, _, err := jenkinsClient(baseUrl, req)
return resBody, err
}
func jenkinsClient(baseUrl string, req *http.Request) ([]byte, http.Header, error) {
newReqUrl, err := url.Parse(baseUrl)
if err != nil {
log.Error(err)
return nil, err
return nil, nil, err
}
client := &http.Client{Timeout: 30 * time.Second}
@@ -678,19 +692,22 @@ func sendJenkinsRequest(baseUrl string, req *http.Request) ([]byte, error) {
resp, err := client.Do(newRequest)
if err != nil {
log.Error(err)
return nil, err
return nil, nil, err
}
defer resp.Body.Close()
resBody, _ := getRespBody(resp)
defer resp.Body.Close()
if resp.StatusCode >= http.StatusBadRequest {
log.Errorf("%+v", string(resBody))
jkerr := new(JkError)
jkerr.Code = resp.StatusCode
jkerr.Message = http.StatusText(resp.StatusCode)
return nil, jkerr
return nil, nil, jkerr
}
return resBody, err
return resBody, resp.Header, nil
}
// Decompress response.body of JenkinsAPIResponse

894
pkg/models/devops/json.go
View File

File diff suppressed because it is too large Load Diff

68
pkg/models/devops/urlconfig.go
View File

@@ -19,43 +19,45 @@ package devops
// Some apis for Jenkins.
const (
GetPipeBranchUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/?"
GetPipeBranchUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/?"
GetBranchPipeUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/"
GetPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/"
GetPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/"
SearchPipelineUrl = "/blue/rest/search/?"
RunBranchPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/"
RunPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/"
GetPipelineRunUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/"
GetPipeBranchRunUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/"
SearchPipelineRunUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/?"
GetBranchPipeRunNodesUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/nodes/?"
GetPipeRunNodesUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/nodes/?"
GetBranchRunLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/log/?"
GetRunLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/log/?"
GetBranchStepLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/nodes/%s/steps/%s/log/?"
GetStepLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/nodes/%s/steps/%s/log/?"
RunBranchPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/"
RunPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/"
GetPipelineRunUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/"
GetPipeBranchRunUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/"
SearchPipelineRunUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/?"
GetBranchPipeRunNodesUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/nodes/?"
GetPipeRunNodesUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/nodes/?"
GetBranchRunLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/log/?"
GetRunLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/log/?"
GetBranchStepLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/nodes/%s/steps/%s/log/?"
GetStepLogUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/nodes/%s/steps/%s/log/?"
StopBranchPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/stop/?"
StopPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/stop/?"
ReplayBranchPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/replay/"
ReplayPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/replay/"
GetBranchArtifactsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/artifacts/?"
GetArtifactsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/artifacts/?"
GetBranchStepsStatusUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/nodes/%s/steps/?"
GetStepsStatusUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/nodes/%s/steps/?"
CheckBranchPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/nodes/%s/steps/%s/"
CheckPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/nodes/%s/steps/%s/"
GetBranchNodeStepsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/branches/%s/runs/%s/nodes/%s/steps/?"
GetNodeStepsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/%s/runs/%s/nodes/%s/steps/?"
ValidateUrl = "/blue/rest/organizations/jenkins/scm/%s/validate"
GetSCMOrgUrl = "/blue/rest/organizations/jenkins/scm/%s/organizations/?"
GetOrgRepoUrl = "/blue/rest/organizations/jenkins/scm/%s/organizations/%s/repositories/?"
GetConsoleLogUrl = "/job/%s/job/%s/indexing/consoleText"
ScanBranchUrl = "/job/%s/job/%s/build?"
GetCrumbUrl = "/crumbIssuer/api/json/"
CheckScriptCompileUrl = "/job/init-job/descriptorByName/org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition/checkScriptCompile"
CheckCronUrl = "/job/init-job/descriptorByName/hudson.triggers.TimerTrigger/checkSpec?"
ToJenkinsfileUrl = "/pipeline-model-converter/toJenkinsfile"
ToJsonUrl = "/pipeline-model-converter/toJson"
GetNotifyCommitUrl = "/git/notifyCommit/?"
GithubWebhookUrl = "/github-webhook/"
GetBranchArtifactsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/artifacts/?"
GetArtifactsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/artifacts/?"
GetBranchStepsStatusUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/nodes/%s/steps/?"
GetStepsStatusUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/nodes/%s/steps/?"
CheckBranchPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/nodes/%s/steps/%s/"
CheckPipelineUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/nodes/%s/steps/%s/"
GetBranchNodeStepsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/branches/%s/runs/%s/nodes/%s/steps/?"
GetNodeStepsUrl = "/blue/rest/organizations/jenkins/pipelines/%s/pipelines/%s/runs/%s/nodes/%s/steps/?"
ValidateUrl = "/blue/rest/organizations/jenkins/scm/%s/validate"
GetSCMOrgUrl = "/blue/rest/organizations/jenkins/scm/%s/organizations/?"
GetOrgRepoUrl = "/blue/rest/organizations/jenkins/scm/%s/organizations/%s/repositories/?"
GetConsoleLogUrl = "/job/%s/job/%s/indexing/consoleText"
ScanBranchUrl = "/job/%s/job/%s/build?"
GetCrumbUrl = "/crumbIssuer/api/json/"
CheckScriptCompileUrl = "/job/init-job/descriptorByName/org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition/checkScriptCompile"
CheckCronUrl = "/job/init-job/descriptorByName/hudson.triggers.TimerTrigger/checkSpec?"
ToJenkinsfileUrl = "/pipeline-model-converter/toJenkinsfile"
ToJsonUrl = "/pipeline-model-converter/toJson"
GetNotifyCommitUrl = "/git/notifyCommit/?"
GithubWebhookUrl = "/github-webhook/"
)

2
pkg/models/iam/im.go
View File

@@ -284,7 +284,7 @@ func ListUsers(conditions *params.Conditions, orderBy string, reverse bool, limi
defer conn.Close()
pageControl := ldap.NewControlPaging(80)
pageControl := ldap.NewControlPaging(1000)
users := make([]models.User, 0)

6
pkg/models/iam/policy/policy.go
View File

@@ -905,7 +905,7 @@ var (
Rules: []v1.PolicyRule{
{
Verbs: []string{"get", "list"},
APIGroups: []string{"resources.kubesphere.io"},
APIGroups: []string{"resources.kubesphere.io", "app.k8s.io"},
Resources: []string{"applications"},
}, {
Verbs: []string{"get", "list"},
@@ -927,7 +927,7 @@ var (
Rules: []v1.PolicyRule{
{
Verbs: []string{"create", "update", "patch"},
APIGroups: []string{"resources.kubesphere.io"},
APIGroups: []string{"resources.kubesphere.io", "app.k8s.io"},
Resources: []string{"applications"},
}, {
Verbs: []string{"create", "update", "patch"},
@@ -940,7 +940,7 @@ var (
Rules: []v1.PolicyRule{
{
Verbs: []string{"delete"},
APIGroups: []string{"resources.kubesphere.io"},
APIGroups: []string{"resources.kubesphere.io", "app.k8s.io"},
Resources: []string{"applications"},
},
{

10
pkg/models/metrics/metricsruleconst.go
View File

@@ -400,10 +400,10 @@ var RulePromQLTmplMap = MetricMap{
"cluster_disk_read_throughput": "sum(node:data_volume_throughput_bytes_read:sum)",
"cluster_disk_write_throughput": "sum(node:data_volume_throughput_bytes_written:sum)",
"cluster_disk_size_usage": `sum(max((node_filesystem_size_bytes{device=~"/dev/.+", job="node-exporter"} - node_filesystem_avail_bytes{device=~"/dev/.+", job="node-exporter"}) * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:) by (node))`,
"cluster_disk_size_usage": `sum(max(node_filesystem_size_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"} - node_filesystem_avail_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"}) by (device, instance))`,
"cluster_disk_size_utilisation": `cluster:disk_utilization:ratio`,
"cluster_disk_size_capacity": `sum(max(node_filesystem_size_bytes{device=~"/dev/.+", job="node-exporter"} * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:) by (node))`,
"cluster_disk_size_available": `sum(max(node_filesystem_avail_bytes{device=~"/dev/.+", job="node-exporter"} * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:) by (node))`,
"cluster_disk_size_capacity": `sum(max(node_filesystem_size_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"}) by (device, instance))`,
"cluster_disk_size_available": `sum(max(node_filesystem_avail_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"}) by (device, instance))`,
"cluster_disk_inode_total": `sum(node:node_inodes_total:)`,
"cluster_disk_inode_usage": `sum(node:node_inodes_total:) - sum(node:node_inodes_free:)`,
@@ -481,9 +481,9 @@ var RulePromQLTmplMap = MetricMap{
"node_disk_read_throughput": "node:data_volume_throughput_bytes_read:sum",
"node_disk_write_throughput": "node:data_volume_throughput_bytes_written:sum",
"node_disk_size_capacity": `max(node_filesystem_size_bytes{device=~"/dev/.+", job="node-exporter"} * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:$1) by (node)`,
"node_disk_size_capacity": `sum(max(node_filesystem_size_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"} * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:$1) by (device, node)) by (node)`,
"node_disk_size_available": `node:disk_space_available:$1`,
"node_disk_size_usage": `max((node_filesystem_size_bytes{device=~"/dev/.+", job="node-exporter"} - node_filesystem_avail_bytes{device=~"/dev/.+", job="node-exporter"}) * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:$1) by (node)`,
"node_disk_size_usage": `sum(max((node_filesystem_size_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"} - node_filesystem_avail_bytes{device=~"/dev/[vsh]d.+", job="node-exporter"}) * on (namespace, pod) group_left(node) node_namespace_pod:kube_pod_info:$1) by (device, node)) by (node)`,
"node_disk_size_utilisation": `node:disk_space_utilization:ratio$1`,
"node_disk_inode_total": `node:node_inodes_total:$1`,

4
pkg/models/routers/routers.go
View File

@@ -112,6 +112,10 @@ func getMasterNodeIp() string {
func addLoadBalancerIp(service *corev1.Service) {
if service == nil {
return
}
// append selected node ip as loadbalancer ingress ip
if service.Spec.Type != corev1.ServiceTypeLoadBalancer && len(service.Status.LoadBalancer.Ingress) == 0 {
rip := getMasterNodeIp()

48
pkg/simple/client/elasticsearch/esclient.go
View File

@@ -137,12 +137,9 @@ type ContainerHighLightField struct {
type EmptyField struct {
}
// The aggs object holds two aggregations to be computed by Elasticsearch
// ContainterAgg is a cardinality aggregation to calculate the count of distinct containers
// StartTimeAgg is a top hits aggregation to retrieve the first record
// StatisticsAggs, the struct for `aggs` of type Request, holds a cardinality aggregation for distinct container counting
type StatisticsAggs struct {
ContainerAgg ContainerAgg `json:"containers"`
StartTimeAgg StartTimeAgg `json:"starttime"`
}
type ContainerAgg struct {
@@ -153,23 +150,6 @@ type AggField struct {
Field string `json:"field"`
}
type StartTimeAgg struct {
TopHits TopHits `json:"top_hits"`
}
type TopHits struct {
Sort []TopHitsSort `json:"sort"`
Size int `json:"size"`
}
type TopHitsSort struct {
Order TopHitsSortOrder `json:"time"`
}
type TopHitsSortOrder struct {
Type string `json:"order"`
}
type HistogramAggs struct {
HistogramAgg HistogramAgg `json:"histogram"`
}
@@ -263,8 +243,7 @@ func createQueryRequest(param QueryParameters) (int, []byte, error) {
if param.Operation == "statistics" {
operation = OperationStatistics
containerAgg := AggField{"kubernetes.docker_id.keyword"}
startTimeAgg := TopHits{[]TopHitsSort{{TopHitsSortOrder{"asc"}}}, 1}
statisticAggs := StatisticsAggs{ContainerAgg{containerAgg}, StartTimeAgg{startTimeAgg}}
statisticAggs := StatisticsAggs{ContainerAgg{containerAgg}}
request.Aggs = statisticAggs
request.Size = 0
} else if param.Operation == "histogram" {
@@ -369,20 +348,15 @@ type ReadResult struct {
Records []LogRecord `json:"records,omitempty"`
}
// The aggregations object represents the return from an aggregation (see StatisticsAggs type)
// StatisticsResponseAggregations, the struct for `aggregations` of type Reponse, holds return results from the aggregation StatisticsAggs
type StatisticsResponseAggregations struct {
ContainerCount ContainerCount `json:"containers"`
StartTime StartTimeTopHit `json:"starttime"`
ContainerCount ContainerCount `json:"containers"`
}
type ContainerCount struct {
Value int64 `json:"value"`
}
type StartTimeTopHit struct {
Hits Hits `json:"hits"`
}
type HistogramAggregations struct {
HistogramAggregation HistogramAggregation `json:"histogram"`
}
@@ -404,7 +378,6 @@ type HistogramRecord struct {
type StatisticsResult struct {
Containers int64 `json:"containers"`
Logs int64 `json:"logs"`
StartTime int64 `json:"starttime"`
}
type HistogramResult struct {
@@ -468,13 +441,14 @@ func parseQueryResult(operation int, param QueryParameters, body []byte, query [
if response.Status != 0 {
//Elastic error, eg, es_rejected_execute_exception
queryResult.Status = response.Status
glog.Errorln("The query failed with no response")
return &queryResult
}
if response.Shards.Successful != response.Shards.Total {
//Elastic some shards error
queryResult.Status = http.StatusInternalServerError
return &queryResult
glog.Warningf("Not all shards succeed, successful shards: %d, skipped shards: %d, failed shards: %d",
response.Shards.Successful, response.Shards.Skipped, response.Shards.Failed)
}
switch operation {
@@ -504,8 +478,7 @@ func parseQueryResult(operation int, param QueryParameters, body []byte, query [
queryResult.Status = http.StatusInternalServerError
return &queryResult
}
queryResult.Statistics = &StatisticsResult{Containers: statisticsResponse.ContainerCount.Value,
Logs: statisticsResponse.StartTime.Hits.Total, StartTime: statisticsResponse.StartTime.Hits.Hits[0].Sort[0]}
queryResult.Statistics = &StatisticsResult{Containers: statisticsResponse.ContainerCount.Value, Logs: response.Hits.Total}
case OperationHistogram:
var histogramResult HistogramResult
@@ -574,14 +547,10 @@ func stubResult() *QueryResult {
func Query(param QueryParameters) *QueryResult {
var queryResult *QueryResult
//queryResult = stubResult()
//return queryResult
client := &http.Client{}
operation, query, err := createQueryRequest(param)
if err != nil {
//fmt.Println("Create query error ", err.Error())
queryResult = new(QueryResult)
queryResult.Status = http.StatusNotFound
return queryResult
@@ -595,6 +564,7 @@ func Query(param QueryParameters) *QueryResult {
}
url := fmt.Sprintf("http://%s:%s/%s*/_search", es.Host, es.Port, es.Index)
request, err := http.NewRequest("GET", url, bytes.NewBuffer(query))
if err != nil {
glog.Errorln(err)

30
pkg/simple/client/kubesphere/kubesphereclient.go
View File

@@ -27,7 +27,6 @@ import (
"kubesphere.io/kubesphere/pkg/constants"
"kubesphere.io/kubesphere/pkg/models"
"kubesphere.io/kubesphere/pkg/models/devops"
"log"
"net/http"
"strings"
"sync"
@@ -84,20 +83,22 @@ func (c client) CreateGroup(group *models.Group) (*models.Group, error) {
req, err := http.NewRequest(http.MethodPost, fmt.Sprintf("%s/kapis/iam.kubesphere.io/v1alpha2/groups", accountAPIServer), bytes.NewReader(data))
if err != nil {
glog.Error(err)
return nil, err
}
req.Header.Add("Content-Type", "application/json")
log.Println(req.Method, req.URL, string(data))
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return nil, err
}
defer resp.Body.Close()
data, err = ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -108,6 +109,7 @@ func (c client) CreateGroup(group *models.Group) (*models.Group, error) {
err = json.Unmarshal(data, group)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -118,20 +120,18 @@ func (c client) UpdateGroup(group *models.Group) (*models.Group, error) {
data, err := json.Marshal(group)
if err != nil {
glog.Error(err)
return nil, err
}
req, err := http.NewRequest(http.MethodPut, fmt.Sprintf("%s/kapis/iam.kubesphere.io/v1alpha2/groups/%s", accountAPIServer, group.Name), bytes.NewReader(data))
if err != nil {
glog.Error(err)
return nil, err
}
req.Header.Add("Content-Type", "application/json")
if err != nil {
return nil, err
}
log.Println(req.Method, req.URL, string(data))
resp, err := c.client.Do(req)
if err != nil {
@@ -141,6 +141,7 @@ func (c client) UpdateGroup(group *models.Group) (*models.Group, error) {
data, err = ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -151,6 +152,7 @@ func (c client) UpdateGroup(group *models.Group) (*models.Group, error) {
err = json.Unmarshal(data, group)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -161,19 +163,21 @@ func (c client) DeleteGroup(name string) error {
req, err := http.NewRequest(http.MethodDelete, fmt.Sprintf("%s/kapis/iam.kubesphere.io/v1alpha2/groups/%s", accountAPIServer, name), nil)
if err != nil {
glog.Error(err)
return err
}
log.Println(req.Method, req.URL)
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return err
}
defer resp.Body.Close()
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return err
}
@@ -188,18 +192,20 @@ func (c client) DescribeGroup(name string) (*models.Group, error) {
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/kapis/iam.kubesphere.io/v1alpha2/groups/%s", accountAPIServer, name), nil)
if err != nil {
glog.Error(err)
return nil, err
}
log.Println(req.Method, req.URL)
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return nil, err
}
defer resp.Body.Close()
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -211,6 +217,7 @@ func (c client) DescribeGroup(name string) (*models.Group, error) {
err = json.Unmarshal(data, &group)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -224,19 +231,17 @@ func (c client) ListUsers() (*models.PageableResponse, error) {
return nil, err
}
req.Header.Add("Authorization", accountAPIServer)
if err != nil {
return nil, err
}
log.Println(req.Method, req.URL)
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return nil, err
}
defer resp.Body.Close()
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return nil, err
}
@@ -248,6 +253,7 @@ func (c client) ListUsers() (*models.PageableResponse, error) {
err = json.Unmarshal(data, &result)
if err != nil {
glog.Error(err)
return nil, err
}

168
pkg/simple/client/openpitrix/openpitrixclient.go
View File

@@ -22,9 +22,11 @@ import (
"encoding/json"
"flag"
"fmt"
"github.com/golang/glog"
"io/ioutil"
"log"
"kubesphere.io/kubesphere/pkg/utils/sliceutil"
"net/http"
"strings"
"sync"
)
@@ -48,6 +50,10 @@ type Interface interface {
CreateRuntime(runtime *RunTime) error
DeleteRuntime(runtimeId string) error
}
type cluster struct {
Status string `json:"status"`
ClusterId string `json:"cluster_id"`
}
type Error struct {
status int
@@ -89,54 +95,174 @@ func (c client) CreateRuntime(runtime *RunTime) error {
req.Header.Add("Content-Type", "application/json")
req.Header.Add("Authorization", openpitrixProxyToken)
log.Println(req.Method, req.URL, openpitrixProxyToken, string(data))
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return err
}
defer resp.Body.Close()
data, err = ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return err
}
if resp.StatusCode > http.StatusOK {
return Error{resp.StatusCode, string(data)}
err = Error{resp.StatusCode, string(data)}
glog.Error(err)
return err
}
return nil
}
func (c client) deleteClusters(clusters []cluster) error {
clusterId := make([]string, 0)
for _, cluster := range clusters {
if cluster.Status != "deleted" && cluster.Status != "deleting" && !sliceutil.HasString(clusterId, cluster.ClusterId) {
clusterId = append(clusterId, cluster.ClusterId)
}
}
if len(clusterId) == 0 {
return nil
}
deleteRequest := struct {
ClusterId []string `json:"cluster_id"`
}{
ClusterId: clusterId,
}
data, _ := json.Marshal(deleteRequest)
req, err := http.NewRequest(http.MethodPost, fmt.Sprintf("%s/v1/clusters/delete", openpitrixAPIServer), bytes.NewReader(data))
if err != nil {
return err
}
req.Header.Add("Authorization", openpitrixProxyToken)
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return err
}
defer resp.Body.Close()
data, err = ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return err
}
if resp.StatusCode > http.StatusOK {
err = Error{resp.StatusCode, string(data)}
glog.Error(err)
return err
}
return nil
}
func (c client) listClusters(runtimeId string) ([]cluster, error) {
limit := 200
offset := 0
clusters := make([]cluster, 0)
for {
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/v1/clusters?runtime_id=%s&limit=%d&offset=%d", openpitrixAPIServer, runtimeId, limit, offset), nil)
if err != nil {
glog.Error(err)
return nil, err
}
req.Header.Add("Authorization", openpitrixProxyToken)
resp, err := c.client.Do(req)
if err != nil {
glog.Error(err)
return nil, err
}
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
glog.Error(err)
return nil, err
}
resp.Body.Close()
if resp.StatusCode > http.StatusOK {
err = Error{resp.StatusCode, string(data)}
glog.Error(err)
return nil, err
}
listClusterResponse := struct {
TotalCount int `json:"total_count"`
ClusterSet []cluster `json:"cluster_set"`
}{}
err = json.Unmarshal(data, &listClusterResponse)
if err != nil {
glog.Error(err)
return nil, err
}
clusters = append(clusters, listClusterResponse.ClusterSet...)
if listClusterResponse.TotalCount <= limit+offset {
break
}
offset += limit
}
return clusters, nil
}
func (c client) DeleteRuntime(runtimeId string) error {
data := []byte(fmt.Sprintf(`{"runtime_id":"%s"}`, runtimeId))
req, err := http.NewRequest(http.MethodDelete, fmt.Sprintf("%s/v1/runtimes", openpitrixAPIServer), bytes.NewReader(data))
clusters, err := c.listClusters(runtimeId)
if err != nil {
glog.Error(err)
return err
}
req.Header.Add("Authorization", openpitrixProxyToken)
if err != nil {
return err
}
log.Println(req.Method, req.URL)
resp, err := c.client.Do(req)
err = c.deleteClusters(clusters)
if err != nil {
glog.Error(err)
return err
}
defer resp.Body.Close()
data, err = ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
if resp.StatusCode > http.StatusOK {
return Error{resp.StatusCode, string(data)}
}
return nil
}
func IsNotFound(err error) bool {
if e, ok := err.(Error); ok {
if e.status == http.StatusNotFound {
return true
}
if strings.Contains(e.message, "not exist") {
return true
}
if strings.Contains(e.message, "not found") {
return true
}
}
return false
}
func IsDeleted(err error) bool {
if e, ok := err.(Error); ok {
if strings.Contains(e.message, "is [deleted]") {
return true
}
}
return false
}

52
vendor/github.com/kubernetes-sigs/application/pkg/genericreconciler/genericreconciler.go generated vendored
View File

@@ -71,7 +71,6 @@ func (gr *Reconciler) observe(observables ...resource.Observable) (*resource.Obj
types.NamespacedName{Name: name, Namespace: namespace},
obs.Obj.(runtime.Object))
if err == nil {
log.Printf(" >>get: %s", otype+"/"+namespace+"/"+name)
resources = append(resources, resource.Object{Obj: obs.Obj})
} else {
log.Printf(" >>>ERR get: %s", otype+"/"+namespace+"/"+name)
@@ -147,11 +146,9 @@ func (gr *Reconciler) ReconcileCR(namespacedname types.NamespacedName, handle cr
err := gr.Get(context.TODO(), namespacedname, rsrc.(runtime.Object))
if err == nil {
o := rsrc.(metav1.Object)
log.Printf("%s Validating spec\n", name)
err = rsrc.Validate()
status = rsrc.NewStatus()
if err == nil {
log.Printf("%s Applying defaults\n", name)
rsrc.ApplyDefaults()
components := rsrc.Components()
for _, component := range components {
@@ -230,8 +227,8 @@ func (gr *Reconciler) ObserveAndMutate(crname string, c component.Component, sta
// FinalizeComponent is a function that finalizes component
func (gr *Reconciler) FinalizeComponent(crname string, c component.Component, status interface{}, aggregated *resource.ObjectBag) error {
cname := crname + "(cmpnt:" + c.Name + ")"
log.Printf("%s { finalizing component\n", cname)
defer log.Printf("%s } finalizing component\n", cname)
log.Printf("%s finalizing component\n", cname)
defer log.Printf("%s finalizing component completed", cname)
expected, observed, err := gr.ObserveAndMutate(crname, c, status, false, aggregated)
@@ -249,8 +246,8 @@ func (gr *Reconciler) ReconcileComponent(crname string, c component.Component, s
var reconciled *resource.ObjectBag = new(resource.ObjectBag)
cname := crname + "(cmpnt:" + c.Name + ")"
log.Printf("%s { reconciling component\n", cname)
defer log.Printf("%s } reconciling component\n", cname)
log.Printf("%s reconciling component\n", cname)
defer log.Printf("%s reconciling component completed\n", cname)
expected, observed, err := gr.ObserveAndMutate(crname, c, status, true, aggregated)
@@ -270,17 +267,8 @@ func (gr *Reconciler) ReconcileComponent(crname string, c component.Component, s
errs = handleErrorArr("", crname, err, errs)
} else {
aggregated.Add(expected.Items()...)
log.Printf("%s Expected Resources:\n", cname)
for _, e := range expected.Items() {
log.Printf("%s exp: %s/%s/%s\n", cname, e.Obj.GetNamespace(), reflect.TypeOf(e.Obj).String(), e.Obj.GetName())
}
log.Printf("%s Observed Resources:\n", cname)
for _, e := range observed.Items() {
log.Printf("%s obs: %s/%s/%s\n", cname, e.Obj.GetNamespace(), reflect.TypeOf(e.Obj).String(), e.Obj.GetName())
}
log.Printf("%s Reconciling Resources:\n", cname)
}
for _, e := range expected.Items() {
seen := false
eNamespace := e.Obj.GetNamespace()
@@ -298,11 +286,7 @@ func (gr *Reconciler) ReconcileComponent(crname string, c component.Component, s
if e.Lifecycle == resource.LifecycleManaged && (specDiffers(e.Obj, o.Obj) && c.Differs(e.Obj, o.Obj) || injectOwnerRefs(e.Obj, c.OwnerRef)) {
if err := gr.Update(context.TODO(), e.Obj.(runtime.Object).DeepCopyObject()); err != nil {
errs = handleErrorArr("update", eRsrcInfo, err, errs)
} else {
log.Printf("%s update: %s\n", cname, eRsrcInfo)
}
} else {
log.Printf("%s nochange: %s\n", cname, eRsrcInfo)
}
reconciled.Add(o)
seen = true
@@ -315,7 +299,6 @@ func (gr *Reconciler) ReconcileComponent(crname string, c component.Component, s
if err := gr.Create(context.TODO(), e.Obj.(runtime.Object)); err != nil {
errs = handleErrorArr("Create", cname, err, errs)
} else {
log.Printf("%s +create: %s\n", cname, eRsrcInfo)
reconciled.Add(e)
}
} else {
@@ -325,31 +308,6 @@ func (gr *Reconciler) ReconcileComponent(crname string, c component.Component, s
}
}
// delete(observed - expected)
for _, o := range observed.Items() {
seen := false
oNamespace := o.Obj.GetNamespace()
oName := o.Obj.GetName()
oKind := reflect.TypeOf(o.Obj).String()
oRsrcInfo := oKind + "/" + oNamespace + "/" + oName
for _, e := range expected.Items() {
if (e.Obj.GetName() == oName) &&
(e.Obj.GetNamespace() == oNamespace) &&
(reflect.TypeOf(o.Obj).String() == oKind) {
seen = true
break
}
}
// rsrc is in observed but not in expected - delete
if !seen {
if err := gr.Delete(context.TODO(), o.Obj.(runtime.Object)); err != nil {
errs = handleErrorArr("delete", oRsrcInfo, err, errs)
} else {
log.Printf("%s -delete: %s\n", cname, oRsrcInfo)
}
}
}
err = utilerrors.NewAggregate(errs)
c.UpdateComponentStatus(c.CR, status, reconciled, err)
return err