High severityNVD Advisory· Published Apr 25, 2025· Updated May 23, 2025
Rancher users who can create Projects can gain access to arbitrary projects
CVE-2024-22031
Description
Impact
A vulnerability has been identified within Rancher where a user with the ability to create a project, on a certain cluster, can create a project with the same name as an existing project in a different cluster. This results in the user gaining access to the other project in the different cluster, resulting in a privilege escalation. This happens because the namespace used on the local cluster to store related resources (PRTBs and secrets) is the name of the project.
Please consult the associated MITRE ATT&CK - Technique - Privilege Escalation for further information about this category of attack.
Patches
Patched versions include releases v2.11.1, v2.10.5, v2.9.9.
The fix involves the following changes:
Rancher: - Instead of using the project name as the namespace, Rancher will instead be using a new field on the project spec called backingNamespace. If that field exists, use that for the project namespace going forward. However, if the project does not have that field filled out (likely because it existed before this change), Rancher will continue using the name for the namespace.
Rancher Webhook: - New mutation on create project.Status.BackingNamespace to be SafeConcatName(project.Spec.ClusterName, project.Name); - Generate the name manually within the mutating webhook, because normally, name generation happens after the mutating webhooks; - Removed a validation where projectName and Namespace had to be the same for PRTBs, since PRTBs now go in project.BackingNamespace; - On update, if BackingNamespace isn't set, set it to project.Name. For existing objects after update this will help unify them to the new projects. - The BackingNamespace can't be edited after it's set.
**Note: Rancher v2.8 release line does not have the fix for this CVE. The fix for v2.8 was considered too complex and with the risk of introducing instabilities right before this version goes into end-of-life (EOL), as documented in SUSE’s Product Support Lifecycle page. Please see the section below for workarounds or consider upgrading to a newer and patched version of Rancher.**
Workarounds
If you can't upgrade to a fixed version, please make sure that: - Users are not allowed to create projects with the same object names from another cluster.
To identify if this security issue could have been abused within your system, you need to find if there are any projects with the same name but on different clusters. To do that, run the following command in the local cluster as an administrator: `` kubectl get projects -A -o=custom-columns='NAME:metadata.name' | sort | uniq -c ``
That command will list all project names, and show the instances of each name. Any project with more than 1 instance is affected by this security issue. To remedy the situation, the projects will need to be deleted and re-created to ensure no namespace collisions happen. While it would be possible to delete all but 1 of the projects with the same name, this is unadvisable because a user could have given themselves access to the wrong project.
References
If you have any questions or comments about this advisory: - Reach out to the SUSE Rancher Security team for security related inquiries. - Open an issue in the Rancher repository. - Verify with our support matrix and product support lifecycle.
Affected packages
Versions sourced from the GitHub Security Advisory.
| Package | Affected versions | Patched versions |
|---|---|---|
github.com/rancher/rancherGo | >= 2.8.0, < 2.9.9 | 2.9.9 |
github.com/rancher/rancherGo | >= 2.10.0, < 2.10.5 | 2.10.5 |
github.com/rancher/rancherGo | >= 2.11.0, < 2.11.1 | 2.11.1 |
Patches
4b0be28f86fc5[v2.9] Add project namespace handling (#49859)
16 files changed · +441 −135
pkg/api/norman/customization/globalnamespaceaccess/access_common.go+20 −8 modified@@ -5,14 +5,13 @@ import ( "fmt" "strings" - v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - "github.com/rancher/norman/api/access" "github.com/rancher/norman/httperror" "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" "github.com/rancher/norman/types/set" "github.com/rancher/norman/types/slice" + v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" client "github.com/rancher/rancher/pkg/client/generated/management/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" "github.com/rancher/rancher/pkg/rbac" @@ -59,7 +58,7 @@ func (ma *MemberAccess) IsAdmin(callerID string) (bool, error) { return false, err } if u == nil { - return false, fmt.Errorf("No user found with ID %v", callerID) + return false, fmt.Errorf("no user found with ID %v", callerID) } // Get globalRoleBinding for this user grbs, err := ma.GrbLister.List("", labels.NewSelector()) @@ -176,9 +175,17 @@ func (ma *MemberAccess) EnsureRoleInTargets(targetProjects, roleTemplates []stri callerIsProjectOwner := false callerIsProjectMember := false callerIsClusterOwner := false - prtbs, err := ma.PrtbLister.List(pname, labels.NewSelector()) + + p, err := ma.ProjectLister.Get(cname, pname) if err != nil { - return err + return fmt.Errorf("unable to get project %s in namespace %s: %w", pname, cname, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) + if err != nil { + return fmt.Errorf("unable to get PRTBs in namespace %s: %w", backingNamespace, err) } for _, prtb := range prtbs { if prtb.UserName == callerID { @@ -520,21 +527,26 @@ func (ma *MemberAccess) RemoveRolesFromTargets(targetProjects, rolesToRemove []s return httperror.NewAPIError(httperror.InvalidBodyContent, errMsg) } clusterName, projectName := split[0], split[1] - clustersCovered := make(map[string]bool) - prtbs, err := ma.PrtbLister.List(projectName, labels.NewSelector()) + p, err := ma.ProjectLister.Get(clusterName, projectName) + if err != nil { + return fmt.Errorf("unable to get project %s in namespace %s: %w", projectName, clusterName, err) + } + backingNamespace := p.GetProjectBackingNamespace() + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) if err != nil { return err } for _, prtb := range prtbs { if prtb.UserPrincipalName == systemUserPrincipalID { if removeAllRoles || rolesToRemoveMap[prtb.RoleTemplateName] { - if err = ma.Prtbs.DeleteNamespaced(projectName, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { + if err = ma.Prtbs.DeleteNamespaced(backingNamespace, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { return err } } } } + clustersCovered := make(map[string]bool) if !clustersCovered[clusterName] { crtbs, err := ma.CrtbLister.List(clusterName, labels.NewSelector()) if err != nil {
pkg/api/norman/server/managementstored/setup.go+4 −3 modified@@ -177,7 +177,7 @@ func Setup(ctx context.Context, apiContext *config.ScaledContext, clusterManager authn.SetRTBStore(ctx, schemas.Schema(&managementschema.Version, client.ProjectRoleTemplateBindingType), apiContext) nodeStore.SetupStore(schemas.Schema(&managementschema.Version, client.NodeType)) projectaction.SetProjectStore(schemas.Schema(&managementschema.Version, client.ProjectType), apiContext) - setupScopedTypes(schemas) + setupScopedTypes(schemas, apiContext) setupPasswordTypes(ctx, schemas, apiContext) multiclusterapp.SetMemberStore(ctx, schemas.Schema(&managementschema.Version, client.MultiClusterAppType), apiContext) @@ -192,7 +192,8 @@ func setupPasswordTypes(ctx context.Context, schemas *types.Schemas, management passwordStore.SetPasswordStore(schemas, secretStore, nsStore) } -func setupScopedTypes(schemas *types.Schemas) { +func setupScopedTypes(schemas *types.Schemas, management *config.ScaledContext) { + projectLister := management.Management.Projects("").Controller().Lister() for _, schema := range schemas.Schemas() { if schema.Scope != types.NamespaceScope || schema.Store == nil || schema.Store.Context() != config.ManagementStorageContext { continue @@ -208,7 +209,7 @@ func setupScopedTypes(schemas *types.Schemas) { continue } - schema.Store = scoped.NewScopedStore(key, schema.Store) + schema.Store = scoped.NewScopedStore(key, schema.Store, projectLister) ns.Required = false schema.ResourceFields["namespaceId"] = ns break
pkg/api/norman/store/scoped/store.go+22 −7 modified@@ -7,14 +7,16 @@ import ( "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" client "github.com/rancher/rancher/pkg/client/generated/management/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" ) type Store struct { types.Store - key string + key string + projectCache v3.ProjectLister } -func NewScopedStore(key string, store types.Store) *Store { +func NewScopedStore(key string, store types.Store, pLister v3.ProjectLister) *Store { return &Store{ Store: &transform.Store{ Store: store, @@ -23,21 +25,34 @@ func NewScopedStore(key string, store types.Store) *Store { return data, nil } v := convert.ToString(data[key]) - if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) { + if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) && v != strings.Replace(convert.ToString(data[client.ProjectFieldNamespaceId]), "-", ":", 1) { data[key] = data[client.ProjectFieldNamespaceId] } + data[client.ProjectFieldNamespaceId] = nil return data, nil }, }, - key: key, + key: key, + projectCache: pLister, } } func (s *Store) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { - if data != nil { - parts := strings.Split(convert.ToString(data[s.key]), ":") - data["namespaceId"] = parts[len(parts)-1] + if data == nil { + return s.Store.Create(apiContext, schema, data) + } + + clusterName, projectName, isProject := strings.Cut(convert.ToString(data[s.key]), ":") + if isProject { + p, err := s.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, err + } + data[client.ProjectFieldNamespaceId] = p.GetProjectBackingNamespace() + } else { + data[client.ProjectFieldNamespaceId] = data[s.key] } + return s.Store.Create(apiContext, schema, data) }
pkg/api/norman/store/scoped/store_test.go+142 −0 added@@ -0,0 +1,142 @@ +package scoped + +import ( + "fmt" + "reflect" + "testing" + + "github.com/rancher/norman/types" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" + "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type fakeStore struct { +} + +func (f fakeStore) Context() types.StorageContext { + return "" +} +func (f fakeStore) ByID(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) List(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) ([]map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { + return data, nil +} +func (f fakeStore) Update(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Delete(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Watch(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) (chan map[string]interface{}, error) { + return nil, nil +} + +func TestStoreCreate(t *testing.T) { + store := fakeStore{} + + p := fakes.ProjectListerMock{} + + tests := []struct { + name string + key string + getFunc func(string, string) (*v3.Project, error) + data map[string]interface{} + want map[string]interface{} + wantErr bool + }{ + { + name: "nil data returns nil", + data: nil, + want: nil, + }, + { + name: "project: set namespace no backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "project-XYZ", + }, + }, + { + name: "project: set namespace to backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "c-ABC-p-XYZ", + }, + }, + { + name: "error getting project", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + return nil, fmt.Errorf("error") + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + wantErr: true, + }, + { + name: "cluster: set namespaceId", + key: "clusterId", + data: map[string]interface{}{ + "clusterId": "cluster-ABC", + }, + want: map[string]interface{}{ + "clusterId": "cluster-ABC", + "namespaceId": "cluster-ABC", + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p.GetFunc = tt.getFunc + s := &Store{ + Store: store, + key: tt.key, + projectCache: &p, + } + got, err := s.Create(nil, nil, tt.data) + if (err != nil) != tt.wantErr { + t.Errorf("Store.Create() error = %v, wantErr %v", err, tt.wantErr) + return + } + if !reflect.DeepEqual(got, tt.want) { + t.Errorf("Store.Create() = %v, want %v", got, tt.want) + } + }) + } +}
pkg/apis/management.cattle.io/v3/authz_types.go+8 −0 modified@@ -48,6 +48,14 @@ func (p *Project) ObjClusterName() string { return p.Spec.ObjClusterName() } +// GetProjectBackingNamespace returns the namespace a project uses in the local cluster to store PRTBs and Project Scoped Secrets. +func (p *Project) GetProjectBackingNamespace() string { + if p.Status.BackingNamespace != "" { + return p.Status.BackingNamespace + } + return p.Name +} + // ProjectStatus represents the most recently observed status of the project. type ProjectStatus struct { // Conditions are a set of indicators about aspects of the project.
pkg/controllers/management/auth/crtb_handler.go+8 −5 modified@@ -190,11 +190,13 @@ func (c *crtbLifecycle) reconcileBindings(binding *v3.ClusterRoleTemplateBinding return err } for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() if p.DeletionTimestamp != nil { - logrus.Warnf("Project %v is being deleted, not creating membership bindings", p.Name) + logrus.Warnf("Project %v is being deleted, not creating membership bindings", backingNamespace) continue } - if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, p.Name, projectManagementPlaneResources, subject, binding); err != nil { + + if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, backingNamespace, projectManagementPlaneResources, subject, binding); err != nil { return err } } @@ -208,14 +210,15 @@ func (c *crtbLifecycle) removeMGMTClusterScopedPrivilegesInProjectNamespace(bind } bindingKey := pkgrbac.GetRTBLabel(binding.ObjectMeta) for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() set := labels.Set(map[string]string{bindingKey: CrtbInProjectBindingOwner}) - rbs, err := c.rbLister.List(p.Name, set.AsSelector()) + rbs, err := c.rbLister.List(backingNamespace, set.AsSelector()) if err != nil { return err } for _, rb := range rbs { - logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, p.Name, binding.Name) - if err := c.rbClient.DeleteNamespaced(p.Name, rb.Name, &metav1.DeleteOptions{}); err != nil { + logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, backingNamespace, binding.Name) + if err := c.rbClient.DeleteNamespaced(backingNamespace, rb.Name, &metav1.DeleteOptions{}); err != nil { return err } }
pkg/controllers/management/auth/crtb_handler_test.go+160 −9 modified@@ -5,16 +5,20 @@ import ( "testing" "time" - v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + rbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" + corefakes "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1/fakes" + "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" ) var ( - e = fmt.Errorf("error") + errDefault = fmt.Errorf("error") defaultCRTB = v3.ClusterRoleTemplateBinding{ UserName: "test", GroupName: "", @@ -37,12 +41,25 @@ var ( Name: "test-project", }, } + backingNamespaceProject = v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-project", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } deletingProject = v3.Project{ ObjectMeta: v1.ObjectMeta{ Name: "deleting-project", DeletionTimestamp: &v1.Time{Time: time.Now()}, }, } + defaultBinding = rbacv1.RoleBinding{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-binding", + }, + } ) type crtbTestState struct { @@ -66,7 +83,7 @@ func TestReconcileBindings(t *testing.T) { name: "error getting cluster", stateSetup: func(cts crtbTestState) { cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -91,7 +108,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). - Return(true, e) + Return(true, errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -108,7 +125,7 @@ func TestReconcileBindings(t *testing.T) { Return(true, nil) cts.managerMock.EXPECT(). ensureClusterMembershipBinding("clustername-clusterowner", gomock.Any(), gomock.Any(), true, gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -129,7 +146,7 @@ func TestReconcileBindings(t *testing.T) { Return(nil) cts.managerMock.EXPECT(). grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -151,7 +168,7 @@ func TestReconcileBindings(t *testing.T) { grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). Return(nil) cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -179,7 +196,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "test-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -237,6 +254,32 @@ func TestReconcileBindings(t *testing.T) { }, crtb: defaultCRTB.DeepCopy(), }, + { + name: "successfully reconcile clustermember with backingNamespace", + stateSetup: func(cts crtbTestState) { + cts.managerMock.EXPECT(). + checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). + Return(false, nil) + cts.managerMock.EXPECT(). + ensureClusterMembershipBinding("clustername-clustermember", gomock.Any(), gomock.Any(), false, gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "c-ABC-p-XYZ", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { + c := defaultCluster.DeepCopy() + return c, nil + } + cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { + p := backingNamespaceProject.DeepCopy() + return []*v3.Project{p}, nil + } + }, + crtb: defaultCRTB.DeepCopy(), + }, { name: "skip projects that are deleting", stateSetup: func(cts crtbTestState) { @@ -252,7 +295,7 @@ func TestReconcileBindings(t *testing.T) { // This should not be called cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "deleting-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e).AnyTimes() + Return(errDefault).AnyTimes() cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { c := defaultCluster.DeepCopy() return c, nil @@ -302,3 +345,111 @@ func setupTest(t *testing.T) crtbTestState { } return state } + +func Test_removeMGMTClusterScopedPrivilegesInProjectNamespace(t *testing.T) { + tests := []struct { + name string + projectListFunc func(string, labels.Selector) ([]*apisv3.Project, error) + roleBindingListFunc func(string, labels.Selector) ([]*rbacv1.RoleBinding, error) + roleBindingDeleteFunc func(string, string, *v1.DeleteOptions) error + binding *v3.ClusterRoleTemplateBinding + wantErr bool + }{ + { + name: "error listing projects", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error listing rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error deleting rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + return errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "successfully delete rolebindings no backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, defaultProject.Name, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, defaultProject.Name, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + { + name: "successfully delete rolebindings with backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + backingNamespaceProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p := fakes.ProjectListerMock{} + p.ListFunc = tt.projectListFunc + rbl := corefakes.RoleBindingListerMock{} + rbl.ListFunc = tt.roleBindingListFunc + rbi := corefakes.RoleBindingInterfaceMock{} + rbi.DeleteNamespacedFunc = tt.roleBindingDeleteFunc + + c := &crtbLifecycle{ + projectLister: &p, + rbLister: &rbl, + rbClient: &rbi, + } + if err := c.removeMGMTClusterScopedPrivilegesInProjectNamespace(tt.binding); (err != nil) != tt.wantErr { + t.Errorf("crtbLifecycle.removeMGMTClusterScopedPrivilegesInProjectNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +}
pkg/controllers/management/auth/project_cluster_handler.go+29 −38 modified@@ -11,7 +11,6 @@ import ( "github.com/rancher/norman/condition" apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/rancher/pkg/controllers" "github.com/rancher/rancher/pkg/controllers/managementuserlegacy/systemimage" wranglerv3 "github.com/rancher/rancher/pkg/generated/controllers/management.cattle.io/v3" @@ -27,7 +26,6 @@ import ( "github.com/sirupsen/logrus" v12 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" - kerrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/api/meta" v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" @@ -91,13 +89,14 @@ func (l *projectLifecycle) sync(key string, orig *apisv3.Project) (runtime.Objec } obj := orig.DeepCopyObject() + backingNamespace := orig.GetProjectBackingNamespace() - obj, err := l.mgr.reconcileResourceToNamespace(obj, projectCreateController) + obj, err := l.mgr.reconcileResourceToNamespace(obj, projectCreateController, backingNamespace) if err != nil { return nil, err } - obj, err = l.mgr.reconcileCreatorRTB(obj) + obj, err = l.mgr.reconcileCreatorRTB(obj, backingNamespace) if err != nil { return nil, err } @@ -110,7 +109,7 @@ func (l *projectLifecycle) sync(key string, orig *apisv3.Project) (runtime.Objec return nil, err } } - if err != nil && !kerrors.IsAlreadyExists(err) { + if err != nil && !apierrors.IsAlreadyExists(err) { return nil, err } @@ -155,7 +154,8 @@ func (l *projectLifecycle) Remove(obj *apisv3.Project) (runtime.Object, error) { err := l.mgr.roleBindings.DeleteNamespaced(obj.Name, rb.Name, &v1.DeleteOptions{}) returnErr = errors.Join(returnErr, err) } - err = l.mgr.deleteNamespace(obj, projectRemoveController) + backingNamespace := obj.GetProjectBackingNamespace() + err = l.mgr.deleteNamespace(projectRemoveController, backingNamespace) returnErr = errors.Join(returnErr, err) return obj, returnErr } @@ -170,7 +170,7 @@ func (l *clusterLifecycle) sync(key string, orig *apisv3.Cluster) (runtime.Objec } obj := orig.DeepCopyObject() - obj, err := l.mgr.reconcileResourceToNamespace(obj, clusterCreateController) + obj, err := l.mgr.reconcileResourceToNamespace(obj, clusterCreateController, orig.Name) if err != nil { return nil, err } @@ -198,7 +198,7 @@ func (l *clusterLifecycle) sync(key string, orig *apisv3.Cluster) (runtime.Objec } } - obj, err = l.mgr.reconcileCreatorRTB(obj) + obj, err = l.mgr.reconcileCreatorRTB(obj, orig.Name) if err != nil { return nil, err } @@ -242,7 +242,7 @@ func (l *clusterLifecycle) Remove(obj *apisv3.Cluster) (runtime.Object, error) { } returnErr = errors.Join( l.mgr.deleteSystemProject(obj, clusterRemoveController), - l.mgr.deleteNamespace(obj, clusterRemoveController), + l.mgr.deleteNamespace(clusterRemoveController, obj.Name), ) return obj, returnErr } @@ -264,11 +264,11 @@ type mgr struct { } func (m *mgr) createDefaultProject(obj runtime.Object) (runtime.Object, error) { - return m.createProject(project.Default, v32.ClusterConditionDefaultProjectCreated, obj, defaultProjectLabels) + return m.createProject(project.Default, apisv3.ClusterConditionDefaultProjectCreated, obj, defaultProjectLabels) } func (m *mgr) createSystemProject(obj runtime.Object) (runtime.Object, error) { - return m.createProject(project.System, v32.ClusterConditionSystemProjectCreated, obj, systemProjectLabels) + return m.createProject(project.System, apisv3.ClusterConditionSystemProjectCreated, obj, systemProjectLabels) } func (m *mgr) createProject(name string, cond condition.Cond, obj runtime.Object, labels labels.Set) (runtime.Object, error) { @@ -310,7 +310,7 @@ func (m *mgr) createProject(name string, cond condition.Cond, obj runtime.Object Annotations: annotation, Labels: labels, }, - Spec: v32.ProjectSpec{ + Spec: apisv3.ProjectSpec{ DisplayName: name, Description: fmt.Sprintf("%s project created for the cluster", name), ClusterName: metaAccessor.GetName(), @@ -352,8 +352,8 @@ func (m *mgr) deleteSystemProject(cluster *apisv3.Cluster, controller string) er return deleteError } -func (m *mgr) reconcileCreatorRTB(obj runtime.Object) (runtime.Object, error) { - return v32.CreatorMadeOwner.DoUntilTrue(obj, func() (runtime.Object, error) { +func (m *mgr) reconcileCreatorRTB(obj runtime.Object, nsName string) (runtime.Object, error) { + return apisv3.CreatorMadeOwner.DoUntilTrue(obj, func() (runtime.Object, error) { metaAccessor, err := meta.Accessor(obj) if err != nil { return obj, err @@ -374,7 +374,7 @@ func (m *mgr) reconcileCreatorRTB(obj runtime.Object) (runtime.Object, error) { case v3.ProjectGroupVersionKind.Kind: project := obj.(*apisv3.Project) - if v32.ProjectConditionInitialRolesPopulated.IsTrue(project) { + if apisv3.ProjectConditionInitialRolesPopulated.IsTrue(project) { // The projectRoleBindings are already completed, no need to check break } @@ -407,7 +407,7 @@ func (m *mgr) reconcileCreatorRTB(obj runtime.Object) (runtime.Object, error) { // The projectRoleBinding doesn't exist yet so create it om := v1.ObjectMeta{ Name: rtbName, - Namespace: metaAccessor.GetName(), + Namespace: nsName, } logrus.Infof("[%v] Creating creator projectRoleTemplateBinding for user %v for project %v", projectCreateController, creatorID, metaAccessor.GetName()) @@ -433,7 +433,7 @@ func (m *mgr) reconcileCreatorRTB(obj runtime.Object) (runtime.Object, error) { project.Annotations[roleTemplatesRequired] = string(d) if reflect.DeepEqual(roleMap["required"], createdRoles) { - v32.ProjectConditionInitialRolesPopulated.True(project) + apisv3.ProjectConditionInitialRolesPopulated.True(project) logrus.Infof("[%v] Setting InitialRolesPopulated condition on project %v", ctrbMGMTController, project.Name) } if _, err := m.mgmt.Management.Projects("").Update(project); err != nil { @@ -443,7 +443,7 @@ func (m *mgr) reconcileCreatorRTB(obj runtime.Object) (runtime.Object, error) { case v3.ClusterGroupVersionKind.Kind: cluster := obj.(*apisv3.Cluster) - if v32.ClusterConditionInitialRolesPopulated.IsTrue(cluster) { + if apisv3.ClusterConditionInitialRolesPopulated.IsTrue(cluster) { // The clusterRoleBindings are already completed, no need to check break } @@ -507,52 +507,43 @@ func (m *mgr) reconcileCreatorRTB(obj runtime.Object) (runtime.Object, error) { }) } -func (m *mgr) deleteNamespace(obj runtime.Object, controller string) error { - o, err := meta.Accessor(obj) - if err != nil { - return condition.Error("MissingMetadata", err) - } - +func (m *mgr) deleteNamespace(controller string, nsName string) error { nsClient := m.mgmt.K8sClient.CoreV1().Namespaces() - ns, err := nsClient.Get(context.TODO(), o.GetName(), v1.GetOptions{}) + ns, err := nsClient.Get(context.TODO(), nsName, v1.GetOptions{}) if apierrors.IsNotFound(err) { return nil } if ns.Status.Phase != v12.NamespaceTerminating { - logrus.Infof("[%s] Deleting namespace %s", controller, o.GetName()) - err = nsClient.Delete(context.TODO(), o.GetName(), v1.DeleteOptions{}) + logrus.Infof("[%s] Deleting namespace %s", controller, nsName) + err = nsClient.Delete(context.TODO(), nsName, v1.DeleteOptions{}) if apierrors.IsNotFound(err) { return nil } } return err } -func (m *mgr) reconcileResourceToNamespace(obj runtime.Object, controller string) (runtime.Object, error) { - return v32.NamespaceBackedResource.Do(obj, func() (runtime.Object, error) { - o, err := meta.Accessor(obj) - if err != nil { - return obj, condition.Error("MissingMetadata", err) - } +func (m *mgr) reconcileResourceToNamespace(obj runtime.Object, controller string, nsName string) (runtime.Object, error) { + return apisv3.NamespaceBackedResource.Do(obj, func() (runtime.Object, error) { t, err := meta.TypeAccessor(obj) if err != nil { return obj, condition.Error("MissingTypeMetadata", err) } - ns, _ := m.nsLister.Get("", o.GetName()) + ns, _ := m.nsLister.Get("", nsName) if ns == nil { nsClient := m.mgmt.K8sClient.CoreV1().Namespaces() - logrus.Infof("[%v] Creating namespace %v", controller, o.GetName()) + logrus.Infof("[%v] Creating namespace %v", controller, nsName) _, err := nsClient.Create(context.TODO(), &v12.Namespace{ ObjectMeta: v1.ObjectMeta{ - Name: o.GetName(), + Name: nsName, Annotations: map[string]string{ "management.cattle.io/system-namespace": "true", }, }, }, v1.CreateOptions{}) if err != nil { - return obj, condition.Error("NamespaceCreationFailure", fmt.Errorf("failed to create namespace for %v %v: %w", t.GetKind(), o.GetName(), err)) + return obj, condition.Error("NamespaceCreationFailure", fmt.Errorf("failed to create namespace for %s %s: %w", t.GetKind(), nsName, err)) } } @@ -640,7 +631,7 @@ func (m *mgr) updateClusterAnnotationandCondition(cluster *apisv3.Cluster, anno c.Annotations[roleTemplatesRequired] = anno if updateCondition { - v32.ClusterConditionInitialRolesPopulated.True(c) + apisv3.ClusterConditionInitialRolesPopulated.True(c) } _, err = m.mgmt.Management.Clusters("").Update(c) if err != nil {
pkg/controllers/management/auth/prtb_handler.go+1 −1 modified@@ -139,7 +139,7 @@ func (p *prtbLifecycle) reconcileSubject(binding *v3.ProjectRoleTemplateBinding) return binding, nil } - return nil, fmt.Errorf("Binding %v has no subject", binding.Name) + return nil, fmt.Errorf("binding %s has no subject", binding.Name) } // When a PRTB is created or updated, translate it into several k8s roles and bindings to actually enforce the RBAC.
pkg/controllers/managementuser/rbac/namespace_handler.go+7 −4 modified@@ -15,7 +15,6 @@ import ( v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" namespaceutil "github.com/rancher/rancher/pkg/namespace" "github.com/rancher/rancher/pkg/project" - projectpkg "github.com/rancher/rancher/pkg/project" "github.com/sirupsen/logrus" v1 "k8s.io/api/core/v1" rbacv1 "k8s.io/api/rbac/v1" @@ -165,15 +164,15 @@ func (n *nsLifecycle) assignToInitialProject(ns *v1.Namespace) error { } func (n *nsLifecycle) GetSystemProjectName() (string, error) { - projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[projectpkg.System].AsSelector()) + projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[project.System].AsSelector()) if err != nil { return "", err } if len(projects) == 0 { return "", nil } if len(projects) > 1 { - return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, projectpkg.System) + return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, project.System) } if projects[0] == nil { return "", nil @@ -261,7 +260,11 @@ func (n *nsLifecycle) ensurePRTBAddToNamespace(ns *v1.Namespace) (bool, error) { var namespace string if parts := strings.SplitN(projectID, ":", 2); len(parts) == 2 && len(parts[1]) > 0 { - namespace = parts[1] + project, err := n.rq.ProjectLister.Get(parts[0], parts[1]) + if err != nil { + return hasPRTBs, err + } + namespace = project.GetProjectBackingNamespace() } else { return hasPRTBs, nil }
pkg/controllers/managementuser/secret/secret.go+32 −25 modified@@ -270,6 +270,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { clusterSecretsClient: clusterSecretsClient, clusterSecretsLister: clusterSecretsClient.Controller().Lister(), managementSecrets: cluster.Management.Core.Secrets("").Controller().Lister(), + projectCache: cluster.Management.Management.Projects("").Controller().Lister(), } cluster.Core.Namespaces("").AddHandler(ctx, "secretsController", n.sync) @@ -306,6 +307,7 @@ type NamespaceController struct { clusterSecretsClient v1.SecretInterface clusterSecretsLister v1.SecretLister managementSecrets v1.SecretLister + projectCache v3.ProjectLister } func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.Object, error) { @@ -316,33 +318,38 @@ func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.O // field.cattle.io/projectId value is <cluster name>:<project name> logrus.Tracef("secretsController: sync: key [%s], obj.Annotations[projectIDLabel]: [%s]", key, obj.Annotations[projectIDLabel]) if obj.Annotations[projectIDLabel] != "" { - parts := strings.Split(obj.Annotations[projectIDLabel], ":") - if len(parts) == 2 { - if parts[1] == "" { - logrus.Debugf("[NamspaceController|sync] empty project name found in obj.Annotations[projectIDLabel] for cluster: %s", parts[0]) - return nil, nil + clusterName, projectName, found := strings.Cut(obj.Annotations[projectIDLabel], ":") + if !found { + logrus.Debugf("secretsController: sync: namespace %s projectId annotation %s is malformed, should be <cluster name>:<project name>", obj.Name, obj.Annotations[projectIDLabel]) + return nil, nil + } + + p, err := n.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, fmt.Errorf("secretsController: sync: error getting project %s for namespace %s: %w", projectName, obj.Name, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + secrets, err := n.managementSecrets.List(backingNamespace, labels.NewSelector()) + if err != nil { + return nil, err + } + + logrus.Tracef("secretsController: sync: length of secrets for project [%s] in namespace [%s] is %d", projectName, obj.Name, len(secrets)) + for _, secret := range secrets { + // skip service account token secrets + if secret.Type == corev1.SecretTypeServiceAccountToken { + logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) + continue } - // on the management side, secret's namespace name equals to project name - secrets, err := n.managementSecrets.List(parts[1], labels.NewSelector()) - if err != nil { - return nil, err + namespacedSecret := getNamespacedSecret(secret, obj.Name) + if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { + continue } - logrus.Tracef("secretsController: sync: length of secrets for [%s] in namespace [%s] is %d", parts[1], obj.Name, len(secrets)) - for _, secret := range secrets { - // skip service account token secrets - if secret.Type == corev1.SecretTypeServiceAccountToken { - logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) - continue - } - namespacedSecret := getNamespacedSecret(secret, obj.Name) - if _, err := n.clusterSecretsLister.Get(namespacedSecret.Namespace, namespacedSecret.Name); err == nil { - continue - } - logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) - _, err := n.clusterSecretsClient.Create(namespacedSecret) - if err != nil && !errors.IsAlreadyExists(err) { - return nil, err - } + logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) + _, err := n.clusterSecretsClient.Create(namespacedSecret) + if err != nil && !errors.IsAlreadyExists(err) { + return nil, err } } }
pkg/systemaccount/systemaccount.go+0 −35 modified@@ -6,7 +6,6 @@ import ( v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" - "github.com/rancher/rancher/pkg/ref" "github.com/rancher/rancher/pkg/types/config" "github.com/rancher/rancher/pkg/types/config/systemtokens" "github.com/rancher/rancher/pkg/user" @@ -28,7 +27,6 @@ func NewManager(management *config.ManagementContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), users: management.Management.Users(""), @@ -41,7 +39,6 @@ func NewManagerFromScale(management *config.ScaledContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), tokenLister: management.Management.Tokens("").Controller().Lister(), @@ -54,7 +51,6 @@ type Manager struct { systemTokens systemtokens.Interface crtbs v3.ClusterRoleTemplateBindingInterface crts v3.ClusterRegistrationTokenInterface - prtbs v3.ProjectRoleTemplateBindingInterface prtbLister v3.ProjectRoleTemplateBindingLister tokens v3.TokenInterface tokenLister v3.TokenLister @@ -124,37 +120,6 @@ func (s *Manager) GetOrCreateSystemClusterToken(clusterName string) (string, err return token, nil } -func (s *Manager) GetOrCreateProjectSystemAccount(projectID string) error { - _, projectName := ref.Parse(projectID) - - u, err := s.GetProjectSystemUser(projectName) - if err != nil { - return err - } - - bindingName := u.Name + "-member" - _, err = s.prtbLister.Get(projectName, bindingName) - if err != nil { - if !errors2.IsNotFound(err) { - return err - } - // prtb does not exist in cache, attempt to create it - prtb := &v3.ProjectRoleTemplateBinding{ - ObjectMeta: v1.ObjectMeta{ - Name: bindingName, - Namespace: projectName, - }, - ProjectName: projectID, - UserName: u.Name, - RoleTemplateName: projectMemberRole, - } - if _, err := s.prtbs.Create(prtb); err != nil && !errors2.IsAlreadyExists(err) { - return err - } - } - return nil -} - func (s *Manager) GetProjectSystemUser(projectName string) (*v3.User, error) { return s.userManager.EnsureUser(fmt.Sprintf("system://%s", projectName), ProjectSystemAccountPrefix+projectName) }
tests/integration/suite/test_app.py+1 −0 modified@@ -104,6 +104,7 @@ def test_app_istio(admin_cc, admin_pc, admin_mc): wait_for_monitor_metric(admin_cc, admin_mc) +@pytest.mark.skip def test_prehook_chart(admin_pc, admin_mc): client = admin_pc.client name = random_str()
tests/integration/suite/test_catalog.py+1 −0 modified@@ -178,6 +178,7 @@ def test_template_version_links(admin_mc, admin_pc, custom_catalog, assert len(templates.data[0]['versionLinks']) == 0 +@pytest.mark.skip def test_relative_paths(admin_mc, admin_pc, remove_resource): """ This test adds a catalog's index.yaml with a relative chart url and ensures that rancher can resolve the relative url"""
tests/integration/suite/test_project_catalog.py+5 −0 modified@@ -3,8 +3,10 @@ from rancher import ApiError from .common import random_str import time +import pytest +@pytest.mark.skip def test_project_catalog_creation(admin_mc, remove_resource, user_mc, user_factory, admin_pc, admin_cc): @@ -91,6 +93,7 @@ def test_project_catalog_creation(admin_mc, remove_resource, wait_for_projectcatalog_template_to_be_deleted(client, catalog_name) +@pytest.mark.skip def test_create_project_catalog_after_user_addition(admin_mc, user_factory, remove_resource, @@ -150,6 +153,7 @@ def test_create_project_catalog_after_user_addition(admin_mc, wait_for_projectcatalog_template_to_be_deleted(client, catalog_name) +@pytest.mark.skip def test_user_addition_after_creating_project_catalog(admin_mc, user_factory, remove_resource, @@ -212,6 +216,7 @@ def test_user_addition_after_creating_project_catalog(admin_mc, wait_for_projectcatalog_template_to_be_deleted(client, catalog_name) +@pytest.mark.skip def test_project_catalog_access_before_app_creation(admin_mc, admin_pc, remove_resource, user_factory):
tests/integration/suite/test_rbac.py+1 −0 modified@@ -796,6 +796,7 @@ def test_readonly_cannot_edit_secret(admin_mc, user_mc, admin_pc, assert e.value.error.status == 404 +@pytest.mark.skip def test_member_can_edit_secret(admin_mc, admin_pc, remove_resource, user_mc): """Tests that a user with project-member role is able to create/update
9c1d1c2bfcba[v2.10] Add project namespace handling (#49849)
20 files changed · +605 −144
pkg/api/norman/customization/globalnamespaceaccess/access_common.go+20 −8 modified@@ -5,14 +5,13 @@ import ( "fmt" "strings" - v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - "github.com/rancher/norman/api/access" "github.com/rancher/norman/httperror" "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" "github.com/rancher/norman/types/set" "github.com/rancher/norman/types/slice" + v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" client "github.com/rancher/rancher/pkg/client/generated/management/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" "github.com/rancher/rancher/pkg/rbac" @@ -63,7 +62,7 @@ func (ma *MemberAccess) IsAdmin(callerID string) (bool, error) { return false, err } if u == nil { - return false, fmt.Errorf("No user found with ID %v", callerID) + return false, fmt.Errorf("no user found with ID %v", callerID) } // Get globalRoleBinding for this user grbs, err := ma.GrbLister.List("", labels.NewSelector()) @@ -180,9 +179,17 @@ func (ma *MemberAccess) EnsureRoleInTargets(targetProjects, roleTemplates []stri callerIsProjectOwner := false callerIsProjectMember := false callerIsClusterOwner := false - prtbs, err := ma.PrtbLister.List(pname, labels.NewSelector()) + + p, err := ma.ProjectLister.Get(cname, pname) if err != nil { - return err + return fmt.Errorf("unable to get project %s in namespace %s: %w", pname, cname, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) + if err != nil { + return fmt.Errorf("unable to get PRTBs in namespace %s: %w", backingNamespace, err) } for _, prtb := range prtbs { if prtb.UserName == callerID { @@ -524,21 +531,26 @@ func (ma *MemberAccess) RemoveRolesFromTargets(targetProjects, rolesToRemove []s return httperror.NewAPIError(httperror.InvalidBodyContent, errMsg) } clusterName, projectName := split[0], split[1] - clustersCovered := make(map[string]bool) - prtbs, err := ma.PrtbLister.List(projectName, labels.NewSelector()) + p, err := ma.ProjectLister.Get(clusterName, projectName) + if err != nil { + return fmt.Errorf("unable to get project %s in namespace %s: %w", projectName, clusterName, err) + } + backingNamespace := p.GetProjectBackingNamespace() + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) if err != nil { return err } for _, prtb := range prtbs { if prtb.UserPrincipalName == systemUserPrincipalID { if removeAllRoles || rolesToRemoveMap[prtb.RoleTemplateName] { - if err = ma.Prtbs.DeleteNamespaced(projectName, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { + if err = ma.Prtbs.DeleteNamespaced(backingNamespace, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { return err } } } } + clustersCovered := make(map[string]bool) if !clustersCovered[clusterName] { crtbs, err := ma.CrtbLister.List(clusterName, labels.NewSelector()) if err != nil {
pkg/api/norman/server/managementstored/setup.go+4 −3 modified@@ -171,7 +171,7 @@ func Setup(ctx context.Context, apiContext *config.ScaledContext, clusterManager authn.SetRTBStore(ctx, schemas.Schema(&managementschema.Version, client.ProjectRoleTemplateBindingType), apiContext) nodeStore.SetupStore(schemas.Schema(&managementschema.Version, client.NodeType)) projectaction.SetProjectStore(schemas.Schema(&managementschema.Version, client.ProjectType), apiContext) - setupScopedTypes(schemas) + setupScopedTypes(schemas, apiContext) setupPasswordTypes(ctx, schemas, apiContext) multiclusterapp.SetMemberStore(ctx, schemas.Schema(&managementschema.Version, client.MultiClusterAppType), apiContext) @@ -185,7 +185,8 @@ func setupPasswordTypes(ctx context.Context, schemas *types.Schemas, management passwordStore.SetPasswordStore(schemas, secretStore, nsStore) } -func setupScopedTypes(schemas *types.Schemas) { +func setupScopedTypes(schemas *types.Schemas, management *config.ScaledContext) { + projectLister := management.Management.Projects("").Controller().Lister() for _, schema := range schemas.Schemas() { if schema.Scope != types.NamespaceScope || schema.Store == nil || schema.Store.Context() != config.ManagementStorageContext { continue @@ -201,7 +202,7 @@ func setupScopedTypes(schemas *types.Schemas) { continue } - schema.Store = scoped.NewScopedStore(key, schema.Store) + schema.Store = scoped.NewScopedStore(key, schema.Store, projectLister) ns.Required = false schema.ResourceFields["namespaceId"] = ns break
pkg/api/norman/store/scoped/store.go+22 −7 modified@@ -7,14 +7,16 @@ import ( "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" client "github.com/rancher/rancher/pkg/client/generated/management/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" ) type Store struct { types.Store - key string + key string + projectCache v3.ProjectLister } -func NewScopedStore(key string, store types.Store) *Store { +func NewScopedStore(key string, store types.Store, pLister v3.ProjectLister) *Store { return &Store{ Store: &transform.Store{ Store: store, @@ -23,21 +25,34 @@ func NewScopedStore(key string, store types.Store) *Store { return data, nil } v := convert.ToString(data[key]) - if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) { + if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) && v != strings.Replace(convert.ToString(data[client.ProjectFieldNamespaceId]), "-", ":", 1) { data[key] = data[client.ProjectFieldNamespaceId] } + data[client.ProjectFieldNamespaceId] = nil return data, nil }, }, - key: key, + key: key, + projectCache: pLister, } } func (s *Store) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { - if data != nil { - parts := strings.Split(convert.ToString(data[s.key]), ":") - data["namespaceId"] = parts[len(parts)-1] + if data == nil { + return s.Store.Create(apiContext, schema, data) + } + + clusterName, projectName, isProject := strings.Cut(convert.ToString(data[s.key]), ":") + if isProject { + p, err := s.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, err + } + data[client.ProjectFieldNamespaceId] = p.GetProjectBackingNamespace() + } else { + data[client.ProjectFieldNamespaceId] = data[s.key] } + return s.Store.Create(apiContext, schema, data) }
pkg/api/norman/store/scoped/store_test.go+142 −0 added@@ -0,0 +1,142 @@ +package scoped + +import ( + "fmt" + "reflect" + "testing" + + "github.com/rancher/norman/types" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" + "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type fakeStore struct { +} + +func (f fakeStore) Context() types.StorageContext { + return "" +} +func (f fakeStore) ByID(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) List(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) ([]map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { + return data, nil +} +func (f fakeStore) Update(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Delete(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Watch(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) (chan map[string]interface{}, error) { + return nil, nil +} + +func TestStoreCreate(t *testing.T) { + store := fakeStore{} + + p := fakes.ProjectListerMock{} + + tests := []struct { + name string + key string + getFunc func(string, string) (*v3.Project, error) + data map[string]interface{} + want map[string]interface{} + wantErr bool + }{ + { + name: "nil data returns nil", + data: nil, + want: nil, + }, + { + name: "project: set namespace no backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "project-XYZ", + }, + }, + { + name: "project: set namespace to backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "c-ABC-p-XYZ", + }, + }, + { + name: "error getting project", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + return nil, fmt.Errorf("error") + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + wantErr: true, + }, + { + name: "cluster: set namespaceId", + key: "clusterId", + data: map[string]interface{}{ + "clusterId": "cluster-ABC", + }, + want: map[string]interface{}{ + "clusterId": "cluster-ABC", + "namespaceId": "cluster-ABC", + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p.GetFunc = tt.getFunc + s := &Store{ + Store: store, + key: tt.key, + projectCache: &p, + } + got, err := s.Create(nil, nil, tt.data) + if (err != nil) != tt.wantErr { + t.Errorf("Store.Create() error = %v, wantErr %v", err, tt.wantErr) + return + } + if !reflect.DeepEqual(got, tt.want) { + t.Errorf("Store.Create() = %v, want %v", got, tt.want) + } + }) + } +}
pkg/apis/management.cattle.io/v3/authz_types.go+8 −0 modified@@ -48,6 +48,14 @@ func (p *Project) ObjClusterName() string { return p.Spec.ObjClusterName() } +// GetProjectBackingNamespace returns the namespace a project uses in the local cluster to store PRTBs and Project Scoped Secrets. +func (p *Project) GetProjectBackingNamespace() string { + if p.Status.BackingNamespace != "" { + return p.Status.BackingNamespace + } + return p.Name +} + // ProjectStatus represents the most recently observed status of the project. type ProjectStatus struct { // Conditions are a set of indicators about aspects of the project.
pkg/controllers/management/auth/crtb_handler.go+7 −5 modified@@ -190,11 +190,12 @@ func (c *crtbLifecycle) reconcileBindings(binding *v3.ClusterRoleTemplateBinding return err } for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() if p.DeletionTimestamp != nil { - logrus.Warnf("Project %v is being deleted, not creating membership bindings", p.Name) + logrus.Warnf("Project %v is being deleted, not creating membership bindings", backingNamespace) continue } - if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, p.Name, projectManagementPlaneResources, subject, binding); err != nil { + if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, backingNamespace, projectManagementPlaneResources, subject, binding); err != nil { return err } } @@ -208,14 +209,15 @@ func (c *crtbLifecycle) removeMGMTClusterScopedPrivilegesInProjectNamespace(bind } bindingKey := pkgrbac.GetRTBLabel(binding.ObjectMeta) for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() set := labels.Set(map[string]string{bindingKey: CrtbInProjectBindingOwner}) - rbs, err := c.rbLister.List(p.Name, set.AsSelector()) + rbs, err := c.rbLister.List(backingNamespace, set.AsSelector()) if err != nil { return err } for _, rb := range rbs { - logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, p.Name, binding.Name) - if err := c.rbClient.DeleteNamespaced(p.Name, rb.Name, &metav1.DeleteOptions{}); err != nil { + logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, backingNamespace, binding.Name) + if err := c.rbClient.DeleteNamespaced(backingNamespace, rb.Name, &metav1.DeleteOptions{}); err != nil { return err } }
pkg/controllers/management/auth/crtb_handler_test.go+160 −9 modified@@ -5,16 +5,20 @@ import ( "testing" "time" - v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + rbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" + corefakes "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1/fakes" + "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" ) var ( - e = fmt.Errorf("error") + errDefault = fmt.Errorf("error") defaultCRTB = v3.ClusterRoleTemplateBinding{ UserName: "test", GroupName: "", @@ -37,12 +41,25 @@ var ( Name: "test-project", }, } + backingNamespaceProject = v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-project", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } deletingProject = v3.Project{ ObjectMeta: v1.ObjectMeta{ Name: "deleting-project", DeletionTimestamp: &v1.Time{Time: time.Now()}, }, } + defaultBinding = rbacv1.RoleBinding{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-binding", + }, + } ) type crtbTestState struct { @@ -66,7 +83,7 @@ func TestReconcileBindings(t *testing.T) { name: "error getting cluster", stateSetup: func(cts crtbTestState) { cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -91,7 +108,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). - Return(true, e) + Return(true, errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -108,7 +125,7 @@ func TestReconcileBindings(t *testing.T) { Return(true, nil) cts.managerMock.EXPECT(). ensureClusterMembershipBinding("clustername-clusterowner", gomock.Any(), gomock.Any(), true, gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -129,7 +146,7 @@ func TestReconcileBindings(t *testing.T) { Return(nil) cts.managerMock.EXPECT(). grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -151,7 +168,7 @@ func TestReconcileBindings(t *testing.T) { grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). Return(nil) cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -179,7 +196,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "test-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -237,6 +254,32 @@ func TestReconcileBindings(t *testing.T) { }, crtb: defaultCRTB.DeepCopy(), }, + { + name: "successfully reconcile clustermember with backingNamespace", + stateSetup: func(cts crtbTestState) { + cts.managerMock.EXPECT(). + checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). + Return(false, nil) + cts.managerMock.EXPECT(). + ensureClusterMembershipBinding("clustername-clustermember", gomock.Any(), gomock.Any(), false, gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "c-ABC-p-XYZ", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { + c := defaultCluster.DeepCopy() + return c, nil + } + cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { + p := backingNamespaceProject.DeepCopy() + return []*v3.Project{p}, nil + } + }, + crtb: defaultCRTB.DeepCopy(), + }, { name: "skip projects that are deleting", stateSetup: func(cts crtbTestState) { @@ -252,7 +295,7 @@ func TestReconcileBindings(t *testing.T) { // This should not be called cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "deleting-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e).AnyTimes() + Return(errDefault).AnyTimes() cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { c := defaultCluster.DeepCopy() return c, nil @@ -302,3 +345,111 @@ func setupTest(t *testing.T) crtbTestState { } return state } + +func Test_removeMGMTClusterScopedPrivilegesInProjectNamespace(t *testing.T) { + tests := []struct { + name string + projectListFunc func(string, labels.Selector) ([]*apisv3.Project, error) + roleBindingListFunc func(string, labels.Selector) ([]*rbacv1.RoleBinding, error) + roleBindingDeleteFunc func(string, string, *v1.DeleteOptions) error + binding *v3.ClusterRoleTemplateBinding + wantErr bool + }{ + { + name: "error listing projects", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error listing rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error deleting rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + return errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "successfully delete rolebindings no backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, defaultProject.Name, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, defaultProject.Name, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + { + name: "successfully delete rolebindings with backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + backingNamespaceProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p := fakes.ProjectListerMock{} + p.ListFunc = tt.projectListFunc + rbl := corefakes.RoleBindingListerMock{} + rbl.ListFunc = tt.roleBindingListFunc + rbi := corefakes.RoleBindingInterfaceMock{} + rbi.DeleteNamespacedFunc = tt.roleBindingDeleteFunc + + c := &crtbLifecycle{ + projectLister: &p, + rbLister: &rbl, + rbClient: &rbi, + } + if err := c.removeMGMTClusterScopedPrivilegesInProjectNamespace(tt.binding); (err != nil) != tt.wantErr { + t.Errorf("crtbLifecycle.removeMGMTClusterScopedPrivilegesInProjectNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +}
pkg/controllers/management/auth/project_cluster/cluster_handler.go+5 −5 modified@@ -13,13 +13,13 @@ import ( "github.com/rancher/rancher/pkg/controllers" "github.com/rancher/rancher/pkg/controllers/managementuserlegacy/systemimage" wranglerv3 "github.com/rancher/rancher/pkg/generated/controllers/management.cattle.io/v3" - corev1 "github.com/rancher/rancher/pkg/generated/norman/core/v1" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" rbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" "github.com/rancher/rancher/pkg/project" "github.com/rancher/rancher/pkg/rbac" "github.com/rancher/rancher/pkg/settings" "github.com/rancher/rancher/pkg/types/config" + corev1 "github.com/rancher/wrangler/v3/pkg/generated/controllers/core/v1" "github.com/rancher/wrangler/v3/pkg/generic" "github.com/sirupsen/logrus" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -51,7 +51,7 @@ type clusterLifecycle struct { clusterClient v3.ClusterInterface crtbLister v3.ClusterRoleTemplateBindingLister crtbClient v3.ClusterRoleTemplateBindingInterface - nsLister corev1.NamespaceLister + nsLister corev1.NamespaceCache nsClient k8scorev1.NamespaceInterface projects wranglerv3.ProjectClient projectLister v3.ProjectLister @@ -66,7 +66,7 @@ func NewClusterLifecycle(management *config.ManagementContext) *clusterLifecycle clusterClient: management.Management.Clusters(""), crtbLister: management.Management.ClusterRoleTemplateBindings("").Controller().Lister(), crtbClient: management.Management.ClusterRoleTemplateBindings(""), - nsLister: management.Core.Namespaces("").Controller().Lister(), + nsLister: management.Wrangler.Core.Namespace().Cache(), nsClient: management.K8sClient.CoreV1().Namespaces(), projects: management.Wrangler.Mgmt.Project(), projectLister: management.Management.Projects("").Controller().Lister(), @@ -84,7 +84,7 @@ func (l *clusterLifecycle) Sync(key string, orig *apisv3.Cluster) (runtime.Objec } obj := orig.DeepCopyObject() - obj, err := reconcileResourceToNamespace(obj, ClusterCreateController, l.nsLister, l.nsClient) + obj, err := reconcileResourceToNamespace(obj, ClusterCreateController, orig.Name, l.nsLister, l.nsClient) if err != nil { return nil, err } @@ -157,7 +157,7 @@ func (l *clusterLifecycle) Remove(obj *apisv3.Cluster) (runtime.Object, error) { } returnErr = errors.Join( l.deleteSystemProject(obj, ClusterRemoveController), - deleteNamespace(obj, ClusterRemoveController, l.nsClient), + deleteNamespace(ClusterRemoveController, obj.Name, l.nsClient), ) return obj, returnErr }
pkg/controllers/management/auth/project_cluster/common.go+12 −19 modified@@ -5,7 +5,7 @@ import ( "fmt" apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - corev1 "github.com/rancher/rancher/pkg/generated/norman/core/v1" + corev1 "github.com/rancher/wrangler/v3/pkg/generated/controllers/core/v1" "github.com/sirupsen/logrus" v12 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -25,50 +25,43 @@ const ( var crtbCreatorOwnerAnnotations = map[string]string{creatorOwnerBindingAnnotation: "true"} -func deleteNamespace(obj runtime.Object, controller string, nsClient v1.NamespaceInterface) error { - o, err := meta.Accessor(obj) - if err != nil { - return fmt.Errorf("[%s] error accessing object %v: %w", controller, obj, err) - } - - ns, err := nsClient.Get(context.TODO(), o.GetName(), metav1.GetOptions{}) +func deleteNamespace(controller string, nsName string, nsClient v1.NamespaceInterface) error { + ns, err := nsClient.Get(context.TODO(), nsName, metav1.GetOptions{}) if apierrors.IsNotFound(err) { return nil + } else if err != nil { + return err } if ns.Status.Phase != v12.NamespaceTerminating { - logrus.Infof("[%s] Deleting namespace %s", controller, o.GetName()) - err = nsClient.Delete(context.TODO(), o.GetName(), metav1.DeleteOptions{}) + logrus.Infof("[%s] Deleting namespace %s", controller, nsName) + err = nsClient.Delete(context.TODO(), nsName, metav1.DeleteOptions{}) if apierrors.IsNotFound(err) { return nil } } return err } -func reconcileResourceToNamespace(obj runtime.Object, controller string, nsLister corev1.NamespaceLister, nsClient v1.NamespaceInterface) (runtime.Object, error) { +func reconcileResourceToNamespace(obj runtime.Object, controller string, nsName string, nsLister corev1.NamespaceCache, nsClient v1.NamespaceInterface) (runtime.Object, error) { return apisv3.NamespaceBackedResource.Do(obj, func() (runtime.Object, error) { - o, err := meta.Accessor(obj) - if err != nil { - return obj, fmt.Errorf("[%s] error accessing object %v: %w", controller, obj, err) - } t, err := meta.TypeAccessor(obj) if err != nil { return obj, err } - ns, _ := nsLister.Get("", o.GetName()) + ns, _ := nsLister.Get(nsName) if ns == nil { - logrus.Infof("[%v] Creating namespace %v", controller, o.GetName()) + logrus.Infof("[%v] Creating namespace %v", controller, nsName) _, err := nsClient.Create(context.TODO(), &v12.Namespace{ ObjectMeta: metav1.ObjectMeta{ - Name: o.GetName(), + Name: nsName, Annotations: map[string]string{ "management.cattle.io/system-namespace": "true", }, }, }, metav1.CreateOptions{}) if err != nil { - return obj, fmt.Errorf("[%s] failed to create namespace for %v %v: %w", controller, t.GetKind(), o.GetName(), err) + return obj, fmt.Errorf("[%s] failed to create namespace for %s %s: %w", controller, t.GetKind(), nsName, err) } }
pkg/controllers/management/auth/project_cluster/common_test.go+153 −0 added@@ -0,0 +1,153 @@ +package project_cluster + +import ( + "context" + "fmt" + "testing" + + v1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/watch" + applycorev1 "k8s.io/client-go/applyconfigurations/core/v1" +) + +var errNotFound = apierrors.NewNotFound(schema.GroupResource{}, "") + +var ( + errDefault = fmt.Errorf("error") + errNsNotFound = apierrors.NewNotFound(v1.Resource("namespace"), "") + + defaultNamespace = v1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "default-namespace", + }, + } + terminatingNamespace = v1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "terminating-namespace", + }, + Status: v1.NamespaceStatus{ + Phase: v1.NamespaceTerminating, + }, + } +) + +func Test_deleteNamespace(t *testing.T) { + tests := []struct { + name string + nsGetFunc func(context.Context, string, metav1.GetOptions) (*v1.Namespace, error) + nsDeleteFunc func(context.Context, string, metav1.DeleteOptions) error + wantErr bool + }{ + { + name: "error getting namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return nil, errDefault + }, + wantErr: true, + }, + { + name: "namespace not found", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return nil, errNsNotFound + }, + }, + { + name: "namespace is terminating", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return terminatingNamespace.DeepCopy(), nil + }, + }, + { + name: "successfully delete namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return nil + }, + }, + { + name: "deleting namespace not found", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return errNsNotFound + }, + }, + { + name: "error deleting namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return errDefault + }, + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + nsClientFake := mockNamespaces{ + getter: tt.nsGetFunc, + deleter: tt.nsDeleteFunc, + } + if err := deleteNamespace("", "", nsClientFake); (err != nil) != tt.wantErr { + t.Errorf("deleteNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +} + +type mockNamespaces struct { + getter func(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Namespace, error) + deleter func(ctx context.Context, name string, opts metav1.DeleteOptions) error +} + +func (m mockNamespaces) Get(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Namespace, error) { + return m.getter(ctx, name, opts) +} + +func (m mockNamespaces) Create(ctx context.Context, namespace *v1.Namespace, opts metav1.CreateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) Update(ctx context.Context, namespace *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) UpdateStatus(ctx context.Context, namespace *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) Delete(ctx context.Context, name string, opts metav1.DeleteOptions) error { + return m.deleter(ctx, name, opts) +} + +func (m mockNamespaces) List(ctx context.Context, opts metav1.ListOptions) (*v1.NamespaceList, error) { + panic("implement me") +} + +func (m mockNamespaces) Watch(ctx context.Context, opts metav1.ListOptions) (watch.Interface, error) { + panic("implement me") +} + +func (m mockNamespaces) Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts metav1.PatchOptions, subresources ...string) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) Apply(ctx context.Context, namespace *applycorev1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) ApplyStatus(ctx context.Context, namespace *applycorev1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) Finalize(ctx context.Context, item *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +}
pkg/controllers/management/auth/project_cluster/project_handler.go+21 −20 modified@@ -8,12 +8,12 @@ import ( "strings" apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - corev1 "github.com/rancher/rancher/pkg/generated/norman/core/v1" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" rbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" "github.com/rancher/rancher/pkg/rbac" "github.com/rancher/rancher/pkg/systemaccount" "github.com/rancher/rancher/pkg/types/config" + corev1 "github.com/rancher/wrangler/v3/pkg/generated/controllers/core/v1" "github.com/sirupsen/logrus" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -32,7 +32,7 @@ const ( type projectLifecycle struct { crtbClient v3.ClusterRoleTemplateBindingInterface crtbLister v3.ClusterRoleTemplateBindingLister - nsLister corev1.NamespaceLister + nsLister corev1.NamespaceCache nsClient k8scorev1.NamespaceInterface projects v3.ProjectInterface prtbLister v3.ProjectRoleTemplateBindingLister @@ -47,7 +47,7 @@ func NewProjectLifecycle(management *config.ManagementContext) *projectLifecycle return &projectLifecycle{ crtbClient: management.Management.ClusterRoleTemplateBindings(""), crtbLister: management.Management.ClusterRoleTemplateBindings("").Controller().Lister(), - nsLister: management.Core.Namespaces("").Controller().Lister(), + nsLister: management.Wrangler.Core.Namespace().Cache(), nsClient: management.K8sClient.CoreV1().Namespaces(), projects: management.Management.Projects(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), @@ -77,12 +77,13 @@ func (l *projectLifecycle) Sync(key string, orig *apisv3.Project) (runtime.Objec obj := orig.DeepCopyObject() - obj, err := reconcileResourceToNamespace(obj, ProjectCreateController, l.nsLister, l.nsClient) + backingNamespace := orig.GetProjectBackingNamespace() + obj, err := reconcileResourceToNamespace(obj, ProjectCreateController, backingNamespace, l.nsLister, l.nsClient) if err != nil { return nil, err } - obj, err = l.reconcileProjectCreatorRTB(obj) + obj, err = l.reconcileProjectCreatorRTB(obj, backingNamespace) if err != nil { return nil, err } @@ -142,13 +143,11 @@ func (l *projectLifecycle) Remove(obj *apisv3.Project) (runtime.Object, error) { returnErr = errors.Join(returnErr, err) } - err = deleteNamespace(obj, ProjectRemoveController, l.nsClient) - returnErr = errors.Join(returnErr, err) - - return obj, returnErr + backingNamespace := obj.GetProjectBackingNamespace() + return obj, deleteNamespace(ProjectRemoveController, backingNamespace, l.nsClient) } -func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runtime.Object, error) { +func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object, nsName string) (runtime.Object, error) { project, ok := obj.(*apisv3.Project) if !ok { return obj, fmt.Errorf("expected project, got %T", obj) @@ -157,18 +156,19 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti // If we specify no creator owner RBAC, exit if _, ok := project.Annotations[NoCreatorRBACAnnotation]; ok { logrus.Infof("[%s] annotation %s found. Skipping adding creator as owner", ProjectCreateController, NoCreatorRBACAnnotation) - return obj, nil + return project, nil } - return apisv3.CreatorMadeOwner.DoUntilTrue(obj, func() (runtime.Object, error) { + + return apisv3.CreatorMadeOwner.DoUntilTrue(project, func() (runtime.Object, error) { creatorID := project.Annotations[CreatorIDAnnotation] if creatorID == "" { logrus.Warnf("[%s] project %s has no creatorId annotation. Cannot add creator as owner", ProjectCreateController, project.Name) - return obj, nil + return project, nil } if apisv3.ProjectConditionInitialRolesPopulated.IsTrue(project) { // The projectRoleBindings are already completed, no need to check - return obj, nil + return project, nil } // If the project does not have the annotation it indicates the @@ -181,13 +181,14 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti roleMap := make(map[string][]string) if err := json.Unmarshal([]byte(creatorRoleBindings), &roleMap); err != nil { - return obj, err + return project, err } var createdRoles []string for _, role := range roleMap["required"] { rtbName := "creator-" + role - if rtb, _ := l.prtbLister.Get(project.Name, rtbName); rtb != nil { + + if rtb, _ := l.prtbLister.Get(nsName, rtbName); rtb != nil { createdRoles = append(createdRoles, role) // This projectRoleBinding exists, need to check all of them so keep going continue @@ -197,7 +198,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti prtb := &apisv3.ProjectRoleTemplateBinding{ ObjectMeta: metav1.ObjectMeta{ Name: rtbName, - Namespace: project.Name, + Namespace: nsName, }, ProjectName: project.Namespace + ":" + project.Name, RoleTemplateName: role, @@ -215,7 +216,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti logrus.Infof("[%s] Creating creator projectRoleTemplateBinding for user %s for project %s", ProjectCreateController, creatorID, project.Name) _, err := l.prtbClient.Create(prtb) if err != nil && !apierrors.IsAlreadyExists(err) { - return obj, err + return project, err } createdRoles = append(createdRoles, role) @@ -226,7 +227,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti roleMap["created"] = createdRoles d, err := json.Marshal(roleMap) if err != nil { - return obj, err + return project, err } project.Annotations[roleTemplatesRequiredAnnotation] = string(d) @@ -238,6 +239,6 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti _, err = l.projects.Update(project) - return obj, err + return project, err }) }
pkg/controllers/management/auth/project_cluster/project_handler_test.go+3 −3 modified@@ -87,13 +87,13 @@ func TestReconcileProjectCreatorRTBRespectsUserPrincipalName(t *testing.T) { }, } - obj, err := lifecycle.reconcileProjectCreatorRTB(project) + obj, err := lifecycle.reconcileProjectCreatorRTB(project, clusterID) require.NoError(t, err) require.NotNil(t, obj) require.Len(t, prtbs, 1) assert.Equal(t, "creator-project-owner", prtbs[0].Name) - assert.Equal(t, "p-abcdef", prtbs[0].Namespace) + assert.Equal(t, clusterID, prtbs[0].Namespace) assert.Equal(t, clusterID+":p-abcdef", prtbs[0].ProjectName) assert.Equal(t, "", prtbs[0].UserName) assert.Equal(t, userPrincipalName, prtbs[0].UserPrincipalName) @@ -109,7 +109,7 @@ func TestReconcileProjectCreatorRTBNoCreatorRBAC(t *testing.T) { }, }, } - obj, err := lifecycle.reconcileProjectCreatorRTB(project) + obj, err := lifecycle.reconcileProjectCreatorRTB(project, clusterID) assert.NoError(t, err) assert.NotNil(t, obj) }
pkg/controllers/management/auth/prtb_handler.go+1 −1 modified@@ -139,7 +139,7 @@ func (p *prtbLifecycle) reconcileSubject(binding *v3.ProjectRoleTemplateBinding) return binding, nil } - return nil, fmt.Errorf("Binding %v has no subject", binding.Name) + return nil, fmt.Errorf("binding %s has no subject", binding.Name) } // When a PRTB is created or updated, translate it into several k8s roles and bindings to actually enforce the RBAC.
pkg/controllers/managementuser/rbac/namespace_handler.go+7 −4 modified@@ -15,7 +15,6 @@ import ( v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" namespaceutil "github.com/rancher/rancher/pkg/namespace" "github.com/rancher/rancher/pkg/project" - projectpkg "github.com/rancher/rancher/pkg/project" "github.com/sirupsen/logrus" v1 "k8s.io/api/core/v1" rbacv1 "k8s.io/api/rbac/v1" @@ -166,15 +165,15 @@ func (n *nsLifecycle) assignToInitialProject(ns *v1.Namespace) error { } func (n *nsLifecycle) GetSystemProjectName() (string, error) { - projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[projectpkg.System].AsSelector()) + projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[project.System].AsSelector()) if err != nil { return "", err } if len(projects) == 0 { return "", nil } if len(projects) > 1 { - return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, projectpkg.System) + return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, project.System) } if projects[0] == nil { return "", nil @@ -262,7 +261,11 @@ func (n *nsLifecycle) ensurePRTBAddToNamespace(ns *v1.Namespace) (bool, error) { var namespace string if parts := strings.SplitN(projectID, ":", 2); len(parts) == 2 && len(parts[1]) > 0 { - namespace = parts[1] + project, err := n.rq.ProjectLister.Get(parts[0], parts[1]) + if err != nil { + return hasPRTBs, err + } + namespace = project.GetProjectBackingNamespace() } else { return hasPRTBs, nil }
pkg/controllers/managementuser/secret/secret.go+32 −25 modified@@ -267,6 +267,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { n := &NamespaceController{ clusterSecretsClient: clusterSecretsClient, managementSecrets: cluster.Management.Core.Secrets("").Controller().Lister(), + projectCache: cluster.Management.Management.Projects("").Controller().Lister(), } cluster.Core.Namespaces("").AddHandler(ctx, "secretsController", n.sync) @@ -302,6 +303,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { type NamespaceController struct { clusterSecretsClient v1.SecretInterface managementSecrets v1.SecretLister + projectCache v3.ProjectLister } func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.Object, error) { @@ -312,33 +314,38 @@ func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.O // field.cattle.io/projectId value is <cluster name>:<project name> logrus.Tracef("secretsController: sync: key [%s], obj.Annotations[projectIDLabel]: [%s]", key, obj.Annotations[projectIDLabel]) if obj.Annotations[projectIDLabel] != "" { - parts := strings.Split(obj.Annotations[projectIDLabel], ":") - if len(parts) == 2 { - if parts[1] == "" { - logrus.Debugf("[NamspaceController|sync] empty project name found in obj.Annotations[projectIDLabel] for cluster: %s", parts[0]) - return nil, nil + clusterName, projectName, found := strings.Cut(obj.Annotations[projectIDLabel], ":") + if !found { + logrus.Debugf("secretsController: sync: namespace %s projectId annotation %s is malformed, should be <cluster name>:<project name>", obj.Name, obj.Annotations[projectIDLabel]) + return nil, nil + } + + p, err := n.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, fmt.Errorf("secretsController: sync: error getting project %s for namespace %s: %w", projectName, obj.Name, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + secrets, err := n.managementSecrets.List(backingNamespace, labels.NewSelector()) + if err != nil { + return nil, err + } + + logrus.Tracef("secretsController: sync: length of secrets for project [%s] in namespace [%s] is %d", projectName, obj.Name, len(secrets)) + for _, secret := range secrets { + // skip service account token secrets + if secret.Type == corev1.SecretTypeServiceAccountToken { + logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) + continue } - // on the management side, secret's namespace name equals to project name - secrets, err := n.managementSecrets.List(parts[1], labels.NewSelector()) - if err != nil { - return nil, err + namespacedSecret := getNamespacedSecret(secret, obj.Name) + if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { + continue } - logrus.Tracef("secretsController: sync: length of secrets for [%s] in namespace [%s] is %d", parts[1], obj.Name, len(secrets)) - for _, secret := range secrets { - // skip service account token secrets - if secret.Type == corev1.SecretTypeServiceAccountToken { - logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) - continue - } - namespacedSecret := getNamespacedSecret(secret, obj.Name) - if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { - continue - } - logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) - _, err := n.clusterSecretsClient.Create(namespacedSecret) - if err != nil && !errors.IsAlreadyExists(err) { - return nil, err - } + logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) + _, err := n.clusterSecretsClient.Create(namespacedSecret) + if err != nil && !errors.IsAlreadyExists(err) { + return nil, err } } }
pkg/systemaccount/systemaccount.go+0 −35 modified@@ -6,7 +6,6 @@ import ( v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" - "github.com/rancher/rancher/pkg/ref" "github.com/rancher/rancher/pkg/types/config" "github.com/rancher/rancher/pkg/types/config/systemtokens" "github.com/rancher/rancher/pkg/user" @@ -28,7 +27,6 @@ func NewManager(management *config.ManagementContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), users: management.Management.Users(""), @@ -41,7 +39,6 @@ func NewManagerFromScale(management *config.ScaledContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), tokenLister: management.Management.Tokens("").Controller().Lister(), @@ -54,7 +51,6 @@ type Manager struct { systemTokens systemtokens.Interface crtbs v3.ClusterRoleTemplateBindingInterface crts v3.ClusterRegistrationTokenInterface - prtbs v3.ProjectRoleTemplateBindingInterface prtbLister v3.ProjectRoleTemplateBindingLister tokens v3.TokenInterface tokenLister v3.TokenLister @@ -124,37 +120,6 @@ func (s *Manager) GetOrCreateSystemClusterToken(clusterName string) (string, err return token, nil } -func (s *Manager) GetOrCreateProjectSystemAccount(projectID string) error { - _, projectName := ref.Parse(projectID) - - u, err := s.GetProjectSystemUser(projectName) - if err != nil { - return err - } - - bindingName := u.Name + "-member" - _, err = s.prtbLister.Get(projectName, bindingName) - if err != nil { - if !errors2.IsNotFound(err) { - return err - } - // prtb does not exist in cache, attempt to create it - prtb := &v3.ProjectRoleTemplateBinding{ - ObjectMeta: v1.ObjectMeta{ - Name: bindingName, - Namespace: projectName, - }, - ProjectName: projectID, - UserName: u.Name, - RoleTemplateName: projectMemberRole, - } - if _, err := s.prtbs.Create(prtb); err != nil && !errors2.IsAlreadyExists(err) { - return err - } - } - return nil -} - func (s *Manager) GetProjectSystemUser(projectName string) (*v3.User, error) { return s.userManager.EnsureUser(fmt.Sprintf("system://%s", projectName), ProjectSystemAccountPrefix+projectName) }
tests/integration/suite/test_app.py+1 −0 modified@@ -104,6 +104,7 @@ def test_app_istio(admin_cc, admin_pc, admin_mc): wait_for_monitor_metric(admin_cc, admin_mc) +@pytest.mark.skip def test_prehook_chart(admin_pc, admin_mc): client = admin_pc.client name = random_str()
tests/integration/suite/test_catalog.py+1 −0 modified@@ -178,6 +178,7 @@ def test_template_version_links(admin_mc, admin_pc, custom_catalog, assert len(templates.data[0]['versionLinks']) == 0 +@pytest.mark.skip def test_relative_paths(admin_mc, admin_pc, remove_resource): """ This test adds a catalog's index.yaml with a relative chart url and ensures that rancher can resolve the relative url"""
tests/integration/suite/test_project_catalog.py+5 −0 modified@@ -3,8 +3,10 @@ from rancher import ApiError from .common import random_str import time +import pytest +@pytest.mark.skip def test_project_catalog_creation(admin_mc, remove_resource, user_mc, user_factory, admin_pc, admin_cc): @@ -91,6 +93,7 @@ def test_project_catalog_creation(admin_mc, remove_resource, wait_for_projectcatalog_template_to_be_deleted(client, catalog_name) +@pytest.mark.skip def test_create_project_catalog_after_user_addition(admin_mc, user_factory, remove_resource, @@ -150,6 +153,7 @@ def test_create_project_catalog_after_user_addition(admin_mc, wait_for_projectcatalog_template_to_be_deleted(client, catalog_name) +@pytest.mark.skip def test_user_addition_after_creating_project_catalog(admin_mc, user_factory, remove_resource, @@ -212,6 +216,7 @@ def test_user_addition_after_creating_project_catalog(admin_mc, wait_for_projectcatalog_template_to_be_deleted(client, catalog_name) +@pytest.mark.skip def test_project_catalog_access_before_app_creation(admin_mc, admin_pc, remove_resource, user_factory):
tests/integration/suite/test_rbac.py+1 −0 modified@@ -796,6 +796,7 @@ def test_readonly_cannot_edit_secret(admin_mc, user_mc, admin_pc, assert e.value.error.status == 404 +@pytest.mark.skip def test_member_can_edit_secret(admin_mc, admin_pc, remove_resource, user_mc): """Tests that a user with project-member role is able to create/update
f036e8b6ab72Add project namespace handling (#49797) (#49848)
18 files changed · +600 −146
go.mod+2 −2 modified@@ -65,7 +65,7 @@ replace ( require ( github.com/antihax/optional v1.0.0 - github.com/rancher/rancher/pkg/apis v0.0.0-20241127174121-c051d99dcded + github.com/rancher/rancher/pkg/apis v0.0.0-20250410003522-2a1bf3d05723 go.qase.io/client v0.0.0-20231114201952-65195ec001fa ) @@ -367,7 +367,7 @@ require ( github.com/lib/pq v1.10.9 // indirect github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect github.com/mailru/easyjson v0.7.7 // indirect - github.com/matryer/moq v0.3.4 // indirect + github.com/matryer/moq v0.5.2 // indirect github.com/mattermost/xml-roundtrip-validator v0.1.0 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-runewidth v0.0.15 // indirect
pkg/api/norman/customization/globalnamespaceaccess/access_common.go+20 −8 modified@@ -5,14 +5,13 @@ import ( "fmt" "strings" - v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - "github.com/rancher/norman/api/access" "github.com/rancher/norman/httperror" "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" "github.com/rancher/norman/types/set" "github.com/rancher/norman/types/slice" + v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" client "github.com/rancher/rancher/pkg/client/generated/management/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" "github.com/rancher/rancher/pkg/ref" @@ -62,7 +61,7 @@ func (ma *MemberAccess) IsAdmin(callerID string) (bool, error) { return false, err } if u == nil { - return false, fmt.Errorf("No user found with ID %v", callerID) + return false, fmt.Errorf("no user found with ID %v", callerID) } // Get globalRoleBinding for this user grbs, err := ma.GrbLister.List("", labels.NewSelector()) @@ -142,9 +141,17 @@ func (ma *MemberAccess) EnsureRoleInTargets(targetProjects, roleTemplates []stri callerIsProjectOwner := false callerIsProjectMember := false callerIsClusterOwner := false - prtbs, err := ma.PrtbLister.List(pname, labels.NewSelector()) + + p, err := ma.ProjectLister.Get(cname, pname) if err != nil { - return err + return fmt.Errorf("unable to get project %s in namespace %s: %w", pname, cname, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) + if err != nil { + return fmt.Errorf("unable to get PRTBs in namespace %s: %w", backingNamespace, err) } for _, prtb := range prtbs { if prtb.UserName == callerID { @@ -486,21 +493,26 @@ func (ma *MemberAccess) RemoveRolesFromTargets(targetProjects, rolesToRemove []s return httperror.NewAPIError(httperror.InvalidBodyContent, errMsg) } clusterName, projectName := split[0], split[1] - clustersCovered := make(map[string]bool) - prtbs, err := ma.PrtbLister.List(projectName, labels.NewSelector()) + p, err := ma.ProjectLister.Get(clusterName, projectName) + if err != nil { + return fmt.Errorf("unable to get project %s in namespace %s: %w", projectName, clusterName, err) + } + backingNamespace := p.GetProjectBackingNamespace() + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) if err != nil { return err } for _, prtb := range prtbs { if prtb.UserPrincipalName == systemUserPrincipalID { if removeAllRoles || rolesToRemoveMap[prtb.RoleTemplateName] { - if err = ma.Prtbs.DeleteNamespaced(projectName, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { + if err = ma.Prtbs.DeleteNamespaced(backingNamespace, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { return err } } } } + clustersCovered := make(map[string]bool) if !clustersCovered[clusterName] { crtbs, err := ma.CrtbLister.List(clusterName, labels.NewSelector()) if err != nil {
pkg/api/norman/server/managementstored/setup.go+5 −3 modified@@ -140,12 +140,14 @@ func Setup(ctx context.Context, apiContext *config.ScaledContext, clusterManager authn.SetRTBStore(ctx, schemas.Schema(&managementschema.Version, client.ProjectRoleTemplateBindingType), apiContext) nodeStore.SetupStore(schemas.Schema(&managementschema.Version, client.NodeType)) projectaction.SetProjectStore(schemas.Schema(&managementschema.Version, client.ProjectType), apiContext) - setupScopedTypes(schemas) + setupScopedTypes(schemas, apiContext) return nil } -func setupScopedTypes(schemas *types.Schemas) { +func setupScopedTypes(schemas *types.Schemas, management *config.ScaledContext) { + projectLister := management.Management.Projects("").Controller().Lister() + for _, schema := range schemas.Schemas() { if schema.Scope != types.NamespaceScope || schema.Store == nil || schema.Store.Context() != config.ManagementStorageContext { continue @@ -161,7 +163,7 @@ func setupScopedTypes(schemas *types.Schemas) { continue } - schema.Store = scoped.NewScopedStore(key, schema.Store) + schema.Store = scoped.NewScopedStore(key, schema.Store, projectLister) ns.Required = false schema.ResourceFields["namespaceId"] = ns break
pkg/api/norman/store/scoped/store.go+22 −7 modified@@ -7,14 +7,16 @@ import ( "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" client "github.com/rancher/rancher/pkg/client/generated/management/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" ) type Store struct { types.Store - key string + key string + projectCache v3.ProjectLister } -func NewScopedStore(key string, store types.Store) *Store { +func NewScopedStore(key string, store types.Store, pLister v3.ProjectLister) *Store { return &Store{ Store: &transform.Store{ Store: store, @@ -23,21 +25,34 @@ func NewScopedStore(key string, store types.Store) *Store { return data, nil } v := convert.ToString(data[key]) - if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) { + if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) && v != strings.Replace(convert.ToString(data[client.ProjectFieldNamespaceId]), "-", ":", 1) { data[key] = data[client.ProjectFieldNamespaceId] } + data[client.ProjectFieldNamespaceId] = nil return data, nil }, }, - key: key, + key: key, + projectCache: pLister, } } func (s *Store) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { - if data != nil { - parts := strings.Split(convert.ToString(data[s.key]), ":") - data["namespaceId"] = parts[len(parts)-1] + if data == nil { + return s.Store.Create(apiContext, schema, data) + } + + clusterName, projectName, isProject := strings.Cut(convert.ToString(data[s.key]), ":") + if isProject { + p, err := s.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, err + } + data[client.ProjectFieldNamespaceId] = p.GetProjectBackingNamespace() + } else { + data[client.ProjectFieldNamespaceId] = data[s.key] } + return s.Store.Create(apiContext, schema, data) }
pkg/api/norman/store/scoped/store_test.go+142 −0 added@@ -0,0 +1,142 @@ +package scoped + +import ( + "fmt" + "reflect" + "testing" + + "github.com/rancher/norman/types" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" + "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type fakeStore struct { +} + +func (f fakeStore) Context() types.StorageContext { + return "" +} +func (f fakeStore) ByID(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) List(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) ([]map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { + return data, nil +} +func (f fakeStore) Update(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Delete(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Watch(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) (chan map[string]interface{}, error) { + return nil, nil +} + +func TestStoreCreate(t *testing.T) { + store := fakeStore{} + + p := fakes.ProjectListerMock{} + + tests := []struct { + name string + key string + getFunc func(string, string) (*v3.Project, error) + data map[string]interface{} + want map[string]interface{} + wantErr bool + }{ + { + name: "nil data returns nil", + data: nil, + want: nil, + }, + { + name: "project: set namespace no backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "project-XYZ", + }, + }, + { + name: "project: set namespace to backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "c-ABC-p-XYZ", + }, + }, + { + name: "error getting project", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + return nil, fmt.Errorf("error") + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + wantErr: true, + }, + { + name: "cluster: set namespaceId", + key: "clusterId", + data: map[string]interface{}{ + "clusterId": "cluster-ABC", + }, + want: map[string]interface{}{ + "clusterId": "cluster-ABC", + "namespaceId": "cluster-ABC", + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p.GetFunc = tt.getFunc + s := &Store{ + Store: store, + key: tt.key, + projectCache: &p, + } + got, err := s.Create(nil, nil, tt.data) + if (err != nil) != tt.wantErr { + t.Errorf("Store.Create() error = %v, wantErr %v", err, tt.wantErr) + return + } + if !reflect.DeepEqual(got, tt.want) { + t.Errorf("Store.Create() = %v, want %v", got, tt.want) + } + }) + } +}
pkg/apis/management.cattle.io/v3/authz_types.go+8 −0 modified@@ -48,6 +48,14 @@ func (p *Project) ObjClusterName() string { return p.Spec.ObjClusterName() } +// GetProjectBackingNamespace returns the namespace a project uses in the local cluster to store PRTBs and Project Scoped Secrets. +func (p *Project) GetProjectBackingNamespace() string { + if p.Status.BackingNamespace != "" { + return p.Status.BackingNamespace + } + return p.Name +} + // ProjectStatus represents the most recently observed status of the project. type ProjectStatus struct { // Conditions are a set of indicators about aspects of the project.
pkg/controllers/management/auth/crtb_handler.go+8 −7 modified@@ -6,9 +6,8 @@ import ( "strings" "time" - "github.com/rancher/rancher/pkg/controllers/status" - "github.com/rancher/rancher/pkg/controllers/management/authprovisioningv2" + "github.com/rancher/rancher/pkg/controllers/status" controllersv3 "github.com/rancher/rancher/pkg/generated/controllers/management.cattle.io/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" typesrbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" @@ -234,11 +233,12 @@ func (c *crtbLifecycle) reconcileBindings(binding *v3.ClusterRoleTemplateBinding return err } for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() if p.DeletionTimestamp != nil { - logrus.Warnf("Project %v is being deleted, not creating membership bindings", p.Name) + logrus.Warnf("Project %v is being deleted, not creating membership bindings", backingNamespace) continue } - if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, p.Name, projectManagementPlaneResources, subject, binding); err != nil { + if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, backingNamespace, projectManagementPlaneResources, subject, binding); err != nil { c.s.AddCondition(localConditions, condition, failedToGrantManagementClusterScopedPrivilegesInProjectNamespace, err) return err } @@ -255,14 +255,15 @@ func (c *crtbLifecycle) removeMGMTClusterScopedPrivilegesInProjectNamespace(bind } bindingKey := pkgrbac.GetRTBLabel(binding.ObjectMeta) for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() set := labels.Set(map[string]string{bindingKey: CrtbInProjectBindingOwner}) - rbs, err := c.rbLister.List(p.Name, set.AsSelector()) + rbs, err := c.rbLister.List(backingNamespace, set.AsSelector()) if err != nil { return err } for _, rb := range rbs { - logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, p.Name, binding.Name) - if err := c.rbClient.DeleteNamespaced(p.Name, rb.Name, &metav1.DeleteOptions{}); err != nil { + logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, backingNamespace, binding.Name) + if err := c.rbClient.DeleteNamespaced(backingNamespace, rb.Name, &metav1.DeleteOptions{}); err != nil { return err } }
pkg/controllers/management/auth/crtb_handler_test.go+176 −17 modified@@ -5,21 +5,23 @@ import ( "testing" "time" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/rancher/pkg/controllers/status" controllersv3 "github.com/rancher/rancher/pkg/generated/controllers/management.cattle.io/v3" + "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + rbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" + corefakes "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1/fakes" "github.com/rancher/wrangler/v3/pkg/generic/fake" "github.com/stretchr/testify/assert" - - v3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" ) var ( - e = fmt.Errorf("error") + errDefault = fmt.Errorf("error") defaultCRTB = v3.ClusterRoleTemplateBinding{ UserName: "test", GroupName: "", @@ -42,12 +44,25 @@ var ( Name: "test-project", }, } + backingNamespaceProject = v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-project", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } deletingProject = v3.Project{ ObjectMeta: v1.ObjectMeta{ Name: "deleting-project", DeletionTimestamp: &v1.Time{Time: time.Now()}, }, } + defaultBinding = rbacv1.RoleBinding{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-binding", + }, + } ) type crtbTestState struct { @@ -93,7 +108,7 @@ func TestReconcileBindings(t *testing.T) { name: "error getting cluster", stateSetup: func(cts crtbTestState) { cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -103,7 +118,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToGetCluster, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -140,7 +155,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). - Return(true, e) + Return(true, errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -149,7 +164,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToCheckReferencedRole, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -168,7 +183,7 @@ func TestReconcileBindings(t *testing.T) { Return(true, nil) cts.managerMock.EXPECT(). ensureClusterMembershipBinding("clustername-clusterowner", gomock.Any(), gomock.Any(), true, gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -178,7 +193,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToEnsureClusterMembershipBinding, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -200,7 +215,7 @@ func TestReconcileBindings(t *testing.T) { Return(nil) cts.managerMock.EXPECT(). grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -209,7 +224,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToGrantManagementPlanePrivileges, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -233,7 +248,7 @@ func TestReconcileBindings(t *testing.T) { grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). Return(nil) cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -243,7 +258,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToListProjects, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -272,7 +287,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "test-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -282,7 +297,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToGrantManagementClusterScopedPrivilegesInProjectNamespace, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -361,6 +376,42 @@ func TestReconcileBindings(t *testing.T) { }, }, }, + { + name: "successfully reconcile clustermember with backingNamespace", + stateSetup: func(cts crtbTestState) { + cts.managerMock.EXPECT(). + checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). + Return(false, nil) + cts.managerMock.EXPECT(). + ensureClusterMembershipBinding("clustername-clustermember", gomock.Any(), gomock.Any(), false, gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "c-ABC-p-XYZ", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { + c := defaultCluster.DeepCopy() + return c, nil + } + cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { + p := backingNamespaceProject.DeepCopy() + return []*v3.Project{p}, nil + } + }, + crtb: defaultCRTB.DeepCopy(), + wantConditions: []v1.Condition{ + { + Type: bindingExists, + Status: v1.ConditionTrue, + Reason: bindingExists, + LastTransitionTime: v1.Time{ + Time: mockTime, + }, + }, + }, + }, { name: "skip projects that are deleting", stateSetup: func(cts crtbTestState) { @@ -376,7 +427,7 @@ func TestReconcileBindings(t *testing.T) { // This should not be called cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "deleting-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e).AnyTimes() + Return(errDefault).AnyTimes() cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { c := defaultCluster.DeepCopy() return c, nil @@ -649,3 +700,111 @@ func setupTest(t *testing.T) crtbTestState { } return state } + +func Test_removeMGMTClusterScopedPrivilegesInProjectNamespace(t *testing.T) { + tests := []struct { + name string + projectListFunc func(string, labels.Selector) ([]*apisv3.Project, error) + roleBindingListFunc func(string, labels.Selector) ([]*rbacv1.RoleBinding, error) + roleBindingDeleteFunc func(string, string, *v1.DeleteOptions) error + binding *v3.ClusterRoleTemplateBinding + wantErr bool + }{ + { + name: "error listing projects", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error listing rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error deleting rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + return errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "successfully delete rolebindings no backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, defaultProject.Name, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, defaultProject.Name, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + { + name: "successfully delete rolebindings with backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + backingNamespaceProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p := fakes.ProjectListerMock{} + p.ListFunc = tt.projectListFunc + rbl := corefakes.RoleBindingListerMock{} + rbl.ListFunc = tt.roleBindingListFunc + rbi := corefakes.RoleBindingInterfaceMock{} + rbi.DeleteNamespacedFunc = tt.roleBindingDeleteFunc + + c := &crtbLifecycle{ + projectLister: &p, + rbLister: &rbl, + rbClient: &rbi, + } + if err := c.removeMGMTClusterScopedPrivilegesInProjectNamespace(tt.binding); (err != nil) != tt.wantErr { + t.Errorf("crtbLifecycle.removeMGMTClusterScopedPrivilegesInProjectNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +}
pkg/controllers/management/auth/project_cluster/cluster_handler.go+2 −2 modified@@ -83,7 +83,7 @@ func (l *clusterLifecycle) Sync(key string, orig *apisv3.Cluster) (runtime.Objec } obj := orig.DeepCopyObject() - obj, err := reconcileResourceToNamespace(obj, ClusterCreateController, l.nsLister, l.nsClient) + obj, err := reconcileResourceToNamespace(obj, ClusterCreateController, orig.Name, l.nsLister, l.nsClient) if err != nil { return nil, err } @@ -155,7 +155,7 @@ func (l *clusterLifecycle) Remove(obj *apisv3.Cluster) (runtime.Object, error) { returnErr := errors.Join( l.deleteSystemProject(obj, ClusterRemoveController), - deleteNamespace(obj, ClusterRemoveController, l.nsClient), + deleteNamespace(ClusterRemoveController, obj.Name, l.nsClient), ) return obj, returnErr }
pkg/controllers/management/auth/project_cluster/common.go+11 −18 modified@@ -32,50 +32,43 @@ const ( var crtbCreatorOwnerAnnotations = map[string]string{creatorOwnerBindingAnnotation: "true"} -func deleteNamespace(obj runtime.Object, controller string, nsClient v1.NamespaceInterface) error { - o, err := meta.Accessor(obj) - if err != nil { - return fmt.Errorf("[%s] error accessing object %v: %w", controller, obj, err) - } - - ns, err := nsClient.Get(context.TODO(), o.GetName(), metav1.GetOptions{}) +func deleteNamespace(controller string, nsName string, nsClient v1.NamespaceInterface) error { + ns, err := nsClient.Get(context.TODO(), nsName, metav1.GetOptions{}) if apierrors.IsNotFound(err) { return nil + } else if err != nil { + return err } if ns.Status.Phase != v12.NamespaceTerminating { - logrus.Infof("[%s] Deleting namespace %s", controller, o.GetName()) - err = nsClient.Delete(context.TODO(), o.GetName(), metav1.DeleteOptions{}) + logrus.Infof("[%s] Deleting namespace %s", controller, nsName) + err = nsClient.Delete(context.TODO(), nsName, metav1.DeleteOptions{}) if apierrors.IsNotFound(err) { return nil } } return err } -func reconcileResourceToNamespace(obj runtime.Object, controller string, nsLister corev1.NamespaceCache, nsClient v1.NamespaceInterface) (runtime.Object, error) { +func reconcileResourceToNamespace(obj runtime.Object, controller string, nsName string, nsLister corev1.NamespaceCache, nsClient v1.NamespaceInterface) (runtime.Object, error) { return apisv3.NamespaceBackedResource.Do(obj, func() (runtime.Object, error) { - o, err := meta.Accessor(obj) - if err != nil { - return obj, fmt.Errorf("[%s] error accessing object %v: %w", controller, obj, err) - } t, err := meta.TypeAccessor(obj) if err != nil { return obj, err } - ns, _ := nsLister.Get(o.GetName()) + ns, _ := nsLister.Get(nsName) if ns == nil { - logrus.Infof("[%v] Creating namespace %v", controller, o.GetName()) + logrus.Infof("[%v] Creating namespace %v", controller, nsName) _, err := nsClient.Create(context.TODO(), &v12.Namespace{ ObjectMeta: metav1.ObjectMeta{ - Name: o.GetName(), + Name: nsName, Annotations: map[string]string{ "management.cattle.io/system-namespace": "true", }, }, }, metav1.CreateOptions{}) if err != nil { - return obj, fmt.Errorf("[%s] failed to create namespace for %v %v: %w", controller, t.GetKind(), o.GetName(), err) + return obj, fmt.Errorf("[%s] failed to create namespace for %s %s: %w", controller, t.GetKind(), nsName, err) } }
pkg/controllers/management/auth/project_cluster/common_test.go+142 −0 modified@@ -1,16 +1,22 @@ package project_cluster import ( + "context" + "fmt" "testing" apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/wrangler/v3/pkg/generic/fake" "go.uber.org/mock/gomock" + v1 "k8s.io/api/core/v1" rbacv1 "k8s.io/api/rbac/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/watch" + applycorev1 "k8s.io/client-go/applyconfigurations/core/v1" ) var errNotFound = apierrors.NewNotFound(schema.GroupResource{}, "") @@ -168,3 +174,139 @@ func TestCreateMembershipRoles(t *testing.T) { }) } } + +var ( + errDefault = fmt.Errorf("error") + errNsNotFound = apierrors.NewNotFound(v1.Resource("namespace"), "") + + defaultNamespace = v1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "default-namespace", + }, + } + terminatingNamespace = v1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "terminating-namespace", + }, + Status: v1.NamespaceStatus{ + Phase: v1.NamespaceTerminating, + }, + } +) + +func Test_deleteNamespace(t *testing.T) { + tests := []struct { + name string + nsGetFunc func(context.Context, string, metav1.GetOptions) (*v1.Namespace, error) + nsDeleteFunc func(context.Context, string, metav1.DeleteOptions) error + wantErr bool + }{ + { + name: "error getting namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return nil, errDefault + }, + wantErr: true, + }, + { + name: "namespace not found", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return nil, errNsNotFound + }, + }, + { + name: "namespace is terminating", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return terminatingNamespace.DeepCopy(), nil + }, + }, + { + name: "successfully delete namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return nil + }, + }, + { + name: "deleting namespace not found", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return errNsNotFound + }, + }, + { + name: "error deleting namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return errDefault + }, + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + nsClientFake := mockNamespaces{ + getter: tt.nsGetFunc, + deleter: tt.nsDeleteFunc, + } + if err := deleteNamespace("", "", nsClientFake); (err != nil) != tt.wantErr { + t.Errorf("deleteNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +} + +type mockNamespaces struct { + getter func(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Namespace, error) + deleter func(ctx context.Context, name string, opts metav1.DeleteOptions) error +} + +func (m mockNamespaces) Get(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Namespace, error) { + return m.getter(ctx, name, opts) +} + +func (m mockNamespaces) Create(ctx context.Context, namespace *v1.Namespace, opts metav1.CreateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) Update(ctx context.Context, namespace *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) UpdateStatus(ctx context.Context, namespace *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) Delete(ctx context.Context, name string, opts metav1.DeleteOptions) error { + return m.deleter(ctx, name, opts) +} + +func (m mockNamespaces) List(ctx context.Context, opts metav1.ListOptions) (*v1.NamespaceList, error) { + panic("implement me") +} + +func (m mockNamespaces) Watch(ctx context.Context, opts metav1.ListOptions) (watch.Interface, error) { + panic("implement me") +} + +func (m mockNamespaces) Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts metav1.PatchOptions, subresources ...string) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) Apply(ctx context.Context, namespace *applycorev1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) ApplyStatus(ctx context.Context, namespace *applycorev1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) Finalize(ctx context.Context, item *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +}
pkg/controllers/management/auth/project_cluster/project_handler.go+18 −14 modified@@ -78,7 +78,8 @@ func (l *projectLifecycle) Sync(key string, orig *apisv3.Project) (runtime.Objec obj := orig.DeepCopyObject() - obj, err := reconcileResourceToNamespace(obj, ProjectCreateController, l.nsLister, l.nsClient) + backingNamespace := orig.GetProjectBackingNamespace() + obj, err := reconcileResourceToNamespace(obj, ProjectCreateController, backingNamespace, l.nsLister, l.nsClient) if err != nil { return nil, err } @@ -89,7 +90,7 @@ func (l *projectLifecycle) Sync(key string, orig *apisv3.Project) (runtime.Objec } } - obj, err = l.reconcileProjectCreatorRTB(obj) + obj, err = l.reconcileProjectCreatorRTB(obj, backingNamespace) if err != nil { return nil, err } @@ -140,10 +141,11 @@ func (l *projectLifecycle) Updated(obj *apisv3.Project) (runtime.Object, error) // Remove deletes all backing resources created by the project func (l *projectLifecycle) Remove(obj *apisv3.Project) (runtime.Object, error) { - return obj, deleteNamespace(obj, ProjectRemoveController, l.nsClient) + backingNamespace := obj.GetProjectBackingNamespace() + return obj, deleteNamespace(ProjectRemoveController, backingNamespace, l.nsClient) } -func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runtime.Object, error) { +func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object, nsName string) (runtime.Object, error) { project, ok := obj.(*apisv3.Project) if !ok { return obj, fmt.Errorf("expected project, got %T", obj) @@ -152,18 +154,19 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti // If we specify no creator owner RBAC, exit if _, ok := project.Annotations[NoCreatorRBACAnnotation]; ok { logrus.Infof("[%s] annotation %s found. Skipping adding creator as owner", ProjectCreateController, NoCreatorRBACAnnotation) - return obj, nil + return project, nil } - return apisv3.CreatorMadeOwner.DoUntilTrue(obj, func() (runtime.Object, error) { + + return apisv3.CreatorMadeOwner.DoUntilTrue(project, func() (runtime.Object, error) { creatorID := project.Annotations[CreatorIDAnnotation] if creatorID == "" { logrus.Warnf("[%s] project %s has no creatorId annotation. Cannot add creator as owner", ProjectCreateController, project.Name) - return obj, nil + return project, nil } if apisv3.ProjectConditionInitialRolesPopulated.IsTrue(project) { // The projectRoleBindings are already completed, no need to check - return obj, nil + return project, nil } // If the project does not have the annotation it indicates the @@ -176,13 +179,14 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti roleMap := make(map[string][]string) if err := json.Unmarshal([]byte(creatorRoleBindings), &roleMap); err != nil { - return obj, err + return project, err } var createdRoles []string for _, role := range roleMap["required"] { rtbName := "creator-" + role - if rtb, _ := l.prtbLister.Get(project.Name, rtbName); rtb != nil { + + if rtb, _ := l.prtbLister.Get(nsName, rtbName); rtb != nil { createdRoles = append(createdRoles, role) // This projectRoleBinding exists, need to check all of them so keep going continue @@ -192,7 +196,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti prtb := &apisv3.ProjectRoleTemplateBinding{ ObjectMeta: metav1.ObjectMeta{ Name: rtbName, - Namespace: project.Name, + Namespace: nsName, }, ProjectName: project.Namespace + ":" + project.Name, RoleTemplateName: role, @@ -210,7 +214,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti logrus.Infof("[%s] Creating creator projectRoleTemplateBinding for user %s for project %s", ProjectCreateController, creatorID, project.Name) _, err := l.prtbClient.Create(prtb) if err != nil && !apierrors.IsAlreadyExists(err) { - return obj, err + return project, err } createdRoles = append(createdRoles, role) @@ -221,7 +225,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti roleMap["created"] = createdRoles d, err := json.Marshal(roleMap) if err != nil { - return obj, err + return project, err } project.Annotations[roleTemplatesRequiredAnnotation] = string(d) @@ -233,6 +237,6 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti _, err = l.projects.Update(project) - return obj, err + return project, err }) }
pkg/controllers/management/auth/project_cluster/project_handler_test.go+3 −3 modified@@ -87,13 +87,13 @@ func TestReconcileProjectCreatorRTBRespectsUserPrincipalName(t *testing.T) { }, } - obj, err := lifecycle.reconcileProjectCreatorRTB(project) + obj, err := lifecycle.reconcileProjectCreatorRTB(project, clusterID) require.NoError(t, err) require.NotNil(t, obj) require.Len(t, prtbs, 1) assert.Equal(t, "creator-project-owner", prtbs[0].Name) - assert.Equal(t, "p-abcdef", prtbs[0].Namespace) + assert.Equal(t, clusterID, prtbs[0].Namespace) assert.Equal(t, clusterID+":p-abcdef", prtbs[0].ProjectName) assert.Equal(t, "", prtbs[0].UserName) assert.Equal(t, userPrincipalName, prtbs[0].UserPrincipalName) @@ -109,7 +109,7 @@ func TestReconcileProjectCreatorRTBNoCreatorRBAC(t *testing.T) { }, }, } - obj, err := lifecycle.reconcileProjectCreatorRTB(project) + obj, err := lifecycle.reconcileProjectCreatorRTB(project, clusterID) assert.NoError(t, err) assert.NotNil(t, obj) }
pkg/controllers/management/auth/prtb_handler.go+1 −1 modified@@ -125,7 +125,7 @@ func (p *prtbLifecycle) reconcileSubject(binding *v3.ProjectRoleTemplateBinding) return binding, nil } - return nil, fmt.Errorf("Binding %v has no subject", binding.Name) + return nil, fmt.Errorf("binding %s has no subject", binding.Name) } // When a PRTB is created or updated, translate it into several k8s roles and bindings to actually enforce the RBAC.
pkg/controllers/managementuser/rbac/namespace_handler.go+7 −4 modified@@ -15,7 +15,6 @@ import ( v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" namespaceutil "github.com/rancher/rancher/pkg/namespace" "github.com/rancher/rancher/pkg/project" - projectpkg "github.com/rancher/rancher/pkg/project" "github.com/sirupsen/logrus" v1 "k8s.io/api/core/v1" rbacv1 "k8s.io/api/rbac/v1" @@ -166,15 +165,15 @@ func (n *nsLifecycle) assignToInitialProject(ns *v1.Namespace) error { } func (n *nsLifecycle) GetSystemProjectName() (string, error) { - projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[projectpkg.System].AsSelector()) + projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[project.System].AsSelector()) if err != nil { return "", err } if len(projects) == 0 { return "", nil } if len(projects) > 1 { - return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, projectpkg.System) + return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, project.System) } if projects[0] == nil { return "", nil @@ -262,7 +261,11 @@ func (n *nsLifecycle) ensurePRTBAddToNamespace(ns *v1.Namespace) (bool, error) { var namespace string if parts := strings.SplitN(projectID, ":", 2); len(parts) == 2 && len(parts[1]) > 0 { - namespace = parts[1] + project, err := n.rq.ProjectLister.Get(parts[0], parts[1]) + if err != nil { + return hasPRTBs, err + } + namespace = project.GetProjectBackingNamespace() } else { return hasPRTBs, nil }
pkg/controllers/managementuser/secret/secret.go+32 −25 modified@@ -267,6 +267,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { n := &NamespaceController{ clusterSecretsClient: clusterSecretsClient, managementSecrets: cluster.Management.Core.Secrets("").Controller().Lister(), + projectCache: cluster.Management.Management.Projects("").Controller().Lister(), } cluster.Core.Namespaces("").AddHandler(ctx, "secretsController", n.sync) @@ -302,6 +303,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { type NamespaceController struct { clusterSecretsClient v1.SecretInterface managementSecrets v1.SecretLister + projectCache v3.ProjectLister } func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.Object, error) { @@ -312,33 +314,38 @@ func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.O // field.cattle.io/projectId value is <cluster name>:<project name> logrus.Tracef("secretsController: sync: key [%s], obj.Annotations[projectIDLabel]: [%s]", key, obj.Annotations[projectIDLabel]) if obj.Annotations[projectIDLabel] != "" { - parts := strings.Split(obj.Annotations[projectIDLabel], ":") - if len(parts) == 2 { - if parts[1] == "" { - logrus.Debugf("[NamspaceController|sync] empty project name found in obj.Annotations[projectIDLabel] for cluster: %s", parts[0]) - return nil, nil + clusterName, projectName, found := strings.Cut(obj.Annotations[projectIDLabel], ":") + if !found { + logrus.Debugf("secretsController: sync: namespace %s projectId annotation %s is malformed, should be <cluster name>:<project name>", obj.Name, obj.Annotations[projectIDLabel]) + return nil, nil + } + + p, err := n.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, fmt.Errorf("secretsController: sync: error getting project %s for namespace %s: %w", projectName, obj.Name, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + secrets, err := n.managementSecrets.List(backingNamespace, labels.NewSelector()) + if err != nil { + return nil, err + } + + logrus.Tracef("secretsController: sync: length of secrets for project [%s] in namespace [%s] is %d", projectName, obj.Name, len(secrets)) + for _, secret := range secrets { + // skip service account token secrets + if secret.Type == corev1.SecretTypeServiceAccountToken { + logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) + continue } - // on the management side, secret's namespace name equals to project name - secrets, err := n.managementSecrets.List(parts[1], labels.NewSelector()) - if err != nil { - return nil, err + namespacedSecret := getNamespacedSecret(secret, obj.Name) + if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { + continue } - logrus.Tracef("secretsController: sync: length of secrets for [%s] in namespace [%s] is %d", parts[1], obj.Name, len(secrets)) - for _, secret := range secrets { - // skip service account token secrets - if secret.Type == corev1.SecretTypeServiceAccountToken { - logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) - continue - } - namespacedSecret := getNamespacedSecret(secret, obj.Name) - if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { - continue - } - logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) - _, err := n.clusterSecretsClient.Create(namespacedSecret) - if err != nil && !errors.IsAlreadyExists(err) { - return nil, err - } + logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) + _, err := n.clusterSecretsClient.Create(namespacedSecret) + if err != nil && !errors.IsAlreadyExists(err) { + return nil, err } } }
pkg/systemaccount/systemaccount.go+0 −35 modified@@ -6,7 +6,6 @@ import ( v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" - "github.com/rancher/rancher/pkg/ref" "github.com/rancher/rancher/pkg/types/config" "github.com/rancher/rancher/pkg/types/config/systemtokens" "github.com/rancher/rancher/pkg/user" @@ -28,7 +27,6 @@ func NewManager(management *config.ManagementContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), users: management.Management.Users(""), @@ -41,7 +39,6 @@ func NewManagerFromScale(management *config.ScaledContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), tokenLister: management.Management.Tokens("").Controller().Lister(), @@ -54,7 +51,6 @@ type Manager struct { systemTokens systemtokens.Interface crtbs v3.ClusterRoleTemplateBindingInterface crts v3.ClusterRegistrationTokenInterface - prtbs v3.ProjectRoleTemplateBindingInterface prtbLister v3.ProjectRoleTemplateBindingLister tokens v3.TokenInterface tokenLister v3.TokenLister @@ -124,37 +120,6 @@ func (s *Manager) GetOrCreateSystemClusterToken(clusterName string) (string, err return token, nil } -func (s *Manager) GetOrCreateProjectSystemAccount(projectID string) error { - _, projectName := ref.Parse(projectID) - - u, err := s.GetProjectSystemUser(projectName) - if err != nil { - return err - } - - bindingName := u.Name + "-member" - _, err = s.prtbLister.Get(projectName, bindingName) - if err != nil { - if !errors2.IsNotFound(err) { - return err - } - // prtb does not exist in cache, attempt to create it - prtb := &v3.ProjectRoleTemplateBinding{ - ObjectMeta: v1.ObjectMeta{ - Name: bindingName, - Namespace: projectName, - }, - ProjectName: projectID, - UserName: u.Name, - RoleTemplateName: projectMemberRole, - } - if _, err := s.prtbs.Create(prtb); err != nil && !errors2.IsAlreadyExists(err) { - return err - } - } - return nil -} - func (s *Manager) GetProjectSystemUser(projectName string) (*v3.User, error) { return s.userManager.EnsureUser(fmt.Sprintf("system://%s", projectName), ProjectSystemAccountPrefix+projectName) }
tests/integration/suite/test_rbac.py+1 −0 modified@@ -679,6 +679,7 @@ def test_readonly_cannot_edit_secret(admin_mc, user_mc, admin_pc, assert e.value.error.status == 404 +@pytest.mark.skip def test_member_can_edit_secret(admin_mc, admin_pc, remove_resource, user_mc): """Tests that a user with project-member role is able to create/update
7f16b596120dAdd project namespace handling (#49797)
19 files changed · +602 −148
go.mod+2 −2 modified@@ -63,7 +63,7 @@ replace ( require ( github.com/antihax/optional v1.0.0 - github.com/rancher/rancher/pkg/apis v0.0.0-20241127174121-c051d99dcded + github.com/rancher/rancher/pkg/apis v0.0.0-20250410003522-2a1bf3d05723 go.qase.io/client v0.0.0-20231114201952-65195ec001fa ) @@ -145,7 +145,7 @@ require ( github.com/rancher/rancher/pkg/client v0.0.0 github.com/rancher/remotedialer v0.4.4 github.com/rancher/rke v1.8.0-rc.4 - github.com/rancher/shepherd v0.0.0-20241213222351-98e341c77d0b + github.com/rancher/shepherd v0.0.0-20250411212007-f3f2fd268849 github.com/rancher/steve v0.6.2 github.com/rancher/system-upgrade-controller/pkg/apis v0.0.0-20250306000150-b1a9781accab github.com/rancher/wrangler v1.1.2
go.sum+2 −2 modified@@ -2469,8 +2469,8 @@ github.com/rancher/rke v1.8.0-rc.4 h1:jowVyaF3LsJonC7vNsAwWf3MONHAtEFUD/j3UzNSE5 github.com/rancher/rke v1.8.0-rc.4/go.mod h1:x9N1abruzDFMwTpqq2cnaDYpKCptlNoW8VraNWB6Pc4= github.com/rancher/saml v0.4.14-rancher3 h1:2NN6cPqm9FJeiT25x8+gLHWGdulsEak33cHRkGaJ5v0= github.com/rancher/saml v0.4.14-rancher3/go.mod h1:S4+611dxnKt8z/ulbvaJzcgSHsuhjVc1QHNTcr1R7Fw= -github.com/rancher/shepherd v0.0.0-20241213222351-98e341c77d0b h1:uX+XbQgpqpfxfvDI+6uPb1DeeDqEwa7lE+QtYuPCqzU= -github.com/rancher/shepherd v0.0.0-20241213222351-98e341c77d0b/go.mod h1:urZvZCFSgT+9NVjAV0y8v+pzuqziaS3aYfoMfk9TENw= +github.com/rancher/shepherd v0.0.0-20250411212007-f3f2fd268849 h1:hxa/Y0LRTx8BzMPxirT9Yg3IZg2YXus7+smLLn5n9tw= +github.com/rancher/shepherd v0.0.0-20250411212007-f3f2fd268849/go.mod h1:IVVaLrIQ1/1Fk7KTrkhpKFlgaqhh3uv27CokmEhXHJc= github.com/rancher/steve v0.6.2 h1:vQEvAWfklKWh8LjvHhgrh9kWu0xNg/m+nIbjxAU7Te4= github.com/rancher/steve v0.6.2/go.mod h1:+tzNay6IxktR3wMBfGD6CVdxxEc0kUkMP5yCpj7ib4I= github.com/rancher/system-upgrade-controller/pkg/apis v0.0.0-20250306000150-b1a9781accab h1:Ttxt14bAImsWyFrtQZ314GW2DeExrYRNoAb+u9V3RiA=
pkg/api/norman/customization/globalnamespaceaccess/access_common.go+20 −8 modified@@ -5,14 +5,13 @@ import ( "fmt" "strings" - v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - "github.com/rancher/norman/api/access" "github.com/rancher/norman/httperror" "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" "github.com/rancher/norman/types/set" "github.com/rancher/norman/types/slice" + v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" client "github.com/rancher/rancher/pkg/client/generated/management/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" "github.com/rancher/rancher/pkg/ref" @@ -62,7 +61,7 @@ func (ma *MemberAccess) IsAdmin(callerID string) (bool, error) { return false, err } if u == nil { - return false, fmt.Errorf("No user found with ID %v", callerID) + return false, fmt.Errorf("no user found with ID %v", callerID) } // Get globalRoleBinding for this user grbs, err := ma.GrbLister.List("", labels.NewSelector()) @@ -142,9 +141,17 @@ func (ma *MemberAccess) EnsureRoleInTargets(targetProjects, roleTemplates []stri callerIsProjectOwner := false callerIsProjectMember := false callerIsClusterOwner := false - prtbs, err := ma.PrtbLister.List(pname, labels.NewSelector()) + + p, err := ma.ProjectLister.Get(cname, pname) if err != nil { - return err + return fmt.Errorf("unable to get project %s in namespace %s: %w", pname, cname, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) + if err != nil { + return fmt.Errorf("unable to get PRTBs in namespace %s: %w", backingNamespace, err) } for _, prtb := range prtbs { if prtb.UserName == callerID { @@ -486,21 +493,26 @@ func (ma *MemberAccess) RemoveRolesFromTargets(targetProjects, rolesToRemove []s return httperror.NewAPIError(httperror.InvalidBodyContent, errMsg) } clusterName, projectName := split[0], split[1] - clustersCovered := make(map[string]bool) - prtbs, err := ma.PrtbLister.List(projectName, labels.NewSelector()) + p, err := ma.ProjectLister.Get(clusterName, projectName) + if err != nil { + return fmt.Errorf("unable to get project %s in namespace %s: %w", projectName, clusterName, err) + } + backingNamespace := p.GetProjectBackingNamespace() + prtbs, err := ma.PrtbLister.List(backingNamespace, labels.NewSelector()) if err != nil { return err } for _, prtb := range prtbs { if prtb.UserPrincipalName == systemUserPrincipalID { if removeAllRoles || rolesToRemoveMap[prtb.RoleTemplateName] { - if err = ma.Prtbs.DeleteNamespaced(projectName, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { + if err = ma.Prtbs.DeleteNamespaced(backingNamespace, prtb.Name, &v1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) && !apierrors.IsGone(err) { return err } } } } + clustersCovered := make(map[string]bool) if !clustersCovered[clusterName] { crtbs, err := ma.CrtbLister.List(clusterName, labels.NewSelector()) if err != nil {
pkg/api/norman/server/managementstored/setup.go+5 −3 modified@@ -141,12 +141,14 @@ func Setup(ctx context.Context, apiContext *config.ScaledContext, clusterManager authn.SetRTBStore(ctx, schemas.Schema(&managementschema.Version, client.ProjectRoleTemplateBindingType), apiContext) nodeStore.SetupStore(schemas.Schema(&managementschema.Version, client.NodeType)) projectaction.SetProjectStore(schemas.Schema(&managementschema.Version, client.ProjectType), apiContext) - setupScopedTypes(schemas) + setupScopedTypes(schemas, apiContext) return nil } -func setupScopedTypes(schemas *types.Schemas) { +func setupScopedTypes(schemas *types.Schemas, management *config.ScaledContext) { + projectLister := management.Management.Projects("").Controller().Lister() + for _, schema := range schemas.Schemas() { if schema.Scope != types.NamespaceScope || schema.Store == nil || schema.Store.Context() != config.ManagementStorageContext { continue @@ -162,7 +164,7 @@ func setupScopedTypes(schemas *types.Schemas) { continue } - schema.Store = scoped.NewScopedStore(key, schema.Store) + schema.Store = scoped.NewScopedStore(key, schema.Store, projectLister) ns.Required = false schema.ResourceFields["namespaceId"] = ns break
pkg/api/norman/store/scoped/store.go+22 −7 modified@@ -7,14 +7,16 @@ import ( "github.com/rancher/norman/types" "github.com/rancher/norman/types/convert" client "github.com/rancher/rancher/pkg/client/generated/management/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" ) type Store struct { types.Store - key string + key string + projectCache v3.ProjectLister } -func NewScopedStore(key string, store types.Store) *Store { +func NewScopedStore(key string, store types.Store, pLister v3.ProjectLister) *Store { return &Store{ Store: &transform.Store{ Store: store, @@ -23,21 +25,34 @@ func NewScopedStore(key string, store types.Store) *Store { return data, nil } v := convert.ToString(data[key]) - if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) { + if !strings.HasSuffix(v, ":"+convert.ToString(data[client.ProjectFieldNamespaceId])) && v != strings.Replace(convert.ToString(data[client.ProjectFieldNamespaceId]), "-", ":", 1) { data[key] = data[client.ProjectFieldNamespaceId] } + data[client.ProjectFieldNamespaceId] = nil return data, nil }, }, - key: key, + key: key, + projectCache: pLister, } } func (s *Store) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { - if data != nil { - parts := strings.Split(convert.ToString(data[s.key]), ":") - data["namespaceId"] = parts[len(parts)-1] + if data == nil { + return s.Store.Create(apiContext, schema, data) + } + + clusterName, projectName, isProject := strings.Cut(convert.ToString(data[s.key]), ":") + if isProject { + p, err := s.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, err + } + data[client.ProjectFieldNamespaceId] = p.GetProjectBackingNamespace() + } else { + data[client.ProjectFieldNamespaceId] = data[s.key] } + return s.Store.Create(apiContext, schema, data) }
pkg/api/norman/store/scoped/store_test.go+142 −0 added@@ -0,0 +1,142 @@ +package scoped + +import ( + "fmt" + "reflect" + "testing" + + "github.com/rancher/norman/types" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" + "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type fakeStore struct { +} + +func (f fakeStore) Context() types.StorageContext { + return "" +} +func (f fakeStore) ByID(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) List(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) ([]map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Create(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}) (map[string]interface{}, error) { + return data, nil +} +func (f fakeStore) Update(apiContext *types.APIContext, schema *types.Schema, data map[string]interface{}, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Delete(apiContext *types.APIContext, schema *types.Schema, id string) (map[string]interface{}, error) { + return nil, nil +} +func (f fakeStore) Watch(apiContext *types.APIContext, schema *types.Schema, opt *types.QueryOptions) (chan map[string]interface{}, error) { + return nil, nil +} + +func TestStoreCreate(t *testing.T) { + store := fakeStore{} + + p := fakes.ProjectListerMock{} + + tests := []struct { + name string + key string + getFunc func(string, string) (*v3.Project, error) + data map[string]interface{} + want map[string]interface{} + wantErr bool + }{ + { + name: "nil data returns nil", + data: nil, + want: nil, + }, + { + name: "project: set namespace no backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "project-XYZ", + }, + }, + { + name: "project: set namespace to backing namespace", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + p := v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "project-XYZ", + Namespace: "cluster-ABC", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } + return &p, nil + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + want: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + "namespaceId": "c-ABC-p-XYZ", + }, + }, + { + name: "error getting project", + key: "projectId", + getFunc: func(s1, s2 string) (*v3.Project, error) { + return nil, fmt.Errorf("error") + }, + data: map[string]interface{}{ + "projectId": "cluster-ABC:project-XYZ", + }, + wantErr: true, + }, + { + name: "cluster: set namespaceId", + key: "clusterId", + data: map[string]interface{}{ + "clusterId": "cluster-ABC", + }, + want: map[string]interface{}{ + "clusterId": "cluster-ABC", + "namespaceId": "cluster-ABC", + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p.GetFunc = tt.getFunc + s := &Store{ + Store: store, + key: tt.key, + projectCache: &p, + } + got, err := s.Create(nil, nil, tt.data) + if (err != nil) != tt.wantErr { + t.Errorf("Store.Create() error = %v, wantErr %v", err, tt.wantErr) + return + } + if !reflect.DeepEqual(got, tt.want) { + t.Errorf("Store.Create() = %v, want %v", got, tt.want) + } + }) + } +}
pkg/apis/management.cattle.io/v3/authz_types.go+8 −0 modified@@ -48,6 +48,14 @@ func (p *Project) ObjClusterName() string { return p.Spec.ObjClusterName() } +// GetProjectBackingNamespace returns the namespace a project uses in the local cluster to store PRTBs and Project Scoped Secrets. +func (p *Project) GetProjectBackingNamespace() string { + if p.Status.BackingNamespace != "" { + return p.Status.BackingNamespace + } + return p.Name +} + // ProjectStatus represents the most recently observed status of the project. type ProjectStatus struct { // Conditions are a set of indicators about aspects of the project.
pkg/controllers/management/auth/crtb_handler.go+8 −7 modified@@ -6,9 +6,8 @@ import ( "strings" "time" - "github.com/rancher/rancher/pkg/controllers/status" - "github.com/rancher/rancher/pkg/controllers/management/authprovisioningv2" + "github.com/rancher/rancher/pkg/controllers/status" controllersv3 "github.com/rancher/rancher/pkg/generated/controllers/management.cattle.io/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" typesrbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" @@ -234,11 +233,12 @@ func (c *crtbLifecycle) reconcileBindings(binding *v3.ClusterRoleTemplateBinding return err } for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() if p.DeletionTimestamp != nil { - logrus.Warnf("Project %v is being deleted, not creating membership bindings", p.Name) + logrus.Warnf("Project %v is being deleted, not creating membership bindings", backingNamespace) continue } - if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, p.Name, projectManagementPlaneResources, subject, binding); err != nil { + if err := c.mgr.grantManagementClusterScopedPrivilegesInProjectNamespace(binding.RoleTemplateName, backingNamespace, projectManagementPlaneResources, subject, binding); err != nil { c.s.AddCondition(localConditions, condition, failedToGrantManagementClusterScopedPrivilegesInProjectNamespace, err) return err } @@ -255,14 +255,15 @@ func (c *crtbLifecycle) removeMGMTClusterScopedPrivilegesInProjectNamespace(bind } bindingKey := pkgrbac.GetRTBLabel(binding.ObjectMeta) for _, p := range projects { + backingNamespace := p.GetProjectBackingNamespace() set := labels.Set(map[string]string{bindingKey: CrtbInProjectBindingOwner}) - rbs, err := c.rbLister.List(p.Name, set.AsSelector()) + rbs, err := c.rbLister.List(backingNamespace, set.AsSelector()) if err != nil { return err } for _, rb := range rbs { - logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, p.Name, binding.Name) - if err := c.rbClient.DeleteNamespaced(p.Name, rb.Name, &metav1.DeleteOptions{}); err != nil { + logrus.Infof("[%v] Deleting rolebinding %v in namespace %v for crtb %v", ctrbMGMTController, rb.Name, backingNamespace, binding.Name) + if err := c.rbClient.DeleteNamespaced(backingNamespace, rb.Name, &metav1.DeleteOptions{}); err != nil { return err } }
pkg/controllers/management/auth/crtb_handler_test.go+176 −17 modified@@ -5,21 +5,23 @@ import ( "testing" "time" + apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" + v3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/rancher/pkg/controllers/status" controllersv3 "github.com/rancher/rancher/pkg/generated/controllers/management.cattle.io/v3" + "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" + rbacv1 "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1" + corefakes "github.com/rancher/rancher/pkg/generated/norman/rbac.authorization.k8s.io/v1/fakes" "github.com/rancher/wrangler/v3/pkg/generic/fake" "github.com/stretchr/testify/assert" - - v3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" - "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/fakes" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" ) var ( - e = fmt.Errorf("error") + errDefault = fmt.Errorf("error") defaultCRTB = v3.ClusterRoleTemplateBinding{ UserName: "test", GroupName: "", @@ -42,12 +44,25 @@ var ( Name: "test-project", }, } + backingNamespaceProject = v3.Project{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-project", + }, + Status: apisv3.ProjectStatus{ + BackingNamespace: "c-ABC-p-XYZ", + }, + } deletingProject = v3.Project{ ObjectMeta: v1.ObjectMeta{ Name: "deleting-project", DeletionTimestamp: &v1.Time{Time: time.Now()}, }, } + defaultBinding = rbacv1.RoleBinding{ + ObjectMeta: v1.ObjectMeta{ + Name: "test-binding", + }, + } ) type crtbTestState struct { @@ -93,7 +108,7 @@ func TestReconcileBindings(t *testing.T) { name: "error getting cluster", stateSetup: func(cts crtbTestState) { cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -103,7 +118,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToGetCluster, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -140,7 +155,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). - Return(true, e) + Return(true, errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -149,7 +164,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToCheckReferencedRole, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -168,7 +183,7 @@ func TestReconcileBindings(t *testing.T) { Return(true, nil) cts.managerMock.EXPECT(). ensureClusterMembershipBinding("clustername-clusterowner", gomock.Any(), gomock.Any(), true, gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -178,7 +193,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToEnsureClusterMembershipBinding, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -200,7 +215,7 @@ func TestReconcileBindings(t *testing.T) { Return(nil) cts.managerMock.EXPECT(). grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, crtb: defaultCRTB.DeepCopy(), @@ -209,7 +224,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToGrantManagementPlanePrivileges, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -233,7 +248,7 @@ func TestReconcileBindings(t *testing.T) { grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). Return(nil) cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { - return nil, e + return nil, errDefault } }, wantError: true, @@ -243,7 +258,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToListProjects, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -272,7 +287,7 @@ func TestReconcileBindings(t *testing.T) { } cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "test-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e) + Return(errDefault) }, wantError: true, @@ -282,7 +297,7 @@ func TestReconcileBindings(t *testing.T) { Type: bindingExists, Status: v1.ConditionFalse, Reason: failedToGrantManagementClusterScopedPrivilegesInProjectNamespace, - Message: e.Error(), + Message: errDefault.Error(), LastTransitionTime: v1.Time{ Time: mockTime, }, @@ -361,6 +376,42 @@ func TestReconcileBindings(t *testing.T) { }, }, }, + { + name: "successfully reconcile clustermember with backingNamespace", + stateSetup: func(cts crtbTestState) { + cts.managerMock.EXPECT(). + checkReferencedRoles("roleTemplate", "cluster", gomock.Any()). + Return(false, nil) + cts.managerMock.EXPECT(). + ensureClusterMembershipBinding("clustername-clustermember", gomock.Any(), gomock.Any(), false, gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementPlanePrivileges("roleTemplate", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.managerMock.EXPECT(). + grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "c-ABC-p-XYZ", gomock.Any(), gomock.Any(), gomock.Any()). + Return(nil) + cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { + c := defaultCluster.DeepCopy() + return c, nil + } + cts.projectListerMock.ListFunc = func(namespace string, selector labels.Selector) ([]*v3.Project, error) { + p := backingNamespaceProject.DeepCopy() + return []*v3.Project{p}, nil + } + }, + crtb: defaultCRTB.DeepCopy(), + wantConditions: []v1.Condition{ + { + Type: bindingExists, + Status: v1.ConditionTrue, + Reason: bindingExists, + LastTransitionTime: v1.Time{ + Time: mockTime, + }, + }, + }, + }, { name: "skip projects that are deleting", stateSetup: func(cts crtbTestState) { @@ -376,7 +427,7 @@ func TestReconcileBindings(t *testing.T) { // This should not be called cts.managerMock.EXPECT(). grantManagementClusterScopedPrivilegesInProjectNamespace("roleTemplate", "deleting-project", gomock.Any(), gomock.Any(), gomock.Any()). - Return(e).AnyTimes() + Return(errDefault).AnyTimes() cts.clusterListerMock.GetFunc = func(namespace, name string) (*v3.Cluster, error) { c := defaultCluster.DeepCopy() return c, nil @@ -649,3 +700,111 @@ func setupTest(t *testing.T) crtbTestState { } return state } + +func Test_removeMGMTClusterScopedPrivilegesInProjectNamespace(t *testing.T) { + tests := []struct { + name string + projectListFunc func(string, labels.Selector) ([]*apisv3.Project, error) + roleBindingListFunc func(string, labels.Selector) ([]*rbacv1.RoleBinding, error) + roleBindingDeleteFunc func(string, string, *v1.DeleteOptions) error + binding *v3.ClusterRoleTemplateBinding + wantErr bool + }{ + { + name: "error listing projects", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error listing rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return nil, errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "error deleting rolebindings", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + return errDefault + }, + binding: defaultCRTB.DeepCopy(), + wantErr: true, + }, + { + name: "successfully delete rolebindings no backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + defaultProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, defaultProject.Name, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, defaultProject.Name, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + { + name: "successfully delete rolebindings with backing namespace", + projectListFunc: func(s1 string, s2 labels.Selector) ([]*apisv3.Project, error) { + return []*apisv3.Project{ + backingNamespaceProject.DeepCopy(), + }, nil + }, + roleBindingListFunc: func(s1 string, s2 labels.Selector) ([]*rbacv1.RoleBinding, error) { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return []*rbacv1.RoleBinding{ + defaultBinding.DeepCopy(), + }, nil + }, + roleBindingDeleteFunc: func(s1, s2 string, do *v1.DeleteOptions) error { + assert.Equal(t, backingNamespaceProject.Status.BackingNamespace, s1) + return nil + }, + binding: defaultCRTB.DeepCopy(), + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + p := fakes.ProjectListerMock{} + p.ListFunc = tt.projectListFunc + rbl := corefakes.RoleBindingListerMock{} + rbl.ListFunc = tt.roleBindingListFunc + rbi := corefakes.RoleBindingInterfaceMock{} + rbi.DeleteNamespacedFunc = tt.roleBindingDeleteFunc + + c := &crtbLifecycle{ + projectLister: &p, + rbLister: &rbl, + rbClient: &rbi, + } + if err := c.removeMGMTClusterScopedPrivilegesInProjectNamespace(tt.binding); (err != nil) != tt.wantErr { + t.Errorf("crtbLifecycle.removeMGMTClusterScopedPrivilegesInProjectNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +}
pkg/controllers/management/auth/project_cluster/cluster_handler.go+2 −2 modified@@ -83,7 +83,7 @@ func (l *clusterLifecycle) Sync(key string, orig *apisv3.Cluster) (runtime.Objec } obj := orig.DeepCopyObject() - obj, err := reconcileResourceToNamespace(obj, ClusterCreateController, l.nsLister, l.nsClient) + obj, err := reconcileResourceToNamespace(obj, ClusterCreateController, orig.Name, l.nsLister, l.nsClient) if err != nil { return nil, err } @@ -155,7 +155,7 @@ func (l *clusterLifecycle) Remove(obj *apisv3.Cluster) (runtime.Object, error) { returnErr := errors.Join( l.deleteSystemProject(obj, ClusterRemoveController), - deleteNamespace(obj, ClusterRemoveController, l.nsClient), + deleteNamespace(ClusterRemoveController, obj.Name, l.nsClient), ) return obj, returnErr }
pkg/controllers/management/auth/project_cluster/common.go+11 −18 modified@@ -32,50 +32,43 @@ const ( var crtbCreatorOwnerAnnotations = map[string]string{creatorOwnerBindingAnnotation: "true"} -func deleteNamespace(obj runtime.Object, controller string, nsClient v1.NamespaceInterface) error { - o, err := meta.Accessor(obj) - if err != nil { - return fmt.Errorf("[%s] error accessing object %v: %w", controller, obj, err) - } - - ns, err := nsClient.Get(context.TODO(), o.GetName(), metav1.GetOptions{}) +func deleteNamespace(controller string, nsName string, nsClient v1.NamespaceInterface) error { + ns, err := nsClient.Get(context.TODO(), nsName, metav1.GetOptions{}) if apierrors.IsNotFound(err) { return nil + } else if err != nil { + return err } if ns.Status.Phase != v12.NamespaceTerminating { - logrus.Infof("[%s] Deleting namespace %s", controller, o.GetName()) - err = nsClient.Delete(context.TODO(), o.GetName(), metav1.DeleteOptions{}) + logrus.Infof("[%s] Deleting namespace %s", controller, nsName) + err = nsClient.Delete(context.TODO(), nsName, metav1.DeleteOptions{}) if apierrors.IsNotFound(err) { return nil } } return err } -func reconcileResourceToNamespace(obj runtime.Object, controller string, nsLister corev1.NamespaceCache, nsClient v1.NamespaceInterface) (runtime.Object, error) { +func reconcileResourceToNamespace(obj runtime.Object, controller string, nsName string, nsLister corev1.NamespaceCache, nsClient v1.NamespaceInterface) (runtime.Object, error) { return apisv3.NamespaceBackedResource.Do(obj, func() (runtime.Object, error) { - o, err := meta.Accessor(obj) - if err != nil { - return obj, fmt.Errorf("[%s] error accessing object %v: %w", controller, obj, err) - } t, err := meta.TypeAccessor(obj) if err != nil { return obj, err } - ns, _ := nsLister.Get(o.GetName()) + ns, _ := nsLister.Get(nsName) if ns == nil { - logrus.Infof("[%v] Creating namespace %v", controller, o.GetName()) + logrus.Infof("[%v] Creating namespace %v", controller, nsName) _, err := nsClient.Create(context.TODO(), &v12.Namespace{ ObjectMeta: metav1.ObjectMeta{ - Name: o.GetName(), + Name: nsName, Annotations: map[string]string{ "management.cattle.io/system-namespace": "true", }, }, }, metav1.CreateOptions{}) if err != nil { - return obj, fmt.Errorf("[%s] failed to create namespace for %v %v: %w", controller, t.GetKind(), o.GetName(), err) + return obj, fmt.Errorf("[%s] failed to create namespace for %s %s: %w", controller, t.GetKind(), nsName, err) } }
pkg/controllers/management/auth/project_cluster/common_test.go+142 −0 modified@@ -1,16 +1,22 @@ package project_cluster import ( + "context" + "fmt" "testing" apisv3 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" "github.com/rancher/wrangler/v3/pkg/generic/fake" "go.uber.org/mock/gomock" + v1 "k8s.io/api/core/v1" rbacv1 "k8s.io/api/rbac/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/watch" + applycorev1 "k8s.io/client-go/applyconfigurations/core/v1" ) var errNotFound = apierrors.NewNotFound(schema.GroupResource{}, "") @@ -168,3 +174,139 @@ func TestCreateMembershipRoles(t *testing.T) { }) } } + +var ( + errDefault = fmt.Errorf("error") + errNsNotFound = apierrors.NewNotFound(v1.Resource("namespace"), "") + + defaultNamespace = v1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "default-namespace", + }, + } + terminatingNamespace = v1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "terminating-namespace", + }, + Status: v1.NamespaceStatus{ + Phase: v1.NamespaceTerminating, + }, + } +) + +func Test_deleteNamespace(t *testing.T) { + tests := []struct { + name string + nsGetFunc func(context.Context, string, metav1.GetOptions) (*v1.Namespace, error) + nsDeleteFunc func(context.Context, string, metav1.DeleteOptions) error + wantErr bool + }{ + { + name: "error getting namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return nil, errDefault + }, + wantErr: true, + }, + { + name: "namespace not found", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return nil, errNsNotFound + }, + }, + { + name: "namespace is terminating", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return terminatingNamespace.DeepCopy(), nil + }, + }, + { + name: "successfully delete namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return nil + }, + }, + { + name: "deleting namespace not found", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return errNsNotFound + }, + }, + { + name: "error deleting namespace", + nsGetFunc: func(ctx context.Context, s string, g metav1.GetOptions) (*v1.Namespace, error) { + return defaultNamespace.DeepCopy(), nil + }, + nsDeleteFunc: func(ctx context.Context, s string, do metav1.DeleteOptions) error { + return errDefault + }, + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + nsClientFake := mockNamespaces{ + getter: tt.nsGetFunc, + deleter: tt.nsDeleteFunc, + } + if err := deleteNamespace("", "", nsClientFake); (err != nil) != tt.wantErr { + t.Errorf("deleteNamespace() error = %v, wantErr %v", err, tt.wantErr) + } + }) + } +} + +type mockNamespaces struct { + getter func(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Namespace, error) + deleter func(ctx context.Context, name string, opts metav1.DeleteOptions) error +} + +func (m mockNamespaces) Get(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Namespace, error) { + return m.getter(ctx, name, opts) +} + +func (m mockNamespaces) Create(ctx context.Context, namespace *v1.Namespace, opts metav1.CreateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) Update(ctx context.Context, namespace *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) UpdateStatus(ctx context.Context, namespace *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +} + +func (m mockNamespaces) Delete(ctx context.Context, name string, opts metav1.DeleteOptions) error { + return m.deleter(ctx, name, opts) +} + +func (m mockNamespaces) List(ctx context.Context, opts metav1.ListOptions) (*v1.NamespaceList, error) { + panic("implement me") +} + +func (m mockNamespaces) Watch(ctx context.Context, opts metav1.ListOptions) (watch.Interface, error) { + panic("implement me") +} + +func (m mockNamespaces) Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts metav1.PatchOptions, subresources ...string) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) Apply(ctx context.Context, namespace *applycorev1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) ApplyStatus(ctx context.Context, namespace *applycorev1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *v1.Namespace, err error) { + panic("implement me") +} + +func (m mockNamespaces) Finalize(ctx context.Context, item *v1.Namespace, opts metav1.UpdateOptions) (*v1.Namespace, error) { + panic("implement me") +}
pkg/controllers/management/auth/project_cluster/project_handler.go+18 −14 modified@@ -78,7 +78,8 @@ func (l *projectLifecycle) Sync(key string, orig *apisv3.Project) (runtime.Objec obj := orig.DeepCopyObject() - obj, err := reconcileResourceToNamespace(obj, ProjectCreateController, l.nsLister, l.nsClient) + backingNamespace := orig.GetProjectBackingNamespace() + obj, err := reconcileResourceToNamespace(obj, ProjectCreateController, backingNamespace, l.nsLister, l.nsClient) if err != nil { return nil, err } @@ -89,7 +90,7 @@ func (l *projectLifecycle) Sync(key string, orig *apisv3.Project) (runtime.Objec } } - obj, err = l.reconcileProjectCreatorRTB(obj) + obj, err = l.reconcileProjectCreatorRTB(obj, backingNamespace) if err != nil { return nil, err } @@ -140,10 +141,11 @@ func (l *projectLifecycle) Updated(obj *apisv3.Project) (runtime.Object, error) // Remove deletes all backing resources created by the project func (l *projectLifecycle) Remove(obj *apisv3.Project) (runtime.Object, error) { - return obj, deleteNamespace(obj, ProjectRemoveController, l.nsClient) + backingNamespace := obj.GetProjectBackingNamespace() + return obj, deleteNamespace(ProjectRemoveController, backingNamespace, l.nsClient) } -func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runtime.Object, error) { +func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object, nsName string) (runtime.Object, error) { project, ok := obj.(*apisv3.Project) if !ok { return obj, fmt.Errorf("expected project, got %T", obj) @@ -152,18 +154,19 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti // If we specify no creator owner RBAC, exit if _, ok := project.Annotations[NoCreatorRBACAnnotation]; ok { logrus.Infof("[%s] annotation %s found. Skipping adding creator as owner", ProjectCreateController, NoCreatorRBACAnnotation) - return obj, nil + return project, nil } - return apisv3.CreatorMadeOwner.DoUntilTrue(obj, func() (runtime.Object, error) { + + return apisv3.CreatorMadeOwner.DoUntilTrue(project, func() (runtime.Object, error) { creatorID := project.Annotations[CreatorIDAnnotation] if creatorID == "" { logrus.Warnf("[%s] project %s has no creatorId annotation. Cannot add creator as owner", ProjectCreateController, project.Name) - return obj, nil + return project, nil } if apisv3.ProjectConditionInitialRolesPopulated.IsTrue(project) { // The projectRoleBindings are already completed, no need to check - return obj, nil + return project, nil } // If the project does not have the annotation it indicates the @@ -176,13 +179,14 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti roleMap := make(map[string][]string) if err := json.Unmarshal([]byte(creatorRoleBindings), &roleMap); err != nil { - return obj, err + return project, err } var createdRoles []string for _, role := range roleMap["required"] { rtbName := "creator-" + role - if rtb, _ := l.prtbLister.Get(project.Name, rtbName); rtb != nil { + + if rtb, _ := l.prtbLister.Get(nsName, rtbName); rtb != nil { createdRoles = append(createdRoles, role) // This projectRoleBinding exists, need to check all of them so keep going continue @@ -192,7 +196,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti prtb := &apisv3.ProjectRoleTemplateBinding{ ObjectMeta: metav1.ObjectMeta{ Name: rtbName, - Namespace: project.Name, + Namespace: nsName, }, ProjectName: project.Namespace + ":" + project.Name, RoleTemplateName: role, @@ -210,7 +214,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti logrus.Infof("[%s] Creating creator projectRoleTemplateBinding for user %s for project %s", ProjectCreateController, creatorID, project.Name) _, err := l.prtbClient.Create(prtb) if err != nil && !apierrors.IsAlreadyExists(err) { - return obj, err + return project, err } createdRoles = append(createdRoles, role) @@ -221,7 +225,7 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti roleMap["created"] = createdRoles d, err := json.Marshal(roleMap) if err != nil { - return obj, err + return project, err } project.Annotations[roleTemplatesRequiredAnnotation] = string(d) @@ -233,6 +237,6 @@ func (l *projectLifecycle) reconcileProjectCreatorRTB(obj runtime.Object) (runti _, err = l.projects.Update(project) - return obj, err + return project, err }) }
pkg/controllers/management/auth/project_cluster/project_handler_test.go+3 −3 modified@@ -87,13 +87,13 @@ func TestReconcileProjectCreatorRTBRespectsUserPrincipalName(t *testing.T) { }, } - obj, err := lifecycle.reconcileProjectCreatorRTB(project) + obj, err := lifecycle.reconcileProjectCreatorRTB(project, clusterID) require.NoError(t, err) require.NotNil(t, obj) require.Len(t, prtbs, 1) assert.Equal(t, "creator-project-owner", prtbs[0].Name) - assert.Equal(t, "p-abcdef", prtbs[0].Namespace) + assert.Equal(t, clusterID, prtbs[0].Namespace) assert.Equal(t, clusterID+":p-abcdef", prtbs[0].ProjectName) assert.Equal(t, "", prtbs[0].UserName) assert.Equal(t, userPrincipalName, prtbs[0].UserPrincipalName) @@ -109,7 +109,7 @@ func TestReconcileProjectCreatorRTBNoCreatorRBAC(t *testing.T) { }, }, } - obj, err := lifecycle.reconcileProjectCreatorRTB(project) + obj, err := lifecycle.reconcileProjectCreatorRTB(project, clusterID) assert.NoError(t, err) assert.NotNil(t, obj) }
pkg/controllers/management/auth/prtb_handler.go+1 −1 modified@@ -125,7 +125,7 @@ func (p *prtbLifecycle) reconcileSubject(binding *v3.ProjectRoleTemplateBinding) return binding, nil } - return nil, fmt.Errorf("Binding %v has no subject", binding.Name) + return nil, fmt.Errorf("binding %s has no subject", binding.Name) } // When a PRTB is created or updated, translate it into several k8s roles and bindings to actually enforce the RBAC.
pkg/controllers/managementuser/rbac/namespace_handler.go+7 −4 modified@@ -15,7 +15,6 @@ import ( v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" namespaceutil "github.com/rancher/rancher/pkg/namespace" "github.com/rancher/rancher/pkg/project" - projectpkg "github.com/rancher/rancher/pkg/project" "github.com/sirupsen/logrus" v1 "k8s.io/api/core/v1" rbacv1 "k8s.io/api/rbac/v1" @@ -166,15 +165,15 @@ func (n *nsLifecycle) assignToInitialProject(ns *v1.Namespace) error { } func (n *nsLifecycle) GetSystemProjectName() (string, error) { - projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[projectpkg.System].AsSelector()) + projects, err := n.m.projectLister.List(n.m.clusterName, initialProjectToLabels[project.System].AsSelector()) if err != nil { return "", err } if len(projects) == 0 { return "", nil } if len(projects) > 1 { - return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, projectpkg.System) + return "", fmt.Errorf("cluster [%s] contains more than 1 [%s] project", n.m.clusterName, project.System) } if projects[0] == nil { return "", nil @@ -262,7 +261,11 @@ func (n *nsLifecycle) ensurePRTBAddToNamespace(ns *v1.Namespace) (bool, error) { var namespace string if parts := strings.SplitN(projectID, ":", 2); len(parts) == 2 && len(parts[1]) > 0 { - namespace = parts[1] + project, err := n.rq.ProjectLister.Get(parts[0], parts[1]) + if err != nil { + return hasPRTBs, err + } + namespace = project.GetProjectBackingNamespace() } else { return hasPRTBs, nil }
pkg/controllers/managementuser/secret/secret.go+32 −25 modified@@ -267,6 +267,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { n := &NamespaceController{ clusterSecretsClient: clusterSecretsClient, managementSecrets: cluster.Management.Core.Secrets("").Controller().Lister(), + projectCache: cluster.Management.Management.Projects("").Controller().Lister(), } cluster.Core.Namespaces("").AddHandler(ctx, "secretsController", n.sync) @@ -302,6 +303,7 @@ func registerDeferred(ctx context.Context, cluster *config.UserContext) { type NamespaceController struct { clusterSecretsClient v1.SecretInterface managementSecrets v1.SecretLister + projectCache v3.ProjectLister } func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.Object, error) { @@ -312,33 +314,38 @@ func (n *NamespaceController) sync(key string, obj *corev1.Namespace) (runtime.O // field.cattle.io/projectId value is <cluster name>:<project name> logrus.Tracef("secretsController: sync: key [%s], obj.Annotations[projectIDLabel]: [%s]", key, obj.Annotations[projectIDLabel]) if obj.Annotations[projectIDLabel] != "" { - parts := strings.Split(obj.Annotations[projectIDLabel], ":") - if len(parts) == 2 { - if parts[1] == "" { - logrus.Debugf("[NamspaceController|sync] empty project name found in obj.Annotations[projectIDLabel] for cluster: %s", parts[0]) - return nil, nil + clusterName, projectName, found := strings.Cut(obj.Annotations[projectIDLabel], ":") + if !found { + logrus.Debugf("secretsController: sync: namespace %s projectId annotation %s is malformed, should be <cluster name>:<project name>", obj.Name, obj.Annotations[projectIDLabel]) + return nil, nil + } + + p, err := n.projectCache.Get(clusterName, projectName) + if err != nil { + return nil, fmt.Errorf("secretsController: sync: error getting project %s for namespace %s: %w", projectName, obj.Name, err) + } + + backingNamespace := p.GetProjectBackingNamespace() + secrets, err := n.managementSecrets.List(backingNamespace, labels.NewSelector()) + if err != nil { + return nil, err + } + + logrus.Tracef("secretsController: sync: length of secrets for project [%s] in namespace [%s] is %d", projectName, obj.Name, len(secrets)) + for _, secret := range secrets { + // skip service account token secrets + if secret.Type == corev1.SecretTypeServiceAccountToken { + logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) + continue } - // on the management side, secret's namespace name equals to project name - secrets, err := n.managementSecrets.List(parts[1], labels.NewSelector()) - if err != nil { - return nil, err + namespacedSecret := getNamespacedSecret(secret, obj.Name) + if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { + continue } - logrus.Tracef("secretsController: sync: length of secrets for [%s] in namespace [%s] is %d", parts[1], obj.Name, len(secrets)) - for _, secret := range secrets { - // skip service account token secrets - if secret.Type == corev1.SecretTypeServiceAccountToken { - logrus.Tracef("secretsController: AddHandler: secret [%s] is Service Account token, skipping", secret.Name) - continue - } - namespacedSecret := getNamespacedSecret(secret, obj.Name) - if _, err := n.clusterSecretsClient.GetNamespaced(namespacedSecret.Namespace, namespacedSecret.Name, metav1.GetOptions{}); err == nil { - continue - } - logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) - _, err := n.clusterSecretsClient.Create(namespacedSecret) - if err != nil && !errors.IsAlreadyExists(err) { - return nil, err - } + logrus.Infof("Creating secret [%s] into namespace [%s]", namespacedSecret.Name, obj.Name) + _, err := n.clusterSecretsClient.Create(namespacedSecret) + if err != nil && !errors.IsAlreadyExists(err) { + return nil, err } } }
pkg/systemaccount/systemaccount.go+0 −35 modified@@ -6,7 +6,6 @@ import ( v32 "github.com/rancher/rancher/pkg/apis/management.cattle.io/v3" v3 "github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3" - "github.com/rancher/rancher/pkg/ref" "github.com/rancher/rancher/pkg/types/config" "github.com/rancher/rancher/pkg/types/config/systemtokens" "github.com/rancher/rancher/pkg/user" @@ -28,7 +27,6 @@ func NewManager(management *config.ManagementContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), users: management.Management.Users(""), @@ -41,7 +39,6 @@ func NewManagerFromScale(management *config.ScaledContext) *Manager { systemTokens: management.SystemTokens, crtbs: management.Management.ClusterRoleTemplateBindings(""), crts: management.Management.ClusterRegistrationTokens(""), - prtbs: management.Management.ProjectRoleTemplateBindings(""), prtbLister: management.Management.ProjectRoleTemplateBindings("").Controller().Lister(), tokens: management.Management.Tokens(""), tokenLister: management.Management.Tokens("").Controller().Lister(), @@ -54,7 +51,6 @@ type Manager struct { systemTokens systemtokens.Interface crtbs v3.ClusterRoleTemplateBindingInterface crts v3.ClusterRegistrationTokenInterface - prtbs v3.ProjectRoleTemplateBindingInterface prtbLister v3.ProjectRoleTemplateBindingLister tokens v3.TokenInterface tokenLister v3.TokenLister @@ -124,37 +120,6 @@ func (s *Manager) GetOrCreateSystemClusterToken(clusterName string) (string, err return token, nil } -func (s *Manager) GetOrCreateProjectSystemAccount(projectID string) error { - _, projectName := ref.Parse(projectID) - - u, err := s.GetProjectSystemUser(projectName) - if err != nil { - return err - } - - bindingName := u.Name + "-member" - _, err = s.prtbLister.Get(projectName, bindingName) - if err != nil { - if !errors2.IsNotFound(err) { - return err - } - // prtb does not exist in cache, attempt to create it - prtb := &v3.ProjectRoleTemplateBinding{ - ObjectMeta: v1.ObjectMeta{ - Name: bindingName, - Namespace: projectName, - }, - ProjectName: projectID, - UserName: u.Name, - RoleTemplateName: projectMemberRole, - } - if _, err := s.prtbs.Create(prtb); err != nil && !errors2.IsAlreadyExists(err) { - return err - } - } - return nil -} - func (s *Manager) GetProjectSystemUser(projectName string) (*v3.User, error) { return s.userManager.EnsureUser(fmt.Sprintf("system://%s", projectName), ProjectSystemAccountPrefix+projectName) }
tests/integration/suite/test_rbac.py+1 −0 modified@@ -679,6 +679,7 @@ def test_readonly_cannot_edit_secret(admin_mc, user_mc, admin_pc, assert e.value.error.status == 404 +@pytest.mark.skip def test_member_can_edit_secret(admin_mc, admin_pc, remove_resource, user_mc): """Tests that a user with project-member role is able to create/update
Vulnerability mechanics
Generated by null/stub on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.
References
7- github.com/advisories/GHSA-8h6m-wv39-239mghsaADVISORY
- github.com/rancher/rancher/commit/7f16b596120dd382ce6e9ed0baf83bc23f633054ghsaWEB
- github.com/rancher/rancher/commit/9c1d1c2bfcba36ae4f06c1fd043eb539ad801d4dghsaWEB
- github.com/rancher/rancher/commit/b0be28f86fc556414bd9b323f05b2b4bf8317c2dghsaWEB
- github.com/rancher/rancher/commit/f036e8b6ab726c3abbc03bbf7c8d0d53373c84e5ghsaWEB
- github.com/rancher/rancher/security/advisories/GHSA-8h6m-wv39-239mghsaWEB
- pkg.go.dev/vuln/GO-2025-3647ghsaWEB
News mentions
0No linked articles in our index yet.