Question:
[x] write e2e tests for all admission operations [x] (for resource APIs) [x] (for discovery docs) [x] switch to v1 admission APIs in e2e - [x] remove skips/time.Sleeps - and promote to conformance tests - AdmissionWebhook should have tests for all stable operations. Those tests should go through the conformance process before being released. This ticket is to help track what operations are yet to be covered by conformance tests. These stable endpoints should be hit by e2e tests tagged with [conformance]. There are tests currently hitting 6 of the corresponding beta endpoints that could be updated to use stable. Tests should be written to exercise the remaining 9 untested stable endpoints as well. These twelve tests could be updated to use the stable API, and subsequently promoted to conformance. / ! ! These will need tests written for them.  untested beta/admissionregistration !
/sig api-machinery
/assign
/area conformance
/assign ^ This is a requirement for graduation to GA.
can we get another apisnoop run? We've coverage for admissionregistration and apiextensions and I'd like to get confirmation from apisnoop that we've got full coverage now.
Keeping this open until tests are actually promoted to conformance
is ready for review. It adds our admission e2e tests to conformance as part of graduation to GA.
is updated currently not hit by any tests, one will need to be updated or written to cover it. Might want to ensure you've promoted one test that hit's each endpoint. !
The PR to do that is at

slowWebhookCleanup() }) <ins> ginkgo.It("patching/updating a validating webhook should work", func() { client := f.ClientSet admissionClient := client.AdmissionregistrationV1() ginkgo.By("Creating a validating webhook configuration") hook, err := createValidatingWebhookConfiguration(f, &admissionregistrationv1.ValidatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{ Name: f.UniqueName, }, Webhooks: []admissionregistrationv1.ValidatingWebhook{ newDenyConfigMapWebhookFixture(f, context), }, }) framework.ExpectNoError(err, "Creating validating webhook configuration") defer func() { err := client.AdmissionregistrationV1().ValidatingWebhookConfigurations().Delete(hook.Name, nil) framework.ExpectNoError(err, "Deleting validating webhook configuration") }() ginkgo.By("Creating a configMap that does not comply to the validation webhook rules") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedNonCompliantConfigMap(string(uuid.NewUUID()), f) _, err = client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err == nil { err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") return false, nil } if !strings.Contains(err.Error(), "denied") { return false, err } return true, nil }) ginkgo.By("Updating a validating webhook configuration's rules to not include the create operation") err = retry.RetryOnConflict(retry.DefaultRetry, func() error { h, err := admissionClient.ValidatingWebhookConfigurations().Get(f.UniqueName, metav1.GetOptions{}) framework.ExpectNoError(err, "Getting validating webhook configuration") h.Webhooks[0].Rules[0].Operations = []admissionregistrationv1.OperationType{admissionregistrationv1.Update} _, err = admissionClient.ValidatingWebhookConfigurations().Update(h) return err }) framework.ExpectNoError(err, "Updating validating webhook configuration") ginkgo.By("Creating a configMap that does not comply to the validation webhook rules") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedNonCompliantConfigMap(string(uuid.NewUUID()), f) _, err = client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err != nil { if !strings.Contains(err.Error(), "denied") { return false, err } return false, nil } err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") return true, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s to be allowed creation since webhook was updated to not validate create", f.Namespace.Name) ginkgo.By("Patching a validating webhook configuration's rules to include the create operation") hook, err = admissionClient.ValidatingWebhookConfigurations().Patch( f.UniqueName, types.JSONPatchType, []byte(`[{"op": "replace", "path": "/webhooks/0/rules/0/operations", "value": ["CREATE"]}]`)) framework.ExpectNoError(err, "Patching validating webhook configuration") ginkgo.By("Creating a configMap that does not comply to the validation webhook rules") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedNonCompliantConfigMap(string(uuid.NewUUID()), f) _, err = client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err == nil { err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") return false, nil } if !strings.Contains(err.Error(), "denied") { return false, err } return true, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s to be denied creation by validating webhook", f.Namespace.Name) }) ginkgo.It("patching/updating a mutating webhook should work", func() { client := f.ClientSet admissionClient := client.AdmissionregistrationV1() ginkgo.By("Creating a mutating webhook configuration") hook, err := createMutatingWebhookConfiguration(f, &admissionregistrationv1.MutatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{ Name: f.UniqueName, }, Webhooks: []admissionregistrationv1.MutatingWebhook{ newMutateConfigMapWebhookFixture(f, context, 1), }, }) framework.ExpectNoError(err, "Creating mutating webhook configuration") defer func() { err := client.AdmissionregistrationV1().MutatingWebhookConfigurations().Delete(hook.Name, nil) framework.ExpectNoError(err, "Deleting mutating webhook configuration") }() hook, err = admissionClient.MutatingWebhookConfigurations().Get(f.UniqueName, metav1.GetOptions{}) framework.ExpectNoError(err, "Getting mutating webhook configuration") ginkgo.By("Updating a mutating webhook configuration's rules to not include the create operation") hook.Webhooks[0].Rules[0].Operations = []admissionregistrationv1.OperationType{admissionregistrationv1.Update} hook, err = admissionClient.MutatingWebhookConfigurations().Update(hook) framework.ExpectNoError(err, "Updating mutating webhook configuration") ginkgo.By("Creating a configMap that should not be mutated") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedToBeMutatedConfigMap(string(uuid.NewUUID()), f) created, err := client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err != nil { return false, err } err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") _, ok := created.Data["mutation-stage-1"] return !ok, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s this is not mutated", f.Namespace.Name) ginkgo.By("Patching a mutating webhook configuration's rules to include the create operation") hook, err = admissionClient.MutatingWebhookConfigurations().Patch( f.UniqueName, types.JSONPatchType, []byte(`[{"op": "replace", "path": "/webhooks/0/rules/0/operations", "value": ["CREATE"]}]`)) framework.ExpectNoError(err, "Patching mutating webhook configuration") ginkgo.By("Creating a configMap that should be mutated") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedToBeMutatedConfigMap(string(uuid.NewUUID()), f) created, err := client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err != nil { return false, err } err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") _, ok := created.Data["mutation-stage-1"] return ok, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s to be mutated", f.Namespace.Name) }) ginkgo.It("listing validating webhooks should work", func() { testListSize := 10 testUUID := string(uuid.NewUUID()) for i := 0; i < testListSize; i++ { name := fmt.Sprintf("%s-%d", f.UniqueName, i) _, err := createValidatingWebhookConfiguration(f, &admissionregistrationv1.ValidatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{ Name:   name, Labels: map[string]string{"e2e-list-test-uuid": testUUID}, }, Webhooks: []admissionregistrationv1.ValidatingWebhook{ newDenyConfigMapWebhookFixture(f, context), }, }) framework.ExpectNoError(err, "Creating validating webhook configuration") } selectorListOpts := metav1.ListOptions{LabelSelector: "e2e-list-test-uuid=" + testUUID} ginkgo.By("Listing all of the created validation webhooks") list, err := client.AdmissionregistrationV1beta1().ValidatingWebhookConfigurations().List(selectorListOpts) framework.ExpectNoError(err, "Listing validating webhook configurations") framework.ExpectEqual(len(list.Items), testListSize) ginkgo.By("Creating a configMap that does not comply to the validation webhook rules") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedNonCompliantConfigMap(string(uuid.NewUUID()), f) _, err = client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err == nil { err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") return false, nil } if !strings.Contains(err.Error(), "denied") { return false, err } return true, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s to be denied creation by validating webhook", f.Namespace.Name) ginkgo.By("Deleting the collection of validation webhooks") err = client.AdmissionregistrationV1beta1().ValidatingWebhookConfigurations().DeleteCollection(nil, selectorListOpts) framework.ExpectNoError(err, "Deleting collection of validating webhook configurations") ginkgo.By("Creating a configMap that does not comply to the validation webhook rules") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedNonCompliantConfigMap(string(uuid.NewUUID()), f) _, err = client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err != nil { if !strings.Contains(err.Error(), "denied") { return false, err } return false, nil } err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") return true, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s to be allowed creation since there are no webhooks", f.Namespace.Name) }) ginkgo.It("listing mutating webhooks should work", func() { testListSize := 10 testUUID := string(uuid.NewUUID()) for i := 0; i < testListSize; i++ { name := fmt.Sprintf("%s-%d", f.UniqueName, i) _, err := createMutatingWebhookConfiguration(f, &admissionregistrationv1.MutatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{ Name:   name, Labels: map[string]string{"e2e-list-test-uuid": testUUID}, }, Webhooks: []admissionregistrationv1.MutatingWebhook{ newMutateConfigMapWebhookFixture(f, context, 1), }, }) framework.ExpectNoError(err, "Creating mutating webhook configuration") } selectorListOpts := metav1.ListOptions{LabelSelector: "e2e-list-test-uuid=" + testUUID} ginkgo.By("Listing all of the created validation webhooks") list, err := client.AdmissionregistrationV1beta1().MutatingWebhookConfigurations().List(selectorListOpts) framework.ExpectNoError(err, "Listing mutating webhook configurations") framework.ExpectEqual(len(list.Items), testListSize) ginkgo.By("Creating a configMap that should be mutated") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedToBeMutatedConfigMap(string(uuid.NewUUID()), f) created, err := client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err != nil { return false, err } err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") _, ok := created.Data["mutation-stage-1"] return ok, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s to be mutated", f.Namespace.Name) ginkgo.By("Deleting the collection of validation webhooks") err = client.AdmissionregistrationV1beta1().MutatingWebhookConfigurations().DeleteCollection(nil, selectorListOpts) framework.ExpectNoError(err, "Deleting collection of mutating webhook configurations") ginkgo.By("Creating a configMap that should not be mutated") err = wait.PollImmediate(100*time.Millisecond, 30*time.Second, func() (bool, error) { cm := namedToBeMutatedConfigMap(string(uuid.NewUUID()), f) created, err := client.CoreV1().ConfigMaps(f.Namespace.Name).Create(cm) if err != nil { return false, err } err = client.CoreV1().ConfigMaps(f.Namespace.Name).Delete(cm.Name, nil) framework.ExpectNoError(err, "Deleting successfully created configMap") _, ok := created.Data["mutation-stage-1"] return !ok, nil }) framework.ExpectNoError(err, "Waiting for configMap in namespace %s this is not mutated", f.Namespace.Name) }) </ins> // TODO: add more e2e tests for mutating webhooks // 1. mutating webhook that mutates pod // 2. mutating webhook that sends empty patch


Question:
[x] write e2e tests for all admission operations [x] (for resource APIs) [x] (for discovery docs) [x] switch to v1 admission APIs in e2e - [x] remove skips/time.Sleeps - and promote to conformance tests - AdmissionWebhook should have tests for all stable operations. Those tests should go through the conformance process before being released. This ticket is to help track what operations are yet to be covered by conformance tests. These stable endpoints should be hit by e2e tests tagged with [conformance]. There are tests currently hitting 6 of the corresponding beta endpoints that could be updated to use stable. Tests should be written to exercise the remaining 9 untested stable endpoints as well. These twelve tests could be updated to use the stable API, and subsequently promoted to conformance. / ! ! These will need tests written for them.  untested beta/admissionregistration !
/sig api-machinery
/assign
/area conformance
/assign ^ This is a requirement for graduation to GA.
can we get another apisnoop run? We've coverage for admissionregistration and apiextensions and I'd like to get confirmation from apisnoop that we've got full coverage now.
Keeping this open until tests are actually promoted to conformance
is ready for review. It adds our admission e2e tests to conformance as part of graduation to GA.
is updated currently not hit by any tests, one will need to be updated or written to cover it. Might want to ensure you've promoted one test that hit's each endpoint. !
The PR to do that is at

MatchLabels: map[string]string{f.UniqueName: "true"}, } <del> sideEffectsNone := admissionregistrationv1.SideEffectClassNone </del> _, err := createValidatingWebhookConfiguration(f, &admissionregistrationv1.ValidatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{ Name: configName, }, Webhooks: []admissionregistrationv1.ValidatingWebhook{ <del> { Name: "deny-unwanted-pod-container-name-and-label.k8s.io", Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"pods"}, }, }}, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: namespace, Name:      serviceName, Path:      strPtr("/pods"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, // Scope the webhook to just this namespace NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, }, }, { Name: "deny-unwanted-configmap-data.k8s.io", Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create, admissionregistrationv1.Update, admissionregistrationv1.Delete}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"configmaps"}, }, }}, // The webhook skips the namespace that has label "skip-webhook-admission":"yes" NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, MatchExpressions: []metav1.LabelSelectorRequirement{ { Key:      skipNamespaceLabelKey, Operator: metav1.LabelSelectorOpNotIn, Values:   []string{skipNamespaceLabelValue}, }, }, }, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: namespace, Name:      serviceName, Path:      strPtr("/configmaps"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, }, </del> <ins> newDenyPodWebhookFixture(f, context), newDenyConfigMapWebhookFixture(f, context), </ins> // Server cannot talk to this webhook, so it always fails. // Because this webhook is configured fail-open, request should be admitted after the call fails. failOpenHook,


Question:
[x] write e2e tests for all admission operations [x] (for resource APIs) [x] (for discovery docs) [x] switch to v1 admission APIs in e2e - [x] remove skips/time.Sleeps - and promote to conformance tests - AdmissionWebhook should have tests for all stable operations. Those tests should go through the conformance process before being released. This ticket is to help track what operations are yet to be covered by conformance tests. These stable endpoints should be hit by e2e tests tagged with [conformance]. There are tests currently hitting 6 of the corresponding beta endpoints that could be updated to use stable. Tests should be written to exercise the remaining 9 untested stable endpoints as well. These twelve tests could be updated to use the stable API, and subsequently promoted to conformance. / ! ! These will need tests written for them.  untested beta/admissionregistration !
/sig api-machinery
/assign
/area conformance
/assign ^ This is a requirement for graduation to GA.
can we get another apisnoop run? We've coverage for admissionregistration and apiextensions and I'd like to get confirmation from apisnoop that we've got full coverage now.
Keeping this open until tests are actually promoted to conformance
is ready for review. It adds our admission e2e tests to conformance as part of graduation to GA.
is updated currently not hit by any tests, one will need to be updated or written to cover it. Might want to ensure you've promoted one test that hit's each endpoint. !
The PR to do that is at

ginkgo.By("Registering the mutating configmap webhook via the AdmissionRegistration API") namespace := f.Namespace.Name <del> sideEffectsNone := admissionregistrationv1.SideEffectClassNone </del> _, err := createMutatingWebhookConfiguration(f, &admissionregistrationv1.MutatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{ Name: configName, }, Webhooks: []admissionregistrationv1.MutatingWebhook{ <del> { Name: "adding-configmap-data-stage-1.k8s.io", Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"configmaps"}, }, }}, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: namespace, Name:      serviceName, Path:      strPtr("/mutating-configmaps"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, // Scope the webhook to just this namespace NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, }, }, { Name: "adding-configmap-data-stage-2.k8s.io", Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"configmaps"}, }, }}, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: namespace, Name:      serviceName, Path:      strPtr("/mutating-configmaps"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, // Scope the webhook to just this namespace NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, }, }, </del> <ins> newMutateConfigMapWebhookFixture(f, context, 1), newMutateConfigMapWebhookFixture(f, context, 2), </ins> }, }) framework.ExpectNoError(err, "registering mutating webhook config %s with namespace %s", configName, namespace)


Question:
[x] write e2e tests for all admission operations [x] (for resource APIs) [x] (for discovery docs) [x] switch to v1 admission APIs in e2e - [x] remove skips/time.Sleeps - and promote to conformance tests - AdmissionWebhook should have tests for all stable operations. Those tests should go through the conformance process before being released. This ticket is to help track what operations are yet to be covered by conformance tests. These stable endpoints should be hit by e2e tests tagged with [conformance]. There are tests currently hitting 6 of the corresponding beta endpoints that could be updated to use stable. Tests should be written to exercise the remaining 9 untested stable endpoints as well. These twelve tests could be updated to use the stable API, and subsequently promoted to conformance. / ! ! These will need tests written for them.  untested beta/admissionregistration !
/sig api-machinery
/assign
/area conformance
/assign ^ This is a requirement for graduation to GA.
can we get another apisnoop run? We've coverage for admissionregistration and apiextensions and I'd like to get confirmation from apisnoop that we've got full coverage now.
Keeping this open until tests are actually promoted to conformance
is ready for review. It adds our admission e2e tests to conformance as part of graduation to GA.
is updated currently not hit by any tests, one will need to be updated or written to cover it. Might want to ensure you've promoted one test that hit's each endpoint. !
The PR to do that is at

} func nonCompliantConfigMap(f *framework.Framework) *v1.ConfigMap { <ins> return namedNonCompliantConfigMap(disallowedConfigMapName, f) } func namedNonCompliantConfigMap(name string, f *framework.Framework) *v1.ConfigMap { </ins> return &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ <del> Name: disallowedConfigMapName, </del> <ins> Name: name, </ins> }, Data: map[string]string{ "webhook-e2e-test": "webhook-disallow",


Question:
[x] write e2e tests for all admission operations [x] (for resource APIs) [x] (for discovery docs) [x] switch to v1 admission APIs in e2e - [x] remove skips/time.Sleeps - and promote to conformance tests - AdmissionWebhook should have tests for all stable operations. Those tests should go through the conformance process before being released. This ticket is to help track what operations are yet to be covered by conformance tests. These stable endpoints should be hit by e2e tests tagged with [conformance]. There are tests currently hitting 6 of the corresponding beta endpoints that could be updated to use stable. Tests should be written to exercise the remaining 9 untested stable endpoints as well. These twelve tests could be updated to use the stable API, and subsequently promoted to conformance. / ! ! These will need tests written for them.  untested beta/admissionregistration !
/sig api-machinery
/assign
/area conformance
/assign ^ This is a requirement for graduation to GA.
can we get another apisnoop run? We've coverage for admissionregistration and apiextensions and I'd like to get confirmation from apisnoop that we've got full coverage now.
Keeping this open until tests are actually promoted to conformance
is ready for review. It adds our admission e2e tests to conformance as part of graduation to GA.
is updated currently not hit by any tests, one will need to be updated or written to cover it. Might want to ensure you've promoted one test that hit's each endpoint. !
The PR to do that is at

} func toBeMutatedConfigMap(f *framework.Framework) *v1.ConfigMap { <ins> return namedToBeMutatedConfigMap("to-be-mutated", f) } func namedToBeMutatedConfigMap(name string, f *framework.Framework) *v1.ConfigMap { </ins> return &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ <del> Name: "to-be-mutated", </del> <ins> Name: name, </ins> }, Data: map[string]string{ "mutation-start": "yes",


Question:
[x] write e2e tests for all admission operations [x] (for resource APIs) [x] (for discovery docs) [x] switch to v1 admission APIs in e2e - [x] remove skips/time.Sleeps - and promote to conformance tests - AdmissionWebhook should have tests for all stable operations. Those tests should go through the conformance process before being released. This ticket is to help track what operations are yet to be covered by conformance tests. These stable endpoints should be hit by e2e tests tagged with [conformance]. There are tests currently hitting 6 of the corresponding beta endpoints that could be updated to use stable. Tests should be written to exercise the remaining 9 untested stable endpoints as well. These twelve tests could be updated to use the stable API, and subsequently promoted to conformance. / ! ! These will need tests written for them.  untested beta/admissionregistration !
/sig api-machinery
/assign
/area conformance
/assign ^ This is a requirement for graduation to GA.
can we get another apisnoop run? We've coverage for admissionregistration and apiextensions and I'd like to get confirmation from apisnoop that we've got full coverage now.
Keeping this open until tests are actually promoted to conformance
is ready for review. It adds our admission e2e tests to conformance as part of graduation to GA.
is updated currently not hit by any tests, one will need to be updated or written to cover it. Might want to ensure you've promoted one test that hit's each endpoint. !
The PR to do that is at

} return f.ClientSet.AdmissionregistrationV1().MutatingWebhookConfigurations().Create(config) } <ins> func newDenyPodWebhookFixture(f *framework.Framework, context *certContext) admissionregistrationv1.ValidatingWebhook { sideEffectsNone := admissionregistrationv1.SideEffectClassNone return admissionregistrationv1.ValidatingWebhook{ Name: "deny-unwanted-pod-container-name-and-label.k8s.io", Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"pods"}, }, }}, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: f.Namespace.Name, Name:      serviceName, Path:      strPtr("/pods"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, // Scope the webhook to just this namespace NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, }, } } func newDenyConfigMapWebhookFixture(f *framework.Framework, context *certContext) admissionregistrationv1.ValidatingWebhook { sideEffectsNone := admissionregistrationv1.SideEffectClassNone return admissionregistrationv1.ValidatingWebhook{ Name: "deny-unwanted-configmap-data.k8s.io", Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create, admissionregistrationv1.Update, admissionregistrationv1.Delete}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"configmaps"}, }, }}, // The webhook skips the namespace that has label "skip-webhook-admission":"yes" NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, MatchExpressions: []metav1.LabelSelectorRequirement{ { Key:      skipNamespaceLabelKey, Operator: metav1.LabelSelectorOpNotIn, Values:   []string{skipNamespaceLabelValue}, }, }, }, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: f.Namespace.Name, Name:      serviceName, Path:      strPtr("/configmaps"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, } } func newMutateConfigMapWebhookFixture(f *framework.Framework, context *certContext, stage int) admissionregistrationv1.MutatingWebhook { sideEffectsNone := admissionregistrationv1.SideEffectClassNone return admissionregistrationv1.MutatingWebhook{ Name: fmt.Sprintf("adding-configmap-data-stage-%d.k8s.io", stage), Rules: []admissionregistrationv1.RuleWithOperations{{ Operations: []admissionregistrationv1.OperationType{admissionregistrationv1.Create}, Rule: admissionregistrationv1.Rule{ APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"configmaps"}, }, }}, ClientConfig: admissionregistrationv1.WebhookClientConfig{ Service: &admissionregistrationv1.ServiceReference{ Namespace: f.Namespace.Name, Name:      serviceName, Path:      strPtr("/mutating-configmaps"), Port:      pointer.Int32Ptr(servicePort), }, CABundle: context.signingCert, }, SideEffects:             &sideEffectsNone, AdmissionReviewVersions: []string{"v1beta1"}, // Scope the webhook to just this namespace NamespaceSelector: &metav1.LabelSelector{ MatchLabels: map[string]string{f.UniqueName: "true"}, }, } } </ins>


Question:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: With kubectl v1.16.0-alpha.2, completion with override flags fails. then hit TAB will produce error "resource type must be specified" What you expected to happen: It must show the namespaces without error. How to reproduce it (as minimally and precisely as possible): With kubectl v1.16.0-alpha.2, then hit TAB key. Anything else we need to know?: It looks like introduced this regression. Environment: Kubernetes version (use ): kubectl v1.16.0-alpha.2 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig cli
I also reproduced this bug with the following steps:
I'm fixing it.

done for var in "${__kubectl_override_flag_list[@]##*-}"; do if eval "test -n "$${var}""; then <del> eval "echo ${${var}}" </del> <ins> eval "echo -n ${${var}}' '" </ins> fi done }


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We need to enhance Azure cloud provider code to support both AAD and ADFS authentication. Why is this needed: On Azure Stack customer can have two type of deployment .
/sig azure
/assign

"crypto/x509" "fmt" "io/ioutil" <ins> "strings" </ins> "github.com/Azure/go-autorest/autorest/adal" "github.com/Azure/go-autorest/autorest/azure"


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We need to enhance Azure cloud provider code to support both AAD and ADFS authentication. Why is this needed: On Azure Stack customer can have two type of deployment .
/sig azure
/assign

var ( // ErrorNoAuth indicates that no credentials are provided. ErrorNoAuth = fmt.Errorf("no credentials provided for Azure cloud provider") <ins> // ADFSIdentitySystem indicates value of tenantId for ADFS on Azure Stack. ADFSIdentitySystem = "ADFS" </ins> ) // AzureAuthConfig holds auth related part of cloud config


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We need to enhance Azure cloud provider code to support both AAD and ADFS authentication. Why is this needed: On Azure Stack customer can have two type of deployment .
/sig azure
/assign

UserAssignedIdentityID string `json:"userAssignedIdentityID,omitempty" yaml:"userAssignedIdentityID,omitempty"` // The ID of the Azure Subscription that the cluster is deployed in SubscriptionID string `json:"subscriptionId,omitempty" yaml:"subscriptionId,omitempty"` <ins> // Identity system value for the deployment. This gets populate for Azure Stack case. IdentitySystem string `json:"identitySystem,omitempty" yaml:"identitySystem,omitempty"` </ins> } // GetServicePrincipalToken creates a new service principal token based on the configuration func GetServicePrincipalToken(config *AzureAuthConfig, env *azure.Environment) (*adal.ServicePrincipalToken, error) { <ins> var tenantID string if strings.EqualFold(config.IdentitySystem, ADFSIdentitySystem) { tenantID = "adfs" } else { tenantID = config.TenantID } </ins> if config.UseManagedIdentityExtension { klog.V(2).Infoln("azure: using managed identity extension to retrieve access token") msiEndpoint, err := adal.GetMSIVMEndpoint()


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We need to enhance Azure cloud provider code to support both AAD and ADFS authentication. Why is this needed: On Azure Stack customer can have two type of deployment .
/sig azure
/assign

env.ServiceManagementEndpoint) } <del> oauthConfig, err := adal.NewOAuthConfig(env.ActiveDirectoryEndpoint, config.TenantID) </del> <ins> oauthConfig, err := adal.NewOAuthConfig(env.ActiveDirectoryEndpoint, tenantID) </ins> if err != nil { return nil, fmt.Errorf("creating the OAuth config: %v", err) }


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

"//staging/src/k8s.io/client-go/tools/watch:go_default_library", "//test/e2e/framework:go_default_library", "//test/e2e/framework/events:go_default_library", <del> "//test/e2e/framework/kubelet:go_default_library", </del> "//test/e2e/framework/network:go_default_library", "//test/e2e/framework/node:go_default_library", "//test/e2e/framework/pod:go_default_library",


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

podutil "k8s.io/kubernetes/pkg/api/v1/pod" "k8s.io/kubernetes/pkg/kubelet" "k8s.io/kubernetes/test/e2e/framework" <del> e2ekubelet "k8s.io/kubernetes/test/e2e/framework/kubelet" </del> e2epod "k8s.io/kubernetes/test/e2e/framework/pod" e2ewebsocket "k8s.io/kubernetes/test/e2e/framework/websocket" imageutils "k8s.io/kubernetes/test/utils/image"


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

err = podClient.Delete(context.TODO(), pod.Name, *metav1.NewDeleteOptions(30)) framework.ExpectNoError(err, "failed to delete pod") <del> ginkgo.By("verifying the kubelet observed the termination notice") err = wait.Poll(time.Second*5, time.Second*30, func() (bool, error) { podList, err := e2ekubelet.GetKubeletPods(f.ClientSet, pod.Spec.NodeName) if err != nil { framework.Logf("Unable to retrieve kubelet pods for node %v: %v", pod.Spec.NodeName, err) return false, nil } for _, kubeletPod := range podList.Items { if pod.Name != kubeletPod.Name { continue } if kubeletPod.ObjectMeta.DeletionTimestamp == nil { framework.Logf("deletion has not yet been observed") return false, nil } return true, nil } framework.Logf("no pod exists with the name we were looking for, assuming the termination request was observed and completed") return true, nil }) framework.ExpectNoError(err, "kubelet never observed the termination notice") </del> ginkgo.By("verifying pod deletion was observed") deleted := false var lastPod *v1.Pod


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

"//test/e2e/framework:go_default_library", "//test/e2e/framework/gpu:go_default_library", "//test/e2e/framework/job:go_default_library", <del> "//test/e2e/framework/kubelet:go_default_library", </del> "//test/e2e/framework/node:go_default_library", "//test/e2e/framework/pod:go_default_library", "//test/e2e/framework/providers/gce:go_default_library",


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

clientset "k8s.io/client-go/kubernetes" podutil "k8s.io/kubernetes/pkg/api/v1/pod" "k8s.io/kubernetes/test/e2e/framework" <del> e2ekubelet "k8s.io/kubernetes/test/e2e/framework/kubelet" </del> e2enode "k8s.io/kubernetes/test/e2e/framework/node" e2epod "k8s.io/kubernetes/test/e2e/framework/pod" e2erc "k8s.io/kubernetes/test/e2e/framework/rc"


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

framework.ExpectNoError(err) for _, node := range nodeList.Items { <del> framework.Logf("nLogging pods the kubelet thinks is on node %v before test", node.Name) printAllKubeletPods(cs, node.Name) </del> <ins> framework.Logf("nLogging pods the apiserver thinks is on node %v before test", node.Name) printAllPodsOnNode(cs, node.Name) </ins> } })


Question:
What happened: In reviewing a test for promotion to conformance I noticed the kubelet API was being accessed via the API server's node proxy endpoint (ref: ). This seemed off, so I reviewed . It doesn't explicitly call out kubelet API access as allowed or disallowed. . and agreed kubelet API access seems like a bad idea. I'm now surveying what we have that does_ assume kubelet API access. It's definitely used by parts of e2e framework and some Conformance tests. I'm trying to verify how critical its use is. It appears that much of it is related to debugging or verifying whether kubelets and the control plane agree on who is running what. What you expected to happen: Find a blanket policy that bans kubelet API access for Conformance tests, whether directly, or indirectly through the API server's node proxy endpoint, at any time. Alternatively, find a policy that says kubelet API access may be allowed, but only for debugging or failure purposes (must not be on the critical path to make the test pass) Alternatively, find a policy that says kubelet API access is allowed, but only indirectly through the API server, and evaluated against the same constraints we use for control plane access (GA, non-optional) If we want to say kubelet API access is banned, we can use this as a tracking issue to demote whichever tests rely on it, and come up with a plan to verify the same behavior without kubelet API access. /area conformance /sig architecture /priority important-soon /milestone v1.16 /assign
Look for anything that ends up calling calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test requires kubelet API access to pass may end up calling it if the test fails, via: Some Conformance tests may end up calling this if the test fails specifically Other areas this is used: only if the test fails required for a pre-stop hook test to pass required to make sure the readonly kubelet apis aren't exposed via api server proxy resource usage tracking tests require kubelet api access to pass tests call this in
Looking for doesn't turn up any additional conformance tests, and I don't feel compelled to link the two other tests that use it as they are unlikely to ever be eligible.
The examples pointed out by on slack, the kubelet's endpoints is required to be available for these to pass
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
are there further changes for this targeted at 1.16?
Any update on this?
/milestone v1.17
Bug Triage Release Team checking in. Code freeze for the 1.17 release is on November 14. Will this make it in? Thanks!
/assign
hi is this still targetting for 1.17 or can we move to 1.18? Code freeze is Nov 14th. thanks!
/milestone v1.18
So we've updated the documented policy. My suggestion to close this out is to drop the [Conformance] tag from the tests that currently require the kubelet API to pass, and open issues that they need to be rewritten. Ignore whether the API is needed for diagnostic info in failure modes. SchedulerPredicates [Serial] calls this in BeforeEach, which will happen before some Conformance tests (if this fails, the tests will fail) Conformance test Pods should be submitted and removed [NodeConformance] requires kubelet API access to pass
i'd support this ^^
Hello , Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March. Is this issue still intended for milestone 1.18?
Hello There's been no movement on this issue this year. Should this get pushed to the next milestone?
yes, please John T Skarbek <:
/milestone v1.19
/cc

}) }) <del> // printAllKubeletPods outputs status of all kubelet pods into log. func printAllKubeletPods(c clientset.Interface, nodeName string) { podList, err := e2ekubelet.GetKubeletPods(c, nodeName) </del> <ins> // printAllPodsOnNode outputs status of all kubelet pods into log. func printAllPodsOnNode(c clientset.Interface, nodeName string) { podList, err := c.CoreV1().Pods(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{FieldSelector: "spec.nodeName=" + nodeName}) </ins> if err != nil { <del> framework.Logf("Unable to retrieve kubelet pods for node %v: %v", nodeName, err) </del> <ins> framework.Logf("Unable to retrieve pods for node %v: %v", nodeName, err) </ins> return } for _, p := range podList.Items {


Question:
This issue was reported in the Description The container manager used in kubelet checks for docker daemon process either via pidfile or process name. While the pidfile points to the docker daemon process PID, the dockerProcessName constant stores a docker cli name (docker) instead of docker daemon name (dockerd). Figure 30.1: Constants in file. Recommendation Correct the docker process name to dockerd. Anything else we need to know?: See for current status of all issues created from these findings. The vendor gave this issue an ID of TOB-K8S-025 and it was finding 31 of the report. The vendor considers this issue Informational Severity. To view the original finding, begin on page 77 of the Environment: Kubernetes version: 1.13.4
/sig node /kind bug /remove-kind feature
/lgtm
lol, this is the issue. my bad!

) const ( <del> dockerProcessName     = "docker" </del> <ins> dockerProcessName     = "dockerd" </ins> dockerPidFile         = "/var/run/docker.pid" containerdProcessName = "docker-containerd" containerdPidFile     = "/run/docker/libcontainerd/docker-containerd.pid"


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : The scale client () currently works fine with namespaced resources. However, it doesn't work for non-namespaced resources e.g. cluster-scoped CRDs. The current implementation should be extended in order to support non-namespaced resources. The current interface: /area custom-resources /sig autoscaling
I think scale subresource support for CRDs falls more into sig-apimachinery domain. After DirectXMan12 moved on from sig-autoscaling I don't think there is anyone in our sig-autoscaling with knowledge of scale subresource internals. /sig api-machinery
/assign
The bug report sounds legit. Does the generic scale client need to be changed (I think so), or does solve your issue?
No, the dashboard PR doesn't solve this. I discovered this issue while working on that PR so when gets merged, it will only work for namespaced CRDs. I think the generic scale client needs to be updated, maybe by adding another getter to the ScalesGetter interface: /kind bug
PR:

result := c.client.clientBase.Get(). AbsPath(path). <del> Namespace(c.namespace). </del> <ins> NamespaceIfScoped(c.namespace, c.namespace != ""). </ins> Resource(gvr.Resource). Name(name). SubResource("scale").


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : The scale client () currently works fine with namespaced resources. However, it doesn't work for non-namespaced resources e.g. cluster-scoped CRDs. The current implementation should be extended in order to support non-namespaced resources. The current interface: /area custom-resources /sig autoscaling
I think scale subresource support for CRDs falls more into sig-apimachinery domain. After DirectXMan12 moved on from sig-autoscaling I don't think there is anyone in our sig-autoscaling with knowledge of scale subresource internals. /sig api-machinery
/assign
The bug report sounds legit. Does the generic scale client need to be changed (I think so), or does solve your issue?
No, the dashboard PR doesn't solve this. I discovered this issue while working on that PR so when gets merged, it will only work for namespaced CRDs. I think the generic scale client needs to be updated, maybe by adding another getter to the ScalesGetter interface: /kind bug
PR:

result := c.client.clientBase.Put(). AbsPath(path). <del> Namespace(c.namespace). </del> <ins> NamespaceIfScoped(c.namespace, c.namespace != ""). </ins> Resource(gvr.Resource). Name(scale.Name). SubResource("scale").


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : The scale client () currently works fine with namespaced resources. However, it doesn't work for non-namespaced resources e.g. cluster-scoped CRDs. The current implementation should be extended in order to support non-namespaced resources. The current interface: /area custom-resources /sig autoscaling
I think scale subresource support for CRDs falls more into sig-apimachinery domain. After DirectXMan12 moved on from sig-autoscaling I don't think there is anyone in our sig-autoscaling with knowledge of scale subresource internals. /sig api-machinery
/assign
The bug report sounds legit. Does the generic scale client need to be changed (I think so), or does solve your issue?
No, the dashboard PR doesn't solve this. I discovered this issue while working on that PR so when gets merged, it will only work for namespaced CRDs. I think the generic scale client needs to be updated, maybe by adding another getter to the ScalesGetter interface: /kind bug
PR:

groupVersion := gvr.GroupVersion() result := c.client.clientBase.Patch(pt). AbsPath(c.client.apiPathFor(groupVersion)). <del> Namespace(c.namespace). </del> <ins> NamespaceIfScoped(c.namespace, c.namespace != ""). </ins> Resource(gvr.Resource). Name(name). SubResource("scale").


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : The scale client () currently works fine with namespaced resources. However, it doesn't work for non-namespaced resources e.g. cluster-scoped CRDs. The current implementation should be extended in order to support non-namespaced resources. The current interface: /area custom-resources /sig autoscaling
I think scale subresource support for CRDs falls more into sig-apimachinery domain. After DirectXMan12 moved on from sig-autoscaling I don't think there is anyone in our sig-autoscaling with knowledge of scale subresource internals. /sig api-machinery
/assign
The bug report sounds legit. Does the generic scale client need to be changed (I think so), or does solve your issue?
No, the dashboard PR doesn't solve this. I discovered this issue while working on that PR so when gets merged, it will only work for namespaced CRDs. I think the generic scale client needs to be updated, maybe by adding another getter to the ScalesGetter interface: /kind bug
PR:

) // ScalesGetter can produce a ScaleInterface <del> // for a particular namespace. </del> type ScalesGetter interface { <ins> // Scales produces a ScaleInterface for a particular namespace. // Set namespace to the empty string for non-namespaced resources. </ins> Scales(namespace string) ScaleInterface }


Question:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: ci-kubernetes-e2e-gci-gce-alpha-enabled-default Which test(s) are failing: CSI hostpath-v0 tests Since when has it been failing: 8/8 Testgrid link: https://k8s- Reason for failure: Anything else we need to know: We need to update client-go to get the fix for selfLink, build new nfs provisioner images, and update them in the e2e test. The fix for client-go is being cherry-picked to 1.15 if you want to use a tagged release:
/assign
/assign
This implies rebase to 1.15.x in external-storage, which seem to be challenging (at least to me :-) )
14 cherry pick also in progress:
OK, I guess this line in the lib is the culprit: So I need to update lib then external-storage to pull it in
I don't think the lib itself needs to be updated. Only updating go-client to get the fix to is needed.
Right, lib doesn't dictate what version of client-go to use. Will wait for client-go release/commit
- thanks a lot for bumping the dependency. Did you have a chance to build the new container with it so that we could update it here:
yeah, opened . (Not sure how to trigger test, or we can just merge and see.)
You can manually create a PVC on an alpha cluster to test.

Containers: []v1.Container{ { Name:  "nfs-provisioner", <del> Image: "quay.io/kubernetes_incubator/nfs-provisioner:v2.2.0-k8s1.12", </del> <ins> Image: "quay.io/kubernetes_incubator/nfs-provisioner:v2.2.2", </ins> SecurityContext: &v1.SecurityContext{ Capabilities: &v1.Capabilities{ Add: []v1.Capability{"DAC_READ_SEARCH"},


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability

# Print the supported API Resources with more information kubectl api-resources -o wide <ins> # Print the supported API Resources sorted by a column kubectl api-resources --sort-by=name </ins> # Print the supported namespaced resources kubectl api-resources --namespaced=true


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability

// As new fields are added, add them here instead of referencing the cmd.Flags() type APIResourceOptions struct { Output     string <ins> SortBy     string </ins> APIGroup   string Namespaced bool Verbs      []string


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability

cmd.Flags().StringVar(&o.APIGroup, "api-group", o.APIGroup, "Limit to resources in the specified API group.") cmd.Flags().BoolVar(&o.Namespaced, "namespaced", o.Namespaced, "If false, non-namespaced resources will be returned, otherwise returning namespaced resources by default.") cmd.Flags().StringSliceVar(&o.Verbs, "verbs", o.Verbs, "Limit to resources that support the specified verbs.") <ins> cmd.Flags().StringVar(&o.SortBy, "sort-by", o.SortBy, "If non-empty, sort nodes list using specified field. The field can be either 'name' or 'kind'.") </ins> cmd.Flags().BoolVar(&o.Cached, "cached", o.Cached, "Use the cached list of resources if available.") return cmd }


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability

if !supportedOutputTypes.Has(o.Output) { return fmt.Errorf("--output %v is not available", o.Output) } <ins> supportedSortTypes := sets.NewString("", "name", "kind") if len(o.SortBy) > 0 { if !supportedSortTypes.Has(o.SortBy) { return fmt.Errorf("--sort-by accepts only name or kind") } } </ins> return nil }


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability

} } <del> sort.Stable(sortableGroupResource(resources)) </del> <ins> sort.Stable(sortableResource{resources, o.SortBy}) </ins> for _, r := range resources { switch o.Output { case "name":


Question:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a sort parameter (or similar concept) to the command. Why is this needed: With the numerous amount of API resources (not counting custom ones), it is sometimes difficult locating them in the output. By default, API resources are not sorted by the name column (the first on the left) and often this is the column holding the information one wishes to query. A sort parameter would help more easily identify the object to be understood.
/sig usability

return err } <del> type sortableGroupResource []groupResource </del> <ins> type sortableResource struct { resources []groupResource sortBy    string } </ins> <del> func (s sortableGroupResource) Len() int      { return len(s) } func (s sortableGroupResource) Swap(i, j int) { s[i], s[j] = s[j], s[i] } func (s sortableGroupResource) Less(i, j int) bool { ret := strings.Compare(s[i].APIGroup, s[j].APIGroup) </del> <ins> func (s sortableResource) Len() int { return len(s.resources) } func (s sortableResource) Swap(i, j int) { s.resources[i], s.resources[j] = s.resources[j], s.resources[i] } func (s sortableResource) Less(i, j int) bool { ret := strings.Compare(s.compareValues(i, j)) </ins> if ret > 0 { return false } else if ret == 0 { <del> return strings.Compare(s[i].APIResource.Name, s[j].APIResource.Name) < 0 </del> <ins> return strings.Compare(s.resources[i].APIResource.Name, s.resources[j].APIResource.Name) < 0 </ins> } return true } <ins> func (s sortableResource) compareValues(i, j int) (string, string) { switch s.sortBy { case "name": return s.resources[i].APIResource.Name, s.resources[j].APIResource.Name case "kind": return s.resources[i].APIResource.Kind, s.resources[j].APIResource.Kind } return s.resources[i].APIGroup, s.resources[j].APIGroup } </ins>


Question:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: ci-kubernetes-node-kubelet-orphans Which test(s) are failing: [sig-storage] EmptyDir volumes [It] pod should support shared volumes between containers [Conformance] Since when has it been failing: Testgrid link: Reason for failure: Anything else we need to know:
please don't CC sig-testing on every test failure :( we own the infrastructure etc. the sig-storage owns these tests.

import ( "fmt" "path" <del> "time" </del> "github.com/onsi/ginkgo" <del> "k8s.io/api/core/v1" </del> <ins> v1 "k8s.io/api/core/v1" </ins> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/uuid" "k8s.io/kubernetes/test/e2e/framework"


Question:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: ci-kubernetes-node-kubelet-orphans Which test(s) are failing: [sig-storage] EmptyDir volumes [It] pod should support shared volumes between containers [Conformance] Since when has it been failing: Testgrid link: Reason for failure: Anything else we need to know:
please don't CC sig-testing on every test failure :( we own the infrastructure etc. the sig-storage owns these tests.

framework.ExpectNoError(err, "failed to get pod %s", pod.Name) ginkgo.By("Reading file content from the nginx-container") <del> resultString, err = framework.LookForStringInFile(f.Namespace.Name, pod.Name, busyBoxMainContainerName, busyBoxMainVolumeFilePath, message, 30*time.Second) framework.ExpectNoError(err, "failed to match expected string %s with %s", message, resultString) </del> <ins> result := f.ExecShellInContainer(pod.Name, busyBoxMainContainerName, fmt.Sprintf("cat %s", busyBoxMainVolumeFilePath)) framework.ExpectEqual(result, message, "failed to match expected string %s with %s", message, resultString) </ins> }) })


Question: What happened: There's a typo typo in and What you expected to happen: “depreciated” should be “deprecated”. How to reproduce it (as minimally and precisely as possible): View the release notes Anything else we need to know?: /lifecycle active I have a PR to fix this /sig docs Environment: Kubernetes version (use ): v1.16 Cloud provider or hardware configuration: not applicable OS (e.g: ): not applicable Kernel (e.g. ): not applicable Install tools: not applicable

- [Metrics Changes](#metrics-changes) - [Added metrics](#added-metrics) - [Removed metrics](#removed-metrics) <del> - [Depreciated/changed metrics](#depreciatedchanged-metrics) </del> <ins> - [Deprecated/changed metrics](#deprecatedchanged-metrics) </ins> - [Notable Features](#notable-features) - [Beta](#beta) - [Alpha](#alpha)


Question: What happened: There's a typo typo in and What you expected to happen: “depreciated” should be “deprecated”. How to reproduce it (as minimally and precisely as possible): View the release notes Anything else we need to know?: /lifecycle active I have a PR to fix this /sig docs Environment: Kubernetes version (use ): v1.16 Cloud provider or hardware configuration: not applicable OS (e.g: ): not applicable Kernel (e.g. ): not applicable Install tools: not applicable

- Removed cadvisor metric labels `pod_name` and `container_name` to match instrumentation guidelines. Any Prometheus queries that match `pod_name` and `container_name` labels (e.g. cadvisor or kubelet probe metrics) must be updated to use `pod` and `container` instead. ([#80376](https://github.com/kubernetes/kubernetes/pull/80376), [@ehashman](https://github.com/ehashman)) <del> ### Depreciated/changed metrics </del> <ins> ### Deprecated/changed metrics </ins> - kube-controller-manager and cloud-controller-manager metrics are now marked as with the ALPHA stability level. ([#81624](https://github.com/kubernetes/kubernetes/pull/81624), [@logicalhan](https://github.com/logicalhan)) - kube-proxy metrics are now marked as with the ALPHA stability level. ([#81626](https://github.com/kubernetes/kubernetes/pull/81626), [@logicalhan](https://github.com/logicalhan))


Question:
What would you like to be : Currently the apiserver has request latencies metrics, which are a histogram of requests to the apiserver. However, these request metrics do not include healthz. While we have recently the ability to get back health check metrics from the healthz endpoint directly, we should also emit this data as a metric, so that we can monitor health check latencies. /assign /sig api-machinery instrumentation
GitHub didn't allow me to assign the following users: jktomer. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : What would you like to be : /sig api-machinery instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I noticed this is informally assigned. Has there been any progress? Is this a place I can fit in?
Yes, I'm working on a PR.
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale

embed = [":go_default_library"], deps = [ "//staging/src/k8s.io/apimachinery/pkg/util/sets:go_default_library", <ins> "//staging/src/k8s.io/apiserver/pkg/endpoints/metrics:go_default_library", "//staging/src/k8s.io/component-base/metrics/legacyregistry:go_default_library", "//staging/src/k8s.io/component-base/metrics/testutil:go_default_library", </ins> ], )


Question:
What would you like to be : Currently the apiserver has request latencies metrics, which are a histogram of requests to the apiserver. However, these request metrics do not include healthz. While we have recently the ability to get back health check metrics from the healthz endpoint directly, we should also emit this data as a metric, so that we can monitor health check latencies. /assign /sig api-machinery instrumentation
GitHub didn't allow me to assign the following users: jktomer. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : What would you like to be : /sig api-machinery instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I noticed this is informally assigned. Has there been any progress? Is this a place I can fit in?
Yes, I'm working on a PR.
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale

deps = [ "//staging/src/k8s.io/apimachinery/pkg/util/sets:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/util/wait:go_default_library", <ins> "//staging/src/k8s.io/apiserver/pkg/endpoints/metrics:go_default_library", </ins> "//staging/src/k8s.io/apiserver/pkg/server/httplog:go_default_library", "//vendor/k8s.io/klog:go_default_library", ],


Question:
What would you like to be : Currently the apiserver has request latencies metrics, which are a histogram of requests to the apiserver. However, these request metrics do not include healthz. While we have recently the ability to get back health check metrics from the healthz endpoint directly, we should also emit this data as a metric, so that we can monitor health check latencies. /assign /sig api-machinery instrumentation
GitHub didn't allow me to assign the following users: jktomer. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : What would you like to be : /sig api-machinery instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I noticed this is informally assigned. Has there been any progress? Is this a place I can fit in?
Yes, I'm working on a PR.
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale

"k8s.io/apimachinery/pkg/util/sets" "k8s.io/apimachinery/pkg/util/wait" <ins> "k8s.io/apiserver/pkg/endpoints/metrics" </ins> "k8s.io/apiserver/pkg/server/httplog" "k8s.io/klog" )


Question:
What would you like to be : Currently the apiserver has request latencies metrics, which are a histogram of requests to the apiserver. However, these request metrics do not include healthz. While we have recently the ability to get back health check metrics from the healthz endpoint directly, we should also emit this data as a metric, so that we can monitor health check latencies. /assign /sig api-machinery instrumentation
GitHub didn't allow me to assign the following users: jktomer. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : What would you like to be : /sig api-machinery instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I noticed this is informally assigned. Has there been any progress? Is this a place I can fit in?
Yes, I'm working on a PR.
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale

klog.V(5).Infof("Installing health checkers for (%v): %v", path, formatQuoted(checkerNames(checks...)...)) <del> mux.Handle(path, handleRootHealthz(checks...)) </del> <ins> mux.Handle(path, metrics.InstrumentHandlerFunc("GET", /* group = */ "", /* version = */ "", /* resource = */ "", /* subresource = */ path, /* scope = */ "", /* component = */ "", handleRootHealthz(checks...))) </ins> for _, check := range checks { mux.Handle(fmt.Sprintf("%s/%v", path, check.Name()), adaptCheckToHandler(check.Check)) }


Question:
What would you like to be : Currently the apiserver has request latencies metrics, which are a histogram of requests to the apiserver. However, these request metrics do not include healthz. While we have recently the ability to get back health check metrics from the healthz endpoint directly, we should also emit this data as a metric, so that we can monitor health check latencies. /assign /sig api-machinery instrumentation
GitHub didn't allow me to assign the following users: jktomer. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : What would you like to be : /sig api-machinery instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I noticed this is informally assigned. Has there been any progress? Is this a place I can fit in?
Yes, I'm working on a PR.
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale

"net/http/httptest" "net/url" "reflect" <ins> "strings" </ins> "testing" "k8s.io/apimachinery/pkg/util/sets" <ins> "k8s.io/apiserver/pkg/endpoints/metrics" "k8s.io/component-base/metrics/legacyregistry" "k8s.io/component-base/metrics/testutil" </ins> ) func TestInstallHandler(t *testing.T) {


Question:
What would you like to be : Currently the apiserver has request latencies metrics, which are a histogram of requests to the apiserver. However, these request metrics do not include healthz. While we have recently the ability to get back health check metrics from the healthz endpoint directly, we should also emit this data as a metric, so that we can monitor health check latencies. /assign /sig api-machinery instrumentation
GitHub didn't allow me to assign the following users: jktomer. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : What would you like to be : /sig api-machinery instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I noticed this is informally assigned. Has there been any progress? Is this a place I can fit in?
Yes, I'm working on a PR.
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale

} } <ins> func TestMetrics(t *testing.T) { mux := http.NewServeMux() InstallHandler(mux) InstallLivezHandler(mux) InstallReadyzHandler(mux) metrics.Register() metrics.Reset() paths := []string{"/healthz", "/livez", "/readyz"} for _, path := range paths { req, err := http.NewRequest("GET", fmt.Sprintf("http://example.com%s", path), nil) if err != nil { t.Errorf("%v", err) } mux.ServeHTTP(httptest.NewRecorder(), req) } expected := strings.NewReader(` # HELP apiserver_request_total [ALPHA] Counter of apiserver requests broken out for each verb, dry run value, group, version, resource, scope, component, client, and HTTP response contentType and code. # TYPE apiserver_request_total counter apiserver_request_total{client="unknown",code="200",component="",contentType="text/plain; charset=utf-8",dry_run="",group="",resource="",scope="",subresource="/healthz",verb="GET",version=""} 1 apiserver_request_total{client="unknown",code="200",component="",contentType="text/plain; charset=utf-8",dry_run="",group="",resource="",scope="",subresource="/livez",verb="GET",version=""} 1 apiserver_request_total{client="unknown",code="200",component="",contentType="text/plain; charset=utf-8",dry_run="",group="",resource="",scope="",subresource="/readyz",verb="GET",version=""} 1 `) if err := testutil.GatherAndCompare(legacyregistry.DefaultGatherer, expected, "apiserver_request_total"); err != nil { t.Error(err) } } </ins> func createGetRequestWithUrl(rawUrlString string) *http.Request { url, _ := url.Parse(rawUrlString) return &http.Request{


Question:
What happened: Provisioning of a volume with failed and this event were created on a pod: The event does not say anything useful and only confuses users. What you expected to happen: Either no event (event on PVC about failed provisioning could be enough) or something generic about "waiting for provisioning to finish". How to reproduce it (as minimally and precisely as possible): Tested on AWS. It should be the same with GCE and other block-based in-tree volumes. StorageClass with PVC requesting RWX (so provisioning fails): pod that uses the PVC. Pod events: Anything else we need to know?: I've been able to track down what happens: Scheduler adds to the PVC Provisioner tries to create a volume for the PVC and fails. The annotation is removed in  The scheduler expects that the annotation exists here: I am not sure how to fix it though. Ignoring the error might be the easiest thing to do. Environment: Kubernetes version (use ): today's master /sig storage /sig scheduling /assign
yes, the error message is confusing. "FailedScheduling" event will be emitted if the scheduler failed to schedule the pod. How about wrapping the internal error message in function, e.g. ? Users will know the pod failed because pod volumes not bound successfully. Sometimes, users can't just wait for the scheduler to recover from the error state, manual intervention might be required. The internal reason will be useful. In this scenario, .
's suggestion sounds good to me

return err } <del> return wait.Poll(time.Second, b.bindTimeout, func() (bool, error) { </del> <ins> err = wait.Poll(time.Second, b.bindTimeout, func() (bool, error) { </ins> b, err := b.checkBindings(assumedPod, bindings, claimsToProvision) return b, err }) <ins> if err != nil { pvcName := "" if len(claimsToProvision) > 0 { pvcName = claimsToProvision[0].Name } return fmt.Errorf("Failed to bind volumes: provisioning failed for PVC %q: %v", pvcName, err) } return nil </ins> } func getPodName(pod *v1.Pod) string {