Question:
/kind feature What happened: When calling , there is no way to override default flag values for the different components of the controlplane. This is surprising, as in the config structure (and in controlplane phase codebase itself), there are explicit settings for doing that (APIServerExtraArgs, ControllerManagerExtraArgs, SchedulerExtraArgs). What you expected to happen: Those config fields should be exposed as flags on and How to reproduce it (as minimally and precisely as possible): Try to deploy K8s using Kubeadm with a custom Admission Control setting (I'll follow up on this issue with a PR)
btw, I am not sure which sig owns kubeadm.
it's sig-cluster-lifecycle.
/sig sig-cluster-lifecycle
/sig cluster-lifecycle

) flagSet.StringVar(featureGatesString, "feature-gates", *featureGatesString, "A set of key=value pairs that describe feature gates for various features. "+ "Options are:n"+strings.Join(features.KnownFeatures(&features.InitFeatureGates), "n")) <ins> flagSet.Var(utilflag.NewMapStringString(&cfg.APIServerExtraArgs), "apiserver-extra-args", "A set of extra flags to pass to the API Server or override default ones in form of <flagname>=<value>") flagSet.Var(utilflag.NewMapStringString(&cfg.ControllerManagerExtraArgs), "controller-manager-extra-args", "A set of extra flags to pass to the Controller Manager or override default ones in form of <flagname>=<value>") flagSet.Var(utilflag.NewMapStringString(&cfg.SchedulerExtraArgs), "scheduler-extra-args", "A set of extra flags to pass to the Scheduler or override default ones in form of <flagname>=<value>") </ins> } // AddInitOtherFlags adds init flags that are not bound to a configuration file to the given flagset


Question:
/kind feature What happened: When calling , there is no way to override default flag values for the different components of the controlplane. This is surprising, as in the config structure (and in controlplane phase codebase itself), there are explicit settings for doing that (APIServerExtraArgs, ControllerManagerExtraArgs, SchedulerExtraArgs). What you expected to happen: Those config fields should be exposed as flags on and How to reproduce it (as minimally and precisely as possible): Try to deploy K8s using Kubeadm with a custom Admission Control setting (I'll follow up on this issue with a PR)
btw, I am not sure which sig owns kubeadm.
it's sig-cluster-lifecycle.
/sig sig-cluster-lifecycle
/sig cluster-lifecycle

"//vendor/github.com/spf13/pflag:go_default_library", "//vendor/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//vendor/k8s.io/apimachinery/pkg/util/sets:go_default_library", <ins> "//vendor/k8s.io/apiserver/pkg/util/flag:go_default_library", </ins> "//vendor/k8s.io/client-go/kubernetes:go_default_library", "//vendor/k8s.io/client-go/tools/bootstrap/token/api:go_default_library", "//vendor/k8s.io/client-go/tools/bootstrap/token/util:go_default_library",


Question:
/kind feature What happened: When calling , there is no way to override default flag values for the different components of the controlplane. This is surprising, as in the config structure (and in controlplane phase codebase itself), there are explicit settings for doing that (APIServerExtraArgs, ControllerManagerExtraArgs, SchedulerExtraArgs). What you expected to happen: Those config fields should be exposed as flags on and How to reproduce it (as minimally and precisely as possible): Try to deploy K8s using Kubeadm with a custom Admission Control setting (I'll follow up on this issue with a PR)
btw, I am not sure which sig owns kubeadm.
it's sig-cluster-lifecycle.
/sig sig-cluster-lifecycle
/sig cluster-lifecycle

"github.com/spf13/cobra" <ins> utilflag "k8s.io/apiserver/pkg/util/flag" </ins> kubeadmapi "k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm" kubeadmapiext "k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1alpha1" "k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/validation"


Question:
/kind feature What happened: When calling , there is no way to override default flag values for the different components of the controlplane. This is surprising, as in the config structure (and in controlplane phase codebase itself), there are explicit settings for doing that (APIServerExtraArgs, ControllerManagerExtraArgs, SchedulerExtraArgs). What you expected to happen: Those config fields should be exposed as flags on and How to reproduce it (as minimally and precisely as possible): Try to deploy K8s using Kubeadm with a custom Admission Control setting (I'll follow up on this issue with a PR)
btw, I am not sure which sig owns kubeadm.
it's sig-cluster-lifecycle.
/sig sig-cluster-lifecycle
/sig cluster-lifecycle

cmd.Flags().StringVar(&cfg.Networking.ServiceSubnet, "service-cidr", cfg.Networking.ServiceSubnet, "The range of IP address used for service VIPs") cmd.Flags().StringVar(&featureGatesString, "feature-gates", featureGatesString, "A set of key=value pairs that describe feature gates for various features. "+ "Options are:n"+strings.Join(features.KnownFeatures(&features.InitFeatureGates), "n")) <ins> cmd.Flags().Var(utilflag.NewMapStringString(&cfg.APIServerExtraArgs), "apiserver-extra-args", "A set of extra flags to pass to the API Server or override default ones in form of <flagname>=<value>") </ins> } if properties.use == "all" || properties.use == "controller-manager" { cmd.Flags().StringVar(&cfg.Networking.PodSubnet, "pod-network-cidr", cfg.Networking.PodSubnet, "The range of IP addresses used for the Pod network") <ins> cmd.Flags().Var(utilflag.NewMapStringString(&cfg.ControllerManagerExtraArgs), "controller-manager-extra-args", "A set of extra flags to pass to the Controller Manager or override default ones in form of <flagname>=<value>") } if properties.use == "all" || properties.use == "scheduler" { cmd.Flags().Var(utilflag.NewMapStringString(&cfg.SchedulerExtraArgs), "scheduler-extra-args", "A set of extra flags to pass to the Scheduler or override default ones in form of <flagname>=<value>") </ins> } cmd.Flags().StringVar(&cfgPath, "config", cfgPath, "Path to kubeadm config file (WARNING: Usage of a configuration file is experimental)")


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec during . However, in order to recreate newMounter from volumeSpec, volume information are lacking. As a result, filesystem volume's reconstruction always fails. What you expected to happen: Filesystem volume's reconstruction works properly, then a volume is unmounted and detached from kubelet node. How to reproduce it (as minimally and precisely as possible): (1) Create filesystem volume using FC plugin (2) Restart kubelet service which a pod with FC volume is running Anything else we need to know?: cc Environment: Kubernetes version (use ):Install tools: local-cluster-
/assign
/sig storage

} func (plugin *fcPlugin) ConstructVolumeSpec(volumeName, mountPath string) (*volume.Spec, error) { <del> fcVolume := &v1.Volume{ Name: volumeName, VolumeSource: v1.VolumeSource{ FC: &v1.FCVolumeSource{}, }, </del> <ins> // Find globalPDPath from pod volume directory(mountPath) // examples: //   mountPath:     pods/{podUid}/volumes/kubernetes.io~fc/{volumeName} //   globalPDPath : plugins/kubernetes.io/fc/50060e801049cfd1-lun-0 var globalPDPath string mounter := plugin.host.GetMounter(plugin.GetPluginName()) paths, err := mount.GetMountRefs(mounter, mountPath) if err != nil { return nil, err } for _, path := range paths { if strings.Contains(path, plugin.host.GetPluginDir(fcPluginName)) { globalPDPath = path break } } // Couldn't fetch globalPDPath if len(globalPDPath) == 0 { return nil, fmt.Errorf("couldn't fetch globalPDPath. failed to obtain volume spec") } arr := strings.Split(globalPDPath, "/") if len(arr) < 1 { return nil, fmt.Errorf("failed to retrieve volume plugin information from globalPDPath: %v", globalPDPath) } volumeInfo := arr[len(arr)-1] // Create volume from wwn+lun or wwid var fcVolume *v1.Volume if strings.Contains(volumeInfo, "-lun-") { wwnLun := strings.Split(volumeInfo, "-lun-") if len(wwnLun) < 2 { return nil, fmt.Errorf("failed to retrieve TargetWWN and Lun. volumeInfo is invalid: %v", volumeInfo) } lun, err := strconv.Atoi(wwnLun[1]) if err != nil { return nil, err } lun32 := int32(lun) fcVolume = &v1.Volume{ Name: volumeName, VolumeSource: v1.VolumeSource{ FC: &v1.FCVolumeSource{TargetWWNs: []string{wwnLun[0]}, Lun: &lun32}, }, } glog.V(5).Infof("ConstructVolumeSpec: TargetWWNs: %v, Lun: %v", fcVolume.VolumeSource.FC.TargetWWNs, *fcVolume.VolumeSource.FC.Lun) } else { fcVolume = &v1.Volume{ Name: volumeName, VolumeSource: v1.VolumeSource{ FC: &v1.FCVolumeSource{WWIDs: []string{volumeInfo}}, }, } glog.V(5).Infof("ConstructVolumeSpec: WWIDs: %v", fcVolume.VolumeSource.FC.WWIDs) </ins> } return volume.NewSpecFromVolume(fcVolume), nil }


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec during . However, in order to recreate newMounter from volumeSpec, volume information are lacking. As a result, filesystem volume's reconstruction always fails. What you expected to happen: Filesystem volume's reconstruction works properly, then a volume is unmounted and detached from kubelet node. How to reproduce it (as minimally and precisely as possible): (1) Create filesystem volume using FC plugin (2) Restart kubelet service which a pod with FC volume is running Anything else we need to know?: cc Environment: Kubernetes version (use ):Install tools: local-cluster-
/assign
/sig storage

//   - If a file is found, then retreives volumePluginDependentPath from globalMapPathUUID. //   - Once volumePluginDependentPath is obtained, store volume information to VolumeSource // examples: <del> //   mapPath: pods/{podUid}}/{DefaultKubeletVolumeDevicesDirName}/{escapeQualifiedPluginName}/{volumeName} </del> <ins> //   mapPath: pods/{podUid}/{DefaultKubeletVolumeDevicesDirName}/{escapeQualifiedPluginName}/{volumeName} </ins> //   globalMapPathUUID : plugins/kubernetes.io/{PluginName}/{DefaultKubeletVolumeDevicesDirName}/{volumePluginDependentPath}/{pod uuid} func (plugin *fcPlugin) ConstructBlockVolumeSpec(podUID types.UID, volumeName, mapPath string) (*volume.Spec, error) { pluginDir := plugin.host.GetVolumeDevicePluginDir(fcPluginName)


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec during . However, in order to recreate newMounter from volumeSpec, volume information are lacking. As a result, filesystem volume's reconstruction always fails. What you expected to happen: Filesystem volume's reconstruction works properly, then a volume is unmounted and detached from kubelet node. How to reproduce it (as minimally and precisely as possible): (1) Create filesystem volume using FC plugin (2) Restart kubelet service which a pod with FC volume is running Anything else we need to know?: cc Environment: Kubernetes version (use ):Install tools: local-cluster-
/assign
/sig storage

v1.FCVolumeSource{TargetWWNs: []string{wwnLun[0]}, Lun: &lun32}) glog.V(5).Infof("ConstructBlockVolumeSpec: TargetWWNs: %v, Lun: %v", fcPV.Spec.PersistentVolumeSource.FC.TargetWWNs, <del> fcPV.Spec.PersistentVolumeSource.FC.Lun) </del> <ins> *fcPV.Spec.PersistentVolumeSource.FC.Lun) </ins> } else { fcPV = createPersistentVolumeFromFCVolumeSource(volumeName, v1.FCVolumeSource{WWIDs: []string{volumeInfo}})


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec during . However, in order to recreate newMounter from volumeSpec, volume information are lacking. As a result, filesystem volume's reconstruction always fails. What you expected to happen: Filesystem volume's reconstruction works properly, then a volume is unmounted and detached from kubelet node. How to reproduce it (as minimally and precisely as possible): (1) Create filesystem volume using FC plugin (2) Restart kubelet service which a pod with FC volume is running Anything else we need to know?: cc Environment: Kubernetes version (use ):Install tools: local-cluster-
/assign
/sig storage

import ( "fmt" "os" <ins> "strconv" "strings" </ins> "testing" "k8s.io/api/core/v1"


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec during . However, in order to recreate newMounter from volumeSpec, volume information are lacking. As a result, filesystem volume's reconstruction always fails. What you expected to happen: Filesystem volume's reconstruction works properly, then a volume is unmounted and detached from kubelet node. How to reproduce it (as minimally and precisely as possible): (1) Create filesystem volume using FC plugin (2) Restart kubelet service which a pod with FC volume is running Anything else we need to know?: cc Environment: Kubernetes version (use ):Install tools: local-cluster-
/assign
/sig storage

t.Errorf("unexpected fc disk found") } } <ins> func Test_ConstructVolumeSpec(t *testing.T) { fm := &mount.FakeMounter{ MountPoints: []mount.MountPoint{ {Device: "/dev/sdb", Path: "/var/lib/kubelet/pods/some-pod/volumes/kubernetes.io~fc/fc-in-pod1"}, {Device: "/dev/sdb", Path: "/var/lib/kubelet/plugins/kubernetes.io/fc/50060e801049cfd1-lun-0"}, {Device: "/dev/sdc", Path: "/var/lib/kubelet/pods/some-pod/volumes/kubernetes.io~fc/fc-in-pod2"}, {Device: "/dev/sdc", Path: "/var/lib/kubelet/plugins/kubernetes.io/fc/volumeDevices/3600508b400105e210000900000490000"}, }, } mountPaths := []string{ "/var/lib/kubelet/pods/some-pod/volumes/kubernetes.io~fc/fc-in-pod1", "/var/lib/kubelet/pods/some-pod/volumes/kubernetes.io~fc/fc-in-pod2", } for _, path := range mountPaths { refs, _ := mount.GetMountRefs(fm, path) var globalPDPath string for _, ref := range refs { if strings.Contains(ref, "kubernetes.io/fc") { globalPDPath = ref break } } if len(globalPDPath) == 0 { t.Errorf("couldn't fetch mountrefs") } arr := strings.Split(globalPDPath, "/") if len(arr) < 1 { t.Errorf("failed to retrieve volume plugin information from globalPDPath: %v", globalPDPath) } volumeInfo := arr[len(arr)-1] if strings.Contains(volumeInfo, "-lun-") { wwnLun := strings.Split(volumeInfo, "-lun-") if len(wwnLun) < 2 { t.Errorf("failed to retrieve TargetWWN and Lun. volumeInfo is invalid: %v", volumeInfo) } lun, _ := strconv.Atoi(wwnLun[1]) lun32 := int32(lun) if wwnLun[0] != "50060e801049cfd1" || lun32 != 0 { t.Errorf("failed to retrieve TargetWWN and Lun") } } else { if volumeInfo != "3600508b400105e210000900000490000" { t.Errorf("failed to retrieve WWIDs") } } } } func Test_ConstructVolumeSpecNoRefs(t *testing.T) { fm := &mount.FakeMounter{ MountPoints: []mount.MountPoint{ {Device: "/dev/sdd", Path: "/var/lib/kubelet/pods/some-pod/volumes/kubernetes.io~fc/fc-in-pod1"}, }, } mountPaths := []string{ "/var/lib/kubelet/pods/some-pod/volumes/kubernetes.io~fc/fc-in-pod1", } for _, path := range mountPaths { refs, _ := mount.GetMountRefs(fm, path) var globalPDPath string for _, ref := range refs { if strings.Contains(ref, "kubernetes.io/fc") { globalPDPath = ref break } } if len(globalPDPath) != 0 { t.Errorf("invalid globalPDPath") } } } </ins>


Question:
If I run first time it goes fine (after reboot). For subsequent runs I get nodes NotReady. I've tried restarting docker but that won't help. Before christams this used to work all the time. Running Arch Linux if it helps. In I see
/sig node
ditto for me: Looking for a work around I noticed that HugePages recently moved to beta. I can get pods to start again by disabling the HugePages feature gate .
/assign
cc
default Arch kernel doesn't have a hugetlb cgroup controller compiled in(been there done that, I use Arch as well), so you have to prepare your own kernel to run it.
If it were a kernel problem I would have expected the kubelet to fail the first time it was started rather than the second time. This isn't confined to Arch, though. I'm having the same problem with the default Debian kernel:
can you check if your Kernel was compiled with flag? You can do it by running: I bet we have to add documentation, describing this behavior. On Ubuntu 16.04 machine:
Sure, looks like it was not: Documentation (especially in the error message) would be great, but if the kubelet isn't going to survive a restart, I would prefer that it not start the first time. Better to fail immediately than eventually.
can you please check if works (as in fails!) for you?
No, this doesn't resolve the issue. Logs from the first start: Logs from the second start:
Do you have the directory? in that case what i have is insufficient :( If you don't have that directory, then we should probably look for a better spot to put that check? any advice?
No, that directory is missing on my system:
I'm seeing this issue as well.
I am preparing a fix to not fail when hugetlb mount is not present. Will post a fix shortly.
I posted a fix here:

GetStats(path string, stats *libcontainercgroups.Stats) error } <del> // getSupportedSubsystems returns list of subsystems supported func getSupportedSubsystems() []subsystem { supportedSubsystems := []subsystem{ &cgroupfs.MemoryGroup{}, &cgroupfs.CpuGroup{}, </del> <ins> // getSupportedSubsystems returns a map of subsystem and if it must be mounted for the kubelet to function. func getSupportedSubsystems() map[subsystem]bool { supportedSubsystems := map[subsystem]bool{ &cgroupfs.MemoryGroup{}: true, &cgroupfs.CpuGroup{}:    true, </ins> } <ins> // not all hosts support hugetlb cgroup, and in the absent of hugetlb, we will fail silently by reporting no capacity. </ins> if utilfeature.DefaultFeatureGate.Enabled(kubefeatures.HugePages) { <del> supportedSubsystems = append(supportedSubsystems, &cgroupfs.HugetlbGroup{}) </del> <ins> supportedSubsystems[&cgroupfs.HugetlbGroup{}] = false </ins> } if utilfeature.DefaultFeatureGate.Enabled(kubefeatures.SupportPodPidsLimit) { <del> supportedSubsystems = append(supportedSubsystems, &cgroupfs.PidsGroup{}) </del> <ins> supportedSubsystems[&cgroupfs.PidsGroup{}] = true </ins> } return supportedSubsystems }


Question:
If I run first time it goes fine (after reboot). For subsequent runs I get nodes NotReady. I've tried restarting docker but that won't help. Before christams this used to work all the time. Running Arch Linux if it helps. In I see
/sig node
ditto for me: Looking for a work around I noticed that HugePages recently moved to beta. I can get pods to start again by disabling the HugePages feature gate .
/assign
cc
default Arch kernel doesn't have a hugetlb cgroup controller compiled in(been there done that, I use Arch as well), so you have to prepare your own kernel to run it.
If it were a kernel problem I would have expected the kubelet to fail the first time it was started rather than the second time. This isn't confined to Arch, though. I'm having the same problem with the default Debian kernel:
can you check if your Kernel was compiled with flag? You can do it by running: I bet we have to add documentation, describing this behavior. On Ubuntu 16.04 machine:
Sure, looks like it was not: Documentation (especially in the error message) would be great, but if the kubelet isn't going to survive a restart, I would prefer that it not start the first time. Better to fail immediately than eventually.
can you please check if works (as in fails!) for you?
No, this doesn't resolve the issue. Logs from the first start: Logs from the second start:
Do you have the directory? in that case what i have is insufficient :( If you don't have that directory, then we should probably look for a better spot to put that check? any advice?
No, that directory is missing on my system:
I'm seeing this issue as well.
I am preparing a fix to not fail when hugetlb mount is not present. Will post a fix shortly.
I posted a fix here:

// but this is not possible with libcontainers Set() method // See https://github.com/opencontainers/runc/issues/932 func setSupportedSubsystems(cgroupConfig *libcontainerconfigs.Cgroup) error { <del> for _, sys := range getSupportedSubsystems() { </del> <ins> for sys, required := range getSupportedSubsystems() { </ins> if _, ok := cgroupConfig.Paths[sys.Name()]; !ok { <del> return fmt.Errorf("Failed to find subsystem mount for subsystem: %v", sys.Name()) </del> <ins> if required { return fmt.Errorf("Failed to find subsystem mount for required subsystem: %v", sys.Name()) } // the cgroup is not mounted, but its not required so continue... glog.V(6).Infof("Unable to find subsystem mount for optional subsystem: %v", sys.Name()) continue </ins> } if err := sys.Set(cgroupConfig.Paths[sys.Name()], cgroupConfig); err != nil { return fmt.Errorf("Failed to set config for supported subsystems : %v", err)


Question:
If I run first time it goes fine (after reboot). For subsequent runs I get nodes NotReady. I've tried restarting docker but that won't help. Before christams this used to work all the time. Running Arch Linux if it helps. In I see
/sig node
ditto for me: Looking for a work around I noticed that HugePages recently moved to beta. I can get pods to start again by disabling the HugePages feature gate .
/assign
cc
default Arch kernel doesn't have a hugetlb cgroup controller compiled in(been there done that, I use Arch as well), so you have to prepare your own kernel to run it.
If it were a kernel problem I would have expected the kubelet to fail the first time it was started rather than the second time. This isn't confined to Arch, though. I'm having the same problem with the default Debian kernel:
can you check if your Kernel was compiled with flag? You can do it by running: I bet we have to add documentation, describing this behavior. On Ubuntu 16.04 machine:
Sure, looks like it was not: Documentation (especially in the error message) would be great, but if the kubelet isn't going to survive a restart, I would prefer that it not start the first time. Better to fail immediately than eventually.
can you please check if works (as in fails!) for you?
No, this doesn't resolve the issue. Logs from the first start: Logs from the second start:
Do you have the directory? in that case what i have is insufficient :( If you don't have that directory, then we should probably look for a better spot to put that check? any advice?
No, that directory is missing on my system:
I'm seeing this issue as well.
I am preparing a fix to not fail when hugetlb mount is not present. Will post a fix shortly.
I posted a fix here:

func getStatsSupportedSubsystems(cgroupPaths map[string]string) (*libcontainercgroups.Stats, error) { stats := libcontainercgroups.NewStats() <del> for _, sys := range getSupportedSubsystems() { </del> <ins> for sys, required := range getSupportedSubsystems() { </ins> if _, ok := cgroupPaths[sys.Name()]; !ok { <del> return nil, fmt.Errorf("Failed to find subsystem mount for subsystem: %v", sys.Name()) </del> <ins> if required { return nil, fmt.Errorf("Failed to find subsystem mount for required subsystem: %v", sys.Name()) } // the cgroup is not mounted, but its not required so continue... glog.V(6).Infof("Unable to find subsystem mount for optional subsystem: %v", sys.Name()) continue </ins> } if err := sys.GetStats(cgroupPaths[sys.Name()], stats); err != nil { return nil, fmt.Errorf("Failed to get stats for supported subsystems : %v", err)


Question:
/kind bug What happened: When I want to enable both plugins by passing below flag, as they both need config file, and I find I donot know what to set If I set , then I get an error in apiserver
/kind bug /sig auth
/cc
/cc
first, you can't put comments in json. the error was complaining about your embedded comments. this would let the server start: second, the docs are really, really, not right (opened with fixes). if you want to specify config for multiple plugins, should point to a config that references individual config files for each plugin (which there's a bug in loading... fixed in ): example
/assign

if !(runtime.IsMissingVersion(err) || runtime.IsMissingKind(err) || runtime.IsNotRegisteredError(err)) { return nil, err } <ins> // Only tolerate load errors if the file appears to be one of the two legacy plugin configs unstructuredData := map[string]interface{}{} if err2 := yaml.Unmarshal(data, &unstructuredData); err2 != nil { return nil, err } _, isLegacyImagePolicy := unstructuredData["imagePolicy"] _, isLegacyPodNodeSelector := unstructuredData["podNodeSelectorPluginConfig"] if !isLegacyImagePolicy && !isLegacyPodNodeSelector { return nil, err } </ins> // convert the legacy format to the new admission control format // in order to preserve backwards compatibility, we set plugins that // previously read input from a non-versioned file configuration to the


Question:
/kind bug What happened: When I want to enable both plugins by passing below flag, as they both need config file, and I find I donot know what to set If I set , then I get an error in apiserver
/kind bug /sig auth
/cc
/cc
first, you can't put comments in json. the error was complaining about your embedded comments. this would let the server start: second, the docs are really, really, not right (opened with fixes). if you want to specify config for multiple plugins, should point to a config that references individual config files for each plugin (which there's a bug in loading... fixed in ): example
/assign

"k8s.io/client-go/rest" ) <del> var scheme = runtime.NewScheme() </del> <ins> var configScheme = runtime.NewScheme() </ins> func init() { <del> apiserverapi.AddToScheme(scheme) apiserverapiv1alpha1.AddToScheme(scheme) </del> <ins> apiserverapi.AddToScheme(configScheme) apiserverapiv1alpha1.AddToScheme(configScheme) </ins> } // AdmissionOptions holds the admission options


Question:
/kind bug What happened: When I want to enable both plugins by passing below flag, as they both need config file, and I find I donot know what to set If I set , then I get an error in apiserver
/kind bug /sig auth
/cc
/cc
first, you can't put comments in json. the error was complaining about your embedded comments. this would let the server start: second, the docs are really, really, not right (opened with fixes). if you want to specify config for multiple plugins, should point to a config that references individual config files for each plugin (which there's a bug in loading... fixed in ): example
/assign

pluginNames = a.enabledPluginNames() } <del> pluginsConfigProvider, err := admission.ReadAdmissionConfiguration(pluginNames, a.ConfigFile, scheme) </del> <ins> pluginsConfigProvider, err := admission.ReadAdmissionConfiguration(pluginNames, a.ConfigFile, configScheme) </ins> if err != nil { return fmt.Errorf("failed to read plugin config: %v", err) }


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/serializer" <ins> "k8s.io/apimachinery/pkg/runtime/serializer/json" </ins> utilruntime "k8s.io/apimachinery/pkg/util/runtime" "k8s.io/apimachinery/pkg/util/uuid" "k8s.io/apimachinery/pkg/util/wait"


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.

// ConfigFile is the location of the scheduler server's configuration file. ConfigFile string <ins> // WriteConfigTo is the path where the default configuration will be written. WriteConfigTo string </ins> // config is the scheduler server's configuration object. config *componentconfig.KubeSchedulerConfiguration


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.

// AddFlags adds flags for a specific SchedulerServer to the specified FlagSet func (o *Options) AddFlags(fs *pflag.FlagSet) { fs.StringVar(&o.ConfigFile, "config", o.ConfigFile, "The path to the configuration file.") <ins> fs.StringVar(&o.WriteConfigTo, "write-config-to", o.WriteConfigTo, "If set, write the configuration values to this file and exit.") </ins> // All flags below here are deprecated and will eventually be removed.


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.

} func (o *Options) Complete() error { <del> if len(o.ConfigFile) == 0 { glog.Warning("WARNING: all flags other than --config are deprecated. Please begin using a config file ASAP.") </del> <ins> if len(o.ConfigFile) == 0 && len(o.WriteConfigTo) == 0 { glog.Warning("WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP.") </ins> o.applyDeprecatedHealthzAddressToConfig() o.applyDeprecatedHealthzPortToConfig() o.applyDeprecatedAlgorithmSourceOptionsToConfig() } <ins> if len(o.ConfigFile) > 0 { if c, err := o.loadConfigFromFile(o.ConfigFile); err != nil { return err } else { o.config = c } } // Apply algorithms based on feature gates. // TODO: make configurable? algorithmprovider.ApplyFeatureGates() </ins> return nil }


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.

} func (o *Options) Run() error { <del> config := o.config if len(o.ConfigFile) > 0 { if c, err := o.loadConfigFromFile(o.ConfigFile); err != nil { return err } else { config = c } </del> <ins> // If user only want to generate a configure file. if len(o.WriteConfigTo) > 0 { return o.writeConfigFile() </ins> } <del> // Apply algorithms based on feature gates. // TODO: make configurable? algorithmprovider.ApplyFeatureGates() server, err := NewSchedulerServer(config, o.master) </del> <ins> server, err := NewSchedulerServer(o.config, o.master) </ins> if err != nil { return err }


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: We face But there is no way to generate default config file How to reproduce it (as minimally and precisely as possible): Environment: Kubernetes version 1.9.2
I'll take a look it /sig scheduling
/assign
Hi, same issue with controller-manager
Any updates here? , and don't have this flag. Is it even will be supported?
/cc
if not set, component should be use default value; why we'd like to dump all default config?
Because the documentation on the config file format is either nonexistent or difficult to find. If the program itself can generate a template, that fulfills (most of) the need for documentation of the format.
+1 for this, from my expirence, add an option to write the default config is really helpful.
/assign
also in 1.10
Deprecating all flags, prompting folks to use config files in advance is great, except when there's no documentation nor example anywhere. That means that we can't actually migrate to the new settings, and I can't seem to find clear alpha/beta/stable status for the new config file spec, nor timeline for the removal. It is also pretty confusing that the scheduler expects us to use a config file, but the controller-manager does not?
This issue indeed applies to kube-controller-manager. And there should already had deprecation policy.
According to , the flag will be available in . Since I can't find any reference online to an example config file, perhaps one could be provided or linked to here?
will you be able to pick my patch and generate a config file? I do not get my devbox handy so can only post here until Monday.

return server.Run(stop) } <ins> func (o *Options) writeConfigFile() error { var encoder runtime.Encoder mediaTypes := o.codecs.SupportedMediaTypes() for _, info := range mediaTypes { if info.MediaType == "application/yaml" { encoder = info.Serializer break } } if encoder == nil { return errors.New("unable to locate yaml encoder") } encoder = json.NewYAMLSerializer(json.DefaultMetaFactory, o.scheme, o.scheme) encoder = o.codecs.EncoderForVersion(encoder, componentconfigv1alpha1.SchemeGroupVersion) configFile, err := os.Create(o.WriteConfigTo) if err != nil { return err } defer configFile.Close() if err := encoder.Encode(o.config, configFile); err != nil { return err } glog.Infof("Wrote configuration to: %sn", o.WriteConfigTo) return nil } </ins> // NewSchedulerCommand creates a *cobra.Command object with default parameters func NewSchedulerCommand() *cobra.Command { opts, err := NewOptions()


Question:
Is this a BUG REPORT or FEATURE REQUEST?: Bug Report /kind bug What happened: Applying a file that has no changes compared to current will output something like , which muddles the output when using What you expected to happen: Using the -o flag should suppress the unchanged messages in stdout to prevent output from being muddled How to reproduce it (as minimally and precisely as possible): Given the file Will result in proper yaml being generated as expected Run again, and is outputted to stdout, and is invalid yaml Anything else we need to know?: I do need to check whether this still affects the most recent version kubectl Environment: Kubernetes version (use ): 1.8.6 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Darwin Install tools: Others:
confirmed on latest stable version (1.9.2)
Reiterating the mentions to trigger a notification: <detailsIn response to : confirmed on latest stable version (1.9.2) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have a solution to this, still waiting on my company to look over the agreement

return cmdutil.AddSourceToErr(fmt.Sprintf("retrieving modified configuration from:n%vnfor:", info), info.Source, err) } <ins> // Print object only if output format other than "name" is specified printObject := len(output) > 0 && !shortOutput </ins> if err := info.Get(); err != nil { if !errors.IsNotFound(err) { return cmdutil.AddSourceToErr(fmt.Sprintf("retrieving current configuration of:n%vnfrom server for:", info), info.Source, err)


Question:
Is this a BUG REPORT or FEATURE REQUEST?: Bug Report /kind bug What happened: Applying a file that has no changes compared to current will output something like , which muddles the output when using What you expected to happen: Using the -o flag should suppress the unchanged messages in stdout to prevent output from being muddled How to reproduce it (as minimally and precisely as possible): Given the file Will result in proper yaml being generated as expected Run again, and is outputted to stdout, and is invalid yaml Anything else we need to know?: I do need to check whether this still affects the most recent version kubectl Environment: Kubernetes version (use ): 1.8.6 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Darwin Install tools: Others:
confirmed on latest stable version (1.9.2)
Reiterating the mentions to trigger a notification: <detailsIn response to : confirmed on latest stable version (1.9.2) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have a solution to this, still waiting on my company to look over the agreement

} count++ <del> if len(output) > 0 && !shortOutput { </del> <ins> if printObject { </ins> return cmdutil.PrintObject(cmd, info.Object, out) } cmdutil.PrintSuccess(shortOutput, out, info.Object, dryRun, "created")


Question:
Is this a BUG REPORT or FEATURE REQUEST?: Bug Report /kind bug What happened: Applying a file that has no changes compared to current will output something like , which muddles the output when using What you expected to happen: Using the -o flag should suppress the unchanged messages in stdout to prevent output from being muddled How to reproduce it (as minimally and precisely as possible): Given the file Will result in proper yaml being generated as expected Run again, and is outputted to stdout, and is invalid yaml Anything else we need to know?: I do need to check whether this still affects the most recent version kubectl Environment: Kubernetes version (use ): 1.8.6 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Darwin Install tools: Others:
confirmed on latest stable version (1.9.2)
Reiterating the mentions to trigger a notification: <detailsIn response to : confirmed on latest stable version (1.9.2) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I have a solution to this, still waiting on my company to look over the agreement

visitedUids.Insert(string(uid)) } <del> if string(patchBytes) == "{}" { </del> <ins> if string(patchBytes) == "{}" && !printObject { </ins> count++ cmdutil.PrintSuccess(shortOutput, out, info.Object, false, "unchanged") return nil } } count++ <del> if len(output) > 0 && !shortOutput { </del> <ins> if printObject { </ins> return cmdutil.PrintObject(cmd, info.Object, out) } cmdutil.PrintSuccess(shortOutput, out, info.Object, dryRun, "configured")


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

s.GenericServerRunOptions.AddUniversalFlags(fs) s.Etcd.AddFlags(fs) s.SecureServing.AddFlags(fs) <del> s.SecureServing.AddDeprecatedFlags(fs) </del> s.InsecureServing.AddFlags(fs) s.InsecureServing.AddDeprecatedFlags(fs) s.Audit.AddFlags(fs)


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

limitations under the License. */ <del> // Package options contains flags and options for initializing an apiserver </del> <ins> // Package options contains flags and options for initializing kube-apiserver </ins> package options import (


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

kubeserver "k8s.io/kubernetes/pkg/kubeapiserver/server" ) <del> // NewSecureServingOptions gives default values for the kube-apiserver and federation-apiserver which are not the options wanted by </del> <ins> // NewSecureServingOptions gives default values for the kube-apiserver which are not the options wanted by </ins> // "normal" API servers running on the platform func NewSecureServingOptions() *genericoptions.SecureServingOptions { return &genericoptions.SecureServingOptions{


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

if s.AdvertiseAddress == nil || s.AdvertiseAddress.IsUnspecified() { hostIP, err := insecure.DefaultExternalAddress() if err != nil { <del> return fmt.Errorf("Unable to find suitable network address.error='%v'. "+ "Try to set the AdvertiseAddress directly or provide a valid BindAddress to fix this.", err) </del> <ins> return fmt.Errorf("unable to find suitable network address.error='%v'. "+ "Try to set the AdvertiseAddress directly or provide a valid BindAddress to fix this", err) </ins> } s.AdvertiseAddress = hostIP }


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

errors := []error{} if s.BindPort < 0 || s.BindPort > 65535 { <del> errors = append(errors, fmt.Errorf("--insecure-port %v must be between 0 and 65535, inclusive. 0 for turning off insecure (HTTP) port.", s.BindPort)) </del> <ins> errors = append(errors, fmt.Errorf("--insecure-port %v must be between 0 and 65535, inclusive. 0 for turning off insecure (HTTP) port", s.BindPort)) </ins> } return errors


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

func (s *InsecureServingOptions) AddFlags(fs *pflag.FlagSet) { fs.IPVar(&s.BindAddress, "insecure-bind-address", s.BindAddress, ""+ "The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces).") <ins> fs.MarkDeprecated("insecure-bind-address", "This flag will be removed in a future version.") </ins> fs.IntVar(&s.BindPort, "insecure-port", s.BindPort, ""+ "The port on which to serve unsecured, unauthenticated access. It is assumed "+ "that firewall rules are set up such that this port is not reachable from outside of "+ "the cluster and that port 443 on the cluster's public address is proxied to this "+ <del> "port. This is performed by nginx in the default setup. Set to zero to disable") </del> <ins> "port. This is performed by nginx in the default setup. Set to zero to disable.") fs.MarkDeprecated("insecure-port", "This flag will be removed in a future version.") </ins> } <ins> // TODO: remove it until kops stop using `--address` </ins> func (s *InsecureServingOptions) AddDeprecatedFlags(fs *pflag.FlagSet) { fs.IPVar(&s.BindAddress, "address", s.BindAddress, "DEPRECATED: see --insecure-bind-address instead.")


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

errors := []error{} if s.BindPort < 0 || s.BindPort > 65535 { <del> errors = append(errors, fmt.Errorf("--secure-port %v must be between 0 and 65535, inclusive. 0 for turning off secure port.", s.BindPort)) </del> <ins> errors = append(errors, fmt.Errorf("--secure-port %v must be between 0 and 65535, inclusive. 0 for turning off secure port", s.BindPort)) </ins> } return errors


Question:
Currently as authentication and authorization are both standard config for all clusters, the kubeapiserver insecure http handler is unsecure, which means the cluster is vulnerable to the attack. So it should be deprecated, even removed totally.
/sig api-machinery
I think removing the insecure option is a good idea. Can you send an email the sig-apimachinery mailing list with a proposed timeline? I think this should be like other GA features and have a one year clock.
Sure, I will send it ASAP.
:+1:

} <del> func (s *SecureServingOptions) AddDeprecatedFlags(fs *pflag.FlagSet) { fs.IPVar(&s.BindAddress, "public-address-override", s.BindAddress, "DEPRECATED: see --bind-address instead.") fs.MarkDeprecated("public-address-override", "see --bind-address instead.") } </del> // ApplyTo fills up serving information in the server configuration. func (s *SecureServingOptions) ApplyTo(c *server.Config) error { if s == nil {


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

ec.InvalidateCachedPredicateItem(nodeName, invalidPredicates) } <del> // getHashEquivalencePod returns the hash of equivalence pod. // 1. equivalenceHash // 2. if equivalence hash is valid func (ec *EquivalenceCache) getHashEquivalencePod(pod *v1.Pod) (uint64, bool) { </del> <ins> // equivalenceClassInfo holds equivalence hash which is used for checking equivalence cache. // We will pass this to podFitsOnNode to ensure equivalence hash is only calculated per schedule. type equivalenceClassInfo struct { // Equivalence hash. hash uint64 } // getEquivalenceClassInfo returns the equivalence class of given pod. func (ec *EquivalenceCache) getEquivalenceClassInfo(pod *v1.Pod) *equivalenceClassInfo { </ins> equivalencePod := ec.getEquivalencePod(pod) if equivalencePod != nil { hash := fnv.New32a() hashutil.DeepHashObject(hash, equivalencePod) <del> return uint64(hash.Sum32()), true </del> <ins> return &equivalenceClassInfo{ hash: uint64(hash.Sum32()), } </ins> } <del> return 0, false </del> <ins> return nil </ins> }


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

for _, test := range tests { for i, podInfo := range test.podInfoList { testPod := podInfo.pod <del> hash, isValid := ecache.getHashEquivalencePod(testPod) if isValid != podInfo.hashIsValid { </del> <ins> eclassInfo := ecache.getEquivalenceClassInfo(testPod) if eclassInfo == nil && podInfo.hashIsValid { </ins> t.Errorf("Failed: pod %v is expected to have valid hash", testPod) } <del> // NOTE(harry): the first element will be used as target so // this logic can't verify more than two inequivalent pods if i == 0 { targetHash = hash targetPodInfo = podInfo } else { if targetHash != hash { if test.isEquivalent { t.Errorf("Failed: pod: %v is expected to be equivalent to: %v", testPod, targetPodInfo.pod) </del> <ins> if eclassInfo != nil { // NOTE(harry): the first element will be used as target so // this logic can't verify more than two inequivalent pods if i == 0 { targetHash = eclassInfo.hash targetPodInfo = podInfo } else { if targetHash != eclassInfo.hash { if test.isEquivalent { t.Errorf("Failed: pod: %v is expected to be equivalent to: %v", testPod, targetPodInfo.pod) } </ins> } } }


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

// We can use the same metadata producer for all nodes. meta := metadataProducer(pod, nodeNameToInfo) <ins> var equivCacheInfo *equivalenceClassInfo if ecache != nil { // getEquivalenceClassInfo will return immediately if no equivalence pod found equivCacheInfo = ecache.getEquivalenceClassInfo(pod) } </ins> checkNode := func(i int) { nodeName := nodes[i].Name <del> fits, failedPredicates, err := podFitsOnNode(pod, meta, nodeNameToInfo[nodeName], predicateFuncs, ecache, schedulingQueue, alwaysCheckAllPredicates) </del> <ins> fits, failedPredicates, err := podFitsOnNode( pod, meta, nodeNameToInfo[nodeName], predicateFuncs, ecache, schedulingQueue, alwaysCheckAllPredicates, equivCacheInfo, ) </ins> if err != nil { predicateResultLock.Lock() errs[err.Error()]++


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

} // podFitsOnNode checks whether a node given by NodeInfo satisfies the given predicate functions. <ins> // For given pod, podFitsOnNode will check if any equivalent pod exists and try to reuse its cached // predicate results as possible. </ins> // This function is called from two different places: Schedule and Preempt. // When it is called from Schedule, we want to test whether the pod is schedulable // on the node with all the existing pods on the node plus higher and equal priority


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

ecache *EquivalenceCache, queue SchedulingQueue, alwaysCheckAllPredicates bool, <ins> equivCacheInfo *equivalenceClassInfo, </ins> ) (bool, []algorithm.PredicateFailureReason, error) { var ( <del> equivalenceHash  uint64 failedPredicates []algorithm.PredicateFailureReason </del> eCacheAvailable  bool <ins> failedPredicates []algorithm.PredicateFailureReason </ins> invalid          bool fit              bool reasons          []algorithm.PredicateFailureReason


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

) predicateResults := make(map[string]HostPredicate) <del> if ecache != nil { // getHashEquivalencePod will return immediately if no equivalence pod found equivalenceHash, eCacheAvailable = ecache.getHashEquivalencePod(pod) } </del> podsAdded := false // We run predicates twice in some cases. If the node has greater or equal priority // nominated pods, we run them when those pods are added to meta and nodeInfo.


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

// Bypass eCache if node has any nominated pods. // TODO(bsalamat): consider using eCache and adding proper eCache invalidations // when pods are nominated or their nominations change. <del> eCacheAvailable = eCacheAvailable && !podsAdded </del> <ins> eCacheAvailable = equivCacheInfo != nil && !podsAdded </ins> for _, predicateKey := range predicates.PredicatesOrdering() { <del> //TODO (yastij) : compute average predicate restrictiveness to export it as promethus metric </del> <ins> //TODO (yastij) : compute average predicate restrictiveness to export it as Prometheus metric </ins> if predicate, exist := predicateFuncs[predicateKey]; exist { if eCacheAvailable { // PredicateWithECache will return its cached predicate results. <del> fit, reasons, invalid = ecache.PredicateWithECache(pod.GetName(), info.Node().GetName(), predicateKey, equivalenceHash) </del> <ins> fit, reasons, invalid = ecache.PredicateWithECache(pod.GetName(), info.Node().GetName(), predicateKey, equivCacheInfo.hash) </ins> } if !eCacheAvailable || invalid {


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

for predKey, result := range predicateResults { // update equivalence cache with newly computed fit & reasons // TODO(resouer) should we do this in another thread? any race? <del> ecache.UpdateCachedPredicateItem(pod.GetName(), nodeName, predKey, result.Fit, result.FailReasons, equivalenceHash) </del> <ins> ecache.UpdateCachedPredicateItem(pod.GetName(), nodeName, predKey, result.Fit, result.FailReasons, equivCacheInfo.hash) </ins> } } return len(failedPredicates) == 0, failedPredicates, nil


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

// that we should check is if the "pod" is failing to schedule due to pod affinity // failure. // TODO(bsalamat): Consider checking affinity to lower priority pods if feasible with reasonable performance. <del> if fits, _, err := podFitsOnNode(pod, meta, nodeInfoCopy, fitPredicates, nil, queue, false); !fits { </del> <ins> if fits, _, err := podFitsOnNode(pod, meta, nodeInfoCopy, fitPredicates, nil, queue, false, nil); !fits { </ins> if err != nil { glog.Warningf("Encountered error while selecting victims on node %v: %v", nodeInfo.Node().Name, err) }


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Today, . This seems unnecessary and should be done once for all the nodes. Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig scheduling cc:
/assign Thanks, will send fix per slack discussion.
/cc

violatingVictims, nonViolatingVictims := filterPodsWithPDBViolation(potentialVictims.Items, pdbs) reprievePod := func(p *v1.Pod) bool { addPod(p) <del> fits, _, _ := podFitsOnNode(pod, meta, nodeInfoCopy, fitPredicates, nil, queue, false) </del> <ins> fits, _, _ := podFitsOnNode(pod, meta, nodeInfoCopy, fitPredicates, nil, queue, false, nil) </ins> if !fits { removePod(p) victims = append(victims, p)


Question:
https://k8s- https://k8s- cc: /priority failing-test
Sorry, but I will fix it ASAP.
/assign
PR is already sent. Wating for CI report...

"//pkg/util/sysctl:go_default_library", "//pkg/util/version:go_default_library", "//vendor/github.com/golang/glog:go_default_library", <del> "//vendor/golang.org/x/sys/unix:go_default_library", </del> "//vendor/k8s.io/api/core/v1:go_default_library", "//vendor/k8s.io/apimachinery/pkg/types:go_default_library", "//vendor/k8s.io/apimachinery/pkg/util/sets:go_default_library",


Question:
https://k8s- https://k8s- cc: /priority failing-test
Sorry, but I will fix it ASAP.
/assign
PR is already sent. Wating for CI report...

] + select({ "@io_bazel_rules_go//go/platform:linux": [ "//vendor/github.com/vishvananda/netlink:go_default_library", <ins> "//vendor/golang.org/x/sys/unix:go_default_library", </ins> ], "//conditions:default": [], }),


Question:
https://k8s- https://k8s- cc: /priority failing-test
Sorry, but I will fix it ASAP.
/assign
PR is already sent. Wating for CI report...

return false, nil } <del> // UnbindAddress unbind address from the interface </del> <ins> // UnbindAddress makes sure IP address is unbound from the network interface. </ins> func (h *netlinkHandle) UnbindAddress(address, devName string) error { dev, err := h.LinkByName(devName) if err != nil {


Question:
https://k8s- https://k8s- cc: /priority failing-test
Sorry, but I will fix it ASAP.
/assign
PR is already sent. Wating for CI report...

return fmt.Errorf("error parse ip address: %s", address) } if err := h.AddrDel(dev, &netlink.Addr{IPNet: netlink.NewIPNet(addr)}); err != nil { <del> return fmt.Errorf("error unbind address: %s from interface: %s, err: %v", address, devName, err) </del> <ins> if err != unix.ENXIO { return fmt.Errorf("error unbind address: %s from interface: %s, err: %v", address, devName, err) } </ins> } return nil }


Question:
https://k8s- https://k8s- cc: /priority failing-test
Sorry, but I will fix it ASAP.
/assign
PR is already sent. Wating for CI report...

utilipvs "k8s.io/kubernetes/pkg/util/ipvs" utilsysctl "k8s.io/kubernetes/pkg/util/sysctl" utilexec "k8s.io/utils/exec" <del> "golang.org/x/sys/unix" </del> ) const (


Question:
https://k8s- https://k8s- cc: /priority failing-test
Sorry, but I will fix it ASAP.
/assign
PR is already sent. Wating for CI report...

for _, addr := range unbindIPAddr.UnsortedList() { err := proxier.netlinkHandle.UnbindAddress(addr, DefaultDummyDevice) // Ignore no such address error when try to unbind address <del> if err != nil && err != unix.ENXIO { </del> <ins> if err != nil { </ins> glog.Errorf("Failed to unbind service addr %s from dummy interface %s: %v", addr, DefaultDummyDevice, err) } }


Question:
/kind bug What happened: kubectl became very slow, taking 30s to list pods etc. After going deep down the rabbit hole of debugging the controller components, I ran kubectl -v=8 which showed: What you expected to happen: Since caching is required for kubectl to respond in reasonable time and the issue isn't obvious, kubectl should at least print warnings in default log level. How to reproduce it (as minimally and precisely as possible): I got into the situation by running kubectl with sudo, but just chown'ing should be enough to reproduce this. Anything else we need to know?: Environment: kubectl v1.8.3
/sig cli
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale /lifecycle freeze

} if err := d.writeCachedFile(filename, liveResources); err != nil { <del> glog.V(3).Infof("failed to write cache to %v due to %v", filename, err) </del> <ins> glog.V(1).Infof("failed to write cache to %v due to %v", filename, err) </ins> } return liveResources, nil


Question:
/kind bug What happened: kubectl became very slow, taking 30s to list pods etc. After going deep down the rabbit hole of debugging the controller components, I ran kubectl -v=8 which showed: What you expected to happen: Since caching is required for kubectl to respond in reasonable time and the issue isn't obvious, kubectl should at least print warnings in default log level. How to reproduce it (as minimally and precisely as possible): I got into the situation by running kubectl with sudo, but just chown'ing should be enough to reproduce this. Anything else we need to know?: Environment: kubectl v1.8.3
/sig cli
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale /lifecycle freeze

} if err := d.writeCachedFile(filename, liveGroups); err != nil { <del> glog.V(3).Infof("failed to write cache to %v due to %v", filename, err) </del> <ins> glog.V(1).Infof("failed to write cache to %v due to %v", filename, err) </ins> } return liveGroups, nil


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: Fix the following : The basic logic in of using the is continuing reducing the nodes slices through , , and . It needs 4 slices to do this. This is my thought to reuse the slices: Use 2 nodes slices, such as , The reducing process becomes: -Clear -Clear - please give some suggestion, thanks :smile:
/sig scheduling
The challenge is that clearing a slice and reusing it still needs GC clean up, so it may not be effective. The size of slices shrink as we proceed through the function. So, having two slices and over-writing the elements in them can help reduce GC time, but this approach needs some additional checks to ensure we are not using stale elements.
To avoid clearing the slices, this is my thought: Declare to indicate the actual length of the 2 node slices, instead of . Appending a node to a node slice will add 1 to . Clearing a node slice will set to 0. WDYT
Yes, I can't think of any better solution either, but remember that you should append only to the slice in the first for loop of the function. The second slice should be initialized with the same number of elements as the slice used in the first for loop. Setting elements of the slices should be done by indexing them (not by append) in the second, third, and forth loop.
OK, I will submit a PR to implement it.

// 3. Ties are broken by sum of priorities of all victims. // 4. If there are still ties, node with the minimum number of victims is picked. // 5. If there are still ties, the first such node is picked (sort of randomly). <del> //TODO(bsalamat): Try to reuse the "min*Nodes" slices in order to save GC time. </del> <ins> // The 'minNodes1' and 'minNodes2' are being reused here to save the memory // allocation and garbage collection time. </ins> func pickOneNodeForPreemption(nodesToVictims map[*v1.Node]*Victims) *v1.Node { if len(nodesToVictims) == 0 { return nil } minNumPDBViolatingPods := math.MaxInt32 <del> var minPDBViolatingNodes []*v1.Node </del> <ins> var minNodes1 []*v1.Node lenNodes1 := 0 </ins> for node, victims := range nodesToVictims { if len(victims.pods) == 0 { // We found a node that doesn't need any preemption. Return it!


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: Fix the following : The basic logic in of using the is continuing reducing the nodes slices through , , and . It needs 4 slices to do this. This is my thought to reuse the slices: Use 2 nodes slices, such as , The reducing process becomes: -Clear -Clear - please give some suggestion, thanks :smile:
/sig scheduling
The challenge is that clearing a slice and reusing it still needs GC clean up, so it may not be effective. The size of slices shrink as we proceed through the function. So, having two slices and over-writing the elements in them can help reduce GC time, but this approach needs some additional checks to ensure we are not using stale elements.
To avoid clearing the slices, this is my thought: Declare to indicate the actual length of the 2 node slices, instead of . Appending a node to a node slice will add 1 to . Clearing a node slice will set to 0. WDYT
Yes, I can't think of any better solution either, but remember that you should append only to the slice in the first for loop of the function. The second slice should be initialized with the same number of elements as the slice used in the first for loop. Setting elements of the slices should be done by indexing them (not by append) in the second, third, and forth loop.
OK, I will submit a PR to implement it.

numPDBViolatingPods := victims.numPDBViolations if numPDBViolatingPods < minNumPDBViolatingPods { minNumPDBViolatingPods = numPDBViolatingPods <del> minPDBViolatingNodes = nil </del> <ins> minNodes1 = nil lenNodes1 = 0 </ins> } if numPDBViolatingPods == minNumPDBViolatingPods { <del> minPDBViolatingNodes = append(minPDBViolatingNodes, node) </del> <ins> minNodes1 = append(minNodes1, node) lenNodes1++ </ins> } } <del> if len(minPDBViolatingNodes) == 1 { return minPDBViolatingNodes[0] </del> <ins> if lenNodes1 == 1 { return minNodes1[0] </ins> } // There are more than one node with minimum number PDB violating pods. Find // the one with minimum highest priority victim. minHighestPriority := int32(math.MaxInt32) <del> var minPriorityNodes []*v1.Node for _, node := range minPDBViolatingNodes { </del> <ins> var minNodes2 = make([]*v1.Node, lenNodes1) lenNodes2 := 0 for i := 0; i < lenNodes1; i++ { node := minNodes1[i] </ins> victims := nodesToVictims[node] // highestPodPriority is the highest priority among the victims on this node. highestPodPriority := util.GetPodPriority(victims.pods[0]) if highestPodPriority < minHighestPriority { minHighestPriority = highestPodPriority <del> minPriorityNodes = nil </del> <ins> lenNodes2 = 0 </ins> } if highestPodPriority == minHighestPriority { <del> minPriorityNodes = append(minPriorityNodes, node) </del> <ins> minNodes2[lenNodes2] = node lenNodes2++ </ins> } } <del> if len(minPriorityNodes) == 1 { return minPriorityNodes[0] </del> <ins> if lenNodes2 == 1 { return minNodes2[0] </ins> } // There are a few nodes with minimum highest priority victim. Find the // smallest sum of priorities. minSumPriorities := int64(math.MaxInt64) <del> var minSumPriorityNodes []*v1.Node for _, node := range minPriorityNodes { </del> <ins> lenNodes1 = 0 for i := 0; i < lenNodes2; i++ { </ins> var sumPriorities int64 <ins> node := minNodes2[i] </ins> for _, pod := range nodesToVictims[node].pods { // We add MaxInt32+1 to all priorities to make all of them >= 0. This is // needed so that a node with a few pods with negative priority is not


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: Fix the following : The basic logic in of using the is continuing reducing the nodes slices through , , and . It needs 4 slices to do this. This is my thought to reuse the slices: Use 2 nodes slices, such as , The reducing process becomes: -Clear -Clear - please give some suggestion, thanks :smile:
/sig scheduling
The challenge is that clearing a slice and reusing it still needs GC clean up, so it may not be effective. The size of slices shrink as we proceed through the function. So, having two slices and over-writing the elements in them can help reduce GC time, but this approach needs some additional checks to ensure we are not using stale elements.
To avoid clearing the slices, this is my thought: Declare to indicate the actual length of the 2 node slices, instead of . Appending a node to a node slice will add 1 to . Clearing a node slice will set to 0. WDYT
Yes, I can't think of any better solution either, but remember that you should append only to the slice in the first for loop of the function. The second slice should be initialized with the same number of elements as the slice used in the first for loop. Setting elements of the slices should be done by indexing them (not by append) in the second, third, and forth loop.
OK, I will submit a PR to implement it.

} if sumPriorities < minSumPriorities { minSumPriorities = sumPriorities <del> minSumPriorityNodes = nil </del> <ins> lenNodes1 = 0 </ins> } if sumPriorities == minSumPriorities { <del> minSumPriorityNodes = append(minSumPriorityNodes, node) </del> <ins> minNodes1[lenNodes1] = node lenNodes1++ </ins> } } <del> if len(minSumPriorityNodes) == 1 { return minSumPriorityNodes[0] </del> <ins> if lenNodes1 == 1 { return minNodes1[0] </ins> } // There are a few nodes with minimum highest priority victim and sum of priorities. // Find one with the minimum number of pods. minNumPods := math.MaxInt32 <del> var minNumPodNodes []*v1.Node for _, node := range minSumPriorityNodes { </del> <ins> lenNodes2 = 0 for i := 0; i < lenNodes1; i++ { node := minNodes1[i] </ins> numPods := len(nodesToVictims[node].pods) if numPods < minNumPods { minNumPods = numPods <del> minNumPodNodes = nil </del> <ins> lenNodes2 = 0 </ins> } if numPods == minNumPods { <del> minNumPodNodes = append(minNumPodNodes, node) </del> <ins> minNodes2[lenNodes2] = node lenNodes2++ </ins> } } // At this point, even if there are more than one node with the same score, // return the first one. <del> if len(minNumPodNodes) > 0 { return minNumPodNodes[0] </del> <ins> if lenNodes2 > 0 { return minNodes2[0] </ins> } glog.Errorf("Error in logic of node scoring for preemption. We should never reach here!") return nil


Question:
http://k8s- /priority failing-test /kind bug /status approved-for-milestone /sig cluster-lifecycle the job is on the master-blocking dashboard and the e2e test has been failing, please take a look cc
testgrid change suggests
(author of the change in that range) -- any idea whether might have broken reboot tests?
I reverted the change as it may lead to this regression. The other change is really small () and unlikely related to this. Still the continues to say otherwise, I think we can reopen this issue.
The reboot test is still failing.
Just opened to achieve the revert.
Is the test passing now? Can this issue be closed?
looks green now

Status: v1.ConditionTrue, }) <del> hostIP, err := kl.getHostIPAnyWay() if err != nil { glog.V(4).Infof("Cannot get host IP: %v", err) return *s } s.HostIP = hostIP.String() if kubecontainer.IsHostNetworkPod(pod) && s.PodIP == "" { s.PodIP = hostIP.String() </del> <ins> if kl.kubeClient != nil { hostIP, err := kl.getHostIPAnyWay() if err != nil { glog.V(4).Infof("Cannot get host IP: %v", err) } else { s.HostIP = hostIP.String() if kubecontainer.IsHostNetworkPod(pod) && s.PodIP == "" { s.PodIP = hostIP.String() } } </ins> } return *s }


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: A Pod is run with local volume mounted in block mode. When that pod is deleted, the tear down process fails because device unmap fails. The device path appears to be empty, causing the unmap to fail. What you expected to happen: The teardown should have succeeded How to reproduce it (as minimally and precisely as possible): Create a pod with a PV for local volume in block mode. Delete the pod. Look at kubelet log files on the minion node to observe the error messages. Anything else we need to know?: Log messages: Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: E0218 09:21:43. 1510 ] Operation for """ ("-148c-11e8-b315-")" failed. No retries permitted until 2018-02-18 09:21:44. +0000 UTC m=+. (durationBeforeRetry 500ms). Error: "UnmapVolume.UnmapDevice on global map path failed for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") : Failed to unmap device from map path. mapPath is empty" Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: I0218 09:21:43. 1510 ] operationExecutor.UnmountVolume started for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") Environment: Kubernetes version (use ): 1.10+ Cloud provider or hardware configuration: GCE OS (e.g. from /etc/os-release): "Container-Optimized OS from Google" Kernel (e.g. ): Linux 4.4.86+ Install tools: Others:
/sig storage
This error is returned in this . It same that we set the global mount path to only if the volume plugin is attachable(see ). However, local volume plugin is not attachable, so will be empty and the operation will fail. I think we should extract the operation from the block, and execute this operation just before we try to map the device to global map path, then this global map path can be cached in for both attachable and non attachable volume plugins. Any comments? If not, I can help to fix this.
Thank you for the investigation. We've discussed this problem by offline yesterday. I proposed same fix which you mentioned above, then tested the fix works correctly with local volume plugin. I'll push a PR soon.
The device path appears to be empty, causing the unmap to fail. Please note, which was stored in is empty. device path isn't related to this problem.
Sorry about that, this is a typo, what I mean is global mount path.
No, no. My above comment is just note for the section. Your investigation is correct.

glog.Infof(volumeToMount.GenerateMsgDetailed("MapVolume.WaitForAttach succeeded", fmt.Sprintf("DevicePath %q", devicePath))) <del> // Update actual state of world to reflect volume is globally mounted markDeviceMappedErr := actualStateOfWorld.MarkDeviceAsMounted( volumeToMount.VolumeName, devicePath, globalMapPath) if markDeviceMappedErr != nil { // On failure, return error. Caller will log and retry. return volumeToMount.GenerateError("MapVolume.MarkDeviceAsMounted failed", markDeviceMappedErr) } </del> } // A plugin doesn't have attacher also needs to map device to global map path with SetUpDevice() pluginDevicePath, mapErr := blockVolumeMapper.SetUpDevice()


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: A Pod is run with local volume mounted in block mode. When that pod is deleted, the tear down process fails because device unmap fails. The device path appears to be empty, causing the unmap to fail. What you expected to happen: The teardown should have succeeded How to reproduce it (as minimally and precisely as possible): Create a pod with a PV for local volume in block mode. Delete the pod. Look at kubelet log files on the minion node to observe the error messages. Anything else we need to know?: Log messages: Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: E0218 09:21:43. 1510 ] Operation for """ ("-148c-11e8-b315-")" failed. No retries permitted until 2018-02-18 09:21:44. +0000 UTC m=+. (durationBeforeRetry 500ms). Error: "UnmapVolume.UnmapDevice on global map path failed for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") : Failed to unmap device from map path. mapPath is empty" Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: I0218 09:21:43. 1510 ] operationExecutor.UnmountVolume started for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") Environment: Kubernetes version (use ): 1.10+ Cloud provider or hardware configuration: GCE OS (e.g. from /etc/os-release): "Container-Optimized OS from Google" Kernel (e.g. ): Linux 4.4.86+ Install tools: Others:
/sig storage
This error is returned in this . It same that we set the global mount path to only if the volume plugin is attachable(see ). However, local volume plugin is not attachable, so will be empty and the operation will fail. I think we should extract the operation from the block, and execute this operation just before we try to map the device to global map path, then this global map path can be cached in for both attachable and non attachable volume plugins. Any comments? If not, I can help to fix this.
Thank you for the investigation. We've discussed this problem by offline yesterday. I proposed same fix which you mentioned above, then tested the fix works correctly with local volume plugin. I'll push a PR soon.
The device path appears to be empty, causing the unmap to fail. Please note, which was stored in is empty. device path isn't related to this problem.
Sorry about that, this is a typo, what I mean is global mount path.
No, no. My above comment is just note for the section. Your investigation is correct.

return volumeToMount.GenerateError("MapVolume failed", fmt.Errorf("Device path of the volume is empty")) } } <ins> // Update actual state of world to reflect volume is globally mounted markDeviceMappedErr := actualStateOfWorld.MarkDeviceAsMounted( volumeToMount.VolumeName, devicePath, globalMapPath) if markDeviceMappedErr != nil { // On failure, return error. Caller will log and retry. return volumeToMount.GenerateError("MapVolume.MarkDeviceAsMounted failed", markDeviceMappedErr) } </ins> mapErr = og.blkUtil.MapDevice(devicePath, globalMapPath, string(volumeToMount.Pod.UID)) if mapErr != nil {


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: A Pod is run with local volume mounted in block mode. When that pod is deleted, the tear down process fails because device unmap fails. The device path appears to be empty, causing the unmap to fail. What you expected to happen: The teardown should have succeeded How to reproduce it (as minimally and precisely as possible): Create a pod with a PV for local volume in block mode. Delete the pod. Look at kubelet log files on the minion node to observe the error messages. Anything else we need to know?: Log messages: Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: E0218 09:21:43. 1510 ] Operation for """ ("-148c-11e8-b315-")" failed. No retries permitted until 2018-02-18 09:21:44. +0000 UTC m=+. (durationBeforeRetry 500ms). Error: "UnmapVolume.UnmapDevice on global map path failed for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") : Failed to unmap device from map path. mapPath is empty" Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: I0218 09:21:43. 1510 ] operationExecutor.UnmountVolume started for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") Environment: Kubernetes version (use ): 1.10+ Cloud provider or hardware configuration: GCE OS (e.g. from /etc/os-release): "Container-Optimized OS from Google" Kernel (e.g. ): Linux 4.4.86+ Install tools: Others:
/sig storage
This error is returned in this . It same that we set the global mount path to only if the volume plugin is attachable(see ). However, local volume plugin is not attachable, so will be empty and the operation will fail. I think we should extract the operation from the block, and execute this operation just before we try to map the device to global map path, then this global map path can be cached in for both attachable and non attachable volume plugins. Any comments? If not, I can help to fix this.
Thank you for the investigation. We've discussed this problem by offline yesterday. I proposed same fix which you mentioned above, then tested the fix works correctly with local volume plugin. I'll push a PR soon.
The device path appears to be empty, causing the unmap to fail. Please note, which was stored in is empty. device path isn't related to this problem.
Sorry about that, this is a typo, what I mean is global mount path.
No, no. My above comment is just note for the section. Your investigation is correct.

// Try to unmap podUID symlink under global map path dir // plugins/kubernetes.io/{PluginName}/volumeDevices/{volumePluginDependentPath}/{podUID} globalUnmapPath := volumeToUnmount.DeviceMountPath <del> if err != nil { // On failure, return error. Caller will log and retry. return volumeToUnmount.GenerateError("UnmapVolume.GetGlobalUnmapPath failed", err) } </del> unmapDeviceErr = og.blkUtil.UnmapDevice(globalUnmapPath, string(volumeToUnmount.PodUID)) if unmapDeviceErr != nil { // On failure, return error. Caller will log and retry.


Question:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: A Pod is run with local volume mounted in block mode. When that pod is deleted, the tear down process fails because device unmap fails. The device path appears to be empty, causing the unmap to fail. What you expected to happen: The teardown should have succeeded How to reproduce it (as minimally and precisely as possible): Create a pod with a PV for local volume in block mode. Delete the pod. Look at kubelet log files on the minion node to observe the error messages. Anything else we need to know?: Log messages: Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: E0218 09:21:43. 1510 ] Operation for """ ("-148c-11e8-b315-")" failed. No retries permitted until 2018-02-18 09:21:44. +0000 UTC m=+. (durationBeforeRetry 500ms). Error: "UnmapVolume.UnmapDevice on global map path failed for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") : Failed to unmap device from map path. mapPath is empty" Feb 18 09:21:43 kubernetes-minion-group-crj1 kubelet[1510]: I0218 09:21:43. 1510 ] operationExecutor.UnmountVolume started for volume "blkdata" (UniqueName: "") pod "-148c-11e8-b315-" (UID: "-148c-11e8-b315-") Environment: Kubernetes version (use ): 1.10+ Cloud provider or hardware configuration: GCE OS (e.g. from /etc/os-release): "Container-Optimized OS from Google" Kernel (e.g. ): Linux 4.4.86+ Install tools: Others:
/sig storage
This error is returned in this . It same that we set the global mount path to only if the volume plugin is attachable(see ). However, local volume plugin is not attachable, so will be empty and the operation will fail. I think we should extract the operation from the block, and execute this operation just before we try to map the device to global map path, then this global map path can be cached in for both attachable and non attachable volume plugins. Any comments? If not, I can help to fix this.
Thank you for the investigation. We've discussed this problem by offline yesterday. I proposed same fix which you mentioned above, then tested the fix works correctly with local volume plugin. I'll push a PR soon.
The device path appears to be empty, causing the unmap to fail. Please note, which was stored in is empty. device path isn't related to this problem.
Sorry about that, this is a typo, what I mean is global mount path.
No, no. My above comment is just note for the section. Your investigation is correct.

// Search under globalMapPath dir if all symbolic links from pods have been removed already. // If symbolick links are there, pods may still refer the volume. globalMapPath := deviceToDetach.DeviceMountPath <del> if err != nil { // On failure, return error. Caller will log and retry. return deviceToDetach.GenerateError("UnmapDevice.GetGlobalMapPath failed", err) } </del> refs, err := og.blkUtil.GetDeviceSymlinkRefs(deviceToDetach.DevicePath, globalMapPath) if err != nil { return deviceToDetach.GenerateError("UnmapDevice.GetDeviceSymlinkRefs check failed", err)


Question:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: The deployment controller for a deployment with a single replica failed to create a new ReplicaSet when the old ReplicaSet included an Evicted pod. Typically, the eviction was caused by "node low on resource: nodefs". When we manually deleted the evicted pod, the new RS and accompanying pod were created immediately. We have seen this happen at least a dozen times. Here are the audit logs for a recent case: ! What you expected to happen: The deployment controller ignores the evicted pod and creates the updated replicaset/pod. How to reproduce it (as minimally and precisely as possible): To reproduce, you would need a way to trigger a kubelet-driven eviction on a node that is currently running a single-replica Deployment with . Wait for the pod to be evicted and the new pod to be created, confirm that the evicted pod still exists, and then make a trivial change to the deployment spec. The running pod will be deleted, and no new replicaset/pod will be created. This issue does not seem to occur for manual evictions (via ); pods I evict that way disappear from the API server immediately. Anything else we need to know?: Possibly related to and/or Environment: Kubernetes version (use ): v1.8.5-gke.0 Cloud provider or hardware configuration: GKE OS (e.g. from /etc/os-release): Container-Optimized OS
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Thanks for the detailed bug report! From Here's what happened: Deployment was updated by the user, so it scaled all old ReplicaSets to 0 old ReplicaSet was scaled to 0 (ReplicaSet's == 0), but still owned the evicted pod, which wouldn't be killed automatically because of the above comment (ReplicaSet's 0) Deployment found that there were still old Pods, and kept waiting for them to be gone ~(wait for old ReplicaSet's to be 0)~ (Edit: Deployment also checks those old Pods by listing them to make sure there are none, not just by checking ReplicaSet's status) before starting the new rollout. We can either modify Deployment controller (to ignore evicted old pods) or ReplicaSet controller (to kill evicted pods ~when it gets scaled down~), if this is not the intended behavior. WDYT?
I am actually debugging likely the same production issue with Recreate strategy in two clusters right now. (We run on patched 1.9.1.) I have traced it down to broken RS as well. in 1 cluster RS leaves around pod failed matching the selector in the other one those are evicted pods. all replicas are scaled down to 0 and report 0 in status but pods with valid controllerRefs to them are still present. Same is 0 for me but pod is still there I think the correct behaviour is for replicaSet scaled to 0 to delete all of its pod, not to leave them laying around
xref - this should fix it

if newRS != nil && newRS.UID == rsUID { continue } <del> if len(podList.Items) > 0 { return true </del> <ins> for _, pod := range podList.Items { switch pod.Status.Phase { case v1.PodFailed, v1.PodSucceeded: // Don't count pods in terminal state. continue case v1.PodUnknown: // This happens in situation like when the node is temporarily disconnected from the cluster. // If we can't be sure that the pod is not running, we have to count it. return true default: // Pod is not in terminal phase. return true } </ins> } } return false