Question: ip := net.IP(ipData) maskData, err := hex.DecodeString(parts[1]) <ins> if err != nil { return "", err } </ins> mask := net.IPMask(maskData) size, _ := mask.Size()

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): hexCIDR and CIDR Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): latest on master Environment: Cloud provider or hardware configuration: Bare-metal, 3 node config using KubeAdm OS (e.g. from /etc/os-release): CentOS 7 Kernel (e.g. ): Linux devstack-77 3.10.0-514.10.2.el7.x8664 SMP Fri Mar 3 00:04:05 UTC 2017 x8664 x8664 x8664 GNU/Linux Install tools: KubeAdm Others: What happened: Investigating the code, hexCIDR() expects the CIDR passed to be for an IPv4 address and doesn't support an IPv6 CIDR. What you expected to happen: Support either IPv4 or IPv6 CIDRs. How to reproduce it (as minimally and precisely as possible): Anything else we need to know: Will be working on a solution for this. Note: In some of the call paths, a PodIP string is used and passed in as a CIDR with a "/32" appended. At some point, this would need to be updated as well, to use "/128" if an IPv6 PodIP was used.
I see there is an asciiCIDR() method too that needs addressing, but only in the UTs.


Question: ip := net.IP(ipData) maskData, err := hex.DecodeString(parts[1]) <ins> if err != nil { return "", err } </ins> mask := net.IPMask(maskData) size, _ := mask.Size()

Appears that it is expecting to reverse hex back to string, but test is not showing case of mask being properly applied. For example, "2001::5/64" when converted to hex and then back to string will be "2001::/64". Will update test to be more accurate.
/area networking
These labels do not exist in this repository: . <detailsIn response to : /area networking Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. I understand the commands that are listed . </details
/sig network
One question I have... Can the callers (bandwidth?) of the function work with an IPv6 based hex CIDR? In the future, if we support pod IPs that are IPv6 addresses, and we modify the code to add /128 vs /32, as is done currently, this would create an IPv6 based hex CIDR.
The only caller is currently the ingress/egress shaping function in kubelet. That code also needs to be updated for IPv6. For example, assumes IPv4 when adding CIDRs to the shaper list.
Agree.I wanted to separate this out, partly to make the change smaller, but also, when I looked at the callers (this shaping functionality) I wasn't sure if it would support IPv6. I looked a bit at the "tc" command and saw some notes that indicate that it doesn't support IPv6 yet. Is that still accurate? Would it be ok to tackle that separately?


Question: glog.V(vLevel).Infof("Image garbage collection succeeded") } <del> }, ImageGCPeriod, stopChan) </del> <ins> }, ImageGCPeriod, wait.NeverStop) </ins> } // initializeModules will initialize internal modules that do not require the container runtime to be up.

In the function, there is a that always runs and closes the channel when returning. This means that instead of running periodically, the image garbage collection will be terminated immediately when the function returns. The bug was introduced by in 1.11 /cc
[MILESTONENOTIFIER] Milestone Issue: Up-to-date for process <details open<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
BTW, when the threshold is set to 100% (the use case that the PR originally wanted to fix), the code'd try sending to a closed channel and triggers a panic.
i will check it soon, will submit one patch immediately, thanks
does this bug applies to v1.11.2 ? or v.1.11.2 includes the fix?


Question: t.Errorf("expected mount options: %v, got: %v", tc.spec.PersistentVolume.Spec.MountOptions, vol.MountFlags) } } <ins> // Verify the deviceMountPath was created by the plugin if tc.stageUnstageSet { s, err := os.Stat(tc.deviceMountPath) if err != nil { t.Errorf("expected staging directory %s to be created and be a directory, got error: %s", tc.deviceMountPath, err) } else { if !s.IsDir() { t.Errorf("expected staging directory %s to be directory, got something else", tc.deviceMountPath) } } } </ins> }) } }

What happened: A custom CSI driver implementing returns a failure response, indicating that it did not complete its intended steps to prepare the volume for the mounting pod. Let's say that these steps involve a filesystem at the target directory, data from a service, and that data to a set of files in the mounted filesystem. If the first filesystem mounting step succeeds, but either of the subsequent two steps fail, the volume is still not ready for use by the pod. However, if this CSI driver returns a failure response, and the CSI mounter comes back and, before invoking the RPC again, it finds a filesystem already mounted at the target directory, , and ceases further calls to to allow the CSI driver to try again. What you expected to happen: If a CSI driver returns a failure response to a RPC, Kubernetes should honor that declaration of failure and invoke the RPC again, disallowing the mounting pod from starting until this RPC succeeds. Even though it's possible to partially work around the current behavior by having the driver mount the filesystem as late as possible, and then attempting to unmount it if fails to populate the volume with its intended data, doing so makes writing drivers more complicated and forces them to do extra work. How to reproduce it (as minimally and precisely as possible): In a CSI driver (even one that serves just ephemeral, inline volumes), implement to mount a filesystem at the target path, then return a failure response. Confirm that Kubernetes does not invoke again, because there's now a filesystem mounted at the target path. Anything else we need to know?: This topic came up for discussion in the "Kubernetes" Slack team. There, helped investigate this complaint, and to discuss changing this behavior. Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS EC2 OS (e.g: ):Kernel (e.g. ):Install tools: kubeadm Others: csi-node-driver-registrar container image tag: v1.0.2
/sig storage
While the plugin should try to clean up as much as possible, I agree that kubelet should not be assuming that if there's a mount point, the publish is successful. NodePublish is supposed to be idempotent so it should theoretically be safe to call it when it's already mounted, although there is the possibility that plugins may have written against the K8s behavior.


Question: t.Errorf("expected mount options: %v, got: %v", tc.spec.PersistentVolume.Spec.MountOptions, vol.MountFlags) } } <ins> // Verify the deviceMountPath was created by the plugin if tc.stageUnstageSet { s, err := os.Stat(tc.deviceMountPath) if err != nil { t.Errorf("expected staging directory %s to be created and be a directory, got error: %s", tc.deviceMountPath, err) } else { if !s.IsDir() { t.Errorf("expected staging directory %s to be directory, got something else", tc.deviceMountPath) } } } </ins> }) } }

For the mount point checking here before calling NodePublishVolume, I am wondering it is just for performance optimization, or it has other purposes. Have concern about some drivers might depend on this check if they are not implemented with idenpotency.
Those drivers would be operating in violation of the specification. I'd prefer that we don't penalize already conforming and yet-to-be written drivers in order to accommodate these existing drivers—if they even exist.
More fallout in last night's testing: I introduced a bug into my driver that caused it to panic during ("Writing that data to a set of files in the mounted filesystem"). My handler function is set up to unmount a successfully mounted volume via if it doesn't complete steps 2 and 3 above, in order to deal with this issue's principal complaint. My driver container would crash due to this panic. The kubelet would call on my driver twice, it would crash twice, but the pod still started running. I'm not yet sure whether or not the filesystem got unmounted successfully via the on the first call, but somehow the pod started despite my handler function never completing successfully. I'm going to try to get some more debugging time on this later today to see if I can figure out what the kubelet was doing there.
Studying my code more, I see that my function only unmounted the mounted directory when the containing handler function was returning an error. When unwinding due to a panic, the named return value of type I was inspecting was still nil. Revising the criteria in that function covers both failure reasons (returning an and unwinding due to a panic). Regardless, this further underscores the problem: If a driver makes it as far as mounting a filesystem at the target directory, but fails afterward—including the process crashing and never responding positively to the RPC—the mounting pod will start. That's likely to then cause the pod to fail, for lack of the files it expected to find mounted.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity.


Question: t.Errorf("expected mount options: %v, got: %v", tc.spec.PersistentVolume.Spec.MountOptions, vol.MountFlags) } } <ins> // Verify the deviceMountPath was created by the plugin if tc.stageUnstageSet { s, err := os.Stat(tc.deviceMountPath) if err != nil { t.Errorf("expected staging directory %s to be created and be a directory, got error: %s", tc.deviceMountPath, err) } else { if !s.IsDir() { t.Errorf("expected staging directory %s to be directory, got something else", tc.deviceMountPath) } } } </ins> }) } }

Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale


Question: }, func(client kubeclientset.Interface, obj runtime.Object) error { rs := obj.(*extensionsv1.ReplicaSet) <del> err := client.Extensions().ReplicaSets(rs.Namespace).Delete(rs.Name, &metav1.DeleteOptions{}) </del> <ins> orphanDependents := false err := client.Extensions().ReplicaSets(rs.Namespace).Delete(rs.Name, &metav1.DeleteOptions{OrphanDependents: &orphanDependents}) </ins> return err })

Problem: When users sends a cascading deletion request to federation i.e they set orphanDependents=false, then federation should also set orphanDependents=false, when it sends the request to underlying clusters.
Fixed by


Question: <ins> apiVersion: "v1beta1" kind: "Service" id: "monitoring-influxdb" port: 8085 containerPort: 8086 selector: name: "influxGrafana" </ins>

examples/monitoring/influx-grafana- and friends NEED to be replaced by a replication controller ASAP. As it stands they are bad examples. Also, they should probably move to contrib/monitoring to match contrib/logging and contrib/dns


Question: KUBE_PROXY_OPTS=" --hostname-override=${1}  --master=http://${2}:8080  <del> --logtostderr=true" </del> <ins> --logtostderr=true  ${3}" </ins> EOF }

There should be a way to select the iptables proxy-mode.
Are you interested in modifying ubuntu deploy scripts to use a user provided configure file?
This is fixed by and it will be merged in soon.
This fix breaks cluster/kube- while kubernetes provider is rather than ubuntu, as the following variables are only defined in ubuntu folder PROXYSETTING=${PROXYSETTING:-""} DEBUG=${DEBUG:-"false"} When I run KUBERNETES_PROVIDER=vagrant and run cluster/kube- It will fail with error "==master: [ERROR ] {'pid': 5871, 'retcode': 1, 'stderr': ' line 21: DEBUG: unbound variable', 'stdout': ''}" I will open a separated issue to discuss the solution.
I do not understand. I have not tried KUBERNETESPROVIDER=vagrant and run cluster/kube-, can you please clarify what is going wrong? This PR () affected only files in cluster/ubuntu/; are you saying that some of them are used when KUBERNETESPROVIDER=vagrant? This PR did not introduce PROXY_SETTING or DEBUG.
I tried running kube- with the Vagrant provider, but ran into a different (and much prior) problem; see --- any help would be appreciated.
OK, I see you opened issue for this and PR fixed it. Thanks. I will respond to the larger question under .


Question: echo "Found multiple ephemeral block devices, formatting with btrfs as RAID-0" mkfs.btrfs -f --data raid0 ${block_devices[@]} fi <del> echo "${block_devices[0]}  /mnt/ephemeral  btrfs  noatime  0 0" >> /etc/fstab </del> <ins> echo "${block_devices[0]}  /mnt/ephemeral  btrfs  noatime,nofail  0 0" >> /etc/fstab </ins> mkdir -p /mnt/ephemeral mount /mnt/ephemeral

On AWS we (will) now run our bootstrap script on every boot. The function to initialize ephemeral volumes doesn't tolerate being re-run, and we probably shouldn't re-run it anyway for speed. This does preserve the contents, which may be a good thing to save user's data (it doesn't belong on ephemeral drives, but...). On the other hand, if the ephemeral drives are corrupted, we lose the opportunity to reformat them and recover. Arguably, if the ephemeral storage is corrupted the right thing to do is to terminate the instance entirely.


Question: k := kubelet.NewMainKubelet( getHostname(), dockerClient, <del> cadvisorClient, </del> etcdClient, *rootDirectory, *networkContainerImage,

So today, cadvisor became wedged and just stopped paying attention to its socket. Not kubelet's fault. But when I would launch all I would get would be: It took me a bit of trolling through netstat and lsof to figure out that the problem was cadvisor being completely locked up. kill -9 cadvisor and everything started working well. I wonder if we should emit more messages during startup about what operation we are about to attempt, not just the results of operations, as I was blaming docker for a while....
kubelet shouldn't block on this. I'll take a look.
Any ideas what was locking cAdvisor? and was it just not responding to requests?
I don't have any idea why it was locked. If I find another one locked I'll hang onto it to debug. Journal had: obviously the 'kill' came from me. It would not respond to systemctl cadvisor stop. netstat looking at port 4194 showed me At which point I realized cadvisor wasn't handling its sockets right since so many long dead kubelet sockets hadn't been cleaned up, so I kill -9'd it.
I cut some interesting info from the cadvisor journal log (trying to cut hostnames) You can see the last message about failed to update stats for container ... happened Oct 08 01:06:01. It was then silent until I killed it Oct 08 19:20:13, even though I'm pretty sure dnf was still being launched every hour on the hour. So it was wedged for about 18 hours.
What kernel is this on? We've seen there is a kernel bug fixed in 3.12+ that sometimes hangs us forever on exit. We're not quite sure how it gets triggered...
16.3


Question: return nil, fmt.Errorf("%s does not support block volume provisioning", a.plugin.GetPluginName()) } <del> var sku, resourceGroup, location, account string </del> <ins> var sku, resourceGroup, location, account, shareName string </ins> <del> // File share name has a length limit of 63, and it cannot contain two consecutive '-'s. name := util.GenerateVolumeName(a.options.ClusterName, a.options.PVName, 63) name = strings.Replace(name, "--", "-", -1) </del> capacity := a.options.PVC.Spec.Resources.Requests[v1.ResourceName(v1.ResourceStorage)] requestGiB := int(volumehelpers.RoundUpToGiB(capacity)) secretNamespace := a.options.PVC.Namespace

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: use an existing azure file share name should be possible in azure file plugin What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig azure /assign


Question: } // In IPVS proxy mode, the following flags need to be set <del> const sysctlRouteLocalnet = "net/ipv4/conf/all/route_localnet" </del> const sysctlBridgeCallIPTables = "net/bridge/bridge-nf-call-iptables" const sysctlVSConnTrack = "net/ipv4/vs/conntrack" const sysctlConnReuse = "net/ipv4/vs/conn_reuse_mode"

CVSS Rating: In typical clusters: medium (5.4) In clusters where API server insecure port has not been disabled: high (8.8) A security issue was discovered in which allows adjacent hosts to reach TCP and UDP services bound to running on the node or in the node's network namespace. For example, if a cluster administrator runs a TCP service on a node that listens on , because of this bug, that service would be potentially reachable by other hosts on the same LAN as the node, or by containers running on the same node as the service. If the example service on port required no additional authentication (because it assumed that only other localhost processes could reach it), then it could be vulnerable to attacks that make use of this bug. The Kubernetes API Server's default insecure port setting causes the API server to listen on where it will accept requests without authentication. Many Kubernetes installers explicitly disable the API Server's insecure port, but in clusters where it is not disabled, an attacker with access to another system on the same LAN or with control of a container running on the master may be able to reach the API server and execute arbitrary API requests on the cluster. This port is deprecated, and will be removed in Kubernetes v1.20. You may be vulnerable if: You are running a vulnerable version (see below) Your cluster nodes run in an environment where untrusted hosts share the same layer 2 domain (i.e. same LAN) as nodes Your cluster allows untrusted pods to run containers with (the Kubernetes default is to allow this capability). Your nodes (or hostnetwork pods) run any localhost-only services which do not require any further authentication. To list services that are potentially affected, run the following commands on nodes: - On a master node, an lsof entry like this indicates that the API server may be listening with an insecure port: kubelet/kube-proxy v1.18.0-1.18.3 kubelet/kube-proxy v1.17.0-1.17.6 kubelet/kube-proxy <=1.16.10 Prior to upgrading, this vulnerability can be mitigated by manually adding an iptables rule on nodes. This rule will reject traffic to 127.0.0.1 which does not originate on the node. Additionally, if your cluster does not already have the API Server insecure port disabled, we strongly suggest that you disable it.


Question: } // In IPVS proxy mode, the following flags need to be set <del> const sysctlRouteLocalnet = "net/ipv4/conf/all/route_localnet" </del> const sysctlBridgeCallIPTables = "net/bridge/bridge-nf-call-iptables" const sysctlVSConnTrack = "net/ipv4/vs/conntrack" const sysctlConnReuse = "net/ipv4/vs/conn_reuse_mode"

Add the following flag to your kubernetes API server command line: Packets on the wire with an IPv4 destination in the range 127.0.0.0/8 and a layer-2 destination MAC address of a node may indicate that an attack is targeting this vulnerability. Although the issue is caused by , the current fix for the issue is in (although future versions may have the fix in instead). We recommend updating both and to be sure the issue is addressed. The following versions contain the fix: kubelet/kube-proxy master - fixed by kubelet/kube-proxy v1.18.4+ - fixed by kubelet/kube-proxy v1.17.7+ - fixed by kubelet/kube-proxy v1.16.11+ - fixed by To upgrade, refer to the documentation: This issue was originally raised in issue which details how the sets which causes the system not to reject traffic to localhost which originates on other hosts. IPv6-only services that bind to a address are not affected. There may be additional attack vectors possible in addition to those fixed by and its cherry-picks. For those attacks to succeed, the target service would need to be UDP and the attack could only rely upon sending UDP datagrams since it wouldn't receive any replies. Finally, the target node would need to have reverse-path filtering disabled for an attack to have any effect. Work is ongoing to determine whether and how this issue should be fixed. See for up-to-date status on this issue. This vulnerability was reported by János Kövér, Ericsson with additional impacts reported by Rory McCune, NCC Group and Yuval Avrahami and Ariel Zelivansky, Palo Alto Networks. /area security /kind bug /committee product-security /sig network /sig node /area kubelet
For insecure port removal:
The original announcement contained an error. The announcement makes it sound like upgrading kube-proxy will address the issue, but that is incorrect. I have edited the issue summary to add the following paragraph in the "Fixed Versions" section: Additionally, I have changed the components in listed versions to say kubelet/kube-proxy instead of kube-proxy.
Could you add numerical CVSS scores to these? "High" is too vague, frequently varies from vendor to vendor. I can calculate a score from the vector myself, but adding the score would save us some work...
Edited.


Question: } // In IPVS proxy mode, the following flags need to be set <del> const sysctlRouteLocalnet = "net/ipv4/conf/all/route_localnet" </del> const sysctlBridgeCallIPTables = "net/bridge/bridge-nf-call-iptables" const sysctlVSConnTrack = "net/ipv4/vs/conntrack" const sysctlConnReuse = "net/ipv4/vs/conn_reuse_mode"

Also, the vector scores are clickable and will show the CVSS scores too.
To be clear, this is fixed in v1.18.4+, v1.17.7+, and v1.16.11+ correct?
Right. Sorry if the extra paragraph in "Fixed Versions" made that unclear.
Got it. Thank you!
So just to be sure, this wouldn't be a problem in an EKS cluster running in its own VPC, right? (assuming no untrusted hosts on the same VPC)
Are all the conditions listed under , a logical AND ?
Edit: after more testing, I think is indeed required
The attack can be done from any pod with (this is still K8S default), so you might not be safe. a more accurate title: "Node setting allows to bypass localhost boundary"
We believe that of the four conditions, you might be affected if (1 and (2 or 3) and 4) are true for your cluster. Sorry that it's not super clear from the text. Trying and failing to convey info concisely and with precision makes me glad that I'm in software, not law.
Isn't this broadly fixed by ? We're discussing better answers and options for deployment, too, but I am curious what the close-condition for this bug is?
I made a typo in the issue description saying that this is addressed by instead of . I'll edit it now. Sorry about that. Yes, this is mostly fixed by . I marked this as closed due to that PR fixing the majority of security issues caused by the setting. We can re-open if people think that we should address the remaining case that talks about.
Sorry, folks. I thought that this issue was already closed. Maybe that makes my response above to less confusing.
Yes, this being open was one of the major points of confusion for us. Appreciate the clarification
/label official-cve-feed (Related to )


Question: } dc := &DeploymentController{ <del> queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "doesnt-matter"), </del> <ins> queue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "doesnt_matter"), </ins> } dc.enqueueDeployment = dc.enqueue

What happened: From : What you expected to happen: Metrics should register successfully. How to reproduce it (as minimally and precisely as possible): Run kubernetes-integration test Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
There are no sig labels on this issue. Please add a sig label by either: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. See the . The in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: pollInterval           = time.Second * 5 ) <del> type testStatus struct { </del> <ins> type testCase struct { </ins> Name             string RestartPolicy    api.RestartPolicy Phase            api.PodPhase

Slow conformance test has caused issues like We should make runtime/image conformance test run faster without breaking the test coverage, possibly by: Shorten consistently check interval . Remove redundant test cases such as ... /cc


Question: containerName := p.ContainerName if len(containerName) == 0 { if len(pod.Spec.Containers) > 1 { <del> usageString := fmt.Sprintf("Defaulting container name to %s.", pod.Spec.Containers[0].Name) if len(p.SuggestedCmdUsage) > 0 { usageString = fmt.Sprintf("%sn%s", usageString, p.SuggestedCmdUsage) </del> <ins> fmt.Fprintf(p.ErrOut, "Defaulting container name to %s.n", pod.Spec.Containers[0].Name) if p.EnableSuggestedCmdUsage { fmt.Fprintf(p.ErrOut, "Use '%s describe pod/%s -n %s' to see all of the containers in this pod.n", p.ParentCommandName, pod.Name, p.Namespace) </ins> } <del> fmt.Fprintf(p.ErrOut, "%sn", usageString) </del> } containerName = pod.Spec.Containers[0].Name }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: The pod name is missing in suggested cmd usage when executing : What you expected to happen: The suggested cmd usage should be a complete cmd like the one got from How to reproduce it (as minimally and precisely as possible): Use to attach any pod/deployment/service, the PodName will be missing in the suggested cmd usage like above. Anything else we need to know?: It works fine for kubectl < v1.12 Environment: Kubernetes version (use ): v1.13.1 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig cli /assign


Question: // Update uncertain volumes - the new markVolumeOpts may have updated information. // Especially reconstructed volumes (marked as uncertain during reconstruction) need // an update. <del> updateUncertainVolume = utilfeature.DefaultFeatureGate.Enabled(features.SELinuxMountReadWriteOncePod) && podObj.volumeMountStateForPod == operationexecutor.VolumeMountUncertain </del> <ins> updateUncertainVolume = utilfeature.DefaultFeatureGate.Enabled(features.NewVolumeManagerReconstruction) && podObj.volumeMountStateForPod == operationexecutor.VolumeMountUncertain </ins> } if !podExists || updateUncertainVolume { // Add new mountedPod or update existing one.

Kubernetes 1.27 with feature gate enabled and disabled, a rebooted node is not able to re-start pods that were running there before the reboot. The node reports in Pod events: The Pod should start just fine. On Kubernetes 1.27: NewVolumeManagerReconstruction feature gate. a StatefulSet that uses a volume. -reboot the node which runs the StatefulSet Pod. The reason is that kubelet's VolumeManager reconstructs the volume as uncertain during startup and it uses a provisional outer volume name for it, because it has not seen the API server and Pods in it yet. This provisional volume ID is then used in ActualStateOfWorld forever. While kubelet does mount the volume after reboot, it's not able to match it with the Pod that needs it. Kubelet reports , while the volume is mounted with ID in my case (= the provisional name, we use for it) and not (=). It is not reproducible with enabled. <details</details<detailsAWS + AWS EBS CSI driver, but it should not be relevant. </details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage /priority important-soon This is reproducible when SELinuxMountReadWriteOncePod is disabled and NewVolumeManagerReconstruction enabled. They both got enabled by default as beta in 1.28. So someone has to explicitly disable SELinuxMountReadWriteOncePod to hit this bug.


Question: switch { case strings.HasPrefix(subs[0][2], "ci"): <del> // Special case. CI images populated only by ci-cross area urlSuffix = "ci-cross" </del> <ins> // Just use whichever the user specified urlSuffix = subs[0][2] </ins> default: urlSuffix = "release" }

https://k8s- /kind bug /sig cluster-lifecycle /milestone v1.10 /priority failing-test cc also cc should this even be in 1.10 blocking dashboard?
/priority critical-urgent (who owns this?)
(also https://k8s-)
Is this actually running now? /cc
It looks like it's failing to up the cluster, I don't remember what the previous status of this job was, I thought all kubeadm 1.10 jobs were running previously...
The reason this failed was the cross CI job was failing to push the images that are used by that job. /cc
/subscribe
the latest cross build for 1.9 is as following We seem to have only one job that does cross builds on master : https://k8s-*build&i=nope&files=&repos=test-infra So the job fails ( for example in )
/status approved-for-milestone
[MILESTONENOTIFIER] Milestone Issue: Up-to-date for process <details open<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details


Question: if err != nil { return admission.NewForbidden(a, err) } <ins> // refuse to operate on non-existent namespaces </ins> if !exists { <del> return nil </del> <ins> // in case of latency in our caches, make a call direct to storage to verify that it truly exists or not namespaceObj, err = l.client.Namespaces().Get(a.GetNamespace()) if err != nil { return admission.NewForbidden(a, fmt.Errorf("Namespace %s does not exist", a.GetNamespace())) } </ins> } <del> namespace := namespaceObj.(*api.Namespace) if namespace.Status.Phase != api.NamespaceTerminating { return nil </del> <ins> // ensure that we're not trying to create objects in terminating namespaces if a.GetOperation() == admission.Create { namespace := namespaceObj.(*api.Namespace) if namespace.Status.Phase != api.NamespaceTerminating { return nil } return admission.NewForbidden(a, fmt.Errorf("Unable to create new content in namespace %s because it is being terminated.", a.GetNamespace())) </ins> } <del> return admission.NewForbidden(a, fmt.Errorf("Unable to create new content in namespace %s because it is being terminated.", a.GetNamespace())) </del> <ins> return nil </ins> } // NewLifecycle creates a new namespace lifecycle admission control handler

We should have a single NamespaceLifecycle plugin that enforces the Namespace rules now that all providers are off NamespaceAutoProvision. We may also want to make this no longer a user choice to configure and hard-wire the server to always run this check first to simplify configuration errors. For an example


Question: name = "go_default_library", srcs = [ "aws_ebs.go", <ins> "azure_disk.go", </ins> "gce_pd.go", "in_tree_volume.go", "openstack_cinder.go",

<!-- Please only use this template for submitting enhancement requests --What would you like to be : Azure is not listed as "migratable" in could refer to Why is this needed:
/sig azure /assign


Question: var ( inTreePlugins = map[string]plugins.InTreePlugin{ <del> plugins.GCEPDDriverName:  plugins.NewGCEPersistentDiskCSITranslator(), plugins.AWSEBSDriverName: plugins.NewAWSElasticBlockStoreCSITranslator(), plugins.CinderDriverName: plugins.NewOpenStackCinderCSITranslator(), </del> <ins> plugins.GCEPDDriverName:     plugins.NewGCEPersistentDiskCSITranslator(), plugins.AWSEBSDriverName:    plugins.NewAWSElasticBlockStoreCSITranslator(), plugins.CinderDriverName:    plugins.NewOpenStackCinderCSITranslator(), plugins.AzureDiskDriverName: plugins.NewAzureDiskCSITranslator(), </ins> } )


Question: metav1.TypeMeta `json:",inline"` // Standard list metadata // +optional <del> metav1.ListMeta `json:"listMeta" protobuf:"bytes,1,opt,name=listMeta"` </del> <ins> metav1.ListMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"` </ins> // Items is the list of resource ResourceSlices. Items []ResourceSlice `json:"items" protobuf:"bytes,2,rep,name=items"`

Listing s on v1.31.0 returns an object with a field instead of the standard field for (list) objects: Listing s on v1.30.3 behaves correctly: This is also propagated to the Swagger spec (which is how I noticed it, since I maintain the Rust bindings for the client API). v1.31.0: v1.30.3: As explained above. v1.31.0 with feature gate enabled and runtime config . With kind, this is: the API server request as in the above curl command in any way you like as long as it lets you see the raw JSON response. No response <details</details<detailskind 0.23.0 </details<details</details<details</details<details</details<details</details
/sig api-machinery
It seems there may be a discrepancy in the API response format between v1.30.3 and v1.31.0. The object is expected to follow standard Kubernetes API conventions, which typically use the field for list objects.
This deviation might cause compatibility issues for API clients that rely on this structure.
If this deviation from the expected API structure wasn't intentional, it should be aligned with the standard conventions.
The presence of instead of could disrupt client integrations, as you've encountered with the Swagger spec and Rust bindings.
cc
/triage accepted
/assign
this must be fixed to json:"metadata" protobuf:"bytes,1,opt,name=metadata"`` the serialization will not work with standard clients
/priority critical-urgent
Shoot, that's clearly a mistake.


Question: return err } <del> // Anything left in addrs is a new member that needs to be added to a pool for addr := range addrs { _, err := v2_pools.CreateAssociateMember(lbaas.network, poolID, v2_pools.MemberCreateOpts{ Address:      addr, ProtocolPort: int(ports[0].NodePort), SubnetID:     lbaas.opts.SubnetId, }).Extract() </del> <ins> // Compose Set of member (addresses) that _should_ exist addrs := map[string]empty{} for _, host := range hosts { addr, err := getAddressByName(lbaas.compute, host) </ins> if err != nil { return err } <del> waitLoadbalancerActiveProvisioningStatus(lbaas.network, loadbalancer.ID) </del> <ins> addrs[addr] = empty{} </ins> } <ins> // Check for adding/removing members associated with each port for _, port := range ports { // Get listener associated with this port listener, ok := lbListeners[portKey{ Protocol: string(port.Protocol), Port:     int(port.Port), }] if !ok { return fmt.Errorf("Loadbalancer %s does not contain required listener for port %d and protocol %s", loadBalancerName, port.Port, port.Protocol) } // Get pool associated with this listener pool, ok := lbPools[listener.ID] if !ok { return fmt.Errorf("Loadbalancer %s does not contain required pool for listener %s", loadBalancerName, listener.ID) } // Find existing pool members (by address) for this port members := make(map[string]v2_pools.Member) err := v2_pools.ListAssociateMembers(lbaas.network, pool.ID, v2_pools.MemberListOpts{}).EachPage(func(page pagination.Page) (bool, error) { membersList, err := v2_pools.ExtractMembers(page) if err != nil { return false, err } for _, member := range membersList { members[member.Address] = member } return true, nil }) if err != nil { return err } // Add any new members for this port for addr := range addrs { if _, ok := members[addr]; ok { // Already exists, do not create member continue } _, err := v2_pools.CreateAssociateMember(lbaas.network, pool.ID, v2_pools.MemberCreateOpts{ Address:      addr, ProtocolPort: int(port.NodePort), SubnetID:     lbaas.opts.SubnetId, }).Extract() if err != nil { return err } waitLoadbalancerActiveProvisioningStatus(lbaas.network, loadbalancer.ID) } // Remove any old members for this port for _, member := range members { if _, ok := addrs[member.Address]; ok { // Still present, do not delete member continue } err = v2_pools.DeleteMember(lbaas.network, pool.ID, member.ID).ExtractErr() if err != nil && !isNotFound(err) { return err } waitLoadbalancerActiveProvisioningStatus(lbaas.network, loadbalancer.ID) } } </ins> return nil }

LBaaS implementation (v1 or v2) will not support services that require to load balancer more than one port.
Fixed for LBaaS v2 by I suspect LBaaS v1 can't support multiple ports (not sure if reusing the VIP across multiple loadbalancers is a universally supported thing). I also believe LBaaS v1 is also mostly gone in favour of v2 in deployments so trying to address this in v1 is extremely low priority unless someone steps forward to ask for it. So: I think we can close this bug now, but I lack the powers to do that directly.
thank you for the update.


Question: }, func(client kubeclientset.Interface, obj runtime.Object) error { rs := obj.(*extensionsv1.Deployment) <del> err := client.Extensions().Deployments(rs.Namespace).Delete(rs.Name, &metav1.DeleteOptions{}) </del> <ins> orphanDependents := false err := client.Extensions().Deployments(rs.Namespace).Delete(rs.Name, &metav1.DeleteOptions{OrphanDependents: &orphanDependents}) </ins> return err })

Problem: When users sends a cascading deletion request to federation i.e they set orphanDependents=false, then federation should also set orphanDependents=false, when it sends the request to underlying clusters.
Fixed by


Question: // amount of allocations needed to create the fields.Set. If you add any // field here or the number of object-meta related fields changes, this should // be adjusted. <del> podSpecificFieldsSet := make(fields.Set, 8) </del> <ins> podSpecificFieldsSet := make(fields.Set, 9) </ins> podSpecificFieldsSet["spec.nodeName"] = pod.Spec.NodeName podSpecificFieldsSet["spec.restartPolicy"] = string(pod.Spec.RestartPolicy) podSpecificFieldsSet["spec.schedulerName"] = string(pod.Spec.SchedulerName) <ins> podSpecificFieldsSet["spec.serviceAccountName"] = string(pod.Spec.ServiceAccountName) </ins> podSpecificFieldsSet["status.phase"] = string(pod.Status.Phase) podSpecificFieldsSet["status.podIP"] = string(pod.Status.PodIP) podSpecificFieldsSet["status.nominatedNodeName"] = string(pod.Status.NominatedNodeName)

Is this a BUG REPORT or FEATURE REQUEST?: /kind feature What happened: When trying to list object using field spec.ServiceAccountName, we are getting the below error message. field label not supported: spec.serviceAccountName What you expected to happen: To be able to fetch pods using the list and watch based on the field 'spec.serviceAccountName' How to reproduce it (as minimally and precisely as possible): in minikube run Anything else we need to know?: Description: Unable to fetch pods based on the field 'spec.serviceAccountName'. When trying to get an overview of RBAC privileges in a cluster it is hard to find the objects attached to service accounts Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: minikube version: v0.28.0 OS (e.g. from /etc/os-release): NAME=Buildroot VERSION=2018.05 ID=buildroot VERSIONID=2018.05 PRETTYNAME="Buildroot 2018.05" Kernel (e.g. ): Linux minikube 4.16.14 SMP Mon Jun 11 16:20:15 UTC 2018 x86_64 GNU/Linux Install tools: kubeadm Others:
/sig architecture
You might do it by jsonpath filter
I intend to utilize it in the client-go list as an option
Not all fields are supported for field selector. There is a doc about field selector which is still under review: As far as I can see, supported field selectors for pods are listed here:
Didn't find that you have a pull request for it. :) Good luck!


Question: import ( "bytes" "fmt" <ins> "reflect" </ins> "sort" "github.com/golang/glog"

https://k8s- /sig apps /priority critical-urgent /assign PR that the test: /cc
GitHub didn't allow me to assign the following users: mortent. Note that only and repo collaborators can be assigned. For more information please see <detailsIn response to : https://k8s- /priority critical-urgent /assign /cc Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind flake
I'm looking into this.
The test is still flaky, but with different symptom: An example flake: https://k8s- It's likely because the collision count was updated in a previous sync loop, but the in the cache store hasn't been updated and we used a stale collision count while calculating the new hash: /assign


Question: // GetContainerInfo returns stats (from Cadvisor) for a container. func (kl *Kubelet) GetContainerInfo(podFullName, uuid, containerName string, req *info.ContainerInfoRequest) (*info.ContainerInfo, error) { <del> if kl.cadvisorClient == nil { </del> <ins> cc := kl.GetCadvisorClient() if cc == nil { </ins> return nil, nil } dockerContainers, err := dockertools.GetKubeletDockerContainers(kl.dockerClient, false)

So today, cadvisor became wedged and just stopped paying attention to its socket. Not kubelet's fault. But when I would launch all I would get would be: It took me a bit of trolling through netstat and lsof to figure out that the problem was cadvisor being completely locked up. kill -9 cadvisor and everything started working well. I wonder if we should emit more messages during startup about what operation we are about to attempt, not just the results of operations, as I was blaming docker for a while....
kubelet shouldn't block on this. I'll take a look.
Any ideas what was locking cAdvisor? and was it just not responding to requests?
I don't have any idea why it was locked. If I find another one locked I'll hang onto it to debug. Journal had: obviously the 'kill' came from me. It would not respond to systemctl cadvisor stop. netstat looking at port 4194 showed me At which point I realized cadvisor wasn't handling its sockets right since so many long dead kubelet sockets hadn't been cleaned up, so I kill -9'd it.
I cut some interesting info from the cadvisor journal log (trying to cut hostnames) You can see the last message about failed to update stats for container ... happened Oct 08 01:06:01. It was then silent until I killed it Oct 08 19:20:13, even though I'm pretty sure dnf was still being launched every hour on the hour. So it was wedged for about 18 hours.
What kernel is this on? We've seen there is a kernel bug fixed in 3.12+ that sometimes hangs us forever on exit. We're not quite sure how it gets triggered...
16.3


Question: <ins> #!/bin/sh # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A script encapsulating a common Dockerimage pattern for installing packages # and then cleaning up the unnecessary install artifacts. # e.g. clean-install iptables ebtables conntrack set -o errexit if [ $# = 0 ]; then echo >&2 "No packages specified" exit 1 fi apt-get update apt-get install -y --no-install-recommends $@ clean-apt </ins>

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: [root EFK]# kubectl logs -f fluentd-es-jrsf7 -n kube-system /var/lib/gems/2.1.0/gems/ffi- block in ffilib': Could not open library '': cannot open shared object file: No such file or directory. (LoadError) Could not open library '': cannot open shared object file: No such file or directory. Could not open library 'libsystemd-': cannot open shared object file: No such file or directory. Could not open library 'libsystemd-': libsystemd- cannot open shared object file: No such file or directory from /var/lib/gems/2.1.0/gems/ffi-map' from /var/lib/gems/2.1.0/gems/ffi- ffilib' from /var/lib/gems/2.1.0/gems/systemd-journal-<module:Native' from /var/lib/gems/2.1.0/gems/systemd-journal- <module:Id128' from /var/lib/gems/2.1.0/gems/systemd-journal-<module:Systemd' from /var/lib/gems/2.1.0/gems/systemd-journal- <top (required)' fromrequire' from require' from /var/lib/gems/2.1.0/gems/systemd-journal-<top (required)' from require' fromrequire' from /var/lib/gems/2.1.0/gems/systemd-journal- <top (required)' fromrequire' from rescue in require' fromrequire' from /var/lib/gems/2.1.0/gems/fluent-plugin-systemd- <top (required)' fromrequire' from require' from /var/lib/gems/2.1.0/gems/fluentd-block in tryloadplugin' from /var/lib/gems/2.1.0/gems/fluentd- each' from /var/lib/gems/2.1.0/gems/fluentd-tryloadplugin' from /var/lib/gems/2.1.


Question: <ins> #!/bin/sh # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A script encapsulating a common Dockerimage pattern for installing packages # and then cleaning up the unnecessary install artifacts. # e.g. clean-install iptables ebtables conntrack set -o errexit if [ $# = 0 ]; then echo >&2 "No packages specified" exit 1 fi apt-get update apt-get install -y --no-install-recommends $@ clean-apt </ins>

0/gems/fluentd- newimpl' from /var/lib/gems/2.1.0/gems/fluentd-newinput' from /var/lib/gems/2.1.0/gems/fluentd- addsource' from /var/lib/gems/2.1.0/gems/fluentd-block in configure' from /var/lib/gems/2.1.0/gems/fluentd- each' from /var/lib/gems/2.1.0/gems/fluentd-configure' from /var/lib/gems/2.1.0/gems/fluentd- configure' from /var/lib/gems/2.1.0/gems/fluentd-runconfigure' from /var/lib/gems/2.1.0/gems/fluentd- runconfigure' from /var/lib/gems/2.1.0/gems/fluentd-block in start' from /var/lib/gems/2.1.0/gems/fluentd- call' from /var/lib/gems/2.1.0/gems/fluentd-mainprocess' from /var/lib/gems/2.1.0/gems/fluentd- start' from /var/lib/gems/2.1.0/gems/fluentd-<top (required)' from require' fromrequire' from /var/lib/gems/2.1.0/gems/fluentd-0.12.39/bin/fluentd:5:in <top (required)' from /usr/local/bin/fluentd:23:inload' from /usr/local/bin/fluentd:23:in <main' What you expected to happen: fluentd-es run success How to reproduce it (as minimally and precisely as possible): kubectl create -f fluentd-es- the image is google-containers/fluentd-elasticsearch:v2.0.0 Anything else we need to know?


Question: <ins> #!/bin/sh # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A script encapsulating a common Dockerimage pattern for installing packages # and then cleaning up the unnecessary install artifacts. # e.g. clean-install iptables ebtables conntrack set -o errexit if [ $# = 0 ]; then echo >&2 "No packages specified" exit 1 fi apt-get update apt-get install -y --no-install-recommends $@ clean-apt </ins>

: Environment: Kubernetes version (use ):Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-21T08:23:22Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-21T08:08:00Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release):centos 7.3 Kernel (e.g. ):Linux node-21.46 3.10.0-514.26.2.el7.x8664 SMP Tue Jul 4 15:04:05 UTC 2017 x8664 x8664 x86_64 GNU/Linux Install tools: Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
Yup, the situation with fluentd on the system with systemd is very sloppy. There's a hack to make it works sometimes, but it seems that it doesn't work. I'll look into it
/sig instrumentation /assign
Facing same issue.


Question: <ins> #!/bin/sh # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A script encapsulating a common Dockerimage pattern for installing packages # and then cleaning up the unnecessary install artifacts. # e.g. clean-install iptables ebtables conntrack set -o errexit if [ $# = 0 ]; then echo >&2 "No packages specified" exit 1 fi apt-get update apt-get install -y --no-install-recommends $@ clean-apt </ins>

Environment Details:kubectl version Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} OS (e.g. from /etc/os-release):Centos 7 (Core) Kernel (e.g. uname -a): Linux xxxx 3.10.0-327.el7.x8664 Error for reference /var/lib/gems/2.1.0/gems/ffi- block in ffilib': Could not open library '': cannot open shared object file: No such file or directory. (LoadError) Could not open library '': cannot open shared object file: No such file or directory. Could not open library 'libsystemd-': cannot open shared object file: No such file or directory. Could not open library 'libsystemd-': libsystemd- cannot open shared object file: No such file or directory from /var/lib/gems/2.1.0/gems/ffi-map' from /var/lib/gems/2.1.0/gems/ffi- ffilib' from /var/lib/gems/2.1.0/gems/systemd-journal-<module:Native' from /var/lib/gems/2.1.0/gems/systemd-journal- <module:Id128' from /var/lib/gems/2.1.0/gems/systemd-journal-<module:Systemd' from /var/lib/gems/2.1.0/gems/systemd-journal- <top (required)' fromrequire' from require' from /var/lib/gems/2.1.0/gems/systemd-journal-<top (required)' from require' fromrequire' from /var/lib/gems/2.1.


Question: <ins> #!/bin/sh # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A script encapsulating a common Dockerimage pattern for installing packages # and then cleaning up the unnecessary install artifacts. # e.g. clean-install iptables ebtables conntrack set -o errexit if [ $# = 0 ]; then echo >&2 "No packages specified" exit 1 fi apt-get update apt-get install -y --no-install-recommends $@ clean-apt </ins>

0/gems/systemd-journal- <top (required)' fromrequire' from rescue in require' fromrequire' from /var/lib/gems/2.1.0/gems/fluent-plugin-systemd- <top (required)' fromrequire' from require' from /var/lib/gems/2.1.0/gems/fluentd-block in tryloadplugin' from /var/lib/gems/2.1.0/gems/fluentd- each' from /var/lib/gems/2.1.0/gems/fluentd-tryloadplugin' from /var/lib/gems/2.1.0/gems/fluentd- newimpl' from /var/lib/gems/2.1.0/gems/fluentd-newinput' from /var/lib/gems/2.1.0/gems/fluentd- addsource' from /var/lib/gems/2.1.0/gems/fluentd-block in configure' from /var/lib/gems/2.1.0/gems/fluentd- each' from /var/lib/gems/2.1.0/gems/fluentd-configure' from /var/lib/gems/2.1.0/gems/fluentd- configure' from /var/lib/gems/2.1.0/gems/fluentd-runconfigure' from /var/lib/gems/2.1.0/gems/fluentd- runconfigure' from /var/lib/gems/2.1.0/gems/fluentd-block in start' from /var/lib/gems/2.1.0/gems/fluentd- call' from /var/lib/gems/2.1.0/gems/fluentd-mainprocess' from /var/lib/gems/2.1.0/gems/fluentd- start' from /var/lib/gems/2.1.0/gems/fluentd-<top (required)' from require' fromrequire' from /var/lib/gems/2.1.0/gems/fluentd-0.12.39/bin/fluentd:5:in <top (required)' from /usr/local/bin/fluentd:23:inload' from /usr/local/bin/fluentd:23:in <


Question: <ins> #!/bin/sh # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #     http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A script encapsulating a common Dockerimage pattern for installing packages # and then cleaning up the unnecessary install artifacts. # e.g. clean-install iptables ebtables conntrack set -o errexit if [ $# = 0 ]; then echo >&2 "No packages specified" exit 1 fi apt-get update apt-get install -y --no-install-recommends $@ clean-apt </ins>

main'
You can solve it temporary by removing I'm working on the solution
A solution here might be to base fluentd-es off an image with journald supporting XZ and LZ4 compression. I'm looking :)
Thanks


Question: stats := make(map[string]*statsapi.NetworkStats) for _, c := range containers { <del> container, err := hcsshim.OpenContainer(c.ID) </del> <ins> cstats, err := fetchContainerStats(c) </ins> if err != nil { <del> klog.V(4).Infof("Failed to open container %q with error '%v', continue to get stats for other containers", c.ID, err) </del> <ins> klog.V(4).Infof("Failed to fetch statistics for container %q with error '%v', continue to get stats for other containers", c.ID, err) </ins> continue } <del> cstats, err := container.Statistics() if err != nil { klog.V(4).Infof("Failed to get statistics for container %q with error '%v', continue to get stats for other containers", c.ID, err) continue } </del> if len(cstats.Network) > 0 { stats[c.ID] = hcsStatsToNetworkStats(cstats.Timestamp, cstats.Network) }

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: 's memory usage is increasing over time. When more pods are run, it increases even more. What you expected to happen: it should not keep increasing How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): 1.14.0 Cloud provider or hardware configuration: AKS OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: !
/sig windows /sig azure
/assign


Question: resizeError := fmt.Errorf("resize of device %s failed: %v. btrfs output: %s", deviceMountPath, err, string(output)) return false, resizeError } <ins> func (resizefs *ResizeFs) NeedResize(devicePath string, deviceMountPath string) (bool, error) { deviceSize, err := resizefs.getDeviceSize(devicePath) if err != nil { return false, err } var fsSize, blockSize uint64 format, err := getDiskFormat(resizefs.exec, devicePath) if err != nil { formatErr := fmt.Errorf("ResizeFS.Resize - error checking format for device %s: %v", devicePath, err) return false, formatErr } // If disk has no format, there is no need to resize the disk because mkfs.* // by default will use whole disk anyways. if format == "" { return false, nil } klog.V(3).Infof("ResizeFs.needResize - checking mounted volume %s", devicePath) switch format { case "ext3", "ext4": blockSize, fsSize, err = resizefs.getExtSize(devicePath) klog.V(5).Infof("Ext size: filesystem size=%d, block size=%d", fsSize, blockSize) case "xfs": blockSize, fsSize, err = resizefs.getXFSSize(deviceMountPath) klog.V(5).Infof("Xfs size: filesystem size=%d, block size=%d, err=%v", fsSize, blockSize, err) default: klog.Errorf("Not able to parse given filesystem info. fsType: %s, will not resize", format) return false, fmt.Errorf("Could not parse fs info on given filesystem format: %s. Supported fs types are: xfs, ext3, ext4", format) } if err != nil { return false, err } // Tolerate one block difference, just in case of rounding errors somewhere. klog.V(5).Infof("Volume %s: device size=%d, filesystem size=%d, block size=%d", devicePath, deviceSize, fsSize, blockSize) if deviceSize <= fsSize+blockSize { return false, nil } return true, nil } func (resizefs *ResizeFs) getDeviceSize(devicePath string) (uint64, error) { output, err := resizefs.exec.Command("blockdev", "--getsize64", devicePath).CombinedOutput() outStr := strings.TrimSpace(string(output)) if err != nil { return 0, fmt.Errorf("failed to read size of device %s: %s: %s", devicePath, err, outStr) } size, err := strconv.ParseUint(outStr, 10, 64) if err != nil { return 0, fmt.Errorf("failed to parse size of device %s %s: %s", devicePath, outStr, err) } return size, nil } func (resizefs *ResizeFs) getExtSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("dumpe2fs", "-h", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), ":", "block size", "block count") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) getXFSSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("xfs_io", "-c", "statfs", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), "=", "geom.bsize", "geom.datablocks") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) parseFsInfoOutput(cmdOutput string, spliter string, blockSizeKey string, blockCountKey string) (uint64, uint64, error) { lines := strings.Split(cmdOutput, "n") var blockSize, blockCount uint64 var err error for _, line := range lines { tokens := strings.Split(line, spliter) if len(tokens) != 2 { continue } key, value := strings.ToLower(strings.TrimSpace(tokens[0])), strings.ToLower(strings.TrimSpace(tokens[1])) if key == blockSizeKey { blockSize, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block size %s: %s", value, err) } } if key == blockCountKey { blockCount, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block count %s: %s", value, err) } } } return blockSize, blockCount, err } </ins>

Kubernetes should resize filesystem on a volume when it's created as a clone of a smaller volume / smaller snapshot is restored into it. Example: takes a snapshot of 1 GiB volume with ext3 filesystem on it. restores the snapshot into 2 GiB volume. 2.1 CSI driver restores the snapshot into a bigger volume, but ext3 filesystem still uses only 1 GiB of space. driver should make sure that the filesystem on the volume has been resized before the volume is mounted into a pod. There are few steps needed: [x] Clarify behavior of CSI call, [ ] Update CSI drivers to the new behavior. [x] Move filesystem resize code from kubernetes/kuberenetes to kubernetes/mount-utils, so all CSI drivers can re-use the same code [x] Enhance the code above to compare the volume size from block device perspective and from filesystem perspective. See for proof of concept. [ ] File issues in all community CSI drivers that we care about & fix them. [ ] Update csi-tests to (optionally?) check for volume size when cloning / restoring snapshot to a bigger size.
/assign /sig storage /priority important-soon
True, considering the CSI spec allows restore to a bigger size volume, we should have the support from CO.
This is hard, Kubernets PV controller overwrites on binding. Therefore the external-provisioner cannot create PVC as "marked for resize on node", i.e. with = = the real size of the new volume) and with = size of the original volume. We either need to change the binding mechanism not to overwrite or add a flag (annotation?) that marks the volume for resize on node, even when .
Talked to The external-provisioner could mark for resize on node: Set to the original volume size. Add condition to the PVC. As result, the extenal-provisioner needs to modify , which needs new RBAC and is a breaking change. The PV controller could respect , if set. We discussed and rejected adding a new flag / annotation to PV by the external-provisioner for kubelet to call NodeExpandVolume on the first mount. Such a flag would need to be cleared by kubelet and kubelet can't modify PVs.
From latest discussion last week, I believe the conclusion was to have external-provisioner set to the smaller size.


Question: resizeError := fmt.Errorf("resize of device %s failed: %v. btrfs output: %s", deviceMountPath, err, string(output)) return false, resizeError } <ins> func (resizefs *ResizeFs) NeedResize(devicePath string, deviceMountPath string) (bool, error) { deviceSize, err := resizefs.getDeviceSize(devicePath) if err != nil { return false, err } var fsSize, blockSize uint64 format, err := getDiskFormat(resizefs.exec, devicePath) if err != nil { formatErr := fmt.Errorf("ResizeFS.Resize - error checking format for device %s: %v", devicePath, err) return false, formatErr } // If disk has no format, there is no need to resize the disk because mkfs.* // by default will use whole disk anyways. if format == "" { return false, nil } klog.V(3).Infof("ResizeFs.needResize - checking mounted volume %s", devicePath) switch format { case "ext3", "ext4": blockSize, fsSize, err = resizefs.getExtSize(devicePath) klog.V(5).Infof("Ext size: filesystem size=%d, block size=%d", fsSize, blockSize) case "xfs": blockSize, fsSize, err = resizefs.getXFSSize(deviceMountPath) klog.V(5).Infof("Xfs size: filesystem size=%d, block size=%d, err=%v", fsSize, blockSize, err) default: klog.Errorf("Not able to parse given filesystem info. fsType: %s, will not resize", format) return false, fmt.Errorf("Could not parse fs info on given filesystem format: %s. Supported fs types are: xfs, ext3, ext4", format) } if err != nil { return false, err } // Tolerate one block difference, just in case of rounding errors somewhere. klog.V(5).Infof("Volume %s: device size=%d, filesystem size=%d, block size=%d", devicePath, deviceSize, fsSize, blockSize) if deviceSize <= fsSize+blockSize { return false, nil } return true, nil } func (resizefs *ResizeFs) getDeviceSize(devicePath string) (uint64, error) { output, err := resizefs.exec.Command("blockdev", "--getsize64", devicePath).CombinedOutput() outStr := strings.TrimSpace(string(output)) if err != nil { return 0, fmt.Errorf("failed to read size of device %s: %s: %s", devicePath, err, outStr) } size, err := strconv.ParseUint(outStr, 10, 64) if err != nil { return 0, fmt.Errorf("failed to parse size of device %s %s: %s", devicePath, outStr, err) } return size, nil } func (resizefs *ResizeFs) getExtSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("dumpe2fs", "-h", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), ":", "block size", "block count") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) getXFSSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("xfs_io", "-c", "statfs", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), "=", "geom.bsize", "geom.datablocks") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) parseFsInfoOutput(cmdOutput string, spliter string, blockSizeKey string, blockCountKey string) (uint64, uint64, error) { lines := strings.Split(cmdOutput, "n") var blockSize, blockCount uint64 var err error for _, line := range lines { tokens := strings.Split(line, spliter) if len(tokens) != 2 { continue } key, value := strings.ToLower(strings.TrimSpace(tokens[0])), strings.ToLower(strings.TrimSpace(tokens[1])) if key == blockSizeKey { blockSize, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block size %s: %s", value, err) } } if key == blockCountKey { blockCount, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block count %s: %s", value, err) } } } return blockSize, blockCount, err } </ins>

Then we just let the rest of the normal resize flow continue. No need to have provisioner set the conditions.
we agreed to set conditions too because we want to be as close as possible to external-resizer. If there is some tooling that depends on condition for resizing pending on node, we don't want to break any such tooling. currently kubelet does not depend on condition for node expansion though.
Why can't we let resize controller set the condition when it sees pvc size != pv size?
hmm good point. I think that should work but we may have to verify that flow.
I think we can do that, but only with a change to the CSI spec wording, because the current implication is that should not be necessary.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
can you summarize the current conclusion/state of this? /remove-lifecycle rotten
Updated the issue with TODO list.
should this issue stay open to keep track of remaining issues?
/reopen
Reopened this issue. <detailsIn response to : /reopen </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
/remove-lifecycle stale
/triage accepted
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.


Question: resizeError := fmt.Errorf("resize of device %s failed: %v. btrfs output: %s", deviceMountPath, err, string(output)) return false, resizeError } <ins> func (resizefs *ResizeFs) NeedResize(devicePath string, deviceMountPath string) (bool, error) { deviceSize, err := resizefs.getDeviceSize(devicePath) if err != nil { return false, err } var fsSize, blockSize uint64 format, err := getDiskFormat(resizefs.exec, devicePath) if err != nil { formatErr := fmt.Errorf("ResizeFS.Resize - error checking format for device %s: %v", devicePath, err) return false, formatErr } // If disk has no format, there is no need to resize the disk because mkfs.* // by default will use whole disk anyways. if format == "" { return false, nil } klog.V(3).Infof("ResizeFs.needResize - checking mounted volume %s", devicePath) switch format { case "ext3", "ext4": blockSize, fsSize, err = resizefs.getExtSize(devicePath) klog.V(5).Infof("Ext size: filesystem size=%d, block size=%d", fsSize, blockSize) case "xfs": blockSize, fsSize, err = resizefs.getXFSSize(deviceMountPath) klog.V(5).Infof("Xfs size: filesystem size=%d, block size=%d, err=%v", fsSize, blockSize, err) default: klog.Errorf("Not able to parse given filesystem info. fsType: %s, will not resize", format) return false, fmt.Errorf("Could not parse fs info on given filesystem format: %s. Supported fs types are: xfs, ext3, ext4", format) } if err != nil { return false, err } // Tolerate one block difference, just in case of rounding errors somewhere. klog.V(5).Infof("Volume %s: device size=%d, filesystem size=%d, block size=%d", devicePath, deviceSize, fsSize, blockSize) if deviceSize <= fsSize+blockSize { return false, nil } return true, nil } func (resizefs *ResizeFs) getDeviceSize(devicePath string) (uint64, error) { output, err := resizefs.exec.Command("blockdev", "--getsize64", devicePath).CombinedOutput() outStr := strings.TrimSpace(string(output)) if err != nil { return 0, fmt.Errorf("failed to read size of device %s: %s: %s", devicePath, err, outStr) } size, err := strconv.ParseUint(outStr, 10, 64) if err != nil { return 0, fmt.Errorf("failed to parse size of device %s %s: %s", devicePath, outStr, err) } return size, nil } func (resizefs *ResizeFs) getExtSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("dumpe2fs", "-h", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), ":", "block size", "block count") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) getXFSSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("xfs_io", "-c", "statfs", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), "=", "geom.bsize", "geom.datablocks") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) parseFsInfoOutput(cmdOutput string, spliter string, blockSizeKey string, blockCountKey string) (uint64, uint64, error) { lines := strings.Split(cmdOutput, "n") var blockSize, blockCount uint64 var err error for _, line := range lines { tokens := strings.Split(line, spliter) if len(tokens) != 2 { continue } key, value := strings.ToLower(strings.TrimSpace(tokens[0])), strings.ToLower(strings.TrimSpace(tokens[1])) if key == blockSizeKey { blockSize, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block size %s: %s", value, err) } } if key == blockCountKey { blockCount, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block count %s: %s", value, err) } } } return blockSize, blockCount, err } </ins>

This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle rotten
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.


Question: resizeError := fmt.Errorf("resize of device %s failed: %v. btrfs output: %s", deviceMountPath, err, string(output)) return false, resizeError } <ins> func (resizefs *ResizeFs) NeedResize(devicePath string, deviceMountPath string) (bool, error) { deviceSize, err := resizefs.getDeviceSize(devicePath) if err != nil { return false, err } var fsSize, blockSize uint64 format, err := getDiskFormat(resizefs.exec, devicePath) if err != nil { formatErr := fmt.Errorf("ResizeFS.Resize - error checking format for device %s: %v", devicePath, err) return false, formatErr } // If disk has no format, there is no need to resize the disk because mkfs.* // by default will use whole disk anyways. if format == "" { return false, nil } klog.V(3).Infof("ResizeFs.needResize - checking mounted volume %s", devicePath) switch format { case "ext3", "ext4": blockSize, fsSize, err = resizefs.getExtSize(devicePath) klog.V(5).Infof("Ext size: filesystem size=%d, block size=%d", fsSize, blockSize) case "xfs": blockSize, fsSize, err = resizefs.getXFSSize(deviceMountPath) klog.V(5).Infof("Xfs size: filesystem size=%d, block size=%d, err=%v", fsSize, blockSize, err) default: klog.Errorf("Not able to parse given filesystem info. fsType: %s, will not resize", format) return false, fmt.Errorf("Could not parse fs info on given filesystem format: %s. Supported fs types are: xfs, ext3, ext4", format) } if err != nil { return false, err } // Tolerate one block difference, just in case of rounding errors somewhere. klog.V(5).Infof("Volume %s: device size=%d, filesystem size=%d, block size=%d", devicePath, deviceSize, fsSize, blockSize) if deviceSize <= fsSize+blockSize { return false, nil } return true, nil } func (resizefs *ResizeFs) getDeviceSize(devicePath string) (uint64, error) { output, err := resizefs.exec.Command("blockdev", "--getsize64", devicePath).CombinedOutput() outStr := strings.TrimSpace(string(output)) if err != nil { return 0, fmt.Errorf("failed to read size of device %s: %s: %s", devicePath, err, outStr) } size, err := strconv.ParseUint(outStr, 10, 64) if err != nil { return 0, fmt.Errorf("failed to parse size of device %s %s: %s", devicePath, outStr, err) } return size, nil } func (resizefs *ResizeFs) getExtSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("dumpe2fs", "-h", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), ":", "block size", "block count") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) getXFSSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("xfs_io", "-c", "statfs", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), "=", "geom.bsize", "geom.datablocks") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) parseFsInfoOutput(cmdOutput string, spliter string, blockSizeKey string, blockCountKey string) (uint64, uint64, error) { lines := strings.Split(cmdOutput, "n") var blockSize, blockCount uint64 var err error for _, line := range lines { tokens := strings.Split(line, spliter) if len(tokens) != 2 { continue } key, value := strings.ToLower(strings.TrimSpace(tokens[0])), strings.ToLower(strings.TrimSpace(tokens[1])) if key == blockSizeKey { blockSize, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block size %s: %s", value, err) } } if key == blockCountKey { blockCount, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block count %s: %s", value, err) } } } return blockSize, blockCount, err } </ins>

This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close [1]:
Closing this issue. <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-lifecycle stale /reopen
Reopened this issue. <detailsIn response to : /remove-lifecycle stale /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The issue has been marked as an important bug and triaged. Such issues are automatically marked as frozen when hitting the rotten state to avoid missing important bugs. Please send feedback to sig-contributor-experience at . /lifecycle frozen
This issue is labeled with but has not been updated in over 90 days, and should be re-triaged. Important-soon issues must be staffed and worked on either currently, or very soon, ideally in time for the next release. You can: Confirm that this issue is still relevant with (org members only) Deprioritize it with or Close this issue with For more details on the triage process, see /remove-triage accepted
/triage accepted /priority long-term what is the latest status on this?
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /triage accepted /priority long-term what is the latest status on this? Instructions for interacting with me using PR comments are available .


Question: resizeError := fmt.Errorf("resize of device %s failed: %v. btrfs output: %s", deviceMountPath, err, string(output)) return false, resizeError } <ins> func (resizefs *ResizeFs) NeedResize(devicePath string, deviceMountPath string) (bool, error) { deviceSize, err := resizefs.getDeviceSize(devicePath) if err != nil { return false, err } var fsSize, blockSize uint64 format, err := getDiskFormat(resizefs.exec, devicePath) if err != nil { formatErr := fmt.Errorf("ResizeFS.Resize - error checking format for device %s: %v", devicePath, err) return false, formatErr } // If disk has no format, there is no need to resize the disk because mkfs.* // by default will use whole disk anyways. if format == "" { return false, nil } klog.V(3).Infof("ResizeFs.needResize - checking mounted volume %s", devicePath) switch format { case "ext3", "ext4": blockSize, fsSize, err = resizefs.getExtSize(devicePath) klog.V(5).Infof("Ext size: filesystem size=%d, block size=%d", fsSize, blockSize) case "xfs": blockSize, fsSize, err = resizefs.getXFSSize(deviceMountPath) klog.V(5).Infof("Xfs size: filesystem size=%d, block size=%d, err=%v", fsSize, blockSize, err) default: klog.Errorf("Not able to parse given filesystem info. fsType: %s, will not resize", format) return false, fmt.Errorf("Could not parse fs info on given filesystem format: %s. Supported fs types are: xfs, ext3, ext4", format) } if err != nil { return false, err } // Tolerate one block difference, just in case of rounding errors somewhere. klog.V(5).Infof("Volume %s: device size=%d, filesystem size=%d, block size=%d", devicePath, deviceSize, fsSize, blockSize) if deviceSize <= fsSize+blockSize { return false, nil } return true, nil } func (resizefs *ResizeFs) getDeviceSize(devicePath string) (uint64, error) { output, err := resizefs.exec.Command("blockdev", "--getsize64", devicePath).CombinedOutput() outStr := strings.TrimSpace(string(output)) if err != nil { return 0, fmt.Errorf("failed to read size of device %s: %s: %s", devicePath, err, outStr) } size, err := strconv.ParseUint(outStr, 10, 64) if err != nil { return 0, fmt.Errorf("failed to parse size of device %s %s: %s", devicePath, outStr, err) } return size, nil } func (resizefs *ResizeFs) getExtSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("dumpe2fs", "-h", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), ":", "block size", "block count") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) getXFSSize(devicePath string) (uint64, uint64, error) { output, err := resizefs.exec.Command("xfs_io", "-c", "statfs", devicePath).CombinedOutput() if err != nil { return 0, 0, fmt.Errorf("failed to read size of filesystem on %s: %s: %s", devicePath, err, string(output)) } blockSize, blockCount, _ := resizefs.parseFsInfoOutput(string(output), "=", "geom.bsize", "geom.datablocks") if blockSize == 0 { return 0, 0, fmt.Errorf("could not find block size of device %s", devicePath) } if blockCount == 0 { return 0, 0, fmt.Errorf("could not find block count of device %s", devicePath) } return blockSize, blockSize * blockCount, nil } func (resizefs *ResizeFs) parseFsInfoOutput(cmdOutput string, spliter string, blockSizeKey string, blockCountKey string) (uint64, uint64, error) { lines := strings.Split(cmdOutput, "n") var blockSize, blockCount uint64 var err error for _, line := range lines { tokens := strings.Split(line, spliter) if len(tokens) != 2 { continue } key, value := strings.ToLower(strings.TrimSpace(tokens[0])), strings.ToLower(strings.TrimSpace(tokens[1])) if key == blockSizeKey { blockSize, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block size %s: %s", value, err) } } if key == blockCountKey { blockCount, err = strconv.ParseUint(value, 10, 64) if err != nil { return 0, 0, fmt.Errorf("failed to parse block count %s: %s", value, err) } } } return blockSize, blockCount, err } </ins>

If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/priority important-long-term
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority important-long-term Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Hello I've checked what is the current status of the drivers that were involved in this effort and I think we might be able to close this as all upstream work is done. Please take a look at the table below and let me know if anything is missing.
/close
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: <ins> /* Copyright 2017 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package clientcmd import ( "testing" "github.com/spf13/pflag" ) func TestNamespacePrefixStrip(t *testing.T) { testData := map[string]string{ "namespaces/foo": "foo", "NAMESPACES/foo": "foo", "NameSpaces/foo": "foo", "namespace/foo":  "foo", "NAMESPACE/foo":  "foo", "nameSpace/foo":  "foo", "ns/foo":         "foo", "NS/foo":         "foo", "namespaces/":    "namespaces/", "namespace/":     "namespace/", "ns/":            "ns/", } for before, after := range testData { overrides := &ConfigOverrides{} fs := &pflag.FlagSet{} BindOverrideFlags(overrides, fs, RecommendedConfigOverrideFlags("")) fs.Parse([]string{"--namespace", before}) if overrides.Context.Namespace != after { t.Fatalf("Expected %s, got %s", after, overrides.Context.Namespace) } } } </ins>

Because always return names with a prefix like , it cannot be directly used to iterate over all namespaces to execute other commands. I've compiled from source: Reviewing some scripts, I came across a usage of like this: That line fetches namespace names to later run in a loop: That seemed to be a perfect use case for the flag, but it can't be used without extra hacks to get rid of the prefix (FWIW, is invalid). Changing the current output of might not be possible or desirable, for this could impact existing users that rely on the output format. If I'm wrong about the above limitation, than it would make sense to output just the name when and there is a single resource type, while using the ideas introduced in to show the resource type only when there is more than one type in the output, or when is explicitly set. Before: After:
could you help me ping the appropriate team to triage this? Thanks!
Please re-post your question to . We are trying to consolidate the channels to which questions for help/support are posted so that we can improve our efficiency in responding to your requests, and to make it easier for you to find answers to frequently asked questions and how to address common use cases. We regularly see messages posted in multiple forums, with the full response thread only in one place or, worse, spread across multiple forums. Also, the large volume of support issues on github is making it difficult for us to use issues to identify real bugs. The Kubernetes team scans stackoverflow on a regular basis, and will try to ensure your questions don't go unanswered. Before posting a new question, please search stackoverflow for answers to similar questions, and also familiarize yourself with: Again, thanks for using Kubernetes. The Kubernetes Team
perhaps I didn't do a good job at explaining the problem, but this is not a question of how to do something, but a report on a problem with the CLI design.
Sorry about that :-/. I'll try to route to the proper team.
update: console examples to the description.
the -o format is designed for scripting, the use case is just different from yours. You should be able to paste the output of into . Does do what you want? If this is a common use case we could create another -o flag value.


Question: <ins> /* Copyright 2017 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package clientcmd import ( "testing" "github.com/spf13/pflag" ) func TestNamespacePrefixStrip(t *testing.T) { testData := map[string]string{ "namespaces/foo": "foo", "NAMESPACES/foo": "foo", "NameSpaces/foo": "foo", "namespace/foo":  "foo", "NAMESPACE/foo":  "foo", "nameSpace/foo":  "foo", "ns/foo":         "foo", "NS/foo":         "foo", "namespaces/":    "namespaces/", "namespace/":     "namespace/", "ns/":            "ns/", } for before, after := range testData { overrides := &ConfigOverrides{} fs := &pflag.FlagSet{} BindOverrideFlags(overrides, fs, RecommendedConfigOverrideFlags("")) fs.Parse([]string{"--namespace", before}) if overrides.Context.Namespace != after { t.Fatalf("Expected %s, got %s", after, overrides.Context.Namespace) } } } </ins>

the use cases I've observed doing code reviews: over all namespaces to get certain resources from that namespace: over all namespaces, then iterate of over resource types that produce logs and fetch logs one by one: Above we can use for listing pods because later we'll pass it to that accept the syntax. We cannot do the same for namespaces, because doesn't like the format.
Alternatively we could be kubectl recognize as valid
I'll take a look on it if it's not taken by anyone yet. CC
Have you started this?
Only investigated due to being out of office since Thursday last week. I can work on it in upcoming week, but I'll understand if it's piority issue and you'll reassign it.
Ready :-)
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list .
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: test.old.Spec.RestartPolicy = "Always" } <del> errs := ValidatePodUpdate(&test.new, &test.old, PodValidationOptions{}) </del> <ins> errs := ValidatePodUpdate(&test.new, &test.old, test.opts) </ins> if test.err == "" { if len(errs) != 0 { t.Errorf("unexpected invalid: %s (%+v)nA: %+vnB: %+v", test.test, errs, test.new, test.old)

See for context. This issue is creating to track work on the API validation changes. Validation relaxation of certain fields in the pod spec to allow for mutable scheduling directives.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/sig scheduling
Docs PR: