Question: releaseHugepages() ginkgo.By("restarting kubelet to pick up pre-allocated hugepages") <del> restartKubelet() </del> <ins> // stop the kubelet and wait until the server will restart it automatically stopKubelet() </ins> waitForHugepages() })

So the only reliable solution that I found, it just trying to start the pod number of times. I tested it 5 times and the test passed. Can you please review
Can we access kubelet service from hugepage test cases? If we can we can use its APIs like or to check if kubelet is ready after the restart. Overall I have a feeling of duplication test/e2enode/services functionality in This is a main reason for this and similar bugs I believe. It would be nice to get rid of this duplication if possible. That would nicely isolate kubelet service controls in one place and eliminate conflicts like this one. wdyt?
not really, the used mostly to start all needed components to emulate the cluster(not only the kubelet) and the test does not have access to it. I think you can separate the readiness check under the importable method, but it worth the separate PR.
I updated the PR to check the healthiness of the kubelet and looks like it works just fine.
thank you. it looks good to me so far. I'll add lgtm label when is merged and tests are all green.
the job is still flaky. can you reopen this issue?
I opened a different one - , the issue can affect any job, not only the hugepages one.
I can open a fix to W/A an issue(start the pod until it will succeed), it should fix all problems, WDYT?
i think it's ok as a workaround, until is fixed
would it be better to wait until api server reports correct resource value before running the pod?
We are waiting for it, the problem that the scheduler and the kubelet see a different picture regarding resources requested by pods.
ok. In this case restarting the pod until it succeeds seems reasonable to me to fix the flakiness until is fixed.
any progress on the workaround fix?


Question: containers: - name: fluentd-cloud-logging image: gcr.io/google_containers/fluentd-gcp:1.6 <ins> resources: limits: cpu: 100m </ins> env: - name: "FLUENTD_ARGS" value: "-qq"

During debugging , I noticed that one of services pods: monitoring and logging pods stuck at pending sometimes forever: I logged into the node, and found the container is running happily on the node: Looks like status is never reported from kubelet. I checked kubelet log, then I found: Why we cannot delete mirror pod? and Created a new one? Because this regression, once we run into such state, the rest of test won't be triggered, and eventually timeout / failed.
checks those service pods in ready state, and prevent the test to run once it is pending. cc/ I think this is the real issue I talked to you earlier about PodStatus.
I spoke to offline. For some reason, kubelet thinks the spec of the mirror pod does not match the static pod locally. It then decides to delete the mirror pod and recreate it, repeatedly. As a result of that, it slows down (due to the QPS limit?) reporting statuses of other pods. This can be reproduced by running the e2e "Shell" test. I am looking into why the specs would suddenly mismatch. Did any recent PR modified the pod spec in apiserver?
The recently introduced default cpu share limit was not there in the static pod, causing the difference between its spec and the mirror pod's.
Seeing this as well. Makes the whole cluster slow and unusable for me. Sent from my iPhone
Is there a desired fix for this?
said she would add default resource limits to the static pod manifest files. If we go for that, I think it's reasonable to fail (not run the static pod) if resource limit is not specified. Also, the resource limits set in the mirror pods are not meaningful because kubelet creates containers based on the spec of the static pod. In general, static pods do not subject to any defaulting values applied at the apiserver.
Pretty sure that this is fixed by . Certainly testing against head since then works well for me. What version are you testing?
was fixing node status, and the problem had been around for a while. This issue is about pod status, which is still unfixed on head. These two are very different, and my cluster did include the fix :) This issue will slow down kubelet at times, but if you get lucky and the pod status (running) get sent out in time, the tests will still pass.


Question: containers: - name: fluentd-cloud-logging image: gcr.io/google_containers/fluentd-gcp:1.6 <ins> resources: limits: cpu: 100m </ins> env: - name: "FLUENTD_ARGS" value: "-qq"

There is no way the issue is fixed by that PR. I am still seeing the issue on HEAD. The system gets worse the longer its up. I agree with the analysis of the issue in source PR Sent from my iPhone
I am on this one, and here is proposed solution in my mind so far (not final yet since I haven't put too much thoughts into it yet): 1) For all our static pods I am going to update them with some initial cpu limit for now. This should quickly stop flakiness caused by this particular issue. 2) Next step is that for static pods, if there is no initial cpu limit, kubelet should refuse to run. Before I do this, I need to send an announcement to our container-vm users about this potential breakage for the in the future. Another potential fix is we can ignore resource changes when comparing if a mirror pod is updated or not. I think is proposed a similiar one earlier. But I had a concern with it: this is defeat the initial purpose we created mirror pods and sync back to master. So if one made some resource changes on the static pods, you have to always ssh-ed into the node to see if the new value applied or not. Also eventually (post 1.0) those mirrored pod's resource limit is going to be used by scheduler when calculating if user pods are fit or not, we need them to be accurated. Another concern is in the code implementation itself to distinuguish mirror pods future from rest of pods. I guess this shouldn't be the big issue here. Any more suggestions here? One puzzle I had is that why master component pods doesn't have issue at all. I manually restart kube-apiserver pod, still doesn't have the issue.
Depending on time we could just remove the default limit from the limit range as a stop gap. Sent from my iPhone
Can you just bypass defaulting at the apiserver when you create mirror pods? I can't think of any other solution that works in general and is easily maintainable. Approach 2 you described in the previous entry works if we are defaulting cpu limit, but breaks as soon as we default anything else.


Question: containers: - name: fluentd-cloud-logging image: gcr.io/google_containers/fluentd-gcp:1.6 <ins> resources: limits: cpu: 100m </ins> env: - name: "FLUENTD_ARGS" value: "-qq"

Same problem with ignoring resource differences when comparing mirror pod to static pod -- as long as defaulting only touches resources, that works, but if we change it to touch other stuff, then it breaks.
Commit I had can quickly fix current bleeding issue. Currently we only have master component pods and fluentd pod are file based by default. Master component pods doesn't create any issue now because mirror pods aren't created at the first place (due to recent change made by Also file-based pods will have issue only when its spec doesn't have initial value, and need to update (delete miss matched mirror pod, and recreate a new one). Please note that with my commit, everything is working even I manually update its cpu limit from 100m to 200m, there is no issue at all, and query pod shows accurate result with the new limit too: How badly we request all users using file-based pod feature providing default cpu limit for now? Eventually we want to get rid of file-based pods completely, also encouraging the user provides the initial resource limit is aligning with our direction in a long run anyway.
Just to clarify, what I was suggesting was for a "real fix" not the temporary measure to stop the bleeding (which you already have in your PR). But what do you think of what I suggested as the longer-term fix? My point is that today, we default CPU, but in the future, we may default anything. It seems that the most maintainable fix is to just not do defaulting when creating mirror pod from static pod. Is that possible? I wonder if the reason manual update works (what you described in your last comment) is because we don't go through the defaulting when you do an update, only a create? (Though that sounds like a bug.) IMO we don't need CPU default right now, especially since we already have kubelet flag for limiting pods per machine (though unfortunately it's currently set to 100 because we haven't reconfigured the tests that intentionally run lots of pods per node, to override the flag. we can change the default to 30 and add an override mechanism if this is important.)
Comparing a config file with the live state and expecting them to match is doomed. The recommended approach is described by . Kubelet could store the config as an annotation in the mirror pod and compare with that.


Question: containers: - name: fluentd-cloud-logging image: gcr.io/google_containers/fluentd-gcp:1.6 <ins> resources: limits: cpu: 100m </ins> env: - name: "FLUENTD_ARGS" value: "-qq"

Does the scheduler favor nodes with the fewest pods even if they don't specify resources? LimitRange does cause that to happen.
Storing the config as an annotation sounds good to me. Filed
Although this issues was closed, it still appears in my environment
Same here, this happens from time to time in our environments, is there a way to debug to find the root cause?


Question: // Determine if this is tagged as an Internal ELB internalELB := false internalAnnotation := apiService.Annotations[ServiceAnnotationLoadBalancerInternal] <del> if internalAnnotation != "" { </del> <ins> if internalAnnotation == "false" { internalELB = false } else if internalAnnotation != "" { </ins> internalELB = true }

It's useful to have a helm chart that expose a value like . However, the annotation simply ignores , as seen in It's just checking that the value is not empty, so even if it's false - it'll try to create internal LB.
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
GitHub didn't allow me to assign the following users: mcrute. Note that only and repo collaborators can be assigned. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: for i, test := range tests { test.namespace.ObjectMeta.ResourceVersion = "1" test.oldNamespace.ObjectMeta.ResourceVersion = "1" <del> errs := ValidateNamespaceStatusUpdate(&test.oldNamespace, &test.namespace) </del> <ins> errs := ValidateNamespaceStatusUpdate(&test.namespace, &test.oldNamespace) </ins> if test.valid && len(errs) > 0 { t.Errorf("%d: Unexpected error: %v", i, errs) t.Logf("%#v vs %#v", test.oldNamespace.ObjectMeta, test.namespace.ObjectMeta)

There is a funny subtle behvaiour in get namespaces, in which if i call more than one time, it seems that the removal from Terminating state is accelarated. It seems this might indicate a bug, or maybe its not a bug and the eventual semantics of is a little funny sometimes. in any case, here's what we are seeing : 1) when i delete a namespace the first time usning kubectl, we get a state change to . that is correct, of course. 2) but ... if make the call a second time, then the same status line is immediately removed from . Expected behaviour : A second call to "kubectl delete ..." shouldn't have any effect on the timing of removal of a namespace from 'Terminating' state*... So i guess there is some "eventual consistency" subtle bug that is resolved by repeated calls to
suggests this may not be e bug, maybe its just the etcd eventual deleter thingy.
Looks to definitely be a bug. The 2nd delete seems to have interrupted the delete process and artifacts (like events) from the namespace still exist in etcd.
investigating...
oops, see the error, will submit a fix.
I have a fix, but need a test case, will submit PR tomorrow AM


Question: } func TestValidateSubpathMutuallyExclusive(t *testing.T) { <del> // Enable feature VolumeSubpathEnvExpansion and VolumeSubpath defer featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.VolumeSubpathEnvExpansion, true)() </del> <ins> // Enable feature VolumeSubpath </ins> defer featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.VolumeSubpath, true)() volumes := []core.Volume{

/kind cleanup /sig storage /sig node Description: Feature gate VolumeSubpathEnvExpansion was GA in 1.17 and is marked for removal in 1.19 Related Issues: GA Release:
/assign
/cc


Question: - user: root - name: /opt/cni - makedirs: True <del> - source: https://storage.googleapis.com/kubernetes-release/network-plugins/cni-09214926.tar.gz </del> <ins> - source: https://storage.googleapis.com/kubernetes-release/network-plugins/cni-42c4cb842dad606a84e93aad5a4484ded48e3046.tar.gz </ins> - tar_options: v <del> - source_hash: md5=58f8631f912dd88be6a0920775db7274 </del> <ins> - source_hash: md5=8cee1d59f01a27e8c2c10c120dce1e7d </ins> - archive_format: tar - if_missing: /opt/cni/bin

Event for a pod: The cni plugins do exist at /opt/cni/bin. They cannot be executed and bash is complaining:
cc
I'm guessing this is a shared lib problem, since /usr/grte/v4/lib64/ld-linux-x86- doesn't exist on my gce vm. We should update docs to make it clear how someone would install manually (if they run into such problems again) and statically link (if the problem is indeed the shared lib) with something like Unsure how we'd support different GOOS (vend different tarballs vs checking in and building the plugin inline with our make scripts)
uggh. These binaries need to be ultra-static or else we need to containerize them.
Essentially doing would've prevented this bug :)
And see for the CNI_ARGS problems
also , re ... however, I'm not too sure if your comment was mean to be on this ticket.
cc
I have a working version. I uploaded my cni binary with bug fix on GCS. change source to change sourcehash to Welcome to give it a try.
Also see problem:


Question: "//staging/src/k8s.io/client-go/util/testing:go_default_library", "//staging/src/k8s.io/component-base/featuregate/testing:go_default_library", "//staging/src/k8s.io/component-base/version:go_default_library", <del> "//staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2:go_default_library", </del> "//vendor/github.com/google/cadvisor/info/v1:go_default_library", "//vendor/github.com/google/cadvisor/info/v2:go_default_library", "//vendor/github.com/stretchr/testify/assert:go_default_library", "//vendor/github.com/stretchr/testify/require:go_default_library", "//vendor/k8s.io/utils/mount:go_default_library", <del> ], </del> <ins> ] + select({ "@io_bazel_rules_go//go/platform:android": [ "//staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2:go_default_library", ], "@io_bazel_rules_go//go/platform:linux": [ "//staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2:go_default_library", ], "//conditions:default": [], }), </ins> ) filegroup(

What happened: It's most visible in the test <img width="323" alt="Screen Shot 2020-05-20 at 6 09 10 PM" src="https://user-"It shows up in kind, kube- and kops jobs The regression was introduced somewhere within , possible culprits are: (FYI (FYI (FYI There are more tests affected, just not as prominently (visually inspecting ) [sig-apps] StatefulSet [] Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications with PVCs [sig-apps] StatefulSet [] Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications [Conformance] [sig-storage] In-tree Volumes [Driver: nfs] [Testpattern: Inline-volume (default fs)] volumes should store data [sig-storage] In-tree Volumes [Driver: nfs] [Testpattern: Pre-provisioned PV (default fs)] volumes should store data I wasn't sure whether to tag this as kind/regression or not, I'll leave that up to or someone /sig node /sig api-machimery /sig scheduling
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : What happened: It's most visible in the test (FYI (FYI (FYI StatefulSet [] Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications with PVCs [sig-apps] StatefulSet [] Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications [Conformance] [sig-storage] In-tree Volumes [Driver: nfs] [Testpattern: Inline-volume (default fs)] volumes should store data [sig-storage] In-tree Volumes [Driver: nfs] [Testpattern: Pre-provisioned PV (default fs)] volumes should store data /sig api-machimery /sig scheduling Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository.


Question: "//staging/src/k8s.io/client-go/util/testing:go_default_library", "//staging/src/k8s.io/component-base/featuregate/testing:go_default_library", "//staging/src/k8s.io/component-base/version:go_default_library", <del> "//staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2:go_default_library", </del> "//vendor/github.com/google/cadvisor/info/v1:go_default_library", "//vendor/github.com/google/cadvisor/info/v2:go_default_library", "//vendor/github.com/stretchr/testify/assert:go_default_library", "//vendor/github.com/stretchr/testify/require:go_default_library", "//vendor/k8s.io/utils/mount:go_default_library", <del> ], </del> <ins> ] + select({ "@io_bazel_rules_go//go/platform:android": [ "//staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2:go_default_library", ], "@io_bazel_rules_go//go/platform:linux": [ "//staging/src/k8s.io/cri-api/pkg/apis/runtime/v1alpha2:go_default_library", ], "//conditions:default": [], }), </ins> ) filegroup(

</details
/cc
will increase the time for deletion of pod and it should be expected because currently pod is deleted from apiserver before removing the sandbox container but after this fix pod will be deleted from the apiserver only after deleting the sandbox container.
/remove-sig scheduling doesn't affect scheduling running time at all.
Yeah, is almost definitely the culprit. Last time we changed that code path, it also caused a regression. The degradation in performance is expected, but a 5x slowdown is definitely larger than it should be.
do we know if that's because the status used by the path is stale, or if removing sandboxes is actually that slow?
if there's not a quick fix, should we revert and bring it back in once the root issue is understood?
I think a revert is appropriate in this case.
ok, I'll open it
open at
I agree with that 5x is something which shouldn't happen and i can take a look on why thats happening and that might require changing the way how the sandbox is currently deleted which is only during GC.


Question: } return nil, err } <ins> if !matchImageTagOrSHA(resp, image) { return nil, imageNotFoundError{ID: image} } </ins> return &resp, nil }

We had rather strange case today when after rolling update one of the kubelet's started wrong container. For this specific replicationcontroller we are using What happened was that after rolling-update when i started to debug the issue i discovered that one of the kubelet's did not pull the image: When looking into what image it actually started i saw: When checking from the actual node i saw this It seems that instead of starting myimage with tag 2208 kubelet started image tagged as 1393 since the docker image id starts with 2208.


Question: github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.0.2 h1:9yCKha/T5XdGtO0q9Q9a6T5NUCsTn/DrBg0D7ufOcFM= github.com/opencontainers/image-spec v1.0.2/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= <del> github.com/opencontainers/runc v1.1.4 h1:nRCz/8sKg6K6jgYAFLDlXzPeITBZJyX28DBVhWD+5dg= </del> github.com/opencontainers/runc v1.1.4/go.mod h1:1J5XiS+vdZ3wCyZybsuxXZWGrgSr8fFJHLXuG2PsnNg= <ins> github.com/opencontainers/runc v1.1.5 h1:L44KXEpKmfWDcS02aeGm8QNTFXTo2D+8MYGDIJ/GDEs= github.com/opencontainers/runc v1.1.5/go.mod h1:1J5XiS+vdZ3wCyZybsuxXZWGrgSr8fFJHLXuG2PsnNg= </ins> github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/opencontainers/runtime-spec v1.0.3-0.20200929063507-e6143ca7d51d/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=

opencontainers/runc has fixed three CVEs: Latest runc is pulled by k/k Check runc version in file No response n/a n/a <details</details<details</details<details</details<details</details
/assign /sig security architecture release
/triage accepted


Question: <ins> /* Copyright 2023 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package plugin import ( "testing" "github.com/stretchr/testify/assert" v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) func getFakeNode() (*v1.Node, error) { return &v1.Node{ObjectMeta: metav1.ObjectMeta{Name: "worker"}}, nil } func TestRegistrationHandler_ValidatePlugin(t *testing.T) { newRegistrationHandler := func() *RegistrationHandler { return NewRegistrationHandler(nil, getFakeNode) } for _, test := range []struct { description string handler     func() *RegistrationHandler pluginName  string endpoint    string versions    []string shouldError bool }{ { description: "no versions provided", handler:     newRegistrationHandler, shouldError: true, }, { description: "unsupported version", handler:     newRegistrationHandler, versions:    []string{"v2.0.0"}, shouldError: true, }, { description: "plugin already registered with a higher supported version", handler: func() *RegistrationHandler { handler := newRegistrationHandler() if err := handler.RegisterPlugin("this-plugin-already-exists-and-has-a-long-name-so-it-doesnt-collide", "", []string{"v1.1.0"}, nil); err != nil { t.Fatal(err) } return handler }, pluginName:  "this-plugin-already-exists-and-has-a-long-name-so-it-doesnt-collide", versions:    []string{"v1.0.0"}, shouldError: true, }, { description: "should validate the plugin", handler:     newRegistrationHandler, pluginName:  "this-is-a-dummy-plugin-with-a-long-name-so-it-doesnt-collide", versions:    []string{"v1.3.0"}, }, } { t.Run(test.description, func(t *testing.T) { handler := test.handler() err := handler.ValidatePlugin(test.pluginName, test.endpoint, test.versions) if test.shouldError { assert.Error(t, err) } else { assert.NoError(t, err) } }) } t.Cleanup(func() { handler := newRegistrationHandler() handler.DeRegisterPlugin("this-plugin-already-exists-and-has-a-long-name-so-it-doesnt-collide") handler.DeRegisterPlugin("this-is-a-dummy-plugin-with-a-long-name-so-it-doesnt-collide") }) } </ins>

The PR above moved everything related to into because that was where most of the code already was, but didn't renamed the files to avoid a larger diff and merge conflicts. Once is merged, we can do the renaming. Code clarity. /sig node
/triage accepted /priority important-soon
/assign
a PR can be created for this now.
will do


Question: var map_EndpointPort = map[string]string{ "":         "EndpointPort represents a Port used by an EndpointSlice", <del> "name":     "The name of this port. All ports in an EndpointSlice must have a unique name. If the EndpointSlice is dervied from a Kubernetes service, this corresponds to the Service.ports[].name. Name must either be an empty string or pass IANA_SVC_NAME validation: * must be no more than 15 characters long * may contain only [-a-z0-9] * must contain at least one letter [a-z] * it must not start or end with a hyphen, nor contain adjacent hyphens Default is empty string.", </del> <ins> "name":     "The name of this port. All ports in an EndpointSlice must have a unique name. If the EndpointSlice is dervied from a Kubernetes service, this corresponds to the Service.ports[].name. Name must either be an empty string or pass DNS_LABEL validation: * must be no more than 63 characters long. * must consist of lower case alphanumeric characters or '-'. * must start and end with an alphanumeric character. Default is empty string.", </ins> "protocol": "The IP protocol for this port. Must be UDP, TCP, or SCTP. Default is TCP.", "port":     "The port number of the endpoint. If this is not specified, ports are not restricted and must be interpreted in the context of the specific consumer.", }

What happened: No endpoints created in the EndpointSlice for my service due to the port on the service being too long. Logs show: EndpointSlice created has no endpoint: What you expected to happen: Service that works with Endpoints should work with EndpointSlice. If this is intentional, it should be clearly documented How to reproduce it (as minimally and precisely as possible): Create a service with a long port name Environment: Kubernetes version (use ): 1.16 Cloud provider or hardware configuration: kind
/sig networking
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /sig networking Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network These SIGs are my best guesses for this issue. Please comment if I am incorrect about one. I am a bot run by vllry.
For a rough measure of impact, Istio had 5 ports that were over this limit (and many more that were right at 15 characters), so I suspect it may not be terribly rare.
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
This needs to be resolved before endpointslices graduate from alpha
/assign
/remove-triage unresolved


Question: "github.com/GoogleCloudPlatform/kubernetes/pkg/admission" "github.com/GoogleCloudPlatform/kubernetes/pkg/api" <ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/api/errors" </ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/client" "github.com/GoogleCloudPlatform/kubernetes/pkg/client/cache" )

While running one of my soak tests I noticed the error message: I don't see why I should get this error message. Why should it matter if the namespace already exists? It was created as part of the service creation call just beforehand. And the second attempt to create this pod in exactly the same namespace works. Is this a bug?
Running a second time did not produce this error.
I agree it should not matter. The error is produced there, and I suspect there were multiple concurrent calls being made that try to create a resource in a namespace that had not yet existed. We need to update the code in question to swallow the error if the namespace already exists.
FYI - longer term, I would like to eliminate the use of autoprovision of namespace and require you to first create a namespace manually. what is there now is more to preserve existing behavior prior to namespace as object was introduced.


Question: - name: config mountPath: /etc/config nodeSelector: <del> # TODO(liggitt): switch to node.kubernetes.io/masq-agent-ds-ready in 1.16 beta.kubernetes.io/masq-agent-ds-ready: "true" </del> <ins> node.kubernetes.io/masq-agent-ds-ready: "true" </ins> volumes: - name: config configMap:

xref In 1.16: change the metadata-proxy addon to use change the kube-proxy addon to use change the masq-agent addon to use stop setting , , and in kube-up
Hey I'm trying to start to contribute to kubernetes and this seem to be an easy task. Can I help with this one?


Question: // and updating Scale for any resource which implements the `scale` subresource, // as long as that subresource operates on a version of scale convertable to // autoscaling.Scale. <del> package scale </del> <ins> package scale // import "k8s.io/client-go/scale" </ins>

Look at one of the directories under (Please don't pick more than one directory for one PR as it will cause issues for reviewers) Find all the files named (example find -name ) See the following link it has a this tells tools how this package needs to be imported. So we should ensure that each has an import Add a description appropriate to the package (similar to the one below) Look through sub directories and see if there are any other important directories that are likely to be used by outside users and add a Why are we doing this? We perfer to use vanity urls instead of directly using to refer to code, so this helps ensure that tools honor our wish. See It would be good to have some package level documentation since these directories under staging/ are meant to be for use by folks outside of kubernetes github org. /good-first-issue
/kind documentation /sig architecture
currently working my through Not going to use / assign since there could be a number of PRs for this issue.
/lifecycle active /good-first-issue edit: hmm..
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /lifecycle active /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
May I try on
IIUC this means that there needs to be a even at the root of the repo? Then how does work? :thinking:
Hi , can I make a PR for this directory ?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or .


Question: // and updating Scale for any resource which implements the `scale` subresource, // as long as that subresource operates on a version of scale convertable to // autoscaling.Scale. <del> package scale </del> <ins> package scale // import "k8s.io/client-go/scale" </ins>

/lifecycle rotten
I guess this is still not done, was supposed to partially fix this issue. /reopen
Reopened this issue. <detailsIn response to : I guess this is still not done, was supposed to partially fix this issue. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-lifecycle rotten /area code-organization


Question: Reject      = "REJECT" ToDest      = "--to-destination " Recent      = "recent " <ins> MatchSet    = "--match-set " </ins> ) type Rule map[string]string

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Service loadbalancer source range don't work in ipvs proxy mode. Which result in gci-gce-ipvs CI failed. http://k8s- How to reproduce it (as minimally and precisely as possible): In gce environment. a service with backend. service type to service util service external IP is specified. the service from pod inside and pod out of the . Environment: Kubernetes version (use ): master Cloud provider or hardware configuration: gce OS (e.g. from /etc/os-release): ubuntu Kernel (e.g. ): Install tools: cluster/kube- Others:
/kind bug /sig network /label ipvs
/area ipvs
The test seems to be flaky now. Can you take a look please?
I don't know why, but I think this flaky have the similiar reason with other normal service-test cases, like Those flaky are pretty annoying, problem is, I have run those cases in my own environment many times, never fail... : ( But still, I'll do my best to find the reason.


Question: <del> apiVersion: extensions/v1beta1 </del> <ins> apiVersion: networking.k8s.io/v1beta1 </ins> kind: Ingress metadata: name: static-ip

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: Which test(s) are failing: Since when has it been failing: Testgrid link: https://k8s- Reason for failure: Anything else we need to know: /cc /milestone v1.19 /priority important-soon /sig network
Ref:
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
my fu points me to this commit [] Fix multiple conversion tests that belongs to this PR , that seems related can you confirm if that PR is causing this test to fail? (should we gate on this test?)
Yes, that PR is related to the failure. The test utility is trying to convert extensions/v1beta1 data read from various files directly to a go type. This was never a valid thing to do, and happened to silently succeed because the two structs had an identical structure. This is exactly the type of bug was designed to flush out
the correct fix is to update the test/e2e/testing- files to
/assign
It was my PR that broke this - I can also fix that.
Thanks, o it was the commit that was good but the test that was wrong :upsidedownface: , At least I got right the commit :sweat_smile:
is out for review (talked with offline, he was aware that I'll send that out)


Question: <del> linux/amd64=alpine:3.12 linux/arm=arm32v6/alpine:3.12 linux/arm64=arm64v8/alpine:3.12 linux/ppc64le=ppc64le/alpine:3.12 linux/s390x=s390x/alpine:3.12 </del> <ins> linux/amd64=alpine:3.16 linux/arm=arm32v6/alpine:3.16 linux/arm64=arm64v8/alpine:3.16 linux/ppc64le=ppc64le/alpine:3.16 linux/s390x=s390x/alpine:3.16 </ins> windows/amd64/1809=REGISTRY/busybox:1.29-2-windows-amd64-1809 windows/amd64/ltsc2022=REGISTRY/busybox:1.29-2-windows-amd64-ltsc2022

The test image currently uses as its BASEIMAGE: Per , v3.12 reached end of support on 2022-05-01 and is no longer receiving regular bugfixes or security fixes. Current versions of the image appear vulnerable to known CVEs as a result. I don't know if a fix is quite as simple as just The image's base should be an in-support version which is maintained with at least security fixes. No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig testing
is like that, but then there are 2 additional steps the changes the image that has been built with the postsubmit job the repo to use the new image More detail in If you want to try, please make sure you are going to follow up with all the steps, you can see other promotions PRs too /help
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : is like that, but then there are 2 additional steps promote the image that has been built with the postsubmit job update the repo to use the new image </details
Thank you for this information! Is the manual and of images described in that same document something that you would consider a prerequisite for a proposed change like this? Or is it safe to assume that the automated PR test bucket(s) would provide sufficient test coverage?
I always trust the CI system
Good enough for me! Leading off with
/triage accepted


Question: updateMeta(&podMeta, "Pod") setMeta := set.TypeMeta updateMeta(&setMeta, "StatefulSet") <del> policy := set.Spec.PersistentVolumeClaimRetentionPolicy </del> <ins> policy := getPersistentVolumeClaimRetentionPolicy(set) </ins> const retain = apps.RetainPersistentVolumeClaimRetentionPolicyType const delete = apps.DeletePersistentVolumeClaimRetentionPolicyType switch {

The replicas is 3 but just create 1 pod 'web-0' and one pvc, then kube-controller-manager panic happend. Should create 3 pod and 3 pvc add '--feature-gates=StatefulSetAutoDeletePVC=true' arg for kube-controller-manager, then create statefulset with 2 replicas and volumeClaimTemplates. No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig apps
I can reproduce the issue and want to fix it. It seems like variables cause . /assign
/triage accepted
Great find! Does this only happen on an upgraded cluster? I'll spin up a cluster and see if the e2e test catches this problem.
The e2e tests are still passing on a 1.24 cluster, so I suspect this is an upgrade problem. I wonder if there are other holes here.
I installed the Kubernetes 1.25 cluster directly using and still reproduced this issue, so it was probably not caused by upgrading the cluster.
Interesting. I thought that would set the policy to be non-nil. Let's keep this bug open after you submit your PR as I think something else is going on here---I don't think we should be seeing the null policy in the first place.
I suspect that somewhere accidentally set it to nil, and I'll keep looking at this code.
I tried the following (and also with parallel pod management), and it didn't seem to cause the panic. This was a 1.24 cluster, maybe this is only happening on head versions?
I just tried this on a kind cluster running on k8s head, and the problem didn't seem to reproduce. But this was setting the feature flag default in I wonder----does the api server need to have the feature gate as well? That would explain why defaulting wasn't happening.
As far as I'm concerned,I just set the in kube-controller-manager manifests to reproduce this issue. I repeat the yaml used as follows: I tested that in statefulset yaml, setting or not would cause the component to panic. The panic logs are the same as above, so ignore them here.


Question: updateMeta(&podMeta, "Pod") setMeta := set.TypeMeta updateMeta(&setMeta, "StatefulSet") <del> policy := set.Spec.PersistentVolumeClaimRetentionPolicy </del> <ins> policy := getPersistentVolumeClaimRetentionPolicy(set) </ins> const retain = apps.RetainPersistentVolumeClaimRetentionPolicyType const delete = apps.DeletePersistentVolumeClaimRetentionPolicyType switch {

statefulset with : The environment
I believe you---this doesn't seem to be well-documented, but it's my understanding that you need to set the feature gate on all of the system components. Obviously failing to do so shouldn't cause a panic, so thank you again for finding & fixing this bug! Let me ask around to see what the expected behavior is. I know there may be skew between kubelets and master versions, but I didn't expect to have to deal with skew between master components.
I read the code, and now I understand what you mean. This feature gate must be enabled in both the and components. We need to describe it in the to prevent someone from ignoring the component, which also needs to enable the feature gate.
I don't think we need all the system components, like the component, right? I read the original KEP documentation. This feature gate only involves components and :
I had a bit of discussion about this with others. Because upgrades don't occur atomically, any feature gate has to work regardless of how the other system components are configured (things shouldn't panic!). I suppose if the cluster administrator wants to track what needs to be enabled on each feature they can try to sequence which component has which feature gate. If it were me I'd try to make all the components consistent though.
This is certainly a method, but it is a bit more cumbersome for the administrator. It is best to have a place for the document to be queried by the administrator and avoid confusion. I don't think everyone is very familiar with the k8s mechanism. I suggest adding a partial description to this section of the : Before: After: I can submit this PR on .
This is going to be true of any api change. When you tried to use it with only adding the controller manager feature gate, I think you would be able to set the retain policy, but there wouldn't be any defaulting or validation (?).
I've seen other feature gates that say what components are needed to enable it to use the new feature. For example: If it's not necessary, then I close .
only adding the controller manager feature gate, I think you would be able to set the retain policy, but there wouldn't be any defaulting or validation (?). I remember just enabling it in kube-controller-manager.


Question: updateMeta(&podMeta, "Pod") setMeta := set.TypeMeta updateMeta(&setMeta, "StatefulSet") <del> policy := set.Spec.PersistentVolumeClaimRetentionPolicy </del> <ins> policy := getPersistentVolumeClaimRetentionPolicy(set) </ins> const retain = apps.RetainPersistentVolumeClaimRetentionPolicyType const delete = apps.DeletePersistentVolumeClaimRetentionPolicyType switch {

it using the retain policy, but the statefulset does not have the policy from output of (It should be the apiserver authentication forbids it)
Yeah I think that makes sense. Thanks for finding this issue! After your PR with the panic fix is submitted, I think we can close this?
Yes. I agree. /close
Closing this issue. <detailsIn response to : Yes. I agree. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: if len(obj.Type) == 0 { obj.Nullable = false // because this does not roundtrip through go-openapi } <ins> if obj.XIntOrString { obj.Type = "" } </ins> }, func(obj *apiextensions.JSONSchemaPropsOrBool, c fuzz.Continue) { if c.RandBool() {

The server-side validation does not check the type of values with . Create a CRD like: Then create a CR: It is not rejected, but should because must not be a boolean.


Question: client := http.Client{} request, err := http.NewRequest("PATCH", server.URL+"/api/version/namespaces/default/simple/"+ID, bytes.NewReader([]byte(`{"labels":{"foo":"bar"}}`))) <del> request.Header.Set("Content-Type", "application/merge-patch+json") </del> <ins> request.Header.Set("Content-Type", "application/merge-patch+json; charset=UTF-8") </ins> _, err = client.Do(request) if err != nil { t.Errorf("unexpected error: %v", err)

Given a request with a Content-Type header set to , the server will respond with an error because it does not exactly match the existing strings. Some browsers (Firefox, see: ) will automatically add to a Content-Type header when making an XHR request.


Question: "tagline" : "You Know, for Search" } ``` <del> Visiting the URL `http://104.154.91.224:5601` should show the Kibana viewer for the logging information stored in the Elasticsearch service running at `http://104.154.81.135:9200`. </del> <ins> Visiting the URL `https://130.211.122.180/api/v1beta3/proxy/namespaces/default/services/kibana-logging` should show the Kibana viewer for the logging information stored in the Elasticsearch service. </ins>

Hi , I followed the instructions in examples/logging-demo, and there was no "external IP" field for elasticsearch-logging or kibana-logging, as shown below: I start up my cluster via cluster/kube- Did I miss something or is it a bug? Thanks.
Correct. This demo does not create an ELB so there is no external IP address. Access is via service proxy at the master. Type: and you should see the URLs for Elasticsearch and Kibana.
please let us know whether: a. Satnam's comment resolves your issue b. the demo should be updated to prevent this misunderstanding c. the issue isn't resolved Thank you!
Yes, Satnam's comment resolve the issue. I updated the demo in .
Awesome, thanks for sending the patch! We'll discuss there. Always fantastic to have docs updated regularly by folks trying them.


Question: } // Uses the same niceness level as cadvisor.fs does when running du // Uses -B 1 to always scale to a blocksize of 1 byte <del> out, err := exec.Command("nice", "-n", "19", "du", "-s", "-B", "1", path).CombinedOutput() </del> <ins> out, err := exec.Command("nice", "-n", "19", "du", "-x", "-s", "-B", "1", path).CombinedOutput() </ins> if err != nil { <del> return nil, fmt.Errorf("failed command 'du' ($ nice -n 19 du -s -B 1) on path %s with error %v", path, err) </del> <ins> return nil, fmt.Errorf("failed command 'du' ($ nice -n 19 du -x -s -B 1) on path %s with error %v", path, err) </ins> } used, err := resource.ParseQuantity(strings.Fields(string(out))[0]) if err != nil {

I'm using K8S 1.10 and have a specific use case where I'm trying to use an emptyDir to share mounts between two containers in a pod. I'm using the MountPropagation feature successfully, but while debugging another issue I noticed that the node was running a very long running du command. Looking into the source I noted: Which runs du on any emptyDir, do measure disk usage. I recommend that in this case we should add the -x flag that will skip any mounted directories as they do not count against the file usage and would make these scans a lot faster. I could of course add a simple pull request for this but first wanted to see if this was known and on purpose. To reproduce bind mount/loop mount/device mount any large/slow directory into an emptyDir volume and watch as the du process slogs away Environment: Kubernetes 1.10 GKE <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I think it makes sense to ignore submounts.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
do you still want to submit a PR to fix this? Thanks!
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or .


Question: } // Uses the same niceness level as cadvisor.fs does when running du // Uses -B 1 to always scale to a blocksize of 1 byte <del> out, err := exec.Command("nice", "-n", "19", "du", "-s", "-B", "1", path).CombinedOutput() </del> <ins> out, err := exec.Command("nice", "-n", "19", "du", "-x", "-s", "-B", "1", path).CombinedOutput() </ins> if err != nil { <del> return nil, fmt.Errorf("failed command 'du' ($ nice -n 19 du -s -B 1) on path %s with error %v", path, err) </del> <ins> return nil, fmt.Errorf("failed command 'du' ($ nice -n 19 du -x -s -B 1) on path %s with error %v", path, err) </ins> } used, err := resource.ParseQuantity(strings.Fields(string(out))[0]) if err != nil {

/lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: } } if test.disableAttach { <del> gomega.Expect(err).To(gomega.HaveOccurred(), "Unexpected VolumeAttachment found") </del> <ins> framework.ExpectError(err, "Unexpected VolumeAttachment found") </ins> } })

What happened: After fixing golint failures under (now still in progress), we need to put for each and . Then common error checking code has become like: We can replace this code with which is more readable and easy to be understood what the code does. Now we have following e2e test directories which can be replaced: [x] test/e2e/apimachinery/ () [x] test/e2e/apps/ () [x] test/e2e/auth/ () [x] test/e2e/common/ () [x] test/e2e/node/ () [x] test/e2e/storage/ () [x] () [x] () [x] () [x] () [x] () [x] () [x] [x] test/e2e/storage/persistentvolumes- [x] () [x] () [x] [x] [x] [x] [x] ()
/sig testing /kind cleanup
/assign PR will be submitted soon.
Fix the following in
Fix in
Fix in
Hi, maybe I could take files beneath test/e2e/storage/vsphere/
oops, I'm finished and prepare for PR.
Thank you That's OK
Fix the following in
/assign
All related PRs were merged. I think this ticket can be closed. BTW, should we also replace to ?
Thanks for your work. thing is managed with We are almost finishing the work also :-)
/remove-kind bug
I found in these files. I'm working on :
I can work to the following item.


Question: action MUST be validated. release: v1.21 file: test/e2e/network/service.go <ins> - testname: Service, deletes a collection of services codename: '[sig-network] Services should delete a collection of services [Conformance]' description: Create three services with the required labels and ports. It MUST locate three services in the test namespace. It MUST succeed at deleting a collection of services via a label selector. It MUST locate only one service after deleting the service collection. release: v1.23 file: test/e2e/network/service.go </ins> - testname: Find Kubernetes Service in default Namespace codename: '[sig-network] Services should find a service from listing all namespaces [Conformance]'

[X] Two weeks soak end date: Nov 18 According to this APIsnoop query, there are still a remaining Service endpoint which is untested. A service resource didn’t support endpoint until was merged. Create a watch to track service events Create a service with a static label. Confirm via the watch that the service was created. Delete the service via the service with the dynamic client with . Confirm via the watch that the service was deleted. The above steps have been implemented in the follwing . This query shows all the endpoints hit within a short period of running the above e2e test If a test with these calls gets merged, test coverage will go up by 1 point This test is also created with the goal of conformance promotion. /sig testing /sig network /area conformance
/triage accepted
Waiting on 2 week soak before promoting the test to conformance /reopen
Reopened this issue. <detailsIn response to : Waiting on 2 week soak before promoting the test to conformance /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: allErrs = append(allErrs, metavalidation.ValidateLabels(endpoint.Topology, topologyPath)...) if endpoint.Hostname != nil { <del> for _, msg := range validation.IsDNS1123Label(*endpoint.Hostname) { allErrs = append(allErrs, field.Invalid(idxPath.Child("hostname"), *endpoint.Hostname, msg)) } </del> <ins> allErrs = append(allErrs, apivalidation.ValidateDNS1123Label(*endpoint.Hostname, idxPath.Child("hostname"))...) </ins> } }

What happened: No endpoints created in the EndpointSlice for my service due to the port on the service being too long. Logs show: EndpointSlice created has no endpoint: What you expected to happen: Service that works with Endpoints should work with EndpointSlice. If this is intentional, it should be clearly documented How to reproduce it (as minimally and precisely as possible): Create a service with a long port name Environment: Kubernetes version (use ): 1.16 Cloud provider or hardware configuration: kind
/sig networking
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /sig networking Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network These SIGs are my best guesses for this issue. Please comment if I am incorrect about one. I am a bot run by vllry.
For a rough measure of impact, Istio had 5 ports that were over this limit (and many more that were right at 15 characters), so I suspect it may not be terribly rare.
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry.
This needs to be resolved before endpointslices graduate from alpha
/assign
/remove-triage unresolved


Question: <ins> /* Copyright 2016 The Kubernetes Authors All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Code generated by protoc-gen-gogo. // source: k8s.io/kubernetes/pkg/apis/apps/v1alpha1/generated.proto // DO NOT EDIT! /* Package v1alpha1 is a generated protocol buffer package. It is generated from these files: k8s.io/kubernetes/pkg/apis/apps/v1alpha1/generated.proto It has these top-level messages: PetSet PetSetList PetSetSpec PetSetStatus */ package v1alpha1 import proto "github.com/gogo/protobuf/proto" import fmt "fmt" import math "math" import _ "github.com/gogo/protobuf/gogoproto" import _ "k8s.io/kubernetes/pkg/api/resource" import k8s_io_kubernetes_pkg_api_unversioned "k8s.io/kubernetes/pkg/api/unversioned" import k8s_io_kubernetes_pkg_api_v1 "k8s.io/kubernetes/pkg/api/v1" import _ "k8s.io/kubernetes/pkg/util/intstr" import io "io" // Reference imports to suppress errors if they are not otherwise used. var _ = proto.Marshal var _ = fmt.Errorf var _ = math.Inf func (m *PetSet) Reset()         { *m = PetSet{} } func (m *PetSet) String() string { return proto.CompactTextString(m) } func (*PetSet) ProtoMessage()    {} func (m *PetSetList) Reset()         { *m = PetSetList{} } func (m *PetSetList) String() string { return proto.CompactTextString(m) } func (*PetSetList) ProtoMessage()    {} func (m *PetSetSpec) Reset()         { *m = PetSetSpec{} } func (m *PetSetSpec) String() string { return proto.CompactTextString(m) } func (*PetSetSpec) ProtoMessage()    {} func (m *PetSetStatus) Reset()         { *m = PetSetStatus{} } func (m *PetSetStatus) String() string { return proto.CompactTextString(m) } func (*PetSetStatus) ProtoMessage()    {} func init() { proto.RegisterType((*PetSet)(nil), "k8s.io.kubernetes.pkg.apis.apps.v1alpha1.PetSet") proto.RegisterType((*PetSetList)(nil), "k8s.io.kubernetes.pkg.apis.apps.v1alpha1.PetSetList") proto.RegisterType((*PetSetSpec)(nil), "k8s.io.kubernetes.pkg.apis.apps.v1alpha1.PetSetSpec") proto.RegisterType((*PetSetStatus)(nil), "k8s.io.kubernetes.pkg.apis.apps.v1alpha1.PetSetStatus") } func (m *PetSet) Marshal() (data []byte, err error) { size := m.Size() data = make([]byte, size) n, err := m.MarshalTo(data) if err != nil { return nil, err } return data[:n], nil } func (m *PetSet) MarshalTo(data []byte) (int, error) { var i int _ = i var l int _ = l data[i] = 0xa i++ i = encodeVarintGenerated(data, i, uint64(m.ObjectMeta.Size())) n1, err := m.ObjectMeta.MarshalTo(data[i:]) if err != nil { return 0, err } i += n1 data[i] = 0x12 i++ i = encodeVarintGenerated(data, i, uint64(m.Spec.Size())) n2, err := m.Spec.MarshalTo(data[i:]) if err != nil { return 0, err } i += n2 data[i] = 0x1a i++ i = encodeVarintGenerated(data, i, uint64(m.Status.Size())) n3, err := m.Status.MarshalTo(data[i:]) if err != nil { return 0, err } i += n3 return i, nil } func (m *PetSetList) Marshal() (data []byte, err error) { size := m.Size() data = make([]byte, size) n, err := m.MarshalTo(data) if err != nil { return nil, err } return data[:n], nil } func (m *PetSetList) MarshalTo(data []byte) (int, error) { var i int _ = i var l int _ = l data[i] = 0xa i++ i = encodeVarintGenerated(data, i, uint64(m.ListMeta.Size())) n4, err := m.ListMeta.MarshalTo(data[i:]) if err != nil { return 0, err } i += n4 if len(m.Items) > 0 { for _, msg := range m.Items { data[i] = 0x12 i++ i = encodeVarintGenerated(data, i, uint64(msg.Size())) n, err := msg.MarshalTo(data[i:]) if err != nil { return 0, err } i += n } } return i, nil } func (m *PetSetSpec) Marshal() (data []byte, err error) { size := m.Size() data = make([]byte, size) n, err := m.MarshalTo(data) if err != nil { return nil, err } return data[:n], nil } func (m *PetSetSpec) MarshalTo(data []byte) (int, error) { var i int _ = i var l int _ = l if m.Replicas != nil { data[i] = 0x8 i++ i = encodeVarintGenerated(data, i, uint64(*m.Replicas)) } if m.Selector != nil { data[i] = 0x12 i++ i = encodeVarintGenerated(data, i, uint64(m.Selector.Size())) n5, err := m.Selector.MarshalTo(data[i:]) if err != nil { return 0, err } i += n5 } data[i] = 0x1a i++ i = encodeVarintGenerated(data, i, uint64(m.Template.Size())) n6, err := m.Template.MarshalTo(data[i:]) if err != nil { return 0, err } i += n6 if len(m.VolumeClaimTemplates) > 0 { for _, msg := range m.VolumeClaimTemplates { data[i] = 0x22 i++ i = encodeVarintGenerated(data, i, uint64(msg.Size())) n, err := msg.MarshalTo(data[i:]) if err != nil { return 0, err } i += n } } data[i] = 0x2a i++ i = encodeVarintGenerated(data, i, uint64(len(m.ServiceName))) i += copy(data[i:], m.ServiceName) return i, nil } func (m *PetSetStatus) Marshal() (data []byte, err error) { size := m.Size() data = make([]byte, size) n, err := m.MarshalTo(data) if err != nil { return nil, err } return data[:n], nil } func (m *PetSetStatus) MarshalTo(data []byte) (int, error) { var i int _ = i var l int _ = l if m.ObservedGeneration != nil { data[i] = 0x8 i++ i = encodeVarintGenerated(data, i, uint64(*m.ObservedGeneration)) } data[i] = 0x10 i++ i = encodeVarintGenerated(data, i, uint64(m.Replicas)) return i, nil } func encodeFixed64Generated(data []byte, offset int, v uint64) int { data[offset] = uint8(v) data[offset+1] = uint8(v >> 8) data[offset+2] = uint8(v >> 16) data[offset+3] = uint8(v >> 24) data[offset+4] = uint8(v >> 32) data[offset+5] = uint8(v >> 40) data[offset+6] = uint8(v >> 48) data[offset+7] = uint8(v >> 56) return offset + 8 } func encodeFixed32Generated(data []byte, offset int, v uint32) int { data[offset] = uint8(v) data[offset+1] = uint8(v >> 8) data[offset+2] = uint8(v >> 16) data[offset+3] = uint8(v >> 24) return offset + 4 } func encodeVarintGenerated(data []byte, offset int, v uint64) int { for v >= 1<<7 { data[offset] = uint8(v&0x7f | 0x80) v >>= 7 offset++ } data[offset] = uint8(v) return offset + 1 } func (m *PetSet) Size() (n int) { var l int _ = l l = m.ObjectMeta.Size() n += 1 + l + sovGenerated(uint64(l)) l = m.Spec.Size() n += 1 + l + sovGenerated(uint64(l)) l = m.Status.Size() n += 1 + l + sovGenerated(uint64(l)) return n } func (m *PetSetList) Size() (n int) { var l int _ = l l = m.ListMeta.Size() n += 1 + l + sovGenerated(uint64(l)) if len(m.Items) > 0 { for _, e := range m.Items { l = e.Size() n += 1 + l + sovGenerated(uint64(l)) } } return n } func (m *PetSetSpec) Size() (n int) { var l int _ = l if m.Replicas != nil { n += 1 + sovGenerated(uint64(*m.Replicas)) } if m.Selector != nil { l = m.Selector.Size() n += 1 + l + sovGenerated(uint64(l)) } l = m.Template.Size() n += 1 + l + sovGenerated(uint64(l)) if len(m.VolumeClaimTemplates) > 0 { for _, e := range m.VolumeClaimTemplates { l = e.Size() n += 1 + l + sovGenerated(uint64(l)) } } l = len(m.ServiceName) n += 1 + l + sovGenerated(uint64(l)) return n } func (m *PetSetStatus) Size() (n int) { var l int _ = l if m.ObservedGeneration != nil { n += 1 + sovGenerated(uint64(*m.ObservedGeneration)) } n += 1 + sovGenerated(uint64(m.Replicas)) return n } func sovGenerated(x uint64) (n int) { for { n++ x >>= 7 if x == 0 { break } } return n } func sozGenerated(x uint64) (n int) { return sovGenerated(uint64((x << 1) ^ uint64((int64(x) >> 63)))) } func (m *PetSet) Unmarshal(data []byte) error { l := len(data) iNdEx := 0 for iNdEx < l { preIndex := iNdEx var wire uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ wire |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } fieldNum := int32(wire >> 3) wireType := int(wire & 0x7) if wireType == 4 { return fmt.Errorf("proto: PetSet: wiretype end group for non-group") } if fieldNum <= 0 { return fmt.Errorf("proto: PetSet: illegal tag %d (wire type %d)", fieldNum, wire) } switch fieldNum { case 1: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field ObjectMeta", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } if err := m.ObjectMeta.Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex case 2: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field Spec", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } if err := m.Spec.Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex case 3: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field Status", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } if err := m.Status.Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipGenerated(data[iNdEx:]) if err != nil { return err } if skippy < 0 { return ErrInvalidLengthGenerated } if (iNdEx + skippy) > l { return io.ErrUnexpectedEOF } iNdEx += skippy } } if iNdEx > l { return io.ErrUnexpectedEOF } return nil } func (m *PetSetList) Unmarshal(data []byte) error { l := len(data) iNdEx := 0 for iNdEx < l { preIndex := iNdEx var wire uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ wire |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } fieldNum := int32(wire >> 3) wireType := int(wire & 0x7) if wireType == 4 { return fmt.Errorf("proto: PetSetList: wiretype end group for non-group") } if fieldNum <= 0 { return fmt.Errorf("proto: PetSetList: illegal tag %d (wire type %d)", fieldNum, wire) } switch fieldNum { case 1: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field ListMeta", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } if err := m.ListMeta.Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex case 2: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field Items", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } m.Items = append(m.Items, PetSet{}) if err := m.Items[len(m.Items)-1].Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipGenerated(data[iNdEx:]) if err != nil { return err } if skippy < 0 { return ErrInvalidLengthGenerated } if (iNdEx + skippy) > l { return io.ErrUnexpectedEOF } iNdEx += skippy } } if iNdEx > l { return io.ErrUnexpectedEOF } return nil } func (m *PetSetSpec) Unmarshal(data []byte) error { l := len(data) iNdEx := 0 for iNdEx < l { preIndex := iNdEx var wire uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ wire |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } fieldNum := int32(wire >> 3) wireType := int(wire & 0x7) if wireType == 4 { return fmt.Errorf("proto: PetSetSpec: wiretype end group for non-group") } if fieldNum <= 0 { return fmt.Errorf("proto: PetSetSpec: illegal tag %d (wire type %d)", fieldNum, wire) } switch fieldNum { case 1: if wireType != 0 { return fmt.Errorf("proto: wrong wireType = %d for field Replicas", wireType) } var v int32 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ v |= (int32(b) & 0x7F) << shift if b < 0x80 { break } } m.Replicas = &v case 2: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field Selector", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } if m.Selector == nil { m.Selector = &k8s_io_kubernetes_pkg_api_unversioned.LabelSelector{} } if err := m.Selector.Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex case 3: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field Template", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } if err := m.Template.Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex case 4: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field VolumeClaimTemplates", wireType) } var msglen int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ msglen |= (int(b) & 0x7F) << shift if b < 0x80 { break } } if msglen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + msglen if postIndex > l { return io.ErrUnexpectedEOF } m.VolumeClaimTemplates = append(m.VolumeClaimTemplates, k8s_io_kubernetes_pkg_api_v1.PersistentVolumeClaim{}) if err := m.VolumeClaimTemplates[len(m.VolumeClaimTemplates)-1].Unmarshal(data[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex case 5: if wireType != 2 { return fmt.Errorf("proto: wrong wireType = %d for field ServiceName", wireType) } var stringLen uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ stringLen |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } intStringLen := int(stringLen) if intStringLen < 0 { return ErrInvalidLengthGenerated } postIndex := iNdEx + intStringLen if postIndex > l { return io.ErrUnexpectedEOF } m.ServiceName = string(data[iNdEx:postIndex]) iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipGenerated(data[iNdEx:]) if err != nil { return err } if skippy < 0 { return ErrInvalidLengthGenerated } if (iNdEx + skippy) > l { return io.ErrUnexpectedEOF } iNdEx += skippy } } if iNdEx > l { return io.ErrUnexpectedEOF } return nil } func (m *PetSetStatus) Unmarshal(data []byte) error { l := len(data) iNdEx := 0 for iNdEx < l { preIndex := iNdEx var wire uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ wire |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } fieldNum := int32(wire >> 3) wireType := int(wire & 0x7) if wireType == 4 { return fmt.Errorf("proto: PetSetStatus: wiretype end group for non-group") } if fieldNum <= 0 { return fmt.Errorf("proto: PetSetStatus: illegal tag %d (wire type %d)", fieldNum, wire) } switch fieldNum { case 1: if wireType != 0 { return fmt.Errorf("proto: wrong wireType = %d for field ObservedGeneration", wireType) } var v int64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ v |= (int64(b) & 0x7F) << shift if b < 0x80 { break } } m.ObservedGeneration = &v case 2: if wireType != 0 { return fmt.Errorf("proto: wrong wireType = %d for field Replicas", wireType) } m.Replicas = 0 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowGenerated } if iNdEx >= l { return io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ m.Replicas |= (int32(b) & 0x7F) << shift if b < 0x80 { break } } default: iNdEx = preIndex skippy, err := skipGenerated(data[iNdEx:]) if err != nil { return err } if skippy < 0 { return ErrInvalidLengthGenerated } if (iNdEx + skippy) > l { return io.ErrUnexpectedEOF } iNdEx += skippy } } if iNdEx > l { return io.ErrUnexpectedEOF } return nil } func skipGenerated(data []byte) (n int, err error) { l := len(data) iNdEx := 0 for iNdEx < l { var wire uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return 0, ErrIntOverflowGenerated } if iNdEx >= l { return 0, io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ wire |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } wireType := int(wire & 0x7) switch wireType { case 0: for shift := uint(0); ; shift += 7 { if shift >= 64 { return 0, ErrIntOverflowGenerated } if iNdEx >= l { return 0, io.ErrUnexpectedEOF } iNdEx++ if data[iNdEx-1] < 0x80 { break } } return iNdEx, nil case 1: iNdEx += 8 return iNdEx, nil case 2: var length int for shift := uint(0); ; shift += 7 { if shift >= 64 { return 0, ErrIntOverflowGenerated } if iNdEx >= l { return 0, io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ length |= (int(b) & 0x7F) << shift if b < 0x80 { break } } iNdEx += length if length < 0 { return 0, ErrInvalidLengthGenerated } return iNdEx, nil case 3: for { var innerWire uint64 var start int = iNdEx for shift := uint(0); ; shift += 7 { if shift >= 64 { return 0, ErrIntOverflowGenerated } if iNdEx >= l { return 0, io.ErrUnexpectedEOF } b := data[iNdEx] iNdEx++ innerWire |= (uint64(b) & 0x7F) << shift if b < 0x80 { break } } innerWireType := int(innerWire & 0x7) if innerWireType == 4 { break } next, err := skipGenerated(data[start:]) if err != nil { return 0, err } iNdEx = start + next } return iNdEx, nil case 4: return iNdEx, nil case 5: iNdEx += 4 return iNdEx, nil default: return 0, fmt.Errorf("proto: illegal wireType %d", wireType) } } panic("unreachable") } var ( ErrInvalidLengthGenerated = fmt.Errorf("proto: negative length found during unmarshaling") ErrIntOverflowGenerated   = fmt.Errorf("proto: integer overflow") ) </ins>

During
any chance this is related to pass: https://k8s- fail: https://k8s- https://k8s-
This is because for some reason the submit queue rebuild didn't catch the flake before merging. Protobuf landed first, merging a new api object like petset should have failed.
Quick fix is to rebuild the proto marshalers and push.
Also, hack/verify-generated- is broken, which probably is the root cause.
Looking at it now
Three problems: -generated-protobuf wasn't updated when we made update-generated-protobuf use docker, so it didn't fail out script was verifying the old API groups, not any of the new API groups that have been merged without protobuf serializations (although pkg/api/serialization_test should have failed, possibly the job didn't build with the latest master?), which caused the tests to fail
Something about the merge bot is broken - it merged without all the tests being green: . That's why this broke -
Spawned since there is something wrong with update-generated-protobuf - will merge that to fix the failing test and then debug the generator in the morning.
The bot can race with pushes between when it starts testing and when it merges...


Question: } // inClusterClientConfig makes a config that will work from within a kubernetes cluster container environment. <del> type inClusterClientConfig struct{} </del> <ins> // Can take options overrides for flags explicitly provided to the command inside the cluster container. type inClusterClientConfig struct { overrides               *ConfigOverrides inClusterConfigProvider func() (*restclient.Config, error) } </ins> <del> var _ ClientConfig = inClusterClientConfig{} </del> <ins> var _ ClientConfig = &inClusterClientConfig{} </ins> <del> func (inClusterClientConfig) RawConfig() (clientcmdapi.Config, error) { </del> <ins> func (config *inClusterClientConfig) RawConfig() (clientcmdapi.Config, error) { </ins> return clientcmdapi.Config{}, fmt.Errorf("inCluster environment config doesn't support multiple clusters") } <del> func (inClusterClientConfig) ClientConfig() (*restclient.Config, error) { return restclient.InClusterConfig() </del> <ins> func (config *inClusterClientConfig) ClientConfig() (*restclient.Config, error) { if config.inClusterConfigProvider == nil { config.inClusterConfigProvider = restclient.InClusterConfig } icc, err := config.inClusterConfigProvider() if err != nil { return nil, err } // in-cluster configs only takes a host, token, or CA file // if any of them were individually provided, ovewrite anything else if config.overrides != nil { if server := config.overrides.ClusterInfo.Server; len(server) > 0 { icc.Host = server } if token := config.overrides.AuthInfo.Token; len(token) > 0 { icc.BearerToken = token } if certificateAuthorityFile := config.overrides.ClusterInfo.CertificateAuthority; len(certificateAuthorityFile) > 0 { icc.TLSClientConfig.CAFile = certificateAuthorityFile } } return icc, err </ins> } <del> func (inClusterClientConfig) Namespace() (string, bool, error) { </del> <ins> func (config *inClusterClientConfig) Namespace() (string, bool, error) { </ins> // This way assumes you've set the POD_NAMESPACE environment variable using the downward API. // This check has to be done first for backwards compatibility with the way InClusterConfig was originally set up if ns := os.Getenv("POD_NAMESPACE"); ns != "" {

Found in openshift, but I'm pretty sure the codepaths are the same. That means the token is getting ignored and in-cluster config is being used in preference to flags.
Is a similar issue?
Yep, kind of regression.
Sigh.
Yes, most likely the same cause.
I'm on it.
Unit test :)
Probably will want to cherrypick this into 1.5.
WIP in


Question: "The same PVC should be used after failover.") By("verifying the container output has 2 lines, indicating the pod has been created twice using the same regional PD.") <del> pod = getPod(c, ns, regionalPDLabels) </del> logs, err := framework.GetPodLogs(c, ns, pod.Name, "") Expect(err).NotTo(HaveOccurred(), "Error getting logs from pod %s in namespace %s", pod.Name, ns)

<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Testgrid: https://k8s-
/assign
The test's assumption about test parameter is wrong. Previously it assumed the group name is the same across all zones, but they are different in regional clusters. Looking into using node taints as the permanent fix, as suggested by the TODO comment in the test.
Hm I think taint based evictions is still an alpha feature.
OK. I propose the following solution: Get the zone of Schedule the StatefulSet pod to that zone using node affinity; wait for the pod to run Remove the node affinity from StatefulSet Delete
Can it be simplified? Schedule the pod Figure out which zone that pod got scheduled in Delete the instance group of that zone
Unfortunately I don't know of a good way to look up the instance group given the zone, since there could be instance groups from multiple clusters in the same project
How about: Schedule the pod Figure out the zone Taint all the nodes in the zone Kill the pod
Taint with NoSchedule? yeah that sounds good


Question: package plugin import ( <ins> "context" "fmt" "net" "os" "path/filepath" "sync" </ins> "testing" "github.com/stretchr/testify/assert" <del> v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" </del> <ins> "google.golang.org/grpc" drapb "k8s.io/kubelet/pkg/apis/dra/v1alpha4" "k8s.io/kubernetes/test/utils/ktesting" </ins> ) <del> func getFakeNode() (*v1.Node, error) { return &v1.Node{ObjectMeta: metav1.ObjectMeta{Name: "worker"}}, nil </del> <ins> const ( v1alpha4Version = "v1alpha4" ) type fakeV1alpha4GRPCServer struct { drapb.UnimplementedNodeServer } var _ drapb.NodeServer = &fakeV1alpha4GRPCServer{} func (f *fakeV1alpha4GRPCServer) NodePrepareResources(ctx context.Context, in *drapb.NodePrepareResourcesRequest) (*drapb.NodePrepareResourcesResponse, error) { return &drapb.NodePrepareResourcesResponse{Claims: map[string]*drapb.NodePrepareResourceResponse{"claim-uid": { Devices: []*drapb.Device{ { RequestNames: []string{"test-request"}, CDIDeviceIDs: []string{"test-cdi-id"}, }, }, }}}, nil } func (f *fakeV1alpha4GRPCServer) NodeUnprepareResources(ctx context.Context, in *drapb.NodeUnprepareResourcesRequest) (*drapb.NodeUnprepareResourcesResponse, error) { return &drapb.NodeUnprepareResourcesResponse{}, nil } type tearDown func() func setupFakeGRPCServer(version string) (string, tearDown, error) { p, err := os.MkdirTemp("", "dra_plugin") if err != nil { return "", nil, err } closeCh := make(chan struct{}) addr := filepath.Join(p, "server.sock") teardown := func() { close(closeCh) if err := os.RemoveAll(addr); err != nil { panic(err) } } listener, err := net.Listen("unix", addr) if err != nil { teardown() return "", nil, err } s := grpc.NewServer() switch version { case v1alpha4Version: fakeGRPCServer := &fakeV1alpha4GRPCServer{} drapb.RegisterNodeServer(s, fakeGRPCServer) default: return "", nil, fmt.Errorf("unsupported version: %s", version) } go func() { go func() { if err := s.Serve(listener); err != nil { panic(err) } }() <-closeCh s.GracefulStop() }() return addr, teardown, nil </ins> } <del> func TestRegistrationHandler_ValidatePlugin(t *testing.T) { newRegistrationHandler := func() *RegistrationHandler { return NewRegistrationHandler(nil, getFakeNode) </del> <ins> func TestGRPCConnIsReused(t *testing.T) { tCtx := ktesting.Init(t) addr, teardown, err := setupFakeGRPCServer(v1alpha4Version) if err != nil { t.Fatal(err) } defer teardown() reusedConns := make(map[*grpc.ClientConn]int) wg := sync.WaitGroup{} m := sync.Mutex{} p := &Plugin{ backgroundCtx:     tCtx, endpoint:          addr, clientCallTimeout: defaultClientCallTimeout, </ins> } <ins> conn, err := p.getOrCreateGRPCConn() defer func() { err := conn.Close() if err != nil { t.Error(err) } }() if err != nil { t.Fatal(err) } // ensure the plugin we are using is registered draPlugins.add("dummy-plugin", p) defer draPlugins.delete("dummy-plugin") // we call `NodePrepareResource` 2 times and check whether a new connection is created or the same is reused for i := 0; i < 2; i++ { wg.Add(1) go func() { defer wg.Done() client, err := NewDRAPluginClient("dummy-plugin") if err != nil { t.Error(err) return } req := &drapb.NodePrepareResourcesRequest{ Claims: []*drapb.Claim{ { Namespace: "dummy-namespace", UID:       "dummy-uid", Name:      "dummy-claim", }, }, } _, err = client.NodePrepareResources(tCtx, req) assert.NoError(t, err) client.mutex.Lock() conn := client.conn client.mutex.Unlock() m.Lock() defer m.Unlock() reusedConns[conn]++ }() } wg.Wait() // We should have only one entry otherwise it means another gRPC connection has been created if len(reusedConns) != 1 { t.Errorf("expected length to be 1 but got %d", len(reusedConns)) } if counter, ok := reusedConns[conn]; ok && counter != 2 { t.Errorf("expected counter to be 2 but got %d", counter) } } func TestNewDRAPluginClient(t *testing.T) { </ins> for _, test := range []struct { description string <del> handler     func() *RegistrationHandler </del> <ins> setup       func(string) tearDown </ins> pluginName  string <del> endpoint    string versions    []string </del> shouldError bool }{ { <del> description: "no versions provided", handler:     newRegistrationHandler, </del> <ins> description: "plugin name is empty", setup: func(_ string) tearDown { return func() {} }, pluginName:  "", </ins> shouldError: true, }, { <del> description: "unsupported version", handler:     newRegistrationHandler, versions:    []string{"v2.0.0"}, </del> <ins> description: "plugin name not found in the list", setup: func(_ string) tearDown { return func() {} }, pluginName:  "plugin-name-not-found-in-the-list", </ins> shouldError: true, }, { <del> description: "plugin already registered with a higher supported version", handler: func() *RegistrationHandler { handler := newRegistrationHandler() if err := handler.RegisterPlugin("this-plugin-already-exists-and-has-a-long-name-so-it-doesnt-collide", "", []string{"v1.1.0"}, nil); err != nil { t.Fatal(err) </del> <ins> description: "plugin exists", setup: func(name string) tearDown { draPlugins.add(name, &Plugin{}) return func() { draPlugins.delete(name) </ins> } <del> return handler </del> }, <del> pluginName:  "this-plugin-already-exists-and-has-a-long-name-so-it-doesnt-collide", versions:    []string{"v1.0.0"}, shouldError: true, }, { description: "should validate the plugin", handler:     newRegistrationHandler, pluginName:  "this-is-a-dummy-plugin-with-a-long-name-so-it-doesnt-collide", versions:    []string{"v1.3.0"}, </del> <ins> pluginName: "dummy-plugin", </ins> }, } { t.Run(test.description, func(t *testing.T) { <del> handler := test.handler() err := handler.ValidatePlugin(test.pluginName, test.endpoint, test.versions) </del> <ins> teardown := test.setup(test.pluginName) defer teardown() client, err := NewDRAPluginClient(test.pluginName) </ins> if test.shouldError { <ins> assert.Nil(t, client) </ins> assert.Error(t, err) } else { <ins> assert.NotNil(t, client) </ins> assert.Nil(t, err) } }) } <ins> } </ins> <del> t.Cleanup(func() { handler := newRegistrationHandler() handler.DeRegisterPlugin("this-plugin-already-exists-and-has-a-long-name-so-it-doesnt-collide") handler.DeRegisterPlugin("this-is-a-dummy-plugin-with-a-long-name-so-it-doesnt-collide") }) </del> <ins> func TestNodeUnprepareResources(t *testing.T) { for _, test := range []struct { description   string serverSetup   func(string) (string, tearDown, error) serverVersion string request       *drapb.NodeUnprepareResourcesRequest }{ { description:   "server supports v1alpha4", serverSetup:   setupFakeGRPCServer, serverVersion: v1alpha4Version, request:       &drapb.NodeUnprepareResourcesRequest{}, }, } { t.Run(test.description, func(t *testing.T) { tCtx := ktesting.Init(t) addr, teardown, err := setupFakeGRPCServer(test.serverVersion) if err != nil { t.Fatal(err) } defer teardown() p := &Plugin{ backgroundCtx:     tCtx, endpoint:          addr, clientCallTimeout: defaultClientCallTimeout, } conn, err := p.getOrCreateGRPCConn() defer func() { err := conn.Close() if err != nil { t.Error(err) } }() if err != nil { t.Fatal(err) } draPlugins.add("dummy-plugin", p) defer draPlugins.delete("dummy-plugin") client, err := NewDRAPluginClient("dummy-plugin") if err != nil { t.Fatal(err) } _, err = client.NodeUnprepareResources(tCtx, test.request) if err != nil { t.Fatal(err) } }) } </ins> }

The PR above moved everything related to into because that was where most of the code already was, but didn't renamed the files to avoid a larger diff and merge conflicts. Once is merged, we can do the renaming. Code clarity. /sig node
/triage accepted /priority important-soon
/assign
a PR can be created for this now.
will do


Question: } } <del> func TestNominatedNodeCleanUpUponNodeDeletion(t *testing.T) { // Initialize scheduler. testCtx := initTest(t, "preemption") defer testutils.CleanupTest(t, testCtx) cs := testCtx.ClientSet defer cleanupPodsInNamespace(cs, t, testCtx.NS.Name) // Create a node with some resources and a label. nodeRes := &v1.ResourceList{ v1.ResourcePods:   *resource.NewQuantity(32, resource.DecimalSI), v1.ResourceCPU:    *resource.NewMilliQuantity(100, resource.DecimalSI), v1.ResourceMemory: *resource.NewQuantity(100, resource.DecimalSI), } nodeNames := []string{"node1", "node2"} for _, nodeName := range nodeNames { _, err := createNode(testCtx.ClientSet, nodeName, nodeRes) if err != nil { t.Fatalf("Error creating nodes: %v", err) } } // Fill the cluster with one high-priority and one low-priority Pod. highPriPod, err := createPausePod(cs, mkPriorityPodWithGrace(testCtx, "high-pod", highPriority, 60)) if err != nil { t.Fatalf("Error creating high-priority pod: %v", err) } // Make sure the pod is scheduled. if err := testutils.WaitForPodToSchedule(cs, highPriPod); err != nil { t.Fatalf("Pod %v/%v didn't get scheduled: %v", highPriPod.Namespace, highPriPod.Name, err) } lowPriPod, err := createPausePod(cs, mkPriorityPodWithGrace(testCtx, "low-pod", lowPriority, 60)) if err != nil { t.Fatalf("Error creating low-priority pod: %v", err) } // Make sure the pod is scheduled. if err := testutils.WaitForPodToSchedule(cs, lowPriPod); err != nil { t.Fatalf("Pod %v/%v didn't get scheduled: %v", lowPriPod.Namespace, lowPriPod.Name, err) } // Create a medium-priority Pod. medPriPod, err := createPausePod(cs, mkPriorityPodWithGrace(testCtx, "med-pod", mediumPriority, 60)) if err != nil { t.Fatalf("Error creating med-priority pod: %v", err) } // Check its nominatedNodeName field is set properly. if err := waitForNominatedNodeName(cs, medPriPod); err != nil { t.Fatalf("NominatedNodeName was not set for pod %v/%v: %v", medPriPod.Namespace, medPriPod.Name, err) } // Get the live version of low and med pods. lowPriPod, _ = getPod(cs, lowPriPod.Name, lowPriPod.Namespace) medPriPod, _ = getPod(cs, medPriPod.Name, medPriPod.Namespace) want, got := lowPriPod.Spec.NodeName, medPriPod.Status.NominatedNodeName if want != got { t.Fatalf("Expect med-priority's nominatedNodeName to be %v, but got %v.", want, got) } // Delete the node where med-priority pod is nominated to. cs.CoreV1().Nodes().Delete(context.TODO(), got, metav1.DeleteOptions{}) if err := wait.Poll(200*time.Millisecond, wait.ForeverTestTimeout, func() (bool, error) { _, err := cs.CoreV1().Nodes().Get(context.TODO(), got, metav1.GetOptions{}) if apierrors.IsNotFound(err) { return true, nil } return false, err }); err != nil { t.Fatalf("Node %v cannot be deleted: %v.", got, err) } // Finally verify if med-priority pod's nominatedNodeName gets cleared. if err := wait.Poll(200*time.Millisecond, wait.ForeverTestTimeout, func() (bool, error) { pod, err := cs.CoreV1().Pods(medPriPod.Namespace).Get(context.TODO(), medPriPod.Name, metav1.GetOptions{}) if err != nil { t.Errorf("Error getting the medium priority pod info: %v", err) } if len(pod.Status.NominatedNodeName) == 0 { return true, nil } return false, err }); err != nil { t.Errorf("The nominated node name of the medium priority pod was not cleared: %v", err) } } </del> func mkMinAvailablePDB(name, namespace string, uid types.UID, minAvailable int, matchLabels map[string]string) *policy.PodDisruptionBudget { intMinAvailable := intstr.FromInt(minAvailable) return &policy.PodDisruptionBudget{

<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: Conformance - GCE - master (ci-kubernetes-gce-conformance-latest) Which test(s) are failing: [sig-scheduling] SchedulerPreemption [Serial] PreemptionExecutionPath runs ReplicaSets to verify preemption running path [Conformance]Changes Since when has it been failing: 2020-06-08 22:27 PDT Testgrid link: Reason for failure: Regression by Anything else we need to know: /sig scheduling /priority critical-urgent
cc/ /assign


Question: // +optional optional string imagePullPolicy = 14; <del> // Security options the pod should run with. </del> <ins> // SecurityContext defines the security options the container should be run with. // If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext. </ins> // More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ // +optional optional SecurityContext securityContext = 15;

What happened: states that Container.SecurityContext applies to a Pod, which can be misleading since we already have PodSecurityContext. What you expected to happen: Documentation states that it applies to container How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment:
/sig api-machinery /kind documentation
/cc
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Not stale.
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle rotten
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details


Question: } if (opts.Watch || forceWatch) && rw != nil { <ins> glog.Infof("Started to log from %v for %v", ctx, req.Request.URL.RequestURI()) </ins> watcher, err := rw.Watch(ctx, &opts) if err != nil { scope.err(err, res.ResponseWriter, req.Request)

Currently watches only log when they complete, which can make analyzing kube-apiserver log files a bit tricky. (did component x never start watching after that list or did apiserver shut down?)


Question: trap cleanup EXIT <del> while true; do read x; done </del> <ins> while true; do sleep 1; done </ins>

on the end of script there is a line: It produces high CPU load when it is run without tty. However it can be replaced with something like this:
Looks like this has been fixed.


Question: in.Spec.InitContainers = values // Call defaulters explicitly until annotations are removed <del> for i := range in.Spec.InitContainers { c := &in.Spec.InitContainers[i] SetDefaults_Container(c) </del> <ins> tmpPodTemp := &PodTemplate{ Template: PodTemplateSpec{ Spec: PodSpec{ HostNetwork:    in.Spec.HostNetwork, InitContainers: values, }, }, </ins> } <ins> SetObjectDefaults_PodTemplate(tmpPodTemp) in.Spec.InitContainers = tmpPodTemp.Template.Spec.InitContainers </ins> } if err := autoConvert_v1_PodTemplateSpec_To_api_PodTemplateSpec(in, out, s); err != nil {

What happened: kubelet doesn't seem to be able to manage containers started with CRI-O any more I'm testing kubelet + CRI-O on an Ubuntu 18.04 server installation, inside a qemu/kvm VM managed by Vagrant (see ), in order to be able to test k8s + KubeVirt + KataContainers, which are all compatible with CRI-O kubelet will obsess on starting new containers, which looks like and endless process, and the cluster never gets ready. For instance here are some syslogs : What you expected to happen: It used to work (see asciinema recording at ) a few days ago, but I fear some recent changes broke it. How to reproduce it (as minimally and precisely as possible): Testing with vagrant up --provider=libvirt on Linux with a clone of that commit: vagrant ssh into the VM and ps -edf f or tail -f /var/log/syslog would do... Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: qemu/kvm VM +OS (e.g: ): Ubuntu 18.04.2 LTS Kernel (e.g. ): Linux kubernetes-vagrant-01 4.15.0-48-generic -Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x8664 x8664 x8664 GNU/Linux Install tools: kubeadm as in Network plugin and version (if this is a network-related bug): Others:
/sig node
I can confirm this. If I start cri-o from a systemd service it fails with the above mentioned error message. Starting crio with without systemd works just fine. systemd file:
Thanks for your detailed report :) Just want to do some quick version confirming - you've been using k8s 1.14.1 both when it did and did not work? Any other version changes?
Not sure for before, when it used to work, but according to the asciinema recording mentioned above (), it seems the same. However, not sure for CRI-O, underlying OS stuff (systemd, kernel, etc.). Sorry.
I was looking for some information for days. I have exactly the same issue when I use podman + systemd on 18.04.2.


Question: in.Spec.InitContainers = values // Call defaulters explicitly until annotations are removed <del> for i := range in.Spec.InitContainers { c := &in.Spec.InitContainers[i] SetDefaults_Container(c) </del> <ins> tmpPodTemp := &PodTemplate{ Template: PodTemplateSpec{ Spec: PodSpec{ HostNetwork:    in.Spec.HostNetwork, InitContainers: values, }, }, </ins> } <ins> SetObjectDefaults_PodTemplate(tmpPodTemp) in.Spec.InitContainers = tmpPodTemp.Template.Spec.InitContainers </ins> } if err := autoConvert_v1_PodTemplateSpec_To_api_PodTemplateSpec(in, out, s); err != nil {

It is the same for both 14.15 and 14.18 kernels. Podman without systemd runs fine. Podman with systemd fails at runc with the same error podman[]: open /dev/stderr: no such device or address podman[]: NAME: runc - Open Container Initiative runtime podman[]: runc is a command line client for running applications packaged according to ... podman[]: time="2019-05-11T13:04:59-07:00" level=error msg="open /dev/stderr: no such device or addressn" podman[]: open /dev/stderr: no such device or address
I've tried to reproduce it on 18.04.2 using Podman from but I am not able to reproduce it. Do you have any reproducer? How did you install Podman?
I used ceph unit files to run ceph in podman rather than docker. Systemd unit successfully starts docker but fails with podman. I can successfully run the same podman container run command from the command line. How were you trying to replicate this? Can you post your unit file?
Hi, I'am having the same problem with Kubernetes with CRI-O. After joining a second worker node it fails to start flannel. gives the following error: What I've tried: Installing previous version of Kubernetes (from to ) Installing previous version of crio (from to ) Building runc myself from their github page and selecting this in the Running it on Ubunut 18.04.2 Issue: (the one giuseppe linked)
I wonder if the issue is with systemd and runc. In both cases, crio and podman, I see runc help output which tells me that something is not being passed to runc that it needs.
I've only used so it is different than your test and you confirmed it works from the CLI for you as well. Could you share your service file so I test it?
I've tried systemd and cgroupfs but no luck so far.
so it seems that runc fails to re-open . That seems like a runc issue, as it doesn't really need to reopen it. To fix the failure, you could use something like:
Just an example.