Question:
Integration test timed out due to the following error The error appeared 748 times in the log. Looks like lots of churning happens during the apf configuration bootstrapping phase. Is there a race condition between when the type is being registered and the API calls? Stack trace: Integration job link: PR: I searched for this error in CI and this PR, looks like we only see this error in this PR so far. So I am hoping it has not introduced any flake. (maybe we need to let more time to pass to start seeing flakes) We should not see this error I saw it only in the PR mentioned.
/sig api-machinery cc
I see
The fact that it happens to a non-mandatory object implies that the object was received from the server, so the type must be in the server. The APF config-consuming controller does this:
Look for the output from this line:
Another PR with this failure:
the panic means the request scope FieldManager is nil that is initialized here: OpenAPIModels is set here: called from here: and openAPIModels is constructed here: it looks like it is possible for openapiconfig to be unset, which causes the openapi models to be nil, which causes apply to panic. I don't know if it is only the integration tests that are subject to that configuration. cc /priority important-soon (since this is now being exercised in bring-up of integration test servers)
if APF is creating/bootstrapping the objects without using apply, and then updating status using apply, that would explain how the object exists but apply fails
I suspect that APF is not successfully updating status at all in the integration tests with this server configuration
do you want me to revert , while we work on a solution?
I don't know how bad the test impact is. Work with / to determine if we should revert or fix forward
: I do not follow why you talked about server-side stuff then suggested the problem is in the client (a controller in the kube-apiserver) not supplying a fieldManager. The client does supply the fieldManager, for both creates and updates. See the following lines:
Different field manager. is set server-side in the endpoint handler based on the server configuration.
So if the problem is server-side, why did you suggest the problem is in the client?
Regarding the client, I looked in and found this line: actually several of the same form, all with the same fieldManager.
I don't think I did suggest that :) all of the setup and calls in are server-side
: please help me understand . That is about the client side. Are you saying that a client that creates an object using and later updates the object using --- with a non-empty fieldManager in both --- will run into this server-side scenario? If so, is that because this client behavior is wrong or because there is a bug server side?
That was in response to , trying to figure out how the object was created but could not be updated. Yes, when the server is not configured to enable openapi and therefore does not have apply enabled. Because the server is misconfigured (which, from what I can tell, is limited to some of the server instances started for integration tests)
Looking at and it is not obvious to me whether or how openapi could be disabled. Would that be by turning off the feature flag? FYI, the inclusion of "V3" in the flag makes me unsure what the server does about other versions. : I think you are saying that it is possible to disable openapi and that this in turn disables server-side apply. That seems kind of surprising to me. Does this mean that it is simply incorrect to run a client that does server-side apply with a server that has openapi disabled? Is this intended? Is it documented anywhere?
It's not intended as an externalized configuration option. integration tests have the ability to fiddle with many of the config struct options in the server that a kube-apiserver binary invocation could not. They intentionally use this to do things like inject specialized authorizers / authenticators for testing purposes, but also can unintentionally result in a test server that doesn't match a real API server invocation. I think this is an integration test server configuration problem, which should be fixed.
is correct that we have a misconfiguration in the integration tests and SSA is only enabled on a subset of integration tests. I'd like to understand the test failure conditions a bit better though. The can be consistently reproduced on a non SSA targeted test. (Eg: selected from one of the test failures). However, I'm seeing the panic message while the test is still green and passes locally. it's a bit bizarre what contributes to the flakiness of the test because the FieldManager panic can be reproduced 100% of the time.
The panic occurs when APF is updating status (which it just switched to do using SSA). The APF status update failure does not block the API requests the test cares about, so the test passes. I could see extra CPU usage contributing to flakiness.
Thanks for the explanation. Created to fix the integration test configurations.
/triage accepted
The integration test job started timing out dramatically last night (50% of runs) once merged. Since that was the only change made to integration tests recently, it seems related. I think we should revert (and therefore ) until the test impact is understood
revert of -
did this get fully reverted? we're still seeing panics in integration tests over this:
bumping priority and milestoning until we know we're not currently triggering panics
looks like it got reintroduced in , revert open in
For integration tests, maybe we can cheat and cache the OpenAPI data, so that it's not recomputed for each test?
:-/ maybe... but I really don't want to have non-real init paths in integration if at all possible
I wish we could go a little further in lazily initializing openapi (especially v2), so let's see what we can do here (after code-freeze probably)
if SSA is actively using the openapi, it being lazy won't help, right?
does the revert mean that we can remove milestone v1.24 from this issue?
Yes, cleared
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten
Taking a fresh look at this PR and wondering if it has been mostly fixed already? From comments my understanding is that this panic occurred due to two conditions: running as part of the integration test suite is asked to service an SSA request while are nil. The apiservice has logic to panic in this case. From the code path commented is nil when the server is created with is nil. APF worker (running alongside an integration test) issues an SSA request to the test apiserver. Since the apiserver is misconfigured it panics. This panic could happen sporadically because sometimes the tests finish before the APF worker is able to issue the request. As a brief check to see if this might be fixed, I ran the test mentioned was failing from SSA. I logged out the 's created and found that they all had managed fields. I saw this as an indication SSA was now functioning properly. Indeed, to start the test server the function calls . This function has a code path which unconditionally leads to the apiserver's being set. calls before running the returned apiserver: calls before using the returned config to create the returned apiserver: uses which sets : Thus, any test using should no longer be affected by this panic. The question then becomes: do all integration tests start their servers in this way? It's hard to say. I found a PR (, ) which refactors many occurrences of the old stanza: Into something using the new method: ` But it appears is still being used in a few integration tests: Should these occurrences be refactored to use too?
There was a ton of changes I did around that. But the last PR (already removing the old is already out for review): - FYI Thinking about it now, I guess I agree that the problem might have been 10 different way kube-apiserver was set up through different tests. I just don't know why it was causing flakes, not failing consistently...
There were definitely test setups that were not enabling openapi in the recent past. Two possibilities for flakes: Enabling openapi in all integration test setups was expensive and caused spurious timeouts The excessive numbers of background panics handled and logged in integration tests when the API server tried to handle SSA requests bogged down the CI machine (I think something like 40,000 panics with full stack traces each were getting logged)
i don't think that's the case. With the recent changes that I was doing as part of I think OpenAPI is already enabled for all tests having kube-apiserver (almost all) and that seems to work. But you remainded me that back then we also had an issue that we were running many more tests simulanuously, which could have been contributing to this problem significantly. The second reason is definitely possible contributor too. I guess my concern was that if some requests were panicing due to lack of OpenAPI, this means that these aren't really needed. But that actually might make sense - the fact that we weren't able to upgrade status of PriorityLevels/FlowSchemas shouldn't prevent non P&F-tests from succeeding, and that worked in P&F tests...
I think the panics were happening in async controllers, and no functionality blocked by the controllers intersected with the surface being tested, it just (maybe) caused log/perf issues
I believe has resolved the panic issue, I was able to verify with which changes the priority & fairness controller to use server side apply. Before merged, I analyzed the integration log: Integration test log file from an unrelated pr does not contain any trace of the panic as expected: On the other hand, the log file from integration job for (before merged) had plenty of these panics even though the integration job turned green: I wanted to check which tests produced these panics, so I did a breakdown by package Then I narrowed it down to the individual test(s): Before merged, the above tests used , and either is or is which results in to be nil. But a quick code search revealed that many more integration tests used , so why were the other tests not affected by the panic? I inspected the log of a test that was not affected by the panic, and grepped for , and found the following: In the above case, the apf controller had not advanced far enough, the controller was active for about , may be these tests finish quickly and signals the server to shutdown before the apf controller can send an APPLY command. On the other hand, log from a test that was affected by the panic shows that the apf controller was active for more than , and has lot more activities: I am assuming that integration tests have at least log level 4. Also, the size of the log file differed significantly, without panics around , and with panics over and changed all these tests to use and now integration log for shows no panic.
Great! (This also proves why eliminating gazilion different things of initializing stuff in favor of one makes sense). - FYI ^^ - let's merge the PR then - I just one question there, so let's wait for Jordan to respond it.
Can we just force the openapi/fieldmanager to be set at start-time now that SSA is GA, it should basically always be present, rather than fail when SSA is unavailable (that's probably wrong by the way). If that triggers any test flakiness, then we would solve these. We can wait for to be solved since it will almost certainly have a huge impact on this.
Has this been done Jeff?

// if we are going to issue an update, be sure we track every name we update so we know if we update it too often. currResult.updatedItems.Insert(fsu.flowSchema.Name) <del> patchBytes, err := makeFlowSchemaConditionPatch(fsu.condition) if err != nil { // should never happen because these conditions are created here and well formed panic(fmt.Sprintf("Failed to json.Marshall(%#+v): %s", fsu.condition, err.Error())) } </del> if klogV := klog.V(4); klogV.Enabled() { klogV.Infof("%s writing Condition %s to FlowSchema %s, which had ResourceVersion=%s, because its previous value was %s, diff: %s", cfgCtlr.name, fsu.condition, fsu.flowSchema.Name, fsu.flowSchema.ResourceVersion, fcfmt.Fmt(fsu.oldValue), cmp.Diff(fsu.oldValue, fsu.condition)) } <del> fsIfc := cfgCtlr.flowcontrolClient.FlowSchemas() patchOptions := metav1.PatchOptions{FieldManager: cfgCtlr.asFieldManager} _, err = fsIfc.Patch(context.TODO(), fsu.flowSchema.Name, apitypes.StrategicMergePatchType, patchBytes, patchOptions, "status") if err != nil { </del> <ins> if err := apply(cfgCtlr.flowcontrolClient.FlowSchemas(), fsu, cfgCtlr.asFieldManager); err != nil { </ins> if apierrors.IsNotFound(err) { // This object has been deleted.  A notification is coming // and nothing more needs to be done here.


Question:
Integration test timed out due to the following error The error appeared 748 times in the log. Looks like lots of churning happens during the apf configuration bootstrapping phase. Is there a race condition between when the type is being registered and the API calls? Stack trace: Integration job link: PR: I searched for this error in CI and this PR, looks like we only see this error in this PR so far. So I am hoping it has not introduced any flake. (maybe we need to let more time to pass to start seeing flakes) We should not see this error I saw it only in the PR mentioned.
/sig api-machinery cc
I see
The fact that it happens to a non-mandatory object implies that the object was received from the server, so the type must be in the server. The APF config-consuming controller does this:
Look for the output from this line:
Another PR with this failure:
the panic means the request scope FieldManager is nil that is initialized here: OpenAPIModels is set here: called from here: and openAPIModels is constructed here: it looks like it is possible for openapiconfig to be unset, which causes the openapi models to be nil, which causes apply to panic. I don't know if it is only the integration tests that are subject to that configuration. cc /priority important-soon (since this is now being exercised in bring-up of integration test servers)
if APF is creating/bootstrapping the objects without using apply, and then updating status using apply, that would explain how the object exists but apply fails
I suspect that APF is not successfully updating status at all in the integration tests with this server configuration
do you want me to revert , while we work on a solution?
I don't know how bad the test impact is. Work with / to determine if we should revert or fix forward
: I do not follow why you talked about server-side stuff then suggested the problem is in the client (a controller in the kube-apiserver) not supplying a fieldManager. The client does supply the fieldManager, for both creates and updates. See the following lines:
Different field manager. is set server-side in the endpoint handler based on the server configuration.
So if the problem is server-side, why did you suggest the problem is in the client?
Regarding the client, I looked in and found this line: actually several of the same form, all with the same fieldManager.
I don't think I did suggest that :) all of the setup and calls in are server-side
: please help me understand . That is about the client side. Are you saying that a client that creates an object using and later updates the object using --- with a non-empty fieldManager in both --- will run into this server-side scenario? If so, is that because this client behavior is wrong or because there is a bug server side?
That was in response to , trying to figure out how the object was created but could not be updated. Yes, when the server is not configured to enable openapi and therefore does not have apply enabled. Because the server is misconfigured (which, from what I can tell, is limited to some of the server instances started for integration tests)
Looking at and it is not obvious to me whether or how openapi could be disabled. Would that be by turning off the feature flag? FYI, the inclusion of "V3" in the flag makes me unsure what the server does about other versions. : I think you are saying that it is possible to disable openapi and that this in turn disables server-side apply. That seems kind of surprising to me. Does this mean that it is simply incorrect to run a client that does server-side apply with a server that has openapi disabled? Is this intended? Is it documented anywhere?
It's not intended as an externalized configuration option. integration tests have the ability to fiddle with many of the config struct options in the server that a kube-apiserver binary invocation could not. They intentionally use this to do things like inject specialized authorizers / authenticators for testing purposes, but also can unintentionally result in a test server that doesn't match a real API server invocation. I think this is an integration test server configuration problem, which should be fixed.
is correct that we have a misconfiguration in the integration tests and SSA is only enabled on a subset of integration tests. I'd like to understand the test failure conditions a bit better though. The can be consistently reproduced on a non SSA targeted test. (Eg: selected from one of the test failures). However, I'm seeing the panic message while the test is still green and passes locally. it's a bit bizarre what contributes to the flakiness of the test because the FieldManager panic can be reproduced 100% of the time.
The panic occurs when APF is updating status (which it just switched to do using SSA). The APF status update failure does not block the API requests the test cares about, so the test passes. I could see extra CPU usage contributing to flakiness.
Thanks for the explanation. Created to fix the integration test configurations.
/triage accepted
The integration test job started timing out dramatically last night (50% of runs) once merged. Since that was the only change made to integration tests recently, it seems related. I think we should revert (and therefore ) until the test impact is understood
revert of -
did this get fully reverted? we're still seeing panics in integration tests over this:
bumping priority and milestoning until we know we're not currently triggering panics
looks like it got reintroduced in , revert open in
For integration tests, maybe we can cheat and cache the OpenAPI data, so that it's not recomputed for each test?
:-/ maybe... but I really don't want to have non-real init paths in integration if at all possible
I wish we could go a little further in lazily initializing openapi (especially v2), so let's see what we can do here (after code-freeze probably)
if SSA is actively using the openapi, it being lazy won't help, right?
does the revert mean that we can remove milestone v1.24 from this issue?
Yes, cleared
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten
Taking a fresh look at this PR and wondering if it has been mostly fixed already? From comments my understanding is that this panic occurred due to two conditions: running as part of the integration test suite is asked to service an SSA request while are nil. The apiservice has logic to panic in this case. From the code path commented is nil when the server is created with is nil. APF worker (running alongside an integration test) issues an SSA request to the test apiserver. Since the apiserver is misconfigured it panics. This panic could happen sporadically because sometimes the tests finish before the APF worker is able to issue the request. As a brief check to see if this might be fixed, I ran the test mentioned was failing from SSA. I logged out the 's created and found that they all had managed fields. I saw this as an indication SSA was now functioning properly. Indeed, to start the test server the function calls . This function has a code path which unconditionally leads to the apiserver's being set. calls before running the returned apiserver: calls before using the returned config to create the returned apiserver: uses which sets : Thus, any test using should no longer be affected by this panic. The question then becomes: do all integration tests start their servers in this way? It's hard to say. I found a PR (, ) which refactors many occurrences of the old stanza: Into something using the new method: ` But it appears is still being used in a few integration tests: Should these occurrences be refactored to use too?
There was a ton of changes I did around that. But the last PR (already removing the old is already out for review): - FYI Thinking about it now, I guess I agree that the problem might have been 10 different way kube-apiserver was set up through different tests. I just don't know why it was causing flakes, not failing consistently...
There were definitely test setups that were not enabling openapi in the recent past. Two possibilities for flakes: Enabling openapi in all integration test setups was expensive and caused spurious timeouts The excessive numbers of background panics handled and logged in integration tests when the API server tried to handle SSA requests bogged down the CI machine (I think something like 40,000 panics with full stack traces each were getting logged)
i don't think that's the case. With the recent changes that I was doing as part of I think OpenAPI is already enabled for all tests having kube-apiserver (almost all) and that seems to work. But you remainded me that back then we also had an issue that we were running many more tests simulanuously, which could have been contributing to this problem significantly. The second reason is definitely possible contributor too. I guess my concern was that if some requests were panicing due to lack of OpenAPI, this means that these aren't really needed. But that actually might make sense - the fact that we weren't able to upgrade status of PriorityLevels/FlowSchemas shouldn't prevent non P&F-tests from succeeding, and that worked in P&F tests...
I think the panics were happening in async controllers, and no functionality blocked by the controllers intersected with the surface being tested, it just (maybe) caused log/perf issues
I believe has resolved the panic issue, I was able to verify with which changes the priority & fairness controller to use server side apply. Before merged, I analyzed the integration log: Integration test log file from an unrelated pr does not contain any trace of the panic as expected: On the other hand, the log file from integration job for (before merged) had plenty of these panics even though the integration job turned green: I wanted to check which tests produced these panics, so I did a breakdown by package Then I narrowed it down to the individual test(s): Before merged, the above tests used , and either is or is which results in to be nil. But a quick code search revealed that many more integration tests used , so why were the other tests not affected by the panic? I inspected the log of a test that was not affected by the panic, and grepped for , and found the following: In the above case, the apf controller had not advanced far enough, the controller was active for about , may be these tests finish quickly and signals the server to shutdown before the apf controller can send an APPLY command. On the other hand, log from a test that was affected by the panic shows that the apf controller was active for more than , and has lot more activities: I am assuming that integration tests have at least log level 4. Also, the size of the log file differed significantly, without panics around , and with panics over and changed all these tests to use and now integration log for shows no panic.
Great! (This also proves why eliminating gazilion different things of initializing stuff in favor of one makes sense). - FYI ^^ - let's merge the PR then - I just one question there, so let's wait for Jordan to respond it.
Can we just force the openapi/fieldmanager to be set at start-time now that SSA is GA, it should basically always be present, rather than fail when SSA is unavailable (that's probably wrong by the way). If that triggers any test flakiness, then we would solve these. We can wait for to be solved since it will almost certainly have a huge impact on this.
Has this been done Jeff?

return suggestedDelay, utilerrors.NewAggregate(errs) } <del> // makeFlowSchemaConditionPatch takes in a condition and returns the patch status as a json. func makeFlowSchemaConditionPatch(condition flowcontrol.FlowSchemaCondition) ([]byte, error) { o := struct { Status flowcontrol.FlowSchemaStatus `json:"status"` }{ Status: flowcontrol.FlowSchemaStatus{ Conditions: []flowcontrol.FlowSchemaCondition{ condition, }, }, } return json.Marshal(o) </del> <ins> func apply(client flowcontrolclient.FlowSchemaInterface, fsu fsStatusUpdate, asFieldManager string) error { applyOptions := metav1.ApplyOptions{FieldManager: asFieldManager, Force: true} // the condition field in fsStatusUpdate holds the new condition we want to update. // TODO: this will break when we have multiple conditions for a flowschema _, err := client.ApplyStatus(context.TODO(), toFlowSchemaApplyConfiguration(fsu), applyOptions) return err } func toFlowSchemaApplyConfiguration(fsUpdate fsStatusUpdate) *flowcontrolapplyconfiguration.FlowSchemaApplyConfiguration { condition := flowcontrolapplyconfiguration.FlowSchemaCondition(). WithType(fsUpdate.condition.Type). WithStatus(fsUpdate.condition.Status). WithReason(fsUpdate.condition.Reason). WithLastTransitionTime(fsUpdate.condition.LastTransitionTime). WithMessage(fsUpdate.condition.Message) return flowcontrolapplyconfiguration.FlowSchema(fsUpdate.flowSchema.Name). WithStatus(flowcontrolapplyconfiguration.FlowSchemaStatus(). WithConditions(condition), ) </ins> } // shouldDelayUpdate checks to see if a flowschema has been updated too often and returns true if a delay is needed.


Question:
Integration test timed out due to the following error The error appeared 748 times in the log. Looks like lots of churning happens during the apf configuration bootstrapping phase. Is there a race condition between when the type is being registered and the API calls? Stack trace: Integration job link: PR: I searched for this error in CI and this PR, looks like we only see this error in this PR so far. So I am hoping it has not introduced any flake. (maybe we need to let more time to pass to start seeing flakes) We should not see this error I saw it only in the PR mentioned.
/sig api-machinery cc
I see
The fact that it happens to a non-mandatory object implies that the object was received from the server, so the type must be in the server. The APF config-consuming controller does this:
Look for the output from this line:
Another PR with this failure:
the panic means the request scope FieldManager is nil that is initialized here: OpenAPIModels is set here: called from here: and openAPIModels is constructed here: it looks like it is possible for openapiconfig to be unset, which causes the openapi models to be nil, which causes apply to panic. I don't know if it is only the integration tests that are subject to that configuration. cc /priority important-soon (since this is now being exercised in bring-up of integration test servers)
if APF is creating/bootstrapping the objects without using apply, and then updating status using apply, that would explain how the object exists but apply fails
I suspect that APF is not successfully updating status at all in the integration tests with this server configuration
do you want me to revert , while we work on a solution?
I don't know how bad the test impact is. Work with / to determine if we should revert or fix forward
: I do not follow why you talked about server-side stuff then suggested the problem is in the client (a controller in the kube-apiserver) not supplying a fieldManager. The client does supply the fieldManager, for both creates and updates. See the following lines:
Different field manager. is set server-side in the endpoint handler based on the server configuration.
So if the problem is server-side, why did you suggest the problem is in the client?
Regarding the client, I looked in and found this line: actually several of the same form, all with the same fieldManager.
I don't think I did suggest that :) all of the setup and calls in are server-side
: please help me understand . That is about the client side. Are you saying that a client that creates an object using and later updates the object using --- with a non-empty fieldManager in both --- will run into this server-side scenario? If so, is that because this client behavior is wrong or because there is a bug server side?
That was in response to , trying to figure out how the object was created but could not be updated. Yes, when the server is not configured to enable openapi and therefore does not have apply enabled. Because the server is misconfigured (which, from what I can tell, is limited to some of the server instances started for integration tests)
Looking at and it is not obvious to me whether or how openapi could be disabled. Would that be by turning off the feature flag? FYI, the inclusion of "V3" in the flag makes me unsure what the server does about other versions. : I think you are saying that it is possible to disable openapi and that this in turn disables server-side apply. That seems kind of surprising to me. Does this mean that it is simply incorrect to run a client that does server-side apply with a server that has openapi disabled? Is this intended? Is it documented anywhere?
It's not intended as an externalized configuration option. integration tests have the ability to fiddle with many of the config struct options in the server that a kube-apiserver binary invocation could not. They intentionally use this to do things like inject specialized authorizers / authenticators for testing purposes, but also can unintentionally result in a test server that doesn't match a real API server invocation. I think this is an integration test server configuration problem, which should be fixed.
is correct that we have a misconfiguration in the integration tests and SSA is only enabled on a subset of integration tests. I'd like to understand the test failure conditions a bit better though. The can be consistently reproduced on a non SSA targeted test. (Eg: selected from one of the test failures). However, I'm seeing the panic message while the test is still green and passes locally. it's a bit bizarre what contributes to the flakiness of the test because the FieldManager panic can be reproduced 100% of the time.
The panic occurs when APF is updating status (which it just switched to do using SSA). The APF status update failure does not block the API requests the test cares about, so the test passes. I could see extra CPU usage contributing to flakiness.
Thanks for the explanation. Created to fix the integration test configurations.
/triage accepted
The integration test job started timing out dramatically last night (50% of runs) once merged. Since that was the only change made to integration tests recently, it seems related. I think we should revert (and therefore ) until the test impact is understood
revert of -
did this get fully reverted? we're still seeing panics in integration tests over this:
bumping priority and milestoning until we know we're not currently triggering panics
looks like it got reintroduced in , revert open in
For integration tests, maybe we can cheat and cache the OpenAPI data, so that it's not recomputed for each test?
:-/ maybe... but I really don't want to have non-real init paths in integration if at all possible
I wish we could go a little further in lazily initializing openapi (especially v2), so let's see what we can do here (after code-freeze probably)
if SSA is actively using the openapi, it being lazy won't help, right?
does the revert mean that we can remove milestone v1.24 from this issue?
Yes, cleared
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten
Taking a fresh look at this PR and wondering if it has been mostly fixed already? From comments my understanding is that this panic occurred due to two conditions: running as part of the integration test suite is asked to service an SSA request while are nil. The apiservice has logic to panic in this case. From the code path commented is nil when the server is created with is nil. APF worker (running alongside an integration test) issues an SSA request to the test apiserver. Since the apiserver is misconfigured it panics. This panic could happen sporadically because sometimes the tests finish before the APF worker is able to issue the request. As a brief check to see if this might be fixed, I ran the test mentioned was failing from SSA. I logged out the 's created and found that they all had managed fields. I saw this as an indication SSA was now functioning properly. Indeed, to start the test server the function calls . This function has a code path which unconditionally leads to the apiserver's being set. calls before running the returned apiserver: calls before using the returned config to create the returned apiserver: uses which sets : Thus, any test using should no longer be affected by this panic. The question then becomes: do all integration tests start their servers in this way? It's hard to say. I found a PR (, ) which refactors many occurrences of the old stanza: Into something using the new method: ` But it appears is still being used in a few integration tests: Should these occurrences be refactored to use too?
There was a ton of changes I did around that. But the last PR (already removing the old is already out for review): - FYI Thinking about it now, I guess I agree that the problem might have been 10 different way kube-apiserver was set up through different tests. I just don't know why it was causing flakes, not failing consistently...
There were definitely test setups that were not enabling openapi in the recent past. Two possibilities for flakes: Enabling openapi in all integration test setups was expensive and caused spurious timeouts The excessive numbers of background panics handled and logged in integration tests when the API server tried to handle SSA requests bogged down the CI machine (I think something like 40,000 panics with full stack traces each were getting logged)
i don't think that's the case. With the recent changes that I was doing as part of I think OpenAPI is already enabled for all tests having kube-apiserver (almost all) and that seems to work. But you remainded me that back then we also had an issue that we were running many more tests simulanuously, which could have been contributing to this problem significantly. The second reason is definitely possible contributor too. I guess my concern was that if some requests were panicing due to lack of OpenAPI, this means that these aren't really needed. But that actually might make sense - the fact that we weren't able to upgrade status of PriorityLevels/FlowSchemas shouldn't prevent non P&F-tests from succeeding, and that worked in P&F tests...
I think the panics were happening in async controllers, and no functionality blocked by the controllers intersected with the surface being tested, it just (maybe) caused log/perf issues
I believe has resolved the panic issue, I was able to verify with which changes the priority & fairness controller to use server side apply. Before merged, I analyzed the integration log: Integration test log file from an unrelated pr does not contain any trace of the panic as expected: On the other hand, the log file from integration job for (before merged) had plenty of these panics even though the integration job turned green: I wanted to check which tests produced these panics, so I did a breakdown by package Then I narrowed it down to the individual test(s): Before merged, the above tests used , and either is or is which results in to be nil. But a quick code search revealed that many more integration tests used , so why were the other tests not affected by the panic? I inspected the log of a test that was not affected by the panic, and grepped for , and found the following: In the above case, the apf controller had not advanced far enough, the controller was active for about , may be these tests finish quickly and signals the server to shutdown before the apf controller can send an APPLY command. On the other hand, log from a test that was affected by the panic shows that the apf controller was active for more than , and has lot more activities: I am assuming that integration tests have at least log level 4. Also, the size of the log file differed significantly, without panics around , and with panics over and changed all these tests to use and now integration log for shows no panic.
Great! (This also proves why eliminating gazilion different things of initializing stuff in favor of one makes sense). - FYI ^^ - let's merge the PR then - I just one question there, so let's wait for Jordan to respond it.
Can we just force the openapi/fieldmanager to be set at start-time now that SSA is GA, it should basically always be present, rather than fail when SSA is unavailable (that's probably wrong by the way). If that triggers any test flakiness, then we would solve these. We can wait for to be solved since it will almost certainly have a huge impact on this.
Has this been done Jeff?

<del> /* Copyright 2021 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package flowcontrol import ( "fmt" "reflect" "testing" "time" "github.com/google/go-cmp/cmp" flowcontrol "k8s.io/api/flowcontrol/v1beta3" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) func Test_configController_generatePatchBytes(t *testing.T) { now := time.Now().UTC() tests := []struct { name      string condition flowcontrol.FlowSchemaCondition want      []byte }{ { name: "check if only condition is parsed", condition: flowcontrol.FlowSchemaCondition{ Type:               flowcontrol.FlowSchemaConditionDangling, Status:             flowcontrol.ConditionTrue, Reason:             "test reason", Message:            "test none", LastTransitionTime: metav1.NewTime(now), }, want: []byte(fmt.Sprintf(`{"status":{"conditions":[{"type":"Dangling","status":"True","lastTransitionTime":"%s","reason":"test reason","message":"test none"}]}}`, now.Format(time.RFC3339))), }, { name: "check when message has double quotes", condition: flowcontrol.FlowSchemaCondition{ Type:               flowcontrol.FlowSchemaConditionDangling, Status:             flowcontrol.ConditionTrue, Reason:             "test reason", Message:            `test ""none`, LastTransitionTime: metav1.NewTime(now), }, want: []byte(fmt.Sprintf(`{"status":{"conditions":[{"type":"Dangling","status":"True","lastTransitionTime":"%s","reason":"test reason","message":"test ""none"}]}}`, now.Format(time.RFC3339))), }, { name: "check when message has a whitespace character that can be escaped", condition: flowcontrol.FlowSchemaCondition{ Type:               flowcontrol.FlowSchemaConditionDangling, Status:             flowcontrol.ConditionTrue, Reason:             "test reason", Message:            "test u0009u0009none", LastTransitionTime: metav1.NewTime(now), }, want: []byte(fmt.Sprintf(`{"status":{"conditions":[{"type":"Dangling","status":"True","lastTransitionTime":"%s","reason":"test reason","message":"test ttnone"}]}}`, now.Format(time.RFC3339))), }, { name: "check when a few fields (message & lastTransitionTime) are missing", condition: flowcontrol.FlowSchemaCondition{ Type:   flowcontrol.FlowSchemaConditionDangling, Status: flowcontrol.ConditionTrue, Reason: "test reason", }, want: []byte(`{"status":{"conditions":[{"type":"Dangling","status":"True","lastTransitionTime":null,"reason":"test reason"}]}}`), }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { got, _ := makeFlowSchemaConditionPatch(tt.condition) if !reflect.DeepEqual(got, tt.want) { t.Errorf("makeFlowSchemaConditionPatch() got = %s, want %s; diff is %s", got, tt.want, cmp.Diff(tt.want, got)) } }) } } </del>


Question:
Unit test in package flakes there is a race condition between the go routine that executes the request handler and the timeout go routine that waits for the handler to finish /sig api-machinery
/assign
I have a PR in the works
should fix it, I can easily reproduce it with the PR, the flake is not reproducible any more cc
/triage accepted

}() }() <del> postTimeoutFn() </del> <ins> defer postTimeoutFn() </ins> tw.timeout(err) } }


Question:
Unit test in package flakes there is a race condition between the go routine that executes the request handler and the timeout go routine that waits for the handler to finish /sig api-machinery
/assign
I have a PR in the works
should fix it, I can easily reproduce it with the PR, the flake is not reproducible any more cc
/triage accepted

}) } <ins> postTimeoutCh := make(chan struct{}) </ins> ts := httptest.NewServer( withDeadline( WithTimeout(


Question:
Unit test in package flakes there is a race condition between the go routine that executes the request handler and the timeout go routine that waits for the handler to finish /sig api-machinery
/assign
I have a PR in the works
should fix it, I can easily reproduce it with the PR, the flake is not reproducible any more cc
/triage accepted

h := w.Header() // trigger the timeout cancel() <del> // mutate response Headers for j := 0; j < 1000; j++ { h.Set("Test", "post") </del> <ins> // keep mutating response Headers until the request times out for { select { case <-postTimeoutCh: return default: h.Set("Test", "post") } </ins> } }), func(req *http.Request) (*http.Request, bool, func(), *apierrors.StatusError) { <del> return req, false, func() {}, apierrors.NewServerTimeout(schema.GroupResource{Group: "foo", Resource: "bar"}, "get", 0) </del> <ins> return req, false, func() { close(postTimeoutCh) }, apierrors.NewServerTimeout(schema.GroupResource{Group: "foo", Resource: "bar"}, "get", 0) </ins> }, ), ),


Question:
In </span /kind failing-test <!-- If this is a flake, please add: /kind flake --/sig node I believe the issue here is that the test is trying to get 5 restarts, but if we don't start the loop until after a few failures have occurred we start getting caught up in exponential backoff. The 6th backoff iteration is a 2m30s wait (i believe), so we exceed the default timeout of 5m for the test. We may need to change the test so that we are watching for pod status updates and catch the first one. HOWEVER - there's also some head of line blocking in pod status manager updates that I suspect might result in updates getting delayed, leading to timeout. Either way, the test has too little margin with exponential failure backoff to be reliable, and needs to be improved.
/triage accepted /assign

FailureThreshold:    1, } pod := livenessPodSpec(f.Namespace.Name, nil, livenessProbe) <del> RunLivenessTest(f, pod, 5, time.Minute*5) </del> <ins> // ~2 minutes backoff timeouts + 4 minutes defaultObservationTimeout + 2 minutes for each pod restart RunLivenessTest(f, pod, 5, 2*time.Minute+defaultObservationTimeout+4*2*time.Minute) </ins> }) /*


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

framework.PodNominator parallelizer parallelize.Parallelizer <del> // Indicates that RunFilterPlugins should accumulate all failed statuses and not return // after the first failure. runAllFilters bool </del> } // extensionPoint encapsulates desired and applied set of plugins at a specific extension


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

metricsRecorder        *metricsRecorder podNominator           framework.PodNominator extenders              []framework.Extender <del> runAllFilters          bool </del> captureProfile         CaptureProfile clusterEventMap        map[framework.ClusterEvent]sets.String parallelizer           parallelize.Parallelizer


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

} } <del> // WithRunAllFilters sets the runAllFilters flag, which means RunFilterPlugins accumulates // all failure Statuses. func WithRunAllFilters(runAllFilters bool) Option { return func(o *frameworkOptions) { o.runAllFilters = runAllFilters } } </del> // WithPodNominator sets podNominator for the scheduling frameworkImpl. func WithPodNominator(nominator framework.PodNominator) Option { return func(o *frameworkOptions) {


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

eventRecorder:        options.eventRecorder, informerFactory:      options.informerFactory, metricsRecorder:      options.metricsRecorder, <del> runAllFilters:        options.runAllFilters, </del> extenders:            options.extenders, PodNominator:         options.podNominator, parallelizer:         options.parallelizer,


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

} pluginStatus.SetFailedPlugin(pl.Name()) statuses[pl.Name()] = pluginStatus <del> if !f.runAllFilters { // Exit early if we don't need to run all filters. return statuses } </del> } }


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

plugins       []*TestPlugin wantStatus    *framework.Status wantStatusMap framework.PluginToStatus <del> runAllFilters bool </del> }{ { name: "SuccessFilter",


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

"TestPlugin2": framework.NewStatus(framework.Unschedulable, "injected filter status").WithFailedPlugin("TestPlugin2"), }, }, <del> { name: "SuccessFilterWithRunAllFilters", plugins: []*TestPlugin{ { name: "TestPlugin", inj:  injectedResult{FilterStatus: int(framework.Success)}, }, }, runAllFilters: true, wantStatus:    nil, wantStatusMap: framework.PluginToStatus{}, }, { name: "ErrorAndErrorFilters", plugins: []*TestPlugin{ { name: "TestPlugin1", inj:  injectedResult{FilterStatus: int(framework.Error)}, }, { name: "TestPlugin2", inj:  injectedResult{FilterStatus: int(framework.Error)}, }, }, runAllFilters: true, wantStatus:    framework.AsStatus(fmt.Errorf(`running "TestPlugin1" filter plugin: %w`, errInjectedFilterStatus)).WithFailedPlugin("TestPlugin1"), wantStatusMap: framework.PluginToStatus{ "TestPlugin1": framework.AsStatus(fmt.Errorf(`running "TestPlugin1" filter plugin: %w`, errInjectedFilterStatus)).WithFailedPlugin("TestPlugin1"), }, }, { name: "ErrorAndErrorFilters", plugins: []*TestPlugin{ { name: "TestPlugin1", inj:  injectedResult{FilterStatus: int(framework.UnschedulableAndUnresolvable)}, }, { name: "TestPlugin2", inj:  injectedResult{FilterStatus: int(framework.Unschedulable)}, }, }, runAllFilters: true, wantStatus:    framework.NewStatus(framework.UnschedulableAndUnresolvable, "injected filter status", "injected filter status").WithFailedPlugin("TestPlugin1"), wantStatusMap: framework.PluginToStatus{ "TestPlugin1": framework.NewStatus(framework.UnschedulableAndUnresolvable, "injected filter status").WithFailedPlugin("TestPlugin1"), "TestPlugin2": framework.NewStatus(framework.Unschedulable, "injected filter status").WithFailedPlugin("TestPlugin2"), }, }, </del> } for _, tt := range tests {


Question:
Since scheduler Policy API was total removed, we don't support any more. Remove unsupported configuration.
/sig scheduling /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling /cc
/sig scheduling (trying to tell the bots to remove )
The label(s) cannot be applied. These labels are supported: <detailsIn response to : /remove-needs sig /remove-label needs-sig Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It's true scheduler doesn't quite use this field anymore, but I'm wondering if some other project like ClusterAutoscaler or simulator still consumes this via .
There's no place we can config this field now by configure file(before we can apply the change by policy api, but now it's removed). So it seems like unless we support this feature with CC, or this field is useless. To other projects, it's the same. When they are using legacy policy api, they can configure this field, but once they synced with the latest kubernetes, with the policy api removed, they can not configure this anymore. In this situation, we should add support in CC. I don't quite understand the relationship between k/k with k/autoscaling or others, I think they sync with k/k periodic.
As I mentioned above, it's still accessible via a framework runtime programmatically, and thus may be still useful. For example, upon a scheduling failure, a ClusterAutoscaler's implementation may choose to dryrun through all filter plugins to ensure a new provisioned node can fit the pod, instead of resolving the failure while failing at another latter filter plugin.
Oh, I haven't thought of configuration via programmatically, that saved my concern. But a little strange.
That's exactly what I meant. Given it's a public option in framework, and there is no alternative if we disabled it, it'd be good to check with potential consumers. For now I can think of CA ( and k-sigs/scheduler-simulator ( If they can confirm it's not needed in their usecases, I'm fine with removing .
To k/autoscaler, at least these codes are still there together with the legacy policy api. , looking forward to the response.
is not used in sigs/kube-scheduler-simulator.
this isn't used by cluster autoscaler, we can remove it.

config.Plugin{Name: pl.name}) } profile := config.KubeSchedulerProfile{Plugins: cfgPls} <del> f, err := newFrameworkWithQueueSortAndBind(registry, profile, WithRunAllFilters(tt.runAllFilters)) </del> <ins> f, err := newFrameworkWithQueueSortAndBind(registry, profile) </ins> if err != nil { t.Fatalf("fail to create framework: %s", err) }


Question:
In the K8s Conformance meeting of 6 April the testing of Node endpoints and their eligibility for conformance testing was discussed. This issue is created to get more community input on the topic. Should node endpoints be tested? Can we accualy test these endpoints with the current test framework /sig testing /sig architecture /area conformance
/triage accepted /cc
I have lost track of this issue where it have already been discussed. Let's continue to discuss further here. Apologies for the confusion /cc
Clayton We have historically focused conformance on the perceived end user experience deploying applications to a conformant Kubernetes cluster as being consistently successful. From that perspective, while these are actions that are indeed key to conformant behavior on the majority of distributions, the first two are not invoked by end users, only by kubelets and potentially administrators in rare cases. I would expect to be able to create a node object, delete it, mutate its status, and delete a portion of them on a conformant Kubernetes cluster as an admin - but I would not expect a similar end user to do so. If we can make a case that a conformant Kubernetes distribution would reasonably not allow any of these actions via the API, then we could make a case to omit them from conformance. I can see a small path to adding these to conformance under a "they may be necessary as administrative action on the vast majority of kubernetes clusters", but I can't necessarily bump that to 100% of Kube clusters, especially if we put the application deployer as the center of our story (an application deployer does not need to delete node collections in order to succeed at running an application on conformant kubernetes)
Addressed the question in SIG Node
Created to make the remaining node endpoints ineligible. readCoreV1NodeStatus replaceCoreV1NodeStatus deleteCoreV1CollectionNode

- endpoint: replaceNetworkingV1NamespacedNetworkPolicyStatus reason: endpoints is currently feature gated and and will only receive e2e & conformance test in 1.25 link: https://github.com/kubernetes/kubernetes/pull/107963 <ins> - endpoint: readCoreV1NodeStatus reason: Kubernetes distribution would reasonably not allow this action via the API link: https://github.com/kubernetes/kubernetes/issues/109379 - endpoint: replaceCoreV1NodeStatus reason: Kubernetes distribution would reasonably not allow this action via the API link: https://github.com/kubernetes/kubernetes/issues/109379 - endpoint: deleteCoreV1CollectionNode reason: Kubernetes distribution would reasonably not allow this action via the API link: https://github.com/kubernetes/kubernetes/issues/109379 </ins>


Question:
ci-kubernetes-e2e-gce-master-new-gci-kubectl-skew kubetest.SkewTest 2022-04-14 15:13:42 +0000 UTC The error message is It's complaining about "file contents" and not "url contents". And because this is a skew test I am assuming this is due to The fix might be to cherry pick this to 1.23. No response /sig cli
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I've opened the cherry pick at if we decide that's the right route. The other options I can think of are to revert this change on master or to add the old environment variable back in as a file path. /assign /priority critical-urgent
Tracked the root of this to This should start passing once that merges. The cherry pick will also fix. We really need to get skew tests running on PRs.

OPENSTACK_IMAGE_NAME=${OPENSTACK_IMAGE_NAME:-CentOS7} # Downloaded image name for Openstack project <del> IMAGE_FILE=${IMAGE_FILE:-CentOS-7-x86_64-GenericCloud-1510.qcow2} </del> <ins> IMAGE_FILE=${IMAGE_FILE:-CentOS-7-x86_64-GenericCloud-1604.qcow2} </ins> # Absolute path where image file is stored. IMAGE_PATH=${IMAGE_PATH:-~/Downloads/openstack}


Question:
ci-kubernetes-e2e-gce-master-new-gci-kubectl-skew kubetest.SkewTest 2022-04-14 15:13:42 +0000 UTC The error message is It's complaining about "file contents" and not "url contents". And because this is a skew test I am assuming this is due to The fix might be to cherry pick this to 1.23. No response /sig cli
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I've opened the cherry pick at if we decide that's the right route. The other options I can think of are to revert this change on master or to add the old environment variable back in as a file path. /assign /priority critical-urgent
Tracked the root of this to This should start passing once that merges. The cherry pick will also fix. We really need to get skew tests running on PRs.

# Verify prereqs on host machine function verify-prereqs() { # Check the OpenStack command-line clients <del> for client in swift glance nova openstack; </del> <ins> for client in swift glance nova heat openstack; </ins> do if which $client >/dev/null 2>&1; then echo "${client} client installed"


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

# etcd - name: "etcd" <del> version: 3.5.3 </del> <ins> version: 3.5.4 </ins> refPaths: - path: cluster/gce/manifests/etcd.manifest match: etcd_docker_tag|etcd_version


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

{ "name": "etcd-container", {{security_context}} <del> "image": "{{ pillar.get('etcd_docker_repository', 'registry.k8s.io/etcd') }}:{{ pillar.get('etcd_docker_tag', '3.5.3-0') }}", </del> <ins> "image": "{{ pillar.get('etcd_docker_repository', 'registry.k8s.io/etcd') }}:{{ pillar.get('etcd_docker_tag', '3.5.4-0') }}", </ins> "resources": { "requests": { "cpu": {{ cpulimit }}


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

"value": "{{ pillar.get('storage_backend', 'etcd3') }}" }, { "name": "TARGET_VERSION", <del> "value": "{{ pillar.get('etcd_version', '3.5.3') }}" </del> <ins> "value": "{{ pillar.get('etcd_version', '3.5.4') }}" </ins> }, { "name": "DO_NOT_MOVE_BINARIES",


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

export SECONDARY_RANGE_NAME="pods-default" export STORAGE_BACKEND="etcd3" export STORAGE_MEDIA_TYPE="application/vnd.kubernetes.protobuf" <del> export ETCD_IMAGE=3.5.3-0 export ETCD_VERSION=3.5.3 </del> <ins> export ETCD_IMAGE=3.5.4-0 export ETCD_VERSION=3.5.4 </ins> # Upgrade master with updated kube envs "${KUBE_ROOT}/cluster/gce/upgrade.sh" -M -l


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

MinExternalEtcdVersion = "3.2.18" // DefaultEtcdVersion indicates the default etcd version that kubeadm uses <del> DefaultEtcdVersion = "3.5.3-0" </del> <ins> DefaultEtcdVersion = "3.5.4-0" </ins> // Etcd defines variable used internally when referring to etcd component Etcd = "etcd"


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

19: "3.4.13-0", 20: "3.4.13-0", 21: "3.4.13-0", <del> 22: "3.5.3-0", 23: "3.5.3-0", 24: "3.5.3-0", </del> <ins> 22: "3.5.4-0", 23: "3.5.4-0", 24: "3.5.4-0", 25: "3.5.4-0", </ins> } // KubeadmCertsClusterRoleName sets the name for the ClusterRole that allows


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

# A set of helpers for starting/running etcd for tests <del> ETCD_VERSION=${ETCD_VERSION:-3.5.3} </del> <ins> ETCD_VERSION=${ETCD_VERSION:-3.5.4} </ins> ETCD_HOST=${ETCD_HOST:-127.0.0.1} ETCD_PORT=${ETCD_PORT:-2379} # This is intentionally not called ETCD_LOG_LEVEL:


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

imagePullPolicy: Never args: [ "--etcd-servers=http://localhost:2379" ] - name: etcd <del> image: gcr.io/etcd-development/etcd:v3.5.3 </del> <ins> image: gcr.io/etcd-development/etcd:v3.5.4 </ins>


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

e2essh "k8s.io/kubernetes/test/e2e/framework/ssh" ) <del> const etcdImage = "3.5.3-0" </del> <ins> const etcdImage = "3.5.4-0" </ins> // EtcdUpgrade upgrades etcd on GCE. func EtcdUpgrade(targetStorage, targetVersion string) error {


Question:
Please publish 3.5.4-0 image and update to to benefit from fix for , while still keeping v3.5.3 critical correctness fixes (, ). For recent PRs for update to etcd v3.5.3 see and
/sig release /sig api-machinery /kind bug
/cc /triage accepted
Hey, Can I work on this ? I'm a beginner, but all the good first issues are mostly assigned. Can you guide me on doing this ?
I am working on this. /assign
Hi I would like to work on Make sense to backport in v1.22, v1.23 & v1.24? Thanks!!
hi, it makes sense to me, but requires approval from dependency maintainers. i can only approve the kubeadm changes.
was this backported ?

configs[CudaVectorAdd2] = Config{list.PromoterE2eRegistry, "cuda-vector-add", "2.2"} configs[DebianIptables] = Config{list.BuildImageRegistry, "debian-iptables", "bullseye-v1.3.0"} configs[EchoServer] = Config{list.PromoterE2eRegistry, "echoserver", "2.4"} <del> configs[Etcd] = Config{list.GcEtcdRegistry, "etcd", "3.5.3-0"} </del> <ins> configs[Etcd] = Config{list.GcEtcdRegistry, "etcd", "3.5.4-0"} </ins> configs[GlusterDynamicProvisioner] = Config{list.PromoterE2eRegistry, "glusterdynamic-provisioner", "v1.3"} configs[Httpd] = Config{list.PromoterE2eRegistry, "httpd", "2.4.38-2"} configs[HttpdNew] = Config{list.PromoterE2eRegistry, "httpd", "2.4.39-2"}


Question:
/kind failing-test <!-- If this is a flake, please add: /kind flake --/kind flake the test case was add in /cc smarterclayton <!-- Please assign a SIG using: /sig SIG-NAME --/sig node
/assign
bumping priority, seeing this in PR presubmit runs
/triage accepted

Pod:        newNamedPod("5-static", "test1", "pod1", true), UpdateType: kubetypes.SyncPodUpdate, }) <ins> // Wait for the previous work to be delivered to the worker drainAllWorkers(podWorkers) </ins> channels.Channel("5-static").Hold() podWorkers.UpdatePod(UpdatePodOptions{ Pod:        newNamedPod("5-static", "test1", "pod1", true),


Question:
Follow-up cleanup work for: Most of the information about this issue can be found in , where the work merged with a to change the function signature of to return a func() as well as an error. This issue will help to track that. All TODOs introduced in are resolved. N/A N/A <detailsN/A </details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority low Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc /triage accepted
/priority important-longterm
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

return true, binding, nil }) controllers := make(map[string]string) <del> stopFn := broadcaster.StartEventWatcher(func(obj runtime.Object) { </del> <ins> stopFn, err := broadcaster.StartEventWatcher(func(obj runtime.Object) { </ins> e, ok := obj.(*eventsv1.Event) if !ok || e.Reason != "Scheduled" { return


Question:
Follow-up cleanup work for: Most of the information about this issue can be found in , where the work merged with a to change the function signature of to return a func() as well as an error. This issue will help to track that. All TODOs introduced in are resolved. N/A N/A <detailsN/A </details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority low Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc /triage accepted
/priority important-longterm
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

controllers[e.Regarding.Name] = e.ReportingController wg.Done() }) <ins> if err != nil { t.Fatal(err) } </ins> defer stopFn() // Run scheduler.


Question:
Follow-up cleanup work for: Most of the information about this issue can be found in , where the work merged with a to change the function signature of to return a func() as well as an error. This issue will help to track that. All TODOs introduced in are resolved. N/A N/A <detailsN/A </details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority low Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc /triage accepted
/priority important-longterm
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

fwk.EventRecorder().Eventf(p.Pod, nil, v1.EventTypeWarning, "FailedScheduling", "Scheduling", msg) } called := make(chan struct{}) <del> stopFunc := eventBroadcaster.StartEventWatcher(func(obj runtime.Object) { </del> <ins> stopFunc, err := eventBroadcaster.StartEventWatcher(func(obj runtime.Object) { </ins> e, _ := obj.(*eventsv1.Event) if e.Reason != item.eventReason { t.Errorf("got event %v, want %v", e.Reason, item.eventReason) } close(called) }) <ins> if err != nil { t.Fatal(err) } </ins> sched.scheduleOne(ctx) <-called if e, a := item.expectAssumedPod, gotAssumedPod; !reflect.DeepEqual(e, a) {


Question:
Follow-up cleanup work for: Most of the information about this issue can be found in , where the work merged with a to change the function signature of to return a func() as well as an error. This issue will help to track that. All TODOs introduced in are resolved. N/A N/A <detailsN/A </details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority low Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc /triage accepted
/priority important-longterm
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

fakeVolumeBinder := volumebinding.NewFakeVolumeBinder(item.volumeBinderConfig) s, bindingChan, errChan := setupTestSchedulerWithVolumeBinding(ctx, fakeVolumeBinder, eventBroadcaster) eventChan := make(chan struct{}) <del> stopFunc := eventBroadcaster.StartEventWatcher(func(obj runtime.Object) { </del> <ins> stopFunc, err := eventBroadcaster.StartEventWatcher(func(obj runtime.Object) { </ins> e, _ := obj.(*eventsv1.Event) if e, a := item.eventReason, e.Reason; e != a { t.Errorf("expected %v, got %v", e, a) } close(eventChan) }) <ins> if err != nil { t.Fatal(err) } </ins> s.scheduleOne(ctx) // Wait for pod to succeed or fail scheduling select {


Question:
Follow-up cleanup work for: Most of the information about this issue can be found in , where the work merged with a to change the function signature of to return a func() as well as an error. This issue will help to track that. All TODOs introduced in are resolved. N/A N/A <detailsN/A </details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority low Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc /triage accepted
/priority important-longterm
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

// StartStructuredLogging starts sending events received from this EventBroadcaster to the structured logging function. // The return value can be ignored or used to stop recording, if desired. <ins> // TODO: this function should also return an error. </ins> func (e *eventBroadcasterImpl) StartStructuredLogging(verbosity klog.Level) func() { <del> return e.StartEventWatcher( </del> <ins> stopWatcher, err := e.StartEventWatcher( </ins> func(obj runtime.Object) { event, ok := obj.(*eventsv1.Event) if !ok {


Question:
Follow-up cleanup work for: Most of the information about this issue can be found in , where the work merged with a to change the function signature of to return a func() as well as an error. This issue will help to track that. All TODOs introduced in are resolved. N/A N/A <detailsN/A </details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority low Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc /triage accepted
/priority important-longterm
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

} klog.V(verbosity).InfoS("Event occurred", "object", klog.KRef(event.Regarding.Namespace, event.Regarding.Name), "kind", event.Regarding.Kind, "apiVersion", event.Regarding.APIVersion, "type", event.Type, "reason", event.Reason, "action", event.Action, "note", event.Note) }) <ins> if err != nil { klog.Errorf("failed to start event watcher: '%v'", err) return func() {} } return stopWatcher </ins> } // StartEventWatcher starts sending events received from this EventBroadcaster to the given event handler function. // The return value is used to stop recording <del> func (e *eventBroadcasterImpl) StartEventWatcher(eventHandler func(event runtime.Object)) func() { </del> <ins> func (e *eventBroadcasterImpl) StartEventWatcher(eventHandler func(event runtime.Object)) (func(), error) { </ins> watcher, err := e.Watch() if err != nil { klog.Errorf("Unable start event watcher: '%v' (will not retry!)", err) <del> // TODO: Rewrite the function signature to return an error, for // now just return a no-op function return func() { klog.Error("The event watcher failed to start") } </del> <ins> return nil, err </ins> } go func() { defer utilruntime.HandleCrash()