Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

} // Complete completes all the required options. <del> func (o *Options) Complete() error { </del> <ins> func (o *Options) Complete(fs *pflag.FlagSet) error { </ins> if len(o.ConfigFile) == 0 && len(o.WriteConfigTo) == 0 { o.config.HealthzBindAddress = addressFromDeprecatedFlags(o.config.HealthzBindAddress, o.healthzPort) o.config.MetricsBindAddress = addressFromDeprecatedFlags(o.config.MetricsBindAddress, o.metricsPort)


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

if err != nil { return err } <ins> // Before we overwrite the config which holds the parsed // command line parameters, we need to copy all modified // logging settings over to the loaded config (i.e.  logging // command line flags have priority). Otherwise `--config // ... -v=5` doesn't work (config resets verbosity even // when it contains no logging settings). copyLogsFromFlags(fs, &c.Logging) </ins> o.config = c if err := o.initWatcher(); err != nil {


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

return utilfeature.DefaultMutableFeatureGate.SetFromMap(o.config.FeatureGates) } <ins> // copyLogsFromFlags applies the logging flags from the given flag set to the given // configuration. Fields for which the corresponding flag was not used are left // unmodified. For fields that have multiple values (like vmodule), the values from // the flags get joined so that the command line flags have priority. // // TODO (pohly): move this to logsapi func copyLogsFromFlags(from *pflag.FlagSet, to *logsapi.LoggingConfiguration) error { var cloneFS pflag.FlagSet logsapi.AddFlags(to, &cloneFS) vmodule := to.VModule to.VModule = nil var err error cloneFS.VisitAll(func(f *pflag.Flag) { if err != nil { return } fsFlag := from.Lookup(f.Name) if fsFlag == nil { err = fmt.Errorf("logging flag %s not found in flag set", f.Name) return } if !fsFlag.Changed { return } if setErr := f.Value.Set(fsFlag.Value.String()); setErr != nil { err = fmt.Errorf("copying flag %s value: %v", f.Name, setErr) return } }) to.VModule = append(to.VModule, vmodule...) return err } </ins> // Creates a new filesystem watcher and adds watches for the config file. func (o *Options) initWatcher() error { fswatcher := filesystem.NewFsnotifyWatcher()


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

return fmt.Errorf("failed os init: %w", err) } <del> if err := opts.Complete(); err != nil { </del> <ins> if err := opts.Complete(cmd.Flags()); err != nil { </ins> return fmt.Errorf("failed complete: %w", err) }


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

"testing" "time" <ins> "github.com/spf13/pflag" </ins> v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime"


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

opt := NewOptions() opt.ConfigFile = file.Name() <del> err = opt.Complete() </del> <ins> err = opt.Complete(new(pflag.FlagSet)) </ins> if err != nil { t.Fatal(err) }


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

import ( "errors" "fmt" <ins> "io/ioutil" "path" </ins> "testing" "time" "github.com/google/go-cmp/cmp" <ins> "github.com/spf13/pflag" </ins> "github.com/stretchr/testify/assert" <ins> "github.com/stretchr/testify/require" </ins> v1 "k8s.io/api/core/v1" <ins> "k8s.io/apimachinery/pkg/api/resource" </ins> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" clientsetfake "k8s.io/client-go/kubernetes/fake" componentbaseconfig "k8s.io/component-base/config"


Question:
introduced configuration of additional logging aspects (e.g. format) through command line flags and config file. This was merged for Kubernetes 1.28. Now the following invocation doesn't work as expected when the config file doesn't configure logging: It should run with verbosity 4. Instead it runs with verbosity 0, from the config. Run with this config file: No response 28 rc n/a No response No response No response No response
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The problem is caused by this line: At that point, is 4 (from the command line flag), and then it gets overwritten with == 0. The normal approach would be: set up config with defaults modify config through flags modify config further by parsing a config file into it. Can kube-proxy do this? It has the downside that a config file can only overwrite fields if they are in the file. A config field doesn't get reset to defaults just by parsing the file. /cc
it looks like kube-proxy special-cases the to let a flag override config... can/should it do the same thing for verbosity? are there any other flags like this?
I don't think that's the normal approach... it's more normal to make flags and config file mutually exclusive. That's difficult for things which started as flags and moved into config
/sig network /sig instrumentation /kind bug /kind regression /milestone v1.28
let's go with and warn it and comment as much as possible, I was going with something like but Patrick is working on it so I assign to him the issue
/assign
Hey folks, v1.28 Bug Triage Lead here Just wanted to know if this is already tracked by SIG leads and someone already working actively fixing it and set a priority?
Perhaps I should have said "this is how component-base/logs was meant to be used". Somehow it worked for kubelet. Here's my preliminary solution: It needs more testing, but I wanted to share it early because I am running out of time today.
it is , I can take over Patrick's work in case is needed
It would be nice if the logs API could distinguish between and , how about make to a pointer? But this change is supposed to be a breaking change, I'm not sure of the scope of the impact. For side, if set and at the same time, should the command flag override the config file or warn it and be discarded? I think it should be overridden.
xref:

} } <ins> // TestOptionsComplete checks that command line flags are combined with a // config properly. func TestOptionsComplete(t *testing.T) { header := `apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration ` // Determine default config (depends on platform defaults). o := NewOptions() require.NoError(t, o.Complete(new(pflag.FlagSet))) expected := o.config config := header + `logging: format: json flushFrequency: 1s verbosity: 10 vmodule: - filePattern: foo.go verbosity: 6 - filePattern: bar.go verbosity: 8 ` expectedLoggingConfig := logsapi.LoggingConfiguration{ Format:         "json", FlushFrequency: logsapi.TimeOrMetaDuration{Duration: metav1.Duration{Duration: time.Second}, SerializeAsString: true}, Verbosity:      10, VModule: []logsapi.VModuleItem{ { FilePattern: "foo.go", Verbosity:   6, }, { FilePattern: "bar.go", Verbosity:   8, }, }, Options: logsapi.FormatOptions{ JSON: logsapi.JSONOptions{ InfoBufferSize: resource.QuantityValue{Quantity: resource.MustParse("0")}, }, }, } for name, tc := range map[string]struct { config   string flags    []string expected *kubeproxyconfig.KubeProxyConfiguration }{ "empty": { expected: expected, }, "empty-config": { config:   header, expected: expected, }, "logging-config": { config: config, expected: func() *kubeproxyconfig.KubeProxyConfiguration { c := expected.DeepCopy() c.Logging = *expectedLoggingConfig.DeepCopy() return c }(), }, "flags": { flags: []string{ "-v=7", "--vmodule", "goo.go=8", }, expected: func() *kubeproxyconfig.KubeProxyConfiguration { c := expected.DeepCopy() c.Logging.Verbosity = 7 c.Logging.VModule = append(c.Logging.VModule, logsapi.VModuleItem{ FilePattern: "goo.go", Verbosity:   8, }) return c }(), }, "both": { config: config, flags: []string{ "-v=7", "--vmodule", "goo.go=8", "--ipvs-scheduler", "some-scheduler", // Overwritten by config. }, expected: func() *kubeproxyconfig.KubeProxyConfiguration { c := expected.DeepCopy() c.Logging = *expectedLoggingConfig.DeepCopy() // Flag wins. c.Logging.Verbosity = 7 // Flag and config get merged with command line flags first. c.Logging.VModule = append([]logsapi.VModuleItem{ { FilePattern: "goo.go", Verbosity:   8, }, }, c.Logging.VModule...) return c }(), }, } { t.Run(name, func(t *testing.T) { options := NewOptions() fs := new(pflag.FlagSet) options.AddFlags(fs) flags := tc.flags if len(tc.config) > 0 { tmp := t.TempDir() configFile := path.Join(tmp, "kube-proxy.conf") require.NoError(t, ioutil.WriteFile(configFile, []byte(tc.config), 0666)) flags = append(flags, "--config", configFile) } require.NoError(t, fs.Parse(flags)) require.NoError(t, options.Complete(fs)) assert.Equal(t, tc.expected, options.config) }) } } </ins> type fakeProxyServerLongRun struct{} // Run runs the specified ProxyServer.


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

- baseImportPath: "./vendor/k8s.io/kms/" allowedImports: <del> - k8s.io/apimachinery - k8s.io/klog </del> - k8s.io/kms - baseImportPath: "./vendor/k8s.io/endpointslice/"


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

require ( github.com/gogo/protobuf v1.3.2 google.golang.org/grpc v1.56.3 <del> k8s.io/apimachinery v0.0.0 </del> ) require ( <del> github.com/go-logr/logr v1.2.4 // indirect </del> github.com/golang/protobuf v1.5.3 // indirect golang.org/x/net v0.17.0 // indirect golang.org/x/sys v0.13.0 // indirect golang.org/x/text v0.13.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234030-28d5490b6b19 // indirect google.golang.org/protobuf v1.31.0 // indirect <del> k8s.io/klog/v2 v2.100.1 // indirect k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect </del> ) <del> replace ( k8s.io/apimachinery => ../apimachinery k8s.io/kms => ../kms ) </del> <ins> replace k8s.io/kms => ../kms </ins>


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

cloud.google.com/go/compute v1.19.1/go.mod h1:6ylj3a05WF8leseCdIf77NK0g1ey+nj5IKd5/kvShxE= cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA= <del> github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= </del> github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= <del> github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= </del> github.com/envoyproxy/go-control-plane v0.11.1-0.20230524094728-9239064ad72f/go.mod h1:sfYdkwUW4BA3PbKjySwjJy+O4Pu0h62rlqCMHNk+K+Q= github.com/envoyproxy/protoc-gen-validate v0.10.1/go.mod h1:DRjgyB0I43LtJapqN6NiRwroiAU2PaFuvk/vjgh61ss= <del> github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ= github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs= github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k= github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls= </del> github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/glog v1.1.0/go.mod h1:pfYeQZ3JWZoXTV5sFc986z3HTpwQs9At6P4ImfuP3NQ= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= <del> github.com/google/gnostic-models v0.6.8/go.mod h1:5n7qKqH0f5wFt+aWF8CW6pZLLNOfYuF5OpfBSENuI8U= </del> github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= <del> github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= </del> github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= <del> github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= </del> github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= <del> github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o= github.com/onsi/gomega v1.28.0/go.mod h1:A1H2JE76sI14WIP57LMKj7FVfCHx3g3BcZVjJG8bjX8= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= </del> github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k= golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= <del> golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= </del> golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= <del> golang.org/x/tools v0.12.0/go.mod h1:Sc0INKfu04TlqNoRA1hgpFZbhYXHPr4V5DzpSBTPqQM= </del> <ins> golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= </ins> golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= <del> gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= k8s.io/klog/v2 v2.100.1 h1:7WCHKK6K8fNhTqfBhISHQ97KrnJNFZMcQvKp7gP/tmg= k8s.io/klog/v2 v2.100.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00/go.mod h1:AsvuZPBlUDVuCdzJ87iajxtXuR9oktsTctW/R9wwouA= k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI= k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= sigs.k8s.io/structured-merge-diff/v4 v4.3.0/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08= sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= </del>


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

github.com/ThalesIgnite/crypto11 v1.2.5/go.mod h1:ILDKtnCKiQ7zRoNxcp36Y1ZR8LBPmR2E23+wTQe/MlE= github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= <del> github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ= </del> github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= <del> k8s.io/klog/v2 v2.100.1 h1:7WCHKK6K8fNhTqfBhISHQ97KrnJNFZMcQvKp7gP/tmg= k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI= </del>


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

"google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" <del> "k8s.io/apimachinery/pkg/util/wait" </del> kmsapi "k8s.io/kms/apis/v2" )


Question:
Drop the and dependencies from to keep the import lighter. /sig auth /triage accepted
/assign Hi, , I created a PR to fix this. I am not sure If I am correct due to the repo seems generated. Would you like to provides some hints for me?
This will likely require a new go module in to move the client-go dep and then mark it as internal, ex: . is the only API machinery dep so we can probably come up with some way to re-implement the wait logic locally? Then we drop these lines so that these imports do not come back again:
Whoa, yeah, I'd take a for loop and a time.Sleep in a test to drop those deps
What is kms using client-go for? Tests only?
These were all needed by the KMS repo back when we had a much more complicated reference implementation which required a key hierarchy. Now they are just used in some tests and mock implementations, so we should be able to simplify things.
I found a place for rate limit. #liggitt
that package just delegates to "", switch that mock provider to use that directly?

client := newClient(t, address) // make sure the gRPC server is up before running tests <del> if err := wait.PollImmediateUntilWithContext(ctx, time.Second, func(ctx context.Context) (bool, error) { ctx, cancel := context.WithTimeout(ctx, 3*time.Second) defer cancel() _, err := client.Status(ctx, &kmsapi.StatusRequest{}) if err != nil { t.Logf("failed to get kms status: %v", err) </del> <ins> ready: for { select { case <-ctx.Done(): t.Fatalf("server failed to start in time: %v", ctx.Err()) default: if done := func() bool { ctx, cancel := context.WithTimeout(ctx, 3*time.Second) defer cancel() _, err := client.Status(ctx, &kmsapi.StatusRequest{}) if err != nil { t.Logf("failed to get kms status: %v", err) } return err == nil }(); done { break ready } time.Sleep(time.Second) </ins> } <del> return err == nil, nil }); err != nil { t.Fatal(err) </del> } t.Run("should be able to encrypt and decrypt through unix domain sockets", func(t *testing.T) {


Question:
There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. As suggested by in , we can replace by /sig network /help /kind cleanup
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. /help /kind cleanup </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign I would like to raise a PR to fix this.

if !reflect.DeepEqual(n.podCIDRs, podCIDRs) { klog.ErrorS(nil, "Using NodeCIDR LocalDetector mode, current PodCIDRs are different than previous PodCIDRs, restarting", "node", klog.KObj(node), "newPodCIDRs", podCIDRs, "oldPodCIDRs", n.podCIDRs) <del> panic("Current Node PodCIDRs are different than previous PodCIDRs, restarting") </del> <ins> klog.FlushAndExit(klog.ExitFlushTimeout, 1) </ins> } }


Question:
There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. As suggested by in , we can replace by /sig network /help /kind cleanup
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. /help /kind cleanup </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign I would like to raise a PR to fix this.

if !reflect.DeepEqual(n.podCIDRs, podCIDRs) { klog.ErrorS(nil, "Using NodeCIDR LocalDetector mode, current PodCIDRs are different than previous PodCIDRs, restarting", "node", klog.KObj(node), "newPodCIDRs", podCIDRs, "oldPODCIDRs", n.podCIDRs) <del> panic("Current Node PodCIDRs are different than previous PodCIDRs, restarting") </del> <ins> klog.FlushAndExit(klog.ExitFlushTimeout, 1) </ins> } }


Question:
There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. As suggested by in , we can replace by /sig network /help /kind cleanup
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. /help /kind cleanup </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign I would like to raise a PR to fix this.

package proxy import ( <ins> "strconv" </ins> "testing" v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" <ins> "k8s.io/klog/v2" </ins> ) func TestNodePodCIDRHandlerAdd(t *testing.T) { <ins> oldKlogOsExit := klog.OsExit defer func() { klog.OsExit = oldKlogOsExit }() klog.OsExit = customExit </ins> tests := []struct { name            string oldNodePodCIDRs []string


Question:
There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. As suggested by in , we can replace by /sig network /help /kind cleanup
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. /help /kind cleanup </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign I would like to raise a PR to fix this.

t.Errorf("The code did panic") } }() n.OnNodeAdd(node) }) } } func TestNodePodCIDRHandlerUpdate(t *testing.T) { <ins> oldKlogOsExit := klog.OsExit defer func() { klog.OsExit = oldKlogOsExit }() klog.OsExit = customExit </ins> tests := []struct { name            string oldNodePodCIDRs []string


Question:
There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. As suggested by in , we can replace by /sig network /help /kind cleanup
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : There are some cases when kube-proxy has to exit and restart, in order to do that we panic, but this has the problem that the log output shows a literal "panic" message and engineers, support, customers and automation gets confused, thinking that is a problem when is the expected behavior. /help /kind cleanup </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign I would like to raise a PR to fix this.

t.Errorf("The code did panic") } }() n.OnNodeUpdate(oldNode, node) }) } } <ins> func customExit(exitCode int) { panic(strconv.Itoa(exitCode)) } </ins>


Question:
found this by running tests on this PR: it's problematic for a couple of reasons: prevents kubeadm from moving out (eventually) there are no API guarantees in the given code/structs/functions in pkiutils the logic in cmd/kube- must be reworked to generate cert/key pair without using kubeadm. originally in pending fix PR
/sig api-machinery cluster-lifecycle architecture /area code-organization
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig api-machinery sig-cluster-lifecycle architecture /area code-organization </details
/assign
cc
Please make sure to add an import restriction to prevent a regression in the future.
/triage accepted

<ins> rules: - selectorRegexp: k8s[.]io/kubernetes allowedPrefixes: - k8s.io/kubernetes/cmd/kube-apiserver - k8s.io/kubernetes/pkg - k8s.io/kubernetes/plugin - k8s.io/kubernetes/test/utils - k8s.io/kubernetes/third_party </ins>


Question:
found this by running tests on this PR: it's problematic for a couple of reasons: prevents kubeadm from moving out (eventually) there are no API guarantees in the given code/structs/functions in pkiutils the logic in cmd/kube- must be reworked to generate cert/key pair without using kubeadm. originally in pending fix PR
/sig api-machinery cluster-lifecycle architecture /area code-organization
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig api-machinery sig-cluster-lifecycle architecture /area code-organization </details
/assign
cc
Please make sure to add an import restriction to prevent a regression in the future.
/triage accepted

import ( "context" <ins> "crypto/ecdsa" "crypto/elliptic" "crypto/rand" </ins> "crypto/rsa" "crypto/x509" <ins> "crypto/x509/pkix" "encoding/pem" </ins> "fmt" <ins> "math" "math/big" </ins> "net" "os" "path/filepath"


Question:
found this by running tests on this PR: it's problematic for a couple of reasons: prevents kubeadm from moving out (eventually) there are no API guarantees in the given code/structs/functions in pkiutils the logic in cmd/kube- must be reworked to generate cert/key pair without using kubeadm. originally in pending fix PR
/sig api-machinery cluster-lifecycle architecture /area code-organization
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig api-machinery sig-cluster-lifecycle architecture /area code-organization </details
/assign
cc
Please make sure to add an import restriction to prevent a regression in the future.
/triage accepted

restclient "k8s.io/client-go/rest" clientgotransport "k8s.io/client-go/transport" "k8s.io/client-go/util/cert" <ins> "k8s.io/client-go/util/keyutil" </ins> logsapi "k8s.io/component-base/logs/api/v1" "k8s.io/klog/v2" "k8s.io/kube-aggregator/pkg/apiserver"


Question:
found this by running tests on this PR: it's problematic for a couple of reasons: prevents kubeadm from moving out (eventually) there are no API guarantees in the given code/structs/functions in pkiutils the logic in cmd/kube- must be reworked to generate cert/key pair without using kubeadm. originally in pending fix PR
/sig api-machinery cluster-lifecycle architecture /area code-organization
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig api-machinery sig-cluster-lifecycle architecture /area code-organization </details
/assign
cc
Please make sure to add an import restriction to prevent a regression in the future.
/triage accepted

"k8s.io/kubernetes/cmd/kube-apiserver/app" "k8s.io/kubernetes/cmd/kube-apiserver/app/options" <del> "k8s.io/kubernetes/cmd/kubeadm/app/util/pkiutil" </del> testutil "k8s.io/kubernetes/test/utils" )


Question:
found this by running tests on this PR: it's problematic for a couple of reasons: prevents kubeadm from moving out (eventually) there are no API guarantees in the given code/structs/functions in pkiutils the logic in cmd/kube- must be reworked to generate cert/key pair without using kubeadm. originally in pending fix PR
/sig api-machinery cluster-lifecycle architecture /area code-organization
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig api-machinery sig-cluster-lifecycle architecture /area code-organization </details
/assign
cc
Please make sure to add an import restriction to prevent a regression in the future.
/triage accepted

// give the kube api server an "identity" it can use to for request header auth // so that aggregated api servers can understand who the calling user is s.Authentication.RequestHeader.AllowedNames = []string{"ash", "misty", "brock"} <ins> // create private key signer, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) if err != nil { return result, err } </ins> // make a client certificate for the api server - common name has to match one of our defined names above <ins> serial, err := rand.Int(rand.Reader, new(big.Int).SetInt64(math.MaxInt64-1)) if err != nil { return result, err } serial = new(big.Int).Add(serial, big.NewInt(1)) </ins> tenThousandHoursLater := time.Now().Add(10_000 * time.Hour) <del> clientCrtOfAPIServer, signer, err := pkiutil.NewCertAndKey(proxySigningCert, proxySigningKey, &pkiutil.CertConfig{ Config: cert.Config{ </del> <ins> certTmpl := x509.Certificate{ Subject: pkix.Name{ </ins> CommonName: "misty", <del> Usages: []x509.ExtKeyUsage{ x509.ExtKeyUsageClientAuth, }, </del> }, <del> NotAfter:           &tenThousandHoursLater, PublicKeyAlgorithm: x509.ECDSA, }) </del> <ins> SerialNumber: serial, NotBefore:    proxySigningCert.NotBefore, NotAfter:     tenThousandHoursLater, KeyUsage:     x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{ x509.ExtKeyUsageClientAuth, }, BasicConstraintsValid: true, } certDERBytes, err := x509.CreateCertificate(rand.Reader, &certTmpl, proxySigningCert, signer.Public(), proxySigningKey) </ins> if err != nil { return result, err } <del> if err := pkiutil.WriteCertAndKey(s.SecureServing.ServerCert.CertDirectory, "misty-crt", clientCrtOfAPIServer, signer); err != nil { </del> <ins> clientCrtOfAPIServer, err := x509.ParseCertificate(certDERBytes) if err != nil { </ins> return result, err } <ins> // write the cert to disk certificatePath := filepath.Join(s.SecureServing.ServerCert.CertDirectory, "misty-crt.crt") certBlock := pem.Block{ Type:  "CERTIFICATE", Bytes: clientCrtOfAPIServer.Raw, } certBytes := pem.EncodeToMemory(&certBlock) if err := cert.WriteCert(certificatePath, certBytes); err != nil { return result, err } // write the key to disk privateKeyPath := filepath.Join(s.SecureServing.ServerCert.CertDirectory, "misty-crt.key") encodedPrivateKey, err := keyutil.MarshalPrivateKeyToPEM(signer) if err != nil { return result, err } if err := keyutil.WriteKey(privateKeyPath, encodedPrivateKey); err != nil { return result, err } </ins> s.ProxyClientKeyFile = filepath.Join(s.SecureServing.ServerCert.CertDirectory, "misty-crt.key") s.ProxyClientCertFile = filepath.Join(s.SecureServing.ServerCert.CertDirectory, "misty-crt.crt")


Question:
master-informing: ! 2023-09-07 03:13:59 +0000 UTC No response /sig network
Checking this failure This is running ingress-gce version, logs in failure is It seems the healthceck interval is not updated correctly, by reading the test tagging the people with more expertise in this area /cc
/cc
it seems the test restarts the kube-controller-manager but we landed recently the switch from intree to outtree cloud providers, so this logic now runs in the cloud-controller-manager, can this be the problem, that we have to restart the cloud-controller-manager now instead of the kcm?
/triage accepted This flake started after and kept failing after 09-07.
yeah, the logic that handles the loadbalancer shifted to that cloud controller manager with that PR

"k8s.io/kubernetes/test/e2e/framework" e2edaemonset "k8s.io/kubernetes/test/e2e/framework/daemonset" e2edeployment "k8s.io/kubernetes/test/e2e/framework/deployment" <del> e2ekubesystem "k8s.io/kubernetes/test/e2e/framework/kubesystem" </del> e2enetwork "k8s.io/kubernetes/test/e2e/framework/network" e2enode "k8s.io/kubernetes/test/e2e/framework/node" e2epod "k8s.io/kubernetes/test/e2e/framework/pod"


Question:
master-informing: ! 2023-09-07 03:13:59 +0000 UTC No response /sig network
Checking this failure This is running ingress-gce version, logs in failure is It seems the healthceck interval is not updated correctly, by reading the test tagging the people with more expertise in this area /cc
/cc
it seems the test restarts the kube-controller-manager but we landed recently the switch from intree to outtree cloud providers, so this logic now runs in the cloud-controller-manager, can this be the problem, that we have to restart the cloud-controller-manager now instead of the kcm?
/triage accepted This flake started after and kept failing after 09-07.
yeah, the logic that handles the loadbalancer shifted to that cloud controller manager with that PR

e2eservice "k8s.io/kubernetes/test/e2e/framework/service" e2eskipper "k8s.io/kubernetes/test/e2e/framework/skipper" "k8s.io/kubernetes/test/e2e/network/common" <del> gcecloud "k8s.io/legacy-cloud-providers/gce" </del> admissionapi "k8s.io/pod-security-admission/api" netutils "k8s.io/utils/net" utilpointer "k8s.io/utils/pointer"


Question:
master-informing: ! 2023-09-07 03:13:59 +0000 UTC No response /sig network
Checking this failure This is running ingress-gce version, logs in failure is It seems the healthceck interval is not updated correctly, by reading the test tagging the people with more expertise in this area /cc
/cc
it seems the test restarts the kube-controller-manager but we landed recently the switch from intree to outtree cloud providers, so this logic now runs in the cloud-controller-manager, can this be the problem, that we have to restart the cloud-controller-manager now instead of the kcm?
/triage accepted This flake started after and kept failing after 09-07.
yeah, the logic that handles the loadbalancer shifted to that cloud controller manager with that PR

} }) <del> // This test creates a load balancer, make sure its health check interval // equals to gceHcCheckIntervalSeconds. Then the interval is manipulated // to be something else, see if the interval will be reconciled. ginkgo.It("should reconcile LB health check interval [Slow][Serial][Disruptive]", func(ctx context.Context) { const gceHcCheckIntervalSeconds = int64(8) // This test is for clusters on GCE. // (It restarts kube-controller-manager, which we don't support on GKE) e2eskipper.SkipUnlessProviderIs("gce") e2eskipper.SkipUnlessSSHKeyPresent() clusterID, err := gce.GetClusterID(ctx, cs) if err != nil { framework.Failf("framework.GetClusterID(cs) = _, %v; want nil", err) } gceCloud, err := gce.GetGCECloud() if err != nil { framework.Failf("framework.GetGCECloud() = _, %v; want nil", err) } namespace := f.Namespace.Name serviceName := "lb-hc-int" jig := e2eservice.NewTestJig(cs, namespace, serviceName) ginkgo.By("create load balancer service") // Create loadbalancer service with source range from node[0] and podAccept svc, err := jig.CreateTCPService(ctx, func(svc *v1.Service) { svc.Spec.Type = v1.ServiceTypeLoadBalancer }) framework.ExpectNoError(err) ginkgo.DeferCleanup(func(ctx context.Context) { ginkgo.By("Clean up loadbalancer service") e2eservice.WaitForServiceDeletedWithFinalizer(ctx, cs, svc.Namespace, svc.Name) }) svc, err = jig.WaitForLoadBalancer(ctx, e2eservice.GetServiceLoadBalancerCreationTimeout(ctx, cs)) framework.ExpectNoError(err) hcName := gcecloud.MakeNodesHealthCheckName(clusterID) hc, err := gceCloud.GetHTTPHealthCheck(hcName) if err != nil { framework.Failf("gceCloud.GetHttpHealthCheck(%q) = _, %v; want nil", hcName, err) } gomega.Expect(hc.CheckIntervalSec).To(gomega.Equal(gceHcCheckIntervalSeconds)) ginkgo.By("modify the health check interval") hc.CheckIntervalSec = gceHcCheckIntervalSeconds - 1 if err = gceCloud.UpdateHTTPHealthCheck(hc); err != nil { framework.Failf("gcecloud.UpdateHttpHealthCheck(%#v) = %v; want nil", hc, err) } ginkgo.By("restart kube-controller-manager") if err := e2ekubesystem.RestartControllerManager(ctx); err != nil { framework.Failf("e2ekubesystem.RestartControllerManager() = %v; want nil", err) } if err := e2ekubesystem.WaitForControllerManagerUp(ctx); err != nil { framework.Failf("e2ekubesystem.WaitForControllerManagerUp() = %v; want nil", err) } ginkgo.By("health check should be reconciled") pollInterval := framework.Poll * 10 loadBalancerPropagationTimeout := e2eservice.GetServiceLoadBalancerPropagationTimeout(ctx, cs) if pollErr := wait.PollImmediate(pollInterval, loadBalancerPropagationTimeout, func() (bool, error) { hc, err := gceCloud.GetHTTPHealthCheck(hcName) if err != nil { framework.Logf("ginkgo.Failed to get HttpHealthCheck(%q): %v", hcName, err) return false, err } framework.Logf("hc.CheckIntervalSec = %v", hc.CheckIntervalSec) return hc.CheckIntervalSec == gceHcCheckIntervalSeconds, nil }); pollErr != nil { framework.Failf("Health check %q does not reconcile its check interval to %d.", hcName, gceHcCheckIntervalSeconds) } }) </del> // [LinuxOnly]: Windows does not support session affinity. ginkgo.It("should have session affinity work for LoadBalancer service with ESIPP on [Slow] [LinuxOnly]", func(ctx context.Context) { // L4 load balancer affinity `ClientIP` is not supported on AWS ELB.


Question:
With the migration to podutils and adding some presubmits (from periodic jobs) we are seeing some failures in presubmits. The error seems to have issues finding the right node to create.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node /sig testing
any leads on solving this one?
see notes on slack -
/assign
this looks good now!

indexer.Add(resourceQuota) // old service was a load balancer, but updated version is a node port. <del> oldService := &api.Service{ ObjectMeta: api.ObjectMeta{Name: "service", Namespace: "test"}, </del> <ins> existingService := &api.Service{ ObjectMeta: api.ObjectMeta{Name: "service", Namespace: "test", ResourceVersion: "1"}, </ins> Spec:       api.ServiceSpec{Type: api.ServiceTypeLoadBalancer}, } newService := &api.Service{


Question:
With the migration to podutils and adding some presubmits (from periodic jobs) we are seeing some failures in presubmits. The error seems to have issues finding the right node to create.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node /sig testing
any leads on solving this one?
see notes on slack -
/assign
this looks good now!

Ports: []api.ServicePort{{Port: 1234}}, }, } <del> err := handler.Admit(admission.NewAttributesRecord(newService, oldService, api.Kind("Service").WithVersion("version"), newService.Namespace, newService.Name, api.Resource("services").WithVersion("version"), "", admission.Update, nil)) </del> <ins> err := handler.Admit(admission.NewAttributesRecord(newService, existingService, api.Kind("Service").WithVersion("version"), newService.Namespace, newService.Name, api.Resource("services").WithVersion("version"), "", admission.Update, nil)) </ins> if err != nil { t.Errorf("Unexpected error: %v", err) }


Question:
With the migration to podutils and adding some presubmits (from periodic jobs) we are seeing some failures in presubmits. The error seems to have issues finding the right node to create.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig node /sig testing
any leads on solving this one?
see notes on slack -
/assign
this looks good now!

if prevItem == nil { return nil, admission.NewForbidden(a, fmt.Errorf("unable to get previous usage since prior version of object was not found")) } <del> prevUsage := evaluator.Usage(prevItem) deltaUsage = quota.Subtract(deltaUsage, prevUsage) </del> <ins> // if we can definitively determine that this is not a case of "create on update", // then charge based on the delta.  Otherwise, bill the maximum metadata, err := meta.Accessor(prevItem) if err == nil && len(metadata.GetResourceVersion()) > 0 { prevUsage := evaluator.Usage(prevItem) deltaUsage = quota.Subtract(deltaUsage, prevUsage) } </ins> } if quota.IsZero(deltaUsage) { return quotas, nil


Question:
Two serial tests from sig-node are keep failing: E2eNode Suite. E2eNode Suite. pls see: [1] [2] <fmt.wrapError | 0x40005e7840: failed to update pod "pod--65da-42b7-a6a5-": Pod "pod--65da-42b7-a6a5-" is invalid: spec: Forbidden: pod updates may not change fields other than ,,, (only additions to existing tolerations), (allow it to be set to 1 if it was previously negative) core.PodSpec{ Volumes: []core.Volume{ { Name: "cm", Name: "secret", VolumeSource: core.VolumeSource{ ... // 3 identical fields AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, Secret: &core.SecretVolumeSource{SecretName: "secret", DefaultMode: &420}, NFS: nil, ISCSI: nil, ... // 9 identical fields FC: nil, AzureFile: nil, ConfigMap: &core.ConfigMapVolumeSource{ LocalObjectReference: core.LocalObjectReference{Name: "does-not-exist"}, DefaultMode: &420, }, ConfigMap: nil, VsphereVolume: nil, AzureDisk: nil, ... // 7 identical fields }, }, }, InitContainers: {{Name: "init1", Image: "", Command: {"sh", "-c", "sleep 5s"}, TerminationMessagePath: "/dev/termination-log", ...}}, Containers: []core.Container{ { ... // 9 identical fields ResizePolicy: nil, RestartPolicy: nil, VolumeMounts: []core.VolumeMount{ { Name: "cm", Name: "secret", ReadOnly: false, MountPath: "/config", ... // 3 identical fields }, }, VolumeDevices: nil, LivenessProbe: nil, ... // 10 identical fields }, }, EphemeralContainers: nil, RestartPolicy: "Always", ... // 28 identical fields } happened since 18/10/23 run the e2e testcases manually No response <details</details<detailsImpacts all </details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /cc
cc
failed on x86 as well:
/sig node
/unassign
is merged today. /cc
will take a look at this ASAP /assign
and I are sorry for the issue this caused. We decided to open to fix this issue. I think this test was more complicated than we thought. Sorry for this.

// Verify PodReady is not set (since sandboxcreation is blocked) _, err = getTransitionTimeForPodConditionWithStatus(p, v1.PodReady, false) framework.ExpectNoError(err) <ins> // this testcase is creating the missing volume that unblock the pod above, // and check PodReadyToStartContainer is setting correctly. ginkgo.By("checking pod condition for a pod when volumes source is created") configmap := v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Name: "cm-that-unblock-pod-condition", }, Data: map[string]string{ "key": "value", }, BinaryData: map[string][]byte{ "binaryKey": []byte("value"), }, } _, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(ctx, &configmap, metav1.CreateOptions{}) framework.ExpectNoError(err) defer func() { err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Delete(ctx, "cm-that-unblock-pod-condition", metav1.DeleteOptions{}) framework.ExpectNoError(err, "unable to delete configmap") }() p2 := webserverPodSpec("pod2-"+string(uuid.NewUUID()), "web2", "init2", hasInitContainers) p2.Spec.Volumes = []v1.Volume{ { Name: "cm-2", VolumeSource: v1.VolumeSource{ ConfigMap: &v1.ConfigMapVolumeSource{ LocalObjectReference: v1.LocalObjectReference{Name: "cm-that-unblock-pod-condition"}, }, }, }, } p2.Spec.Containers[0].VolumeMounts = []v1.VolumeMount{ { Name:      "cm-2", MountPath: "/config", }, } p2 = e2epod.NewPodClient(f).Create(ctx, p2) framework.ExpectNoError(e2epod.WaitTimeoutForPodReadyInNamespace(ctx, f.ClientSet, p2.Name, p2.Namespace, framework.PodStartTimeout)) p2, err = e2epod.NewPodClient(f).Get(ctx, p2.Name, metav1.GetOptions{}) framework.ExpectNoError(err) _, err = getTransitionTimeForPodConditionWithStatus(p2, v1.PodScheduled, true) framework.ExpectNoError(err) _, err = getTransitionTimeForPodConditionWithStatus(p2, v1.PodInitialized, true) framework.ExpectNoError(err) // Verify PodReadyToStartContainers is set (since sandboxcreation is unblocked) if checkPodReadyToStart { _, err = getTransitionTimeForPodConditionWithStatus(p2, v1.PodReadyToStartContainers, true) framework.ExpectNoError(err) } </ins> } }


Question:
Two serial tests from sig-node are keep failing: E2eNode Suite. E2eNode Suite. pls see: [1] [2] <fmt.wrapError | 0x40005e7840: failed to update pod "pod--65da-42b7-a6a5-": Pod "pod--65da-42b7-a6a5-" is invalid: spec: Forbidden: pod updates may not change fields other than ,,, (only additions to existing tolerations), (allow it to be set to 1 if it was previously negative) core.PodSpec{ Volumes: []core.Volume{ { Name: "cm", Name: "secret", VolumeSource: core.VolumeSource{ ... // 3 identical fields AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, Secret: &core.SecretVolumeSource{SecretName: "secret", DefaultMode: &420}, NFS: nil, ISCSI: nil, ... // 9 identical fields FC: nil, AzureFile: nil, ConfigMap: &core.ConfigMapVolumeSource{ LocalObjectReference: core.LocalObjectReference{Name: "does-not-exist"}, DefaultMode: &420, }, ConfigMap: nil, VsphereVolume: nil, AzureDisk: nil, ... // 7 identical fields }, }, }, InitContainers: {{Name: "init1", Image: "", Command: {"sh", "-c", "sleep 5s"}, TerminationMessagePath: "/dev/termination-log", ...}}, Containers: []core.Container{ { ... // 9 identical fields ResizePolicy: nil, RestartPolicy: nil, VolumeMounts: []core.VolumeMount{ { Name: "cm", Name: "secret", ReadOnly: false, MountPath: "/config", ... // 3 identical fields }, }, VolumeDevices: nil, LivenessProbe: nil, ... // 10 identical fields }, }, EphemeralContainers: nil, RestartPolicy: "Always", ... // 28 identical fields } happened since 18/10/23 run the e2e testcases manually No response <details</details<detailsImpacts all </details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /cc
cc
failed on x86 as well:
/sig node
/unassign
is merged today. /cc
will take a look at this ASAP /assign
and I are sorry for the issue this caused. We decided to open to fix this issue. I think this test was more complicated than we thought. Sorry for this.

// Verify PodReady is not set (since sandboxcreation is blocked) _, err = getTransitionTimeForPodConditionWithStatus(p, v1.PodReady, false) framework.ExpectNoError(err) <del> ginkgo.By("update pod related volume resource to unblock sandbox creation") e2epod.NewPodClient(f).Update(ctx, p.Name, func(pod *v1.Pod) { pod.Spec.Volumes = []v1.Volume{ { Name: "secret", VolumeSource: v1.VolumeSource{ Secret: &v1.SecretVolumeSource{ SecretName: "secret", }, }, }, } pod.Spec.Containers[0].VolumeMounts = []v1.VolumeMount{ { Name:      "secret", MountPath: "/config", }, } }) // Verify PodReadyToStartContainers is set (since sandboxcreation is unblocked) if checkPodReadyToStart { _, err = getTransitionTimeForPodConditionWithStatus(p, v1.PodReadyToStartContainers, true) framework.ExpectNoError(err) } </del> } }


Question:
CI always has error when using to replace in function in file . No error occurred Use to replace in function in file . No response <details</details<details</details<details</details<details</details<details</details<details</details
/sig api-machinery /area test
/triage accepted /assign /assign

} func (f *fakeService) WaitForReady(ctx context.Context) error { <del> err := wait.PollWithContext(ctx, 1*time.Second, 200*time.Millisecond, func(ctx context.Context) (done bool, err error) { </del> <ins> err := wait.PollUntilContextTimeout(ctx, 200*time.Millisecond, time.Second, false, func(ctx context.Context) (done bool, err error) { </ins> return f.Port() != nil, nil })


Question:
The controller-manager node-lifecycle-controller watches node lease and update probeTimestamp , and it check node probeTimestamp is before . The node-monitor-grace-period default value is 40s. When the client connection lost, the lease watcher cannot receive new event. With the http2 health check, HTTP2PINGTIMEOUTSECONDS default is 30s, HTTP2READIDLETIMEOUTSECONDS default is 15s, it will delay 45s to kill http2 connection. During this period, node-lifecycle-controller will mark all nodes NotReady. Here are some logs When the http2 connection lost, the watch should stop as soon as possible. The reflector should relist resource, and node-lifecycle-controller should not mark all node NotReady. HTTP2PINGTIMEOUTSECONDS+HTTP2READIDLETIMEOUTSECONDS should be less than 40s. the controller-manager node-lifecycle-controller watch client srcIP:srcPort. Drop all tcp package send into srcIP:srcPort. for more than 40 seconds, all node will be marked NotReady node will be Ready after some seconds. No response <details</details<details</details<details</details<details</details<details</details<details</details
How about double-check node lease before marking node NotReady? The node-lifecycle-controller can get reference lease from apiserver with
/sig node
/sig api-machinery
/cc We have discussed this timeouts values before and myself, I also think got some metrics with lower timeouts for the HTTP2 connections. In this situation is clear that the defaults of the node watcher and the timeouts are related and having good defaults will be ideal, but the question is which values do we change ?
I think in internal network, the ping response almost return in 1 second, the HTTP2PINGTIMEOUT_SECONDS default value should be 5 seconds. And the total timeout will 35 seconds. WDYT?
/triage accepted any pointers in other places where those timeouts were changed for http2? Any notes from those discussions?
/priority important-longterm since it is not a regression I will keep it longterm priority and not higher. But we can reevaluate if we hear more reports about it.
/assign
I'm investigating problems similar to this, but I'm looking for a way to confirm this is what is happening - and if there are tips for how to recreate such a scenario, that would be very helpful. We get infrequent reports of "worker goes NotReady for a short time" incidents that appear to be triggered by events like failing over a LB sitting front of the kube-apiservers. We've done some tests with a simple client-go app and have recreated a scenario in which the app logs messages after 45 secs of "context exceeded" errors. The failures reported to us from customers don't have http2 errors in the kubelet logs, and our attempts to recreate with kubelet have never shown more more than a single error which works on subsequent retry.
Hi folks, we have a simulated recreate of the problem. This recreate involves injecting a 40 second delay in "connectivity" between the Kubernetes API server and etcd (we a 40 second delay to etcd defrag and defragged all etcd members at the same time). The connectivity between these two components is obviously critical to nodes successfully updating their leases in order to stay ready. We use 40 seconds because it matches . The test will usually cause one or more of the cluster nodes will flap their readiness state. Setting on the Kubernetes controller manager prevents the flap. Given this, I think that one or more of the Kubernetes default configurations needs to change in order to tolerate such "connectivity" problems in order to maintain healthy node state.
What's the proposal then? Do we update --node-monitor-grace-period to 50s? It seems the less risky change as the client-go defaults are more sensitive and impact the whole ecosystem, beyond kubernetes itself are you open to increase this default? It seems worth it to improve the reliability of cluster
I think we should update .
can someone send a PR to see how it will look like?
I have been following this discussion as we had encountered the bug as well. I have created a PR - Please let me know if it works
I'll add another data point from our testing. Namely, the simulated recreate noted depends on the Kubernetes controller manager (KCM) leader election configuration. Using the following leader election configuration for the KCM yields a KCM that has no leader after the delayed etcd defrag. As a result, nodes won't be marked not ready unless the KCM leadership is restored before the nodes renew their lease. Extending the KCM leader election configuration to the following, makes the problem easier to recreate.
/assign Reassigning the issue to me

fs.DurationVar(&o.NodeMonitorGracePeriod.Duration, "node-monitor-grace-period", o.NodeMonitorGracePeriod.Duration, "Amount of time which we allow running Node to be unresponsive before marking it unhealthy. "+ "Must be N times more than kubelet's nodeStatusUpdateFrequency, "+ <del> "where N means number of retries allowed for kubelet to post node status.") </del> <ins> "where N means number of retries allowed for kubelet to post node status. "+ "This value should also be greater than the sum of HTTP2_PING_TIMEOUT_SECONDS and HTTP2_READ_IDLE_TIMEOUT_SECONDS") </ins> fs.Float32Var(&o.NodeEvictionRate, "node-eviction-rate", 0.1, "Number of nodes per second on which pods are deleted in case of node failure when a zone is healthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters.") fs.Float32Var(&o.SecondaryNodeEvictionRate, "secondary-node-eviction-rate", 0.01, "Number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. This value is implicitly overridden to 0 if the cluster size is smaller than --large-cluster-size-threshold.") fs.Int32Var(&o.LargeClusterSizeThreshold, "large-cluster-size-threshold", 50, fmt.Sprintf("Number of nodes from which %s treats the cluster as large for the eviction logic purposes. --secondary-node-eviction-rate is implicitly overridden to 0 for clusters this size or smaller. Notice: If nodes reside in multiple zones, this threshold will be considered as zone node size threshold for each zone to determine node eviction rate independently.", names.NodeLifecycleController))


Question:
The controller-manager node-lifecycle-controller watches node lease and update probeTimestamp , and it check node probeTimestamp is before . The node-monitor-grace-period default value is 40s. When the client connection lost, the lease watcher cannot receive new event. With the http2 health check, HTTP2PINGTIMEOUTSECONDS default is 30s, HTTP2READIDLETIMEOUTSECONDS default is 15s, it will delay 45s to kill http2 connection. During this period, node-lifecycle-controller will mark all nodes NotReady. Here are some logs When the http2 connection lost, the watch should stop as soon as possible. The reflector should relist resource, and node-lifecycle-controller should not mark all node NotReady. HTTP2PINGTIMEOUTSECONDS+HTTP2READIDLETIMEOUTSECONDS should be less than 40s. the controller-manager node-lifecycle-controller watch client srcIP:srcPort. Drop all tcp package send into srcIP:srcPort. for more than 40 seconds, all node will be marked NotReady node will be Ready after some seconds. No response <details</details<details</details<details</details<details</details<details</details<details</details
How about double-check node lease before marking node NotReady? The node-lifecycle-controller can get reference lease from apiserver with
/sig node
/sig api-machinery
/cc We have discussed this timeouts values before and myself, I also think got some metrics with lower timeouts for the HTTP2 connections. In this situation is clear that the defaults of the node watcher and the timeouts are related and having good defaults will be ideal, but the question is which values do we change ?
I think in internal network, the ping response almost return in 1 second, the HTTP2PINGTIMEOUT_SECONDS default value should be 5 seconds. And the total timeout will 35 seconds. WDYT?
/triage accepted any pointers in other places where those timeouts were changed for http2? Any notes from those discussions?
/priority important-longterm since it is not a regression I will keep it longterm priority and not higher. But we can reevaluate if we hear more reports about it.
/assign
I'm investigating problems similar to this, but I'm looking for a way to confirm this is what is happening - and if there are tips for how to recreate such a scenario, that would be very helpful. We get infrequent reports of "worker goes NotReady for a short time" incidents that appear to be triggered by events like failing over a LB sitting front of the kube-apiservers. We've done some tests with a simple client-go app and have recreated a scenario in which the app logs messages after 45 secs of "context exceeded" errors. The failures reported to us from customers don't have http2 errors in the kubelet logs, and our attempts to recreate with kubelet have never shown more more than a single error which works on subsequent retry.
Hi folks, we have a simulated recreate of the problem. This recreate involves injecting a 40 second delay in "connectivity" between the Kubernetes API server and etcd (we a 40 second delay to etcd defrag and defragged all etcd members at the same time). The connectivity between these two components is obviously critical to nodes successfully updating their leases in order to stay ready. We use 40 seconds because it matches . The test will usually cause one or more of the cluster nodes will flap their readiness state. Setting on the Kubernetes controller manager prevents the flap. Given this, I think that one or more of the Kubernetes default configurations needs to change in order to tolerate such "connectivity" problems in order to maintain healthy node state.
What's the proposal then? Do we update --node-monitor-grace-period to 50s? It seems the less risky change as the client-go defaults are more sensitive and impact the whole ecosystem, beyond kubernetes itself are you open to increase this default? It seems worth it to improve the reliability of cluster
I think we should update .
can someone send a PR to see how it will look like?
I have been following this discussion as we had encountered the bug as well. I have created a PR - Please let me know if it works
I'll add another data point from our testing. Namely, the simulated recreate noted depends on the Kubernetes controller manager (KCM) leader election configuration. Using the following leader election configuration for the KCM yields a KCM that has no leader after the delayed etcd defrag. As a result, nodes won't be marked not ready unless the KCM leadership is restored before the nodes renew their lease. Extending the KCM leader election configuration to the following, makes the problem easier to recreate.
/assign Reassigning the issue to me

// NodeMonitorGracePeriod is the amount of time which we allow a running node to be // unresponsive before marking it unhealthy. Must be N times more than kubelet's // nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet <del> // to post node status. </del> <ins> // to post node status. This value should also be greater than the sum of // HTTP2_PING_TIMEOUT_SECONDS and HTTP2_READ_IDLE_TIMEOUT_SECONDS. </ins> NodeMonitorGracePeriod metav1.Duration // secondaryNodeEvictionRate is implicitly overridden to 0 for clusters smaller than or equal to largeClusterSizeThreshold LargeClusterSizeThreshold int32