Question:
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
Abhishek ought not be responsible for this any more.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention...
[FLAKE-PING] This flaky-test issue would love to have more attention...
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
what is the status of this issue?
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
No idea. This should have nothing to do with me.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
Ah kk just thought you might know something given the label.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
I know it didn't get any attention last release :)
Status is that I'm trying to fix it... I some code to the test to help debug why the flake was occurring as I wasn't able to repro it on my local cluster.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite} Happened on a presubmit run in .
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
This seems to be related to issue

func assertManagedStatus( config *KubeletManagedHostConfig, podName string, expectedIsManaged bool, name string) { <del> // See https://github.com/kubernetes/kubernetes/issues/27023 </del> <ins> // TODO: workaround for https://github.com/kubernetes/kubernetes/issues/34256 </ins> // <del> // Retry until timeout for the right contents of /etc/hosts to show // up. There may be a low probability race here. We still fail the // test if retry was necessary, but at least we will know whether or // not it resolves or seems to be a permanent condition. // // If /etc/hosts is properly mounted, then this will succeed // immediately. </del> <ins> // Retry until timeout for the contents of /etc/hosts to show // up. Note: if /etc/hosts is properly mounted, then this will // succeed immediately. </ins> const retryTimeout = 30 * time.Second retryCount := 0 etcHostsContent := "" <del> matched := false </del> for startTime := time.Now(); time.Since(startTime) < retryTimeout; { etcHostsContent = config.getEtcHostsContent(podName, name) isManaged := strings.Contains(etcHostsContent, etcHostsPartialContent) if expectedIsManaged == isManaged { <del> matched = true break </del> <ins> return </ins> } <del> glog.Errorf( </del> <ins> glog.Warningf( </ins> "For pod: %s, name: %s, expected %t, actual %t (/etc/hosts was %q), retryCount: %d", podName, name, expectedIsManaged, isManaged, etcHostsContent, retryCount)


Question:
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
Abhishek ought not be responsible for this any more.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention...
[FLAKE-PING] This flaky-test issue would love to have more attention...
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
what is the status of this issue?
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
No idea. This should have nothing to do with me.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
Ah kk just thought you might know something given the label.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
I know it didn't get any attention last release :)
Status is that I'm trying to fix it... I some code to the test to help debug why the flake was occurring as I wasn't able to repro it on my local cluster.
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite} Happened on a presubmit run in .
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
https://k8s- Failed: [] KubeletManagedEtcHosts should test kubelet managed /etc/hosts file {Kubernetes e2e suite}
This seems to be related to issue

time.Sleep(100 * time.Millisecond) } <del> if retryCount > 0 { if matched { conditionText := "should" if !expectedIsManaged { conditionText = "should not" } framework.Failf( "/etc/hosts file %s be kubelet managed (name: %s, retries: %d). /etc/hosts contains %q", conditionText, name, retryCount, etcHostsContent) } else { framework.Failf( "had to retry %d times to get matching content in /etc/hosts (name: %s)", retryCount, name) } </del> <ins> if expectedIsManaged { framework.Failf( "/etc/hosts file should be kubelet managed (name: %s, retries: %d). /etc/hosts contains %q", name, retryCount, etcHostsContent) } else { framework.Failf( "/etc/hosts file should no be kubelet managed (name: %s, retries: %d). /etc/hosts contains %q", name, retryCount, etcHostsContent) </ins> } }


Question:
Hello Kubernetes, The order of the columns when you get events confused me. I was thinking, Turns out it's just a visual problem where these events are sorted correctly (by ), but the first column is . I think the first column should be because that's the order the events are listed.
This also confused me when I first saw it - I think this ordering would be more intuitive and help users discover the aggregation feature logically, instead of first having to grapple with why events are "out of order"
Should the order be always LASTSEEN,FIRSTSEEN or just when using -w? I have an ugly PR for the latter, but perhaps we should just go for the former.

var endpointColumns = []string{"NAME", "ENDPOINTS", "AGE"} var nodeColumns = []string{"NAME", "STATUS", "AGE"} var daemonSetColumns = []string{"NAME", "DESIRED", "CURRENT", "NODE-SELECTOR", "AGE"} <del> var eventColumns = []string{"FIRSTSEEN", "LASTSEEN", "COUNT", "NAME", "KIND", "SUBOBJECT", "TYPE", "REASON", "SOURCE", "MESSAGE"} </del> <ins> var eventColumns = []string{"LASTSEEN", "FIRSTSEEN", "COUNT", "NAME", "KIND", "SUBOBJECT", "TYPE", "REASON", "SOURCE", "MESSAGE"} </ins> var limitRangeColumns = []string{"NAME", "AGE"} var resourceQuotaColumns = []string{"NAME", "AGE"} var namespaceColumns = []string{"NAME", "STATUS", "AGE"}


Question:
Hello Kubernetes, The order of the columns when you get events confused me. I was thinking, Turns out it's just a visual problem where these events are sorted correctly (by ), but the first column is . I think the first column should be because that's the order the events are listed.
This also confused me when I first saw it - I think this ordering would be more intuitive and help users discover the aggregation feature logically, instead of first having to grapple with why events are "out of order"
Should the order be always LASTSEEN,FIRSTSEEN or just when using -w? I have an ugly PR for the latter, but perhaps we should just go for the former.

if _, err := fmt.Fprintf( w, "%st%st%dt%st%st%st%st%st%st%s", <del> FirstTimestamp, </del> LastTimestamp, <ins> FirstTimestamp, </ins> event.Count, event.InvolvedObject.Name, event.InvolvedObject.Kind,


Question:
is failing with: Examples:
This is the failing line of code...
PASS: FAIL: Seems like this disappeared: Also this is flipped: we run the following
my leading theory is this is something in
The issue is that the above PR changed a bunch of to be and thus we no longer wait for getArchive() to complete before starting tests. And unfortunately the previous PR cannot be reverted
After merging, kubelet-gce-e2e-ci still failed with the same issue. reopen the issue. Currently the failure blocked the merge-bot.
did stop the bleeding. Now kubelet-gce-e2e-ci is green: and merge-bot is unblocked.
I think the "go arc" piece was a red herring. I suspect the issue is that the same PR tries to copy the binary into the archive directory, but PR stops building ginkgo, so the copy fails. That is why this line then fails...

var k8sBinDir = flag.String("k8s-bin-dir", "", "Directory containing k8s kubelet and kube-apiserver binaries.") <del> var buildTargets = []string{ "cmd/kubelet", "cmd/kube-apiserver", "test/e2e_node/e2e_node.test", } </del> func buildGo() { glog.Infof("Building k8s binaries...") k8sRoot, err := getK8sRootDir() if err != nil { glog.Fatalf("Failed to locate kubernetes root directory %v.", err) } <del> cmd := exec.Command(filepath.Join(k8sRoot, "hack/build-go.sh"), buildTargets...) </del> <ins> cmd := exec.Command(filepath.Join(k8sRoot, "hack/build-go.sh")) </ins> cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr err = cmd.Run()


Question:
is failing with: Examples:
This is the failing line of code...
PASS: FAIL: Seems like this disappeared: Also this is flipped: we run the following
my leading theory is this is something in
The issue is that the above PR changed a bunch of to be and thus we no longer wait for getArchive() to complete before starting tests. And unfortunately the previous PR cannot be reverted
After merging, kubelet-gce-e2e-ci still failed with the same issue. reopen the issue. Currently the failure blocked the merge-bot.
did stop the bleeding. Now kubelet-gce-e2e-ci is green: and merge-bot is unblocked.
I think the "go arc" piece was a red herring. I suspect the issue is that the same PR tries to copy the binary into the archive directory, but PR stops building ginkgo, so the copy fails. That is why this line then fails...

return "", err } if _, err := os.Stat(filepath.Join(*k8sBinDir, bin)); err != nil { <del> return "", fmt.Errorf("Could not find %s under directory %s.", bin, absPath) </del> <ins> return "", fmt.Errorf("Could not find kube-apiserver under directory %s.", absPath) </ins> } return filepath.Join(absPath, bin), nil }


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

kubectl replace "${kube_flags[@]}" --force -f /tmp/tmp-valid-pod.json # Post-condition: spec.container.name = "replaced-k8s-serve-hostname" kube::test::get_object_assert 'pod valid-pod' "{{(index .spec.containers 0).name}}" 'replaced-k8s-serve-hostname' <ins> ## check replace --grace-period requires --force output_message=$(! kubectl replace "${kube_flags[@]}" --grace-period=1 -f /tmp/tmp-valid-pod.json 2>&1) kube::test::if_has_string "${output_message}" '--grace-period must have --force specified' ## check replace --timeout requires --force output_message=$(! kubectl replace "${kube_flags[@]}" --timeout=1s -f /tmp/tmp-valid-pod.json 2>&1) kube::test::if_has_string "${output_message}" '--timeout must have --force specified' </ins> #cleaning rm /tmp/tmp-valid-pod.json


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

"github.com/spf13/cobra" "github.com/golang/glog" <ins> "k8s.io/kubernetes/pkg/api/errors" </ins> "k8s.io/kubernetes/pkg/kubectl" cmdutil "k8s.io/kubernetes/pkg/kubectl/cmd/util" "k8s.io/kubernetes/pkg/kubectl/resource" "k8s.io/kubernetes/pkg/runtime" <ins> "k8s.io/kubernetes/pkg/util/wait" </ins> ) // ReplaceOptions is the start of the data required to perform the operation.  As new fields are added, add them here instead of


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

return forceReplace(f, out, cmd, args, shortOutput, options) } <ins> if cmdutil.GetFlagInt(cmd, "grace-period") >= 0 { return fmt.Errorf("--grace-period must have --force specified") } if cmdutil.GetFlagDuration(cmd, "timeout") != 0 { return fmt.Errorf("--timeout must have --force specified") } </ins> mapper, typer := f.Object(cmdutil.GetIncludeThirdPartyAPIs(cmd)) r := resource.NewBuilder(mapper, typer, resource.ClientMapperFunc(f.ClientForMapping), f.Decoder(true)). Schema(schema).


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

} //Replace will create a resource if it doesn't exist already, so ignore not found error ignoreNotFound := true <ins> timeout := cmdutil.GetFlagDuration(cmd, "timeout") </ins> // By default use a reaper to delete all related resources. if cmdutil.GetFlagBool(cmd, "cascade") { glog.Warningf(""cascade" is set, kubectl will delete and re-create all resources managed by this resource (e.g. Pods created by a ReplicationController). Consider using "kubectl rolling-update" if you want to update a ReplicationController together with its Pods.") <del> err = ReapResult(r, f, out, cmdutil.GetFlagBool(cmd, "cascade"), ignoreNotFound, cmdutil.GetFlagDuration(cmd, "timeout"), cmdutil.GetFlagInt(cmd, "grace-period"), shortOutput, mapper, false) </del> <ins> err = ReapResult(r, f, out, cmdutil.GetFlagBool(cmd, "cascade"), ignoreNotFound, timeout, cmdutil.GetFlagInt(cmd, "grace-period"), shortOutput, mapper, false) </ins> } else { err = DeleteResult(r, out, ignoreNotFound, shortOutput, mapper) }


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

return err } <ins> if timeout == 0 { timeout = kubectl.Timeout } r.Visit(func(info *resource.Info, err error) error { if err != nil { return err } return wait.PollImmediate(kubectl.Interval, timeout, func() (bool, error) { if err := info.Get(); !errors.IsNotFound(err) { return false, err } return true, nil }) }) </ins> r = resource.NewBuilder(mapper, typer, resource.ClientMapperFunc(f.UnstructuredClientForMapping), runtime.UnstructuredJSONScheme). Schema(schema). ContinueOnError().


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

f, tf, codec, _ := NewAPIFactory() ns := dynamic.ContentConfig().NegotiatedSerializer tf.Printer = &testPrinter{} <ins> deleted := false </ins> tf.Client = &fake.RESTClient{ NegotiatedSerializer: ns, Client: fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) { switch p, m := req.URL.Path, req.Method; { <del> case p == "/namespaces/test/replicationcontrollers/redis-master" && (m == http.MethodGet || m == http.MethodPut || m == http.MethodDelete): </del> <ins> case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodDelete: deleted = true fallthrough case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodPut: </ins> return &http.Response{StatusCode: http.StatusOK, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil <ins> case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodGet: statusCode := http.StatusOK if deleted { statusCode = http.StatusNotFound } return &http.Response{StatusCode: statusCode, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil </ins> case p == "/namespaces/test/replicationcontrollers" && m == http.MethodPost: return &http.Response{StatusCode: http.StatusCreated, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil default:


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

f, tf, codec, _ := NewAPIFactory() ns := dynamic.ContentConfig().NegotiatedSerializer tf.Printer = &testPrinter{} <ins> redisMasterDeleted := false frontendDeleted := false </ins> tf.Client = &fake.RESTClient{ NegotiatedSerializer: ns, Client: fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) { switch p, m := req.URL.Path, req.Method; { <del> case p == "/namespaces/test/replicationcontrollers/redis-master" && (m == http.MethodGet || m == http.MethodPut || m == http.MethodDelete): </del> <ins> case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodDelete: redisMasterDeleted = true fallthrough case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodPut: </ins> return &http.Response{StatusCode: http.StatusOK, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil <ins> case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodGet: statusCode := http.StatusOK if redisMasterDeleted { statusCode = http.StatusNotFound } return &http.Response{StatusCode: statusCode, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil </ins> case p == "/namespaces/test/replicationcontrollers" && m == http.MethodPost: return &http.Response{StatusCode: http.StatusCreated, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil <del> case p == "/namespaces/test/services/frontend" && (m == http.MethodGet || m == http.MethodPut || m == http.MethodDelete): </del> <ins> case p == "/namespaces/test/services/frontend" && m == http.MethodDelete: frontendDeleted = true fallthrough case p == "/namespaces/test/services/frontend" && m == http.MethodPut: </ins> return &http.Response{StatusCode: http.StatusOK, Header: defaultHeader(), Body: objBody(codec, &svc.Items[0])}, nil <ins> case p == "/namespaces/test/services/frontend" && m == http.MethodGet: statusCode := http.StatusOK if frontendDeleted { statusCode = http.StatusNotFound } return &http.Response{StatusCode: statusCode, Header: defaultHeader(), Body: objBody(codec, &svc.Items[0])}, nil </ins> case p == "/namespaces/test/services" && m == http.MethodPost: return &http.Response{StatusCode: http.StatusCreated, Header: defaultHeader(), Body: objBody(codec, &svc.Items[0])}, nil default:


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

f, tf, codec, _ := NewAPIFactory() ns := dynamic.ContentConfig().NegotiatedSerializer tf.Printer = &testPrinter{} <ins> created := map[string]bool{} </ins> tf.Client = &fake.RESTClient{ NegotiatedSerializer: ns, Client: fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) { switch p, m := req.URL.Path, req.Method; { <del> case strings.HasPrefix(p, "/namespaces/test/replicationcontrollers/") && (m == http.MethodGet || m == http.MethodPut || m == http.MethodDelete): </del> <ins> case strings.HasPrefix(p, "/namespaces/test/replicationcontrollers/") && m == http.MethodPut: created[p] = true return &http.Response{StatusCode: http.StatusOK, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil case strings.HasPrefix(p, "/namespaces/test/replicationcontrollers/") && m == http.MethodGet: statusCode := http.StatusNotFound if created[p] { statusCode = http.StatusOK } return &http.Response{StatusCode: statusCode, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil case strings.HasPrefix(p, "/namespaces/test/replicationcontrollers/") && m == http.MethodDelete: delete(created, p) </ins> return &http.Response{StatusCode: http.StatusOK, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil case strings.HasPrefix(p, "/namespaces/test/replicationcontrollers") && m == http.MethodPost: return &http.Response{StatusCode: http.StatusCreated, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil


Question:
When replacing an existing pod, the command fails as it tries to create a pod with a name that is still used by terminating pod. How does the option works? deletes a pod (e.g. mypod) creates a new pod with the same name (mypod) What happens? As the command tries to recreate the pod, it fails as the is in state. Thus it can not be created. The problem is the does not wait for a pod to be deleted. Reproducer: run mypod --image=yapei/hello-openshift --generator=run-pod/v1 get pod mypod -o yaml replace -f --force The last command ends with:
option introduce by
With set to positive value, what should be the default behaviour of the command? When set to , should the command hang for and then create the pod? Or is there a way to create a callback (inside a scheduler?) that creates the pod once the current one is deleted?
You could watch for the resource to be deleted and create after you see the deletion happen
have you ever tried to set , I think this will do a non graceful deletion. should make sure the old resource is deleted before it creates a new resource, if not, that's a bug.
Though the help says the is to be used only with , it returns the error above.
Then I think there are two issues we need to track: 1. option should be ignored or just returns error if users are not using option. 2. should always make sure the old resource has been deleted before create the new resource.

NegotiatedSerializer: ns, Client: fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) { switch p, m := req.URL.Path, req.Method; { <del> case p == "/namespaces/test/replicationcontrollers/redis-master" && m == http.MethodDelete: </del> <ins> case p == "/namespaces/test/replicationcontrollers/redis-master" && (m == http.MethodGet || m == http.MethodDelete): </ins> return &http.Response{StatusCode: http.StatusNotFound, Header: defaultHeader(), Body: stringBody("")}, nil case p == "/namespaces/test/replicationcontrollers" && m == http.MethodPost: return &http.Response{StatusCode: http.StatusCreated, Header: defaultHeader(), Body: objBody(codec, &rc.Items[0])}, nil


Question:
As mentioned in , the federation etcd will lose its state any time the pod dies, unless we modify to add a PVC. We'll also need to create a PV and modify the turnup script to create the backing volume in the cloud provider.
FYI I believe this is a showstopper.
OK. implied I could use to get this provisioned on-demand. AFAICT, , and since it makes our life easier I am trying to get this to work in my test cluster, but so far no joy. Any obvious mistake?
I needed to not specify a .
whew! , Matt Liggett wrote:

$host_kubectl create secret generic ${name} --from-file="${dir}/kubeconfig" --namespace="${FEDERATION_NAMESPACE}" done <del> $template "${manifests_root}/federation-apiserver-"{deployment,secrets}".yaml" | $host_kubectl create -f - $template "${manifests_root}/federation-controller-manager-deployment.yaml" | $host_kubectl create -f - </del> <ins> for file in federation-etcd-pvc.yaml federation-apiserver-{deployment,secrets}.yaml federation-controller-manager-deployment.yaml; do $template "${manifests_root}/${file}" | $host_kubectl create -f - done </ins> # Update the users kubeconfig to include federation-apiserver credentials. CONTEXT=federation-cluster


Question:
As mentioned in , the federation etcd will lose its state any time the pod dies, unless we modify to add a PVC. We'll also need to create a PV and modify the turnup script to create the backing volume in the cloud provider.
FYI I believe this is a showstopper.
OK. implied I could use to get this provisioned on-demand. AFAICT, , and since it makes our life easier I am trying to get this to work in my test cluster, but so far no joy. Any obvious mistake?
I needed to not specify a .
whew! , Matt Liggett wrote:

readOnly: true - name: etcd image: quay.io/coreos/etcd:v2.3.3 <ins> command: - /etcd - --data-dir - /var/etcd/data volumeMounts: - mountPath: /var/etcd name: varetcd </ins> volumes: - name: federation-apiserver-secrets secret: secretName: federation-apiserver-secrets <ins> - name: varetcd persistentVolumeClaim: claimName: {{.FEDERATION_APISERVER_DEPLOYMENT_NAME}}-etcd-claim </ins>


Question:
As mentioned in , the federation etcd will lose its state any time the pod dies, unless we modify to add a PVC. We'll also need to create a PV and modify the turnup script to create the backing volume in the cloud provider.
FYI I believe this is a showstopper.
OK. implied I could use to get this provisioned on-demand. AFAICT, , and since it makes our life easier I am trying to get this to work in my test cluster, but so far no joy. Any obvious mistake?
I needed to not specify a .
whew! , Matt Liggett wrote:

<ins> apiVersion: v1 kind: PersistentVolumeClaim metadata: name: {{.FEDERATION_APISERVER_DEPLOYMENT_NAME}}-etcd-claim annotations: volume.alpha.kubernetes.io/storage-class: "yes" namespace: {{.FEDERATION_NAMESPACE}} labels: app: federated-cluster spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi </ins>


Question:
https://k8s- Failed: [] Container Runtime Conformance Test container runtime conformance blackbox test when running a container with a new image should not be able to pull from private registry without secret {E2eNode Suite}
Same with , the fix is under review
https://k8s- Failed: [] Container Runtime Conformance Test container runtime conformance blackbox test when running a container with a new image should not be able to pull from private registry without secret {E2eNode Suite}
https://k8s- Failed: [] Container Runtime Conformance Test container runtime conformance blackbox test when running a container with a new image should not be able to pull from private registry without secret {E2eNode Suite}
The fix has been merged , close this one. Feel free to reopen if this happens again.

Expect(container.Create()).To(Succeed()) defer container.Delete() <del> By("check the pod phase") Eventually(container.GetPhase, retryTimeout, pollInterval).Should(Equal(testCase.phase)) Consistently(container.GetPhase, consistentCheckTimeout, pollInterval).Should(Equal(testCase.phase)) </del> <ins> // We need to check container state first. The default pod status is pending, If we check // pod phase first, and the expected pod phase is Pending, the container status may not // even show up when we check it. </ins> By("check the container state") <del> status, err := container.GetStatus() Expect(err).NotTo(HaveOccurred()) Expect(GetContainerState(status.State)).To(Equal(testCase.state)) </del> <ins> getState := func() (ContainerState, error) { status, err := container.GetStatus() if err != nil { return ContainerStateUnknown, err } return GetContainerState(status.State), nil } Eventually(getState, retryTimeout, pollInterval).Should(Equal(testCase.state)) Consistently(getState, consistentCheckTimeout, pollInterval).Should(Equal(testCase.state)) By("check the pod phase") Expect(container.GetPhase()).To(Equal(testCase.phase)) </ins> By("it should be possible to delete") Expect(container.Delete()).To(Succeed())


Question:
Either include non-arch specific stuff in every tar or break that out too. ref cc
interested in helping with this one?
Yes, I may help
OK, I did not actually understand what was in , apparently. Alternate proposal: [ ] update all references that need just server binaries to download the server tarball directly [ ] same for places that just need (related to ) [ ] remove all arch-specific binary artifacts from and replace with a script which downloads the client/server tarballs as necessary. The alternative option, including all of the non-arch-specific code and docs in an arch-specific starts to get very tricky, since we run into a large cross product of client x server platforms/archs.

kube::release::package_kube_manifests_tarball & kube::util::wait-for-jobs || { kube::log::error "previous tarball phase failed"; return 1; } <del> kube::release::package_full_tarball & # _full depends on all the previous phases </del> <ins> kube::release::package_final_tarball & # _final depends on some of the previous phases </ins> kube::release::package_test_tarball & # _test doesn't depend on anything kube::util::wait-for-jobs || { kube::log::error "previous tarball phase failed"; return 1; } }


Question:
Either include non-arch specific stuff in every tar or break that out too. ref cc
interested in helping with this one?
Yes, I may help
OK, I did not actually understand what was in , apparently. Alternate proposal: [ ] update all references that need just server binaries to download the server tarball directly [ ] same for places that just need (related to ) [ ] remove all arch-specific binary artifacts from and replace with a script which downloads the client/server tarballs as necessary. The alternative option, including all of the non-arch-specific code and docs in an arch-specific starts to get very tricky, since we run into a large cross product of client x server platforms/archs.

kube::release::create_tarball "${package_name}" "${release_stage}/.." } <del> # This is all the stuff you need to run/install kubernetes.  This includes: #   - precompiled binaries for client </del> <ins> # This is all the platform-independent stuff you need to run/install kubernetes. # Arch-specific binaries will need to be downloaded separately (possibly by # using the bundled cluster/get-kube-binaries.sh script). # Included in this tarball: </ins> #   - Cluster spin up/down scripts and configs for various cloud providers <del> #   - tarballs for server binary and salt configs that are ready to be uploaded </del> <ins> #   - Tarballs for salt configs that are ready to be uploaded </ins> #     to master by whatever means appropriate. <del> function kube::release::package_full_tarball() { kube::log::status "Building tarball: full" </del> <ins> #   - Examples (which may or may not still work) #   - The remnants of the docs/ directory function kube::release::package_final_tarball() { kube::log::status "Building tarball: final" # This isn't a "full" tarball anymore, but the release lib still expects # artifacts under "full/kubernetes/" </ins> local release_stage="${RELEASE_STAGE}/full/kubernetes" rm -rf "${release_stage}" mkdir -p "${release_stage}" <del> # Copy all of the client binaries in here, but not test or server binaries. # The server binaries are included with the server binary tarball. local platform for platform in "${KUBE_CLIENT_PLATFORMS[@]}"; do local client_bins=("${KUBE_CLIENT_BINARIES[@]}") if [[ "${platform%/*}" == "windows" ]]; then client_bins=("${KUBE_CLIENT_BINARIES_WIN[@]}") fi mkdir -p "${release_stage}/platforms/${platform}" cp "${client_bins[@]/#/${LOCAL_OUTPUT_BINPATH}/${platform}/}"  "${release_stage}/platforms/${platform}" done </del> <ins> mkdir -p "${release_stage}/client" cat <<EOF > "${release_stage}/client/README" Client binaries are no longer included in the Kubernetes final tarball. Run cluster/get-kube-binaries.sh to download client and server binaries. EOF </ins> # We want everything in /cluster except saltbase.  That is only needed on the # server.


Question:
Either include non-arch specific stuff in every tar or break that out too. ref cc
interested in helping with this one?
Yes, I may help
OK, I did not actually understand what was in , apparently. Alternate proposal: [ ] update all references that need just server binaries to download the server tarball directly [ ] same for places that just need (related to ) [ ] remove all arch-specific binary artifacts from and replace with a script which downloads the client/server tarballs as necessary. The alternative option, including all of the non-arch-specific code and docs in an arch-specific starts to get very tricky, since we run into a large cross product of client x server platforms/archs.

mkdir -p "${release_stage}/server" cp "${RELEASE_DIR}/kubernetes-salt.tar.gz" "${release_stage}/server/" <del> cp "${RELEASE_DIR}"/kubernetes-server-*.tar.gz "${release_stage}/server/" </del> cp "${RELEASE_DIR}/kubernetes-manifests.tar.gz" "${release_stage}/server/" <ins> cat <<EOF > "${release_stage}/server/README" Server binary tarballs are no longer included in the Kubernetes final tarball. Run cluster/get-kube-binaries.sh to download client and server binaries. EOF </ins> mkdir -p "${release_stage}/third_party" cp -R "${KUBE_ROOT}/third_party/htpasswd" "${release_stage}/third_party/htpasswd"


Question:
The code in doesn't work for me: does not pass on SIGTERM when the caller is in the same process group. Then the knocks out the process without giving it a chance to kill its child. The bit doesn't help: either the process has already been killed as above, or it gets killed now with SIGKILL; either way the process is still running. Oddly, it does work about one time in ten. I have no explanation for that. I am running from the command-line like this: , on 0-41-generic -Ubuntu SMP Fri Jun 24 11:28:43 UTC 2016 x8664 x8664 x86_64 GNU/Linux CC
The manual says "This is the signal that the calling process will get when its parent dies."; it is applied to the process, so that will get killed if the process dies. That isn't what we are looking for:

"os/exec" "path" "path/filepath" <del> "reflect" </del> "strconv" "strings" "syscall"


Question:
The code in doesn't work for me: does not pass on SIGTERM when the caller is in the same process group. Then the knocks out the process without giving it a chance to kill its child. The bit doesn't help: either the process has already been killed as above, or it gets killed now with SIGKILL; either way the process is still running. Oddly, it does work about one time in ten. I have no explanation for that. I am running from the command-line like this: , on 0-41-generic -Ubuntu SMP Fri Jun 24 11:28:43 UTC 2016 x8664 x8664 x86_64 GNU/Linux CC
The manual says "This is the signal that the calling process will get when its parent dies."; it is applied to the process, so that will get killed if the process dies. That isn't what we are looking for:

cmd.Cmd.Stdout = outfile cmd.Cmd.Stderr = outfile <del> // Killing the sudo command should kill the server as well. attrs := &syscall.SysProcAttr{} // Hack to set linux-only field without build tags. deathSigField := reflect.ValueOf(attrs).Elem().FieldByName("Pdeathsig") if deathSigField.IsValid() { deathSigField.Set(reflect.ValueOf(syscall.SIGKILL)) } else { cmdErrorChan <- fmt.Errorf("Failed to set Pdeathsig field (non-linux build)") return } cmd.Cmd.SysProcAttr = attrs </del> // Run the command err = cmd.Run() if err != nil {


Question:
The code in doesn't work for me: does not pass on SIGTERM when the caller is in the same process group. Then the knocks out the process without giving it a chance to kill its child. The bit doesn't help: either the process has already been killed as above, or it gets killed now with SIGKILL; either way the process is still running. Oddly, it does work about one time in ten. I have no explanation for that. I am running from the command-line like this: , on 0-41-generic -Ubuntu SMP Fri Jun 24 11:28:43 UTC 2016 x8664 x8664 x86_64 GNU/Linux CC
The manual says "This is the signal that the calling process will get when its parent dies."; it is applied to the process, so that will get killed if the process dies. That isn't what we are looking for:

const timeout = 10 * time.Second for _, signal := range []string{"-TERM", "-KILL"} { glog.V(2).Infof("Killing process %d (%s) with %s", pid, name, signal) <del> _, err := exec.Command("sudo", "kill", signal, strconv.Itoa(pid)).Output() </del> <ins> cmd := exec.Command("sudo", "kill", signal, strconv.Itoa(pid)) // Run the 'kill' command in a separate process group so sudo doesn't ignore it cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} _, err := cmd.Output() </ins> if err != nil { glog.Errorf("Error signaling process %d (%s) with %s: %v", pid, name, signal, err) continue


Question:
The code in doesn't work for me: does not pass on SIGTERM when the caller is in the same process group. Then the knocks out the process without giving it a chance to kill its child. The bit doesn't help: either the process has already been killed as above, or it gets killed now with SIGKILL; either way the process is still running. Oddly, it does work about one time in ten. I have no explanation for that. I am running from the command-line like this: , on 0-41-generic -Ubuntu SMP Fri Jun 24 11:28:43 UTC 2016 x8664 x8664 x86_64 GNU/Linux CC
The manual says "This is the signal that the calling process will get when its parent dies."; it is applied to the process, so that will get killed if the process dies. That isn't what we are looking for:

cmd.Cmd.Stdout = outfile cmd.Cmd.Stderr = outfile <del> // Killing the sudo command should kill the server as well. </del> <ins> // Death of this test process should kill the server as well. </ins> attrs := &syscall.SysProcAttr{} // Hack to set linux-only field without build tags. deathSigField := reflect.ValueOf(attrs).Elem().FieldByName("Pdeathsig") if deathSigField.IsValid() { <del> deathSigField.Set(reflect.ValueOf(syscall.SIGKILL)) </del> <ins> deathSigField.Set(reflect.ValueOf(syscall.SIGTERM)) </ins> } else { cmdErrorChan <- fmt.Errorf("Failed to set Pdeathsig field (non-linux build)") return


Question:
The code in doesn't work for me: does not pass on SIGTERM when the caller is in the same process group. Then the knocks out the process without giving it a chance to kill its child. The bit doesn't help: either the process has already been killed as above, or it gets killed now with SIGKILL; either way the process is still running. Oddly, it does work about one time in ten. I have no explanation for that. I am running from the command-line like this: , on 0-41-generic -Ubuntu SMP Fri Jun 24 11:28:43 UTC 2016 x8664 x8664 x86_64 GNU/Linux CC
The manual says "This is the signal that the calling process will get when its parent dies."; it is applied to the process, so that will get killed if the process dies. That isn't what we are looking for:

const timeout = 10 * time.Second for _, signal := range []string{"-TERM", "-KILL"} { glog.V(2).Infof("Killing process %d (%s) with %s", pid, name, signal) <del> _, err := exec.Command("sudo", "kill", signal, strconv.Itoa(pid)).Output() </del> <ins> cmd := exec.Command("sudo", "kill", signal, strconv.Itoa(pid)) // Run the 'kill' command in a separate process group so sudo doesn't ignore it attrs := &syscall.SysProcAttr{} // Hack to set unix-only field without build tags. setpgidField := reflect.ValueOf(attrs).Elem().FieldByName("Setpgid") if setpgidField.IsValid() { setpgidField.Set(reflect.ValueOf(true)) } else { return fmt.Errorf("Failed to set Setpgid field (non-unix build)") } cmd.SysProcAttr = attrs _, err := cmd.Output() </ins> if err != nil { glog.Errorf("Error signaling process %d (%s) with %s: %v", pid, name, signal, err) continue


Question:
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention...
This has been marked with the "needs attention" label and needs to be addressed for the 1.4 release. Could you please close if it's fixed update on your progress to fix it make a case for it not to block the release or find a new owner who can do one of the above soon. Once you or the new owner update this issue and are actively working on it, you can remove the "needs attention" label.
the following message looks suspicious, looks a DNS issue?
Actually more like a network connectivity issue
do you happen to have the Dockerfile for these images? We better push them into
I don't think this is a blocking issue. It looks a transient issue. Since the example is largely dependent on whether the image can be fetched and how long it takes to pull the image, it is very environment sensitive. I would recommend removing label. Longer term, we should move the images used in examples to gcr.
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
for the last flake, based on , image pull (mattf/zookeeper:latest) started at 42:40, finished at 48:31, about 5min51sec, well passed 5 min
storm-nimbus, storm-worker and zookeeper are from they're all automated builds from github (quick link: and )
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}

workerControllerJson := mkpath("storm-worker-controller.json") nsFlag := fmt.Sprintf("--namespace=%v", ns) zookeeperPod := "zookeeper" <ins> nimbusPod := "nimbus" </ins> By("starting Zookeeper") framework.RunKubectlOrDie("create", "-f", zookeeperPodJson, nsFlag) framework.RunKubectlOrDie("create", "-f", zookeeperServiceJson, nsFlag) <del> err := framework.WaitForPodNameRunningInNamespace(c, zookeeperPod, ns) </del> <ins> err := f.WaitForPodRunningSlow(zookeeperPod) </ins> Expect(err).NotTo(HaveOccurred()) By("checking if zookeeper is up and running")


Question:
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention...
This has been marked with the "needs attention" label and needs to be addressed for the 1.4 release. Could you please close if it's fixed update on your progress to fix it make a case for it not to block the release or find a new owner who can do one of the above soon. Once you or the new owner update this issue and are actively working on it, you can remove the "needs attention" label.
the following message looks suspicious, looks a DNS issue?
Actually more like a network connectivity issue
do you happen to have the Dockerfile for these images? We better push them into
I don't think this is a blocking issue. It looks a transient issue. Since the example is largely dependent on whether the image can be fetched and how long it takes to pull the image, it is very environment sensitive. I would recommend removing label. Longer term, we should move the images used in examples to gcr.
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
for the last flake, based on , image pull (mattf/zookeeper:latest) started at 42:40, finished at 48:31, about 5min51sec, well passed 5 min
storm-nimbus, storm-worker and zookeeper are from they're all automated builds from github (quick link: and )
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] [Feature:Example] [] Storm should create and stop Zookeeper, Nimbus and Storm worker servers {Kubernetes e2e suite}

By("starting Nimbus") framework.RunKubectlOrDie("create", "-f", nimbusPodJson, nsFlag) framework.RunKubectlOrDie("create", "-f", nimbusServiceJson, nsFlag) <del> err = framework.WaitForPodNameRunningInNamespace(c, "nimbus", ns) </del> <ins> err = f.WaitForPodRunningSlow(nimbusPod) </ins> Expect(err).NotTo(HaveOccurred()) err = framework.WaitForEndpoint(c, ns, "nimbus")


Question:
I think got confused between e.g. KUBEMASTEROSDISTRIBUTION and MASTEROS_DISTRIBUTION; it definitely doesn't match GCE. PR coming in once I've tested.
cc

# OS options for minions KUBE_OS_DISTRIBUTION="${KUBE_OS_DISTRIBUTION:-jessie}" <del> KUBE_MASTER_OS_DISTRIBUTION="${KUBE_OS_DISTRIBUTION}" KUBE_NODE_OS_DISTRIBUTION="${KUBE_OS_DISTRIBUTION}" </del> <ins> MASTER_OS_DISTRIBUTION="${KUBE_OS_DISTRIBUTION}" NODE_OS_DISTRIBUTION="${KUBE_OS_DISTRIBUTION}" </ins> KUBE_NODE_IMAGE="${KUBE_NODE_IMAGE:-}" COREOS_CHANNEL="${COREOS_CHANNEL:-alpha}" CONTAINER_RUNTIME="${KUBE_CONTAINER_RUNTIME:-docker}"


Question:
Description of problem Create a petset, when pod is evicted, the petset can't create a new pod on other node and make sure pod is running. Kubernetes version (use ): v1.4.0-alpha.2.1427+ How to reproduce it (as minimally and precisely as possible): kubele eviction-hard="memory.available<${value}" a peset and wait all pod is running node MemoryPressure=true all pod status What happened: is evicted and petset can't create new pod and make sure pod is running. What you expected to happen: create new pod on other node and make sure pod is running.
Also when node come back with sufficient memory, petset can't make pod running again.
FYI
cc
Was able to reproduce it. Pet set controller just loops with the following logs I0830 12:32:52. 9 ] Syncing PetSet default/cockroachdb with 3 pets I0830 12:32:52. 9 ] PetSet cockroachdb blocked from scaling on pet cockroachdb-0 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-0 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-1 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-2
-- do you have the petset YAML that we can evaluate? What is the quality of service tier for the pods produced by the petset? Can you include the of an evicted petset pod so we can see the final resource requirements spec?
I am able to reproduce it with nginx pet set in e2e test. Here is yaml output of it Also it seems that i have working fix and almost finished debugging e2e test (patchset is linked above). Will be grateful for review.
Ah, this helps clear things up... the linked pr shows the problem in petset controller and not kubelet...
Does it make sense to set milestone 1.4 for this issue? I think the fix is ready for review

if err := p.SyncPVCs(pet); err != nil { return err } <del> if exists { </del> <ins> // if pet failed - we need to remove old one because of consistent naming if exists && realPet.pod.Status.Phase == api.PodFailed { glog.V(4).Infof("Delete evicted pod %v", realPet.pod.Name) if err := p.petClient.Delete(realPet); err != nil { return err } } else if exists { </ins> if !p.isHealthy(realPet.pod) { glog.Infof("PetSet %v waiting on unhealthy pet %v", pet.parent.Name, realPet.pod.Name) }


Question:
Description of problem Create a petset, when pod is evicted, the petset can't create a new pod on other node and make sure pod is running. Kubernetes version (use ): v1.4.0-alpha.2.1427+ How to reproduce it (as minimally and precisely as possible): kubele eviction-hard="memory.available<${value}" a peset and wait all pod is running node MemoryPressure=true all pod status What happened: is evicted and petset can't create new pod and make sure pod is running. What you expected to happen: create new pod on other node and make sure pod is running.
Also when node come back with sufficient memory, petset can't make pod running again.
FYI
cc
Was able to reproduce it. Pet set controller just loops with the following logs I0830 12:32:52. 9 ] Syncing PetSet default/cockroachdb with 3 pets I0830 12:32:52. 9 ] PetSet cockroachdb blocked from scaling on pet cockroachdb-0 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-0 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-1 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-2
-- do you have the petset YAML that we can evaluate? What is the quality of service tier for the pods produced by the petset? Can you include the of an evicted petset pod so we can see the final resource requirements spec?
I am able to reproduce it with nginx pet set in e2e test. Here is yaml output of it Also it seems that i have working fix and almost finished debugging e2e test (patchset is linked above). Will be grateful for review.
Ah, this helps clear things up... the linked pr shows the problem in petset controller and not kubelet...
Does it make sense to set milestone 1.4 for this issue? I think the fix is ready for review

"k8s.io/kubernetes/pkg/controller/petset" "k8s.io/kubernetes/pkg/labels" "k8s.io/kubernetes/pkg/runtime" <ins> "k8s.io/kubernetes/pkg/types" </ins> "k8s.io/kubernetes/pkg/util/sets" "k8s.io/kubernetes/pkg/util/wait" utilyaml "k8s.io/kubernetes/pkg/util/yaml" <ins> "k8s.io/kubernetes/pkg/watch" </ins> "k8s.io/kubernetes/test/e2e/framework" ) const ( petsetPoll = 10 * time.Second // Some pets install base packages via wget <del> petsetTimeout           = 10 * time.Minute </del> <ins> petsetTimeout = 10 * time.Minute // Timeout for pet pods to change state petPodTimeout           = 5 * time.Minute </ins> zookeeperManifestPath   = "test/e2e/testing-manifests/petset/zookeeper" mysqlGaleraManifestPath = "test/e2e/testing-manifests/petset/mysql-galera" redisManifestPath       = "test/e2e/testing-manifests/petset/redis"


Question:
Description of problem Create a petset, when pod is evicted, the petset can't create a new pod on other node and make sure pod is running. Kubernetes version (use ): v1.4.0-alpha.2.1427+ How to reproduce it (as minimally and precisely as possible): kubele eviction-hard="memory.available<${value}" a peset and wait all pod is running node MemoryPressure=true all pod status What happened: is evicted and petset can't create new pod and make sure pod is running. What you expected to happen: create new pod on other node and make sure pod is running.
Also when node come back with sufficient memory, petset can't make pod running again.
FYI
cc
Was able to reproduce it. Pet set controller just loops with the following logs I0830 12:32:52. 9 ] Syncing PetSet default/cockroachdb with 3 pets I0830 12:32:52. 9 ] PetSet cockroachdb blocked from scaling on pet cockroachdb-0 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-0 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-1 I0830 12:32:52. 9 ] PetSet cockroachdb waiting on unhealthy pet cockroachdb-2
-- do you have the petset YAML that we can evaluate? What is the quality of service tier for the pods produced by the petset? Can you include the of an evicted petset pod so we can see the final resource requirements spec?
I am able to reproduce it with nginx pet set in e2e test. Here is yaml output of it Also it seems that i have working fix and almost finished debugging e2e test (patchset is linked above). Will be grateful for review.
Ah, this helps clear things up... the linked pr shows the problem in petset controller and not kubelet...
Does it make sense to set milestone 1.4 for this issue? I think the fix is ready for review

}) }) <ins> var _ = framework.KubeDescribe("Pet set recreate [Slow] [Feature:PetSet]", func() { f := framework.NewDefaultFramework("pet-set-recreate") var c *client.Client var ns string labels := map[string]string{ "foo": "bar", "baz": "blah", } headlessSvcName := "test" podName := "test-pod" petSetName := "web" petPodName := "web-0" BeforeEach(func() { framework.SkipUnlessProviderIs("gce", "vagrant") By("creating service " + headlessSvcName + " in namespace " + f.Namespace.Name) headlessService := createServiceSpec(headlessSvcName, "", true, labels) _, err := f.Client.Services(f.Namespace.Name).Create(headlessService) framework.ExpectNoError(err) c = f.Client ns = f.Namespace.Name }) AfterEach(func() { if CurrentGinkgoTestDescription().Failed { dumpDebugInfo(c, ns) } By("Deleting all petset in ns " + ns) deleteAllPetSets(c, ns) }) It("should recreate evicted petset", func() { By("looking for a node to schedule pet set and pod") nodes := framework.GetReadySchedulableNodesOrDie(f.Client) node := nodes.Items[0] By("creating pod with conflicting port in namespace " + f.Namespace.Name) conflictingPort := api.ContainerPort{HostPort: 21017, ContainerPort: 21017, Name: "conflict"} pod := &api.Pod{ ObjectMeta: api.ObjectMeta{ Name: podName, }, Spec: api.PodSpec{ Containers: []api.Container{ { Name:  "nginx", Image: "gcr.io/google_containers/nginx-slim:0.7", Ports: []api.ContainerPort{conflictingPort}, }, }, NodeName: node.Name, }, } pod, err := f.Client.Pods(f.Namespace.Name).Create(pod) framework.ExpectNoError(err) By("creating petset with conflicting port in namespace " + f.Namespace.Name) ps := newPetSet(petSetName, f.Namespace.Name, headlessSvcName, 1, nil, nil, labels) petContainer := &ps.Spec.Template.Spec.Containers[0] petContainer.Ports = append(petContainer.Ports, conflictingPort) ps.Spec.Template.Spec.NodeName = node.Name _, err = f.Client.Apps().PetSets(f.Namespace.Name).Create(ps) framework.ExpectNoError(err) By("waiting until pod " + podName + " will start running in namespace " + f.Namespace.Name) if err := f.WaitForPodRunning(podName); err != nil { framework.Failf("Pod %v did not start running: %v", podName, err) } var initialPetPodUID types.UID By("waiting until pet pod " + petPodName + " will be recreated and deleted at least once in namespace " + f.Namespace.Name) w, err := f.Client.Pods(f.Namespace.Name).Watch(api.SingleObject(api.ObjectMeta{Name: petPodName})) framework.ExpectNoError(err) // we need to get UID from pod in any state and wait until pet set controller will remove pod atleast once _, err = watch.Until(petPodTimeout, w, func(event watch.Event) (bool, error) { pod := event.Object.(*api.Pod) switch event.Type { case watch.Deleted: framework.Logf("Observed delete event for pet pod %v in namespace %v", pod.Name, pod.Namespace) if initialPetPodUID == "" { return false, nil } return true, nil } framework.Logf("Observed pet pod in namespace: %v, name: %v, uid: %v, status phase: %v. Waiting for petset controller to delete.", pod.Namespace, pod.Name, pod.UID, pod.Status.Phase) initialPetPodUID = pod.UID return false, nil }) if err != nil { framework.Failf("Pod %v expected to be re-created atleast once", petPodName) } By("removing pod with conflicting port in namespace " + f.Namespace.Name) err = f.Client.Pods(f.Namespace.Name).Delete(pod.Name, api.NewDeleteOptions(0)) framework.ExpectNoError(err) By("waiting when pet pod " + petPodName + " will be recreated in namespace " + f.Namespace.Name + " and will be in running state") // we may catch delete event, thats why we are waiting for running phase like this, and not with watch.Until Eventually(func() error { petPod, err := f.Client.Pods(f.Namespace.Name).Get(petPodName) if err != nil { return err } if petPod.Status.Phase != api.PodRunning { return fmt.Errorf("Pod %v is not in running phase: %v", petPod.Name, petPod.Status.Phase) } else if petPod.UID == initialPetPodUID { return fmt.Errorf("Pod %v wasn't recreated: %v == %v", petPod.Name, petPod.UID, initialPetPodUID) } return nil }, petPodTimeout, 2*time.Second).Should(BeNil()) }) }) </ins> func dumpDebugInfo(c *client.Client, ns string) { pl, _ := c.Pods(ns).List(api.ListOptions{LabelSelector: labels.Everything()}) for _, p := range pl.Items {


Question:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite} Previous issues for this test:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
I'm pretty sure, those amount of failure are related to: (I don't know why yet).
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}

return err } if waitForReplicas != nil { <del> watchOptions := api.ListOptions{FieldSelector: fields.OneTermEqualSelector("metadata.name", name), ResourceVersion: updatedResourceVersion} watcher, err := scaler.c.ReplicationControllers(namespace).Watch(watchOptions) </del> <ins> checkRC := func(rc *api.ReplicationController) bool { if uint(rc.Spec.Replicas) != newSize { // the size is changed by other party. Don't need to wait for the new change to complete. return true } return rc.Status.ObservedGeneration >= rc.Generation && rc.Status.Replicas == rc.Spec.Replicas } // If number of replicas doesn't change, then the update may not event // be sent to underlying databse (we don't send no-op changes). // In such case, <updatedResourceVersion> will have value of the most // recent update (which may be far in the past) so we may get "too old // RV" error from watch or potentially no ReplicationController events // will be deliver, since it may already be in the expected state. // To protect from these two, we first issue Get() to ensure that we // are not already in the expected state. currentRC, err := scaler.c.ReplicationControllers(namespace).Get(name) </ins> if err != nil { return err } <del> _, err = watch.Until(waitForReplicas.Timeout, watcher, func(event watch.Event) (bool, error) { if event.Type != watch.Added && event.Type != watch.Modified { return false, nil </del> <ins> if !checkRC(currentRC) { watchOptions := api.ListOptions{ FieldSelector:   fields.OneTermEqualSelector("metadata.name", name), ResourceVersion: updatedResourceVersion, </ins> } <del> rc := event.Object.(*api.ReplicationController) if uint(rc.Spec.Replicas) != newSize { // the size is changed by other party. Don't need to wait for the new change to complete. return true, nil </del> <ins> watcher, err := scaler.c.ReplicationControllers(namespace).Watch(watchOptions) if err != nil { return err </ins> } <del> return rc.Status.ObservedGeneration >= rc.Generation && rc.Status.Replicas == rc.Spec.Replicas, nil }) if err == wait.ErrWaitTimeout { return fmt.Errorf("timed out waiting for %q to be synced", name) </del> <ins> _, err = watch.Until(waitForReplicas.Timeout, watcher, func(event watch.Event) (bool, error) { if event.Type != watch.Added && event.Type != watch.Modified { return false, nil } return checkRC(event.Object.(*api.ReplicationController)), nil }) if err == wait.ErrWaitTimeout { return fmt.Errorf("timed out waiting for %q to be synced", name) } return err </ins> } <del> return err </del> } return nil }


Question:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite} Previous issues for this test:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
I'm pretty sure, those amount of failure are related to: (I don't know why yet).
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}

}, }, StopError:       nil, <del> ExpectedActions: []string{"get", "list", "get", "update", "watch", "delete"}, </del> <ins> ExpectedActions: []string{"get", "list", "get", "update", "get", "delete"}, </ins> }, { Name: "NoOverlapping",


Question:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite} Previous issues for this test:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
I'm pretty sure, those amount of failure are related to: (I don't know why yet).
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}

}, }, StopError:       nil, <del> ExpectedActions: []string{"get", "list", "get", "update", "watch", "delete"}, </del> <ins> ExpectedActions: []string{"get", "list", "get", "update", "get", "delete"}, </ins> }, { Name: "OverlappingError",


Question:
<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? Currently, the 4th parameter name of updateServiceListener in kube-proxy is addOrDelete, However, from the func caller, we just pass for AddServiceListener, and for DeleteServiceListener, So, I think the name addOrDelete is unreasonable. I prefer to change the parameter name addOrDelete to add so that avoid making people confused. In addition, there is a typo, We should change addition to deletion! Also, there is a sentence breaking, we should change from No Service Endpoints Not Found =No Service Endpoints are found Thanks!
Fixed!

healthchecker.mutationRequestChannel <- req } <del> func updateServiceListener(serviceName types.NamespacedName, listenPort int, addOrDelete bool) bool { </del> <ins> func updateServiceListener(serviceName types.NamespacedName, listenPort int, add bool) bool { </ins> responseChannel := make(chan bool) req := &proxyListenerRequest{ serviceName:     serviceName, listenPort:      uint16(listenPort), <del> add:             addOrDelete, </del> <ins> add:             add, </ins> responseChannel: responseChannel, } healthchecker.listenerRequestChannel <- req


Question:
<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? Currently, the 4th parameter name of updateServiceListener in kube-proxy is addOrDelete, However, from the func caller, we just pass for AddServiceListener, and for DeleteServiceListener, So, I think the name addOrDelete is unreasonable. I prefer to change the parameter name addOrDelete to add so that avoid making people confused. In addition, there is a typo, We should change addition to deletion! Also, there is a sentence breaking, we should change from No Service Endpoints Not Found =No Service Endpoints are found Thanks!
Fixed!

return updateServiceListener(serviceName, listenPort, true) } <del> // DeleteServiceListener Request addition of a listener for a service's health check </del> <ins> // DeleteServiceListener Request deletion of a listener for a service's health check </ins> func DeleteServiceListener(serviceName types.NamespacedName, listenPort int) bool { return updateServiceListener(serviceName, listenPort, false) }


Question:
<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? Currently, the 4th parameter name of updateServiceListener in kube-proxy is addOrDelete, However, from the func caller, we just pass for AddServiceListener, and for DeleteServiceListener, So, I think the name addOrDelete is unreasonable. I prefer to change the parameter name addOrDelete to add so that avoid making people confused. In addition, there is a typo, We should change addition to deletion! Also, there is a sentence breaking, we should change from No Service Endpoints Not Found =No Service Endpoints are found Thanks!
Fixed!

s, ok := h.serviceEndpointsMap.Get(serviceName) if !ok { glog.V(4).Infof("Service %s not found or has no local endpoints", serviceName) <del> sendHealthCheckResponse(rw, http.StatusServiceUnavailable, "No Service Endpoints Not Found") </del> <ins> sendHealthCheckResponse(rw, http.StatusServiceUnavailable, "No Service Endpoints Found") </ins> return } numEndpoints := len(*s.(*serviceEndpointsList).endpoints)


Question:
In practice this means that it doesn't get to update and thereby may refuse to admit pods which had been assigned to it by the scheduler. For example, if you update labels for that given node, and try to use affinity or on a pod, it won't get to run.
This looks like a sufficient fix for me, but I'm happy to add instead, if that'd help getting it accepted for 1.4.0: cc
cc
That commit makes sense to me.
Thanks, I'll open a PR.

func run(s *options.KubeletServer, kubeDeps *kubelet.KubeletDeps) (err error) { // TODO: this should be replaced by a --standalone flag <del> standaloneMode := (len(s.APIServerList) == 0) </del> <ins> standaloneMode := (len(s.APIServerList) == 0 && !s.RequireKubeConfig) </ins> if s.ExitOnLockContention && s.LockFilePath == "" { return errors.New("cannot exit on lock file contention: no lock file specified")


Question:
While following this to setup a kube-mesos integration env on my laptop, I found the default service account and default token are not created when start the cluster. As a result, I can not create a pod or deployment in this environment. Reproduce steps: the latest Mesos code, build and start 1 single master with 1 single agent. the latest Kubernetes code, use to build. the following commands to start apiserver, controller-manager and scheduler respectively: all the components are started, run command and see that there is no default token created for the default namespace. At this moment, run "kubectl create -f " will see the failure of creating a pod.
This introduced to node controller and now when we start a node controller, if is not nil, we will start this Informer controller as well. But the start of the will block main thread from starting other controllers later in the main thread. we can see that in the kube-mesos integration, we started some other controllers, including serviceaccount controller, after we started node controller. I will submit a patch later to run the in a go routine to solve the issue.

go nc.podController.Run(wait.NeverStop) go nc.daemonSetController.Run(wait.NeverStop) if nc.internalPodInformer != nil { <del> nc.internalPodInformer.Run(wait.NeverStop) </del> <ins> go nc.internalPodInformer.Run(wait.NeverStop) </ins> } // Incorporate the results of node status pushed from kubelet to master.


Question:
https://k8s- Failed: ThirdParty resources Simple Third Party creating/deleting thirdparty objects works [Conformance] {Kubernetes e2e suite}
cc
Seems like the namespace creation/deletion process is flaky. IIRC there have been some recent PRs touching this area, is that right?
We recently code to try to expose the real underlying error for why the ns deletion fails. This is hopefully now showing it.
The issue is with TPR not integrated properly with the namespace controller. Creation of a new type requires a restart of the controller-manager given current state of the code. The test must be marked flaky until the state of the art is improved (which is not happening in 1.4).

Items []Foo `json:"items"` } <del> var _ = Describe("ThirdParty resources", func() { </del> <ins> // This test is marked flaky pending namespace controller observing dynamic creation of new third party types. var _ = Describe("ThirdParty resources [Flaky] [Disruptive]", func() { </ins> f := framework.NewDefaultFramework("thirdparty")


Question:
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite} Previous issues for this test:
This is pretty much the same failure, that are happening in "gce-enormous" suite and we currently debugging them with The main reason of those failures is that we are terminating all watchers for a particular resource type from time to time, and some of them are actually causing a snowball effect. After more looking into it, my hypothesis is that our machinery for aggregating watchers in apiserver sometimes is just to slow. In particular, I some log in my local branch and what I'm observing is that in etcd_watcher, we are: accumulating objects in incoming channel but also accumulating objects in outgoing channel, and the accumulation there is usually higher [I will send a PR to put those logs into master - they seem to be pretty useful for debugging such issues.] So my current hypothesis is that dispatching watch events to watchers is what is too slow. We will continue debugging.
The logs that I seem to confirm my hypothesis - outgoing channel is generally more full then the incoming channel in etcd_watcher: vs I'm working on the PR that should at least improve the situation.
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
https://k8s- Failed: [] Load capacity [Feature:Performance] should be able to handle 30 pods per node {Kubernetes e2e suite}
Please apply the label to this issue if it does not need to be fixed before we launch 1.4. Otherwise please advise on how to proceed with respect to 1.4.
This was a regression - I fixed it by reverting an offending PR: This seems to be fixed now.
I have 33 pods in my container (1 is substantive, the other 32 just run on log files). I get this all the time now. I'm running 1.5.4.
- to be honest, I don't undertsand what your wrote. what is "container" in "33 pods in my container"? Also, if you believe there is a bug somewhere, please open a new issue instead of adding comment to more than half-year-old one.

return result, len(result) > 0 } <ins> // TODO: Most probably splitting this method to a separate thread will visibily // improve throughput of our watch machinery. So what we should do is to: // - OnEvent handler simply put an element to channel // - processEvent be another goroutine processing events from that channel // Additionally, if we make this channel buffered, cacher will be more resistant // to single watchers being slow - see cacheWatcher::add method. </ins> func (c *Cacher) processEvent(event watchCacheEvent) { triggerValues, supported := c.triggerValues(&event)