Question:
Right now, we have a mixture of user documentation and developer documentation in the top-level directory. We should make it easier to find a just user documentation. Step 1 would be to move developer documentation into a subdir like docs/development or something like that. User docs would stay in the top-level docs dir.
move to

<ins> # Development Guide # Releases and Official Builds Official releases are built in Docker containers.  Details are [here](build/README.md).  You can do simple builds and development with just a local Docker installation.  If want to build go locally outside of docker, please continue below. ## Go development environment Kubernetes is written in [Go](http://golang.org) programming language. If you haven't set up Go development environment, please follow [this instruction](http://golang.org/doc/code.html) to install go tool and set up GOPATH. Ensure your version of Go is at least 1.3. ## Put kubernetes into GOPATH We highly recommend to put kubernetes' code into your GOPATH. For example, the following commands will download kubernetes' code under the current user's GOPATH (Assuming there's only one directory in GOPATH.): ``` $ echo $GOPATH /home/user/goproj $ mkdir -p $GOPATH/src/github.com/GoogleCloudPlatform/ $ cd $GOPATH/src/github.com/GoogleCloudPlatform/ $ git clone git@github.com:GoogleCloudPlatform/kubernetes.git ``` The commands above will not work if there are more than one directory in ``$GOPATH``. (Obviously, clone your own fork of Kubernetes if you plan to do development.) ## godep and dependency management Kubernetes uses [godep](https://github.com/tools/godep) to manage dependencies. It is not strictly required for building Kubernetes but it is required when managing dependencies under the Godeps/ tree, and is required by a number of the build and test scripts. Please make sure that ``godep`` is installed and in your ``$PATH``. ### Installing godep There are many ways to build and host go binaries. Here is an easy way to get utilities like ```godep``` installed: 1. Ensure that [mercurial](http://mercurial.selenic.com/wiki/Download) is installed on your system. (some of godep's dependencies use the mercurial source control system).  Use ```apt-get install mercurial``` or ```yum install mercurial``` on Linux, or [brew.sh](http://brew.sh) on OS X, or download directly from mercurial. 2. Create a new GOPATH for your tools and install godep: ``` export GOPATH=$HOME/go-tools mkdir -p $GOPATH go get github.com/tools/godep ``` 3. Add $GOPATH/bin to your path. Typically you'd add this to your ~/.profile: ``` export GOPATH=$HOME/go-tools export PATH=$PATH:$GOPATH/bin ``` ### Using godep Here is a quick summary of `godep`.  `godep` helps manage third party dependencies by copying known versions into Godeps/_workspace.  You can use `godep` in three ways: 1. Use `godep` to call your `go` commands.  For example: `godep go test ./...` 2. Use `godep` to modify your `$GOPATH` so that other tools know where to find the dependencies.  Specifically: `export GOPATH=$GOPATH:$(godep path)` 3. Use `godep` to copy the saved versions of packages into your `$GOPATH`.  This is done with `godep restore`. We recommend using options #1 or #2. ## Hooks Before committing any changes, please link/copy these hooks into your .git directory. This will keep you from accidentally committing non-gofmt'd go code. ``` cd kubernetes ln -s hooks/prepare-commit-msg .git/hooks/prepare-commit-msg ln -s hooks/commit-msg .git/hooks/commit-msg ``` ## Unit tests ``` cd kubernetes hack/test-go.sh ``` Alternatively, you could also run: ``` cd kubernetes godep go test ./... ``` If you only want to run unit tests in one package, you could run ``godep go test`` under the package directory. For example, the following commands will run all unit tests in package kubelet: ``` $ cd kubernetes # step into kubernetes' directory. $ cd pkg/kubelet $ godep go test # some output from unit tests PASS ok      github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet   0.317s ``` ## Coverage ``` cd kubernetes godep go tool cover -html=target/c.out ``` ## Integration tests You need an etcd somewhere in your PATH. To install etcd, run: ``` cd kubernetes hack/install-etcd.sh sudo ln -s $(pwd)/third_party/etcd/bin/etcd /usr/bin/etcd ``` ``` cd kubernetes hack/test-integration.sh ``` ## End-to-End tests You can run an end-to-end test which will bring up a master and two minions, perform some tests, and then tear everything down. Make sure you have followed the getting started steps for your chosen cloud platform (which might involve changing the `KUBERNETES_PROVIDER` environment variable to something other than "gce". ``` cd kubernetes hack/e2e-test.sh ``` Pressing control-C should result in an orderly shutdown but if something goes wrong and you still have some VMs running you can force a cleanup with the magical incantation: ``` hack/e2e-test.sh 1 1 1 ``` ## Testing out flaky tests [Instructions here](docs/devel/flaky-tests.md) ## Add/Update dependencies Kubernetes uses [godep](https://github.com/tools/godep) to manage dependencies. To add or update a package, please follow the instructions on [godep's document](https://github.com/tools/godep). To add a new package ``foo/bar``: - Make sure the kubernetes' root directory is in $GOPATH/github.com/GoogleCloudPlatform/kubernetes - Run ``godep restore`` to make sure you have all dependancies pulled. - Download foo/bar into the first directory in GOPATH: ``go get foo/bar``. - Change code in kubernetes to use ``foo/bar``. - Run ``godep save ./...`` under kubernetes' root directory. To update a package ``foo/bar``: - Make sure the kubernetes' root directory is in $GOPATH/github.com/GoogleCloudPlatform/kubernetes - Run ``godep restore`` to make sure you have all dependancies pulled. - Update the package with ``go get -u foo/bar``. - Change code in kubernetes accordingly if necessary. - Run ``godep update foo/bar`` under kubernetes' root directory. ## Keeping your development fork in sync One time after cloning your forked repo: ``` git remote add upstream https://github.com/GoogleCloudPlatform/kubernetes.git ``` Then each time you want to sync to upstream: ``` git fetch upstream git rebase upstream/master ``` ## Regenerating the API documentation ``` cd kubernetes/api sudo docker build -t kubernetes/raml2html . sudo docker run --name="docgen" kubernetes/raml2html sudo docker cp docgen:/data/kubernetes.html . ``` View the API documentation using htmlpreview (works on your fork, too): ``` http://htmlpreview.github.io/?https://github.com/GoogleCloudPlatform/kubernetes/blob/master/api/kubernetes.html ``` </ins>


Question:
Right now, we have a mixture of user documentation and developer documentation in the top-level directory. We should make it easier to find a just user documentation. Step 1 would be to move developer documentation into a subdir like docs/development or something like that. User docs would stay in the top-level docs dir.
move to

<ins> # Hunting flaky tests in Kubernetes Sometimes unit tests are flaky.  This means that due to (usually) race conditions, they will occasionally fail, even though most of the time they pass. We have a goal of 99.9% flake free tests.  This means that there is only one flake in one thousand runs of a test. Running a test 1000 times on your own machine can be tedious and time consuming.  Fortunately, there is a better way to achieve this using Kubernetes. _Note: these instructions are mildly hacky for now, as we get run once semantics and logging they will get better_ There is a testing image ```brendanburns/flake``` up on the docker hub.  We will use this image to test our fix. Create a replication controller with the following config: ```yaml id: flakeController desiredState: replicas: 24 replicaSelector: name: flake podTemplate: desiredState: manifest: version: v1beta1 id: "" volumes: [] containers: - name: flake image: brendanburns/flake env: - name: TEST_PACKAGE value: pkg/tools - name: REPO_SPEC value: https://github.com/GoogleCloudPlatform/kubernetes restartpolicy: {} labels: name: flake labels: name: flake ``` ```./cluster/kubecfg.sh -c controller.yaml create replicaControllers``` This will spin up 100 instances of the test.  They will run to completion, then exit, the kubelet will restart them, eventually you will have sufficient runs for your purposes, and you can stop the replication controller: ```sh ./cluster/kubecfg.sh stop flakeController ./cluster/kubecfg.sh rm flakeController ``` Now examine the machines with ```docker ps -a``` and look for tasks that exited with non-zero exit codes (ignore those that exited -1, since that's what happens when you stop the replica controller) Happy flake hunting! </ins>


Question:
Right now, we have a mixture of user documentation and developer documentation in the top-level directory. We should make it easier to find a just user documentation. Step 1 would be to move developer documentation into a subdir like docs/development or something like that. User docs would stay in the top-level docs dir.
move to

<ins> // Build it with: //   $ dot -Tsvg releasing.dot >releasing.svg digraph tagged_release { size = "5,5" // Arrows go up. rankdir = BT subgraph left { // Group the left nodes together. ci012abc -> pr101 -> ci345cde -> pr102 style = invis } subgraph right { // Group the right nodes together. version_commit -> dev_commit style = invis } { // Align the version commit and the info about it. rank = same // Align them with pr101 pr101 version_commit // release_info shows the change in the commit. release_info } { // Align the dev commit and the info about it. rank = same // Align them with 345cde ci345cde dev_commit dev_info } // Join the nodes from subgraph left. pr99 -> ci012abc pr102 -> pr100 // Do the version node. pr99 -> version_commit dev_commit -> pr100 tag -> version_commit pr99 [ label = "Merge PR #99" shape = box fillcolor = "#ccccff" style = "filled" fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; ci012abc [ label = "012abc" shape = circle fillcolor = "#ffffcc" style = "filled" fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace" ]; pr101 [ label = "Merge PR #101" shape = box fillcolor = "#ccccff" style = "filled" fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; ci345cde [ label = "345cde" shape = circle fillcolor = "#ffffcc" style = "filled" fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace" ]; pr102 [ label = "Merge PR #102" shape = box fillcolor = "#ccccff" style = "filled" fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; version_commit [ label = "678fed" shape = circle fillcolor = "#ccffcc" style = "filled" fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace" ]; dev_commit [ label = "456dcb" shape = circle fillcolor = "#ffffcc" style = "filled" fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace" ]; pr100 [ label = "Merge PR #100" shape = box fillcolor = "#ccccff" style = "filled" fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; release_info [ label = "pkg/version/base.go:ngitVersion = "v0.5";" shape = none fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; dev_info [ label = "pkg/version/base.go:ngitVersion = "v0.5-dev";" shape = none fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; tag [ label = "$ git tag -a v0.5" fillcolor = "#ffcccc" style = "filled" fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" ]; } </ins>


Question:
Right now, we have a mixture of user documentation and developer documentation in the top-level directory. We should make it easier to find a just user documentation. Step 1 would be to move developer documentation into a subdir like docs/development or something like that. User docs would stay in the top-level docs dir.
move to

<ins> # Releasing Kubernetes This document explains how to create a Kubernetes release (as in version) and how the version information gets embedded into the built binaries. ## Origin of the Sources Kubernetes may be built from either a git tree (using `hack/build-go.sh`) or from a tarball (using either `hack/build-go.sh` or `go install`) or directly by the Go native build system (using `go get`). When building from git, we want to be able to insert specific information about the build tree at build time. In particular, we want to use the output of `git describe` to generate the version of Kubernetes and the status of the build tree (add a `-dirty` prefix if the tree was modified.) When building from a tarball or using the Go build system, we will not have access to the information about the git tree, but we still want to be able to tell whether this build corresponds to an exact release (e.g. v0.3) or is between releases (e.g. at some point in development between v0.3 and v0.4). ## Version Number Format In order to account for these use cases, there are some specific formats that may end up representing the Kubernetes version. Here are a few examples: - **v0.5**: This is official version 0.5 and this version will only be used when building from a clean git tree at the v0.5 git tag, or from a tree extracted from the tarball corresponding to that specific release. - **v0.5-15-g0123abcd4567**: This is the `git describe` output and it indicates that we are 15 commits past the v0.5 release and that the SHA1 of the commit where the binaries were built was `0123abcd4567`. It is only possible to have this level of detail in the version information when building from git, not when building from a tarball. - **v0.5-15-g0123abcd4567-dirty** or **v0.5-dirty**: The extra `-dirty` prefix means that the tree had local modifications or untracked files at the time of the build, so there's no guarantee that the source code matches exactly the state of the tree at the `0123abcd4567` commit or at the `v0.5` git tag (resp.) - **v0.5-dev**: This means we are building from a tarball or using `go get` or, if we have a git tree, we are using `go install` directly, so it is not possible to inject the git version into the build information. Additionally, this is not an official release, so the `-dev` prefix indicates that the version we are building is after `v0.5` but before `v0.6`. (There is actually an exception where a commit with `v0.5-dev` is not present on `v0.6`, see later for details.) ## Injecting Version into Binaries In order to cover the different build cases, we start by providing information that can be used when using only Go build tools or when we do not have the git version information available. To be able to provide a meaningful version in those cases, we set the contents of variables in a Go source file that will be used when no overrides are present. We are using `pkg/version/base.go` as the source of versioning in absence of information from git. Here is a sample of that file's contents: ``` var ( gitVersion   string = "v0.4-dev"  // version from git, output of $(git describe) gitCommit    string = ""          // sha1 from git, output of $(git rev-parse HEAD) ) ``` This means a build with `go install` or `go get` or a build from a tarball will yield binaries that will identify themselves as `v0.4-dev` and will not be able to provide you with a SHA1. To add the extra versioning information when building from git, the `hack/build-go.sh` script will gather that information (using `git describe` and `git rev-parse`) and then create a `-ldflags` string to pass to `go install` and tell the Go linker to override the contents of those variables at build time. It can, for instance, tell it to override `gitVersion` and set it to `v0.4-13-g4567bcdef6789-dirty` and set `gitCommit` to `4567bcdef6789...` which is the complete SHA1 of the (dirty) tree used at build time. ## Handling Official Versions Handling official versions from git is easy, as long as there is an annotated git tag pointing to a specific version then `git describe` will return that tag exactly which will match the idea of an official version (e.g. `v0.5`). Handling it on tarballs is a bit harder since the exact version string must be present in `pkg/version/base.go` for it to get embedded into the binaries. But simply creating a commit with `v0.5` on its own would mean that the commits coming after it would also get the `v0.5` version when built from tarball or `go get` while in fact they do not match `v0.5` (the one that was tagged) exactly. To handle that case, creating a new release should involve creating two adjacent commits where the first of them will set the version to `v0.5` and the second will set it to `v0.5-dev`. In that case, even in the presence of merges, there will be a single comit where the exact `v0.5` version will be used and all others around it will either have `v0.4-dev` or `v0.5-dev`. The diagram below illustrates it. ![Diagram of git commits involved in the release](./releasing.png) After working on `v0.4-dev` and merging PR 99 we decide it is time to release `v0.5`. So we start a new branch, create one commit to update `pkg/version/base.go` to include `gitVersion = "v0.5"` and `git commit` it. We test it and make sure everything is working as expected. Before sending a PR for it, we create a second commit on that same branch, updating `pkg/version/base.go` to include `gitVersion = "v0.5-dev"`. That will ensure that further builds (from tarball or `go install`) on that tree will always include the `-dev` prefix and will not have a `v0.5` version (since they do not match the official `v0.5` exactly.) We then send PR 100 with both commits in it. Once the PR is accepted, we can use `git tag -a` to create an annotated tag *pointing to the one commit* that has `v0.5` in `pkg/version/base.go` and push it to GitHub. (Unfortunately GitHub tags/releases are not annotated tags, so this needs to be done from a git client and pushed to GitHub using SSH.) ## Parallel Commits While we are working on releasing `v0.5`, other development takes place and other PRs get merged. For instance, in the example above, PRs 101 and 102 get merged to the master branch before the versioning PR gets merged. This is not a problem, it is only slightly inaccurate that checking out the tree at commit `012abc` or commit `345cde` or at the commit of the merges of PR 101 or 102 will yield a version of `v0.4-dev` *but* those commits are not present in `v0.5`. In that sense, there is a small window in which commits will get a `v0.4-dev` or `v0.4-N-gXXX` label and while they're indeed later than `v0.4` but they are not really before `v0.5` in that `v0.5` does not contain those commits. Unfortunately, there is not much we can do about it. On the other hand, other projects seem to live with that and it does not really become a large problem. As an example, Docker commit a327d9b91edf has a `v1.1.1-N-gXXX` label but it is not present in Docker `v1.2.0`: ``` $ git describe a327d9b91edf v1.1.1-822-ga327d9b91edf $ git log --oneline v1.2.0..a327d9b91edf a327d9b91edf Fix data space reporting from Kb/Mb to KB/MB (Non-empty output here means the commit is not present on v1.2.0.) ``` </ins>


Question:
Right now, we have a mixture of user documentation and developer documentation in the top-level directory. We should make it easier to find a just user documentation. Step 1 would be to move developer documentation into a subdir like docs/development or something like that. User docs would stay in the top-level docs dir.
move to

<ins> <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!-- Generated by graphviz version 2.36.0 (20140111.2315) --> <!-- Title: tagged_release Pages: 1 --> <svg width="257pt" height="360pt" viewBox="0.00 0.00 257.33 360.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> <g id="graph0" class="graph" transform="scale(0.649819 0.649819) rotate(0) translate(4 550)"> <title>tagged_release</title> <polygon fill="white" stroke="none" points="-4,4 -4,-550 392,-550 392,4 -4,4"/> <!-- ci012abc --> <g id="node1" class="node"><title>ci012abc</title> <ellipse fill="#ffffcc" stroke="black" cx="56" cy="-115" rx="42.7926" ry="42.7926"/> <text text-anchor="middle" x="56" y="-111.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">012abc</text> </g> <!-- pr101 --> <g id="node2" class="node"><title>pr101</title> <polygon fill="#ccccff" stroke="black" points="112,-255 0,-255 0,-219 112,-219 112,-255"/> <text text-anchor="middle" x="56" y="-233.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #101</text> </g> <!-- ci012abc->pr101 --> <g id="edge1" class="edge"><title>ci012abc->pr101</title> <path fill="none" stroke="black" d="M56,-157.97C56,-174.83 56,-193.784 56,-208.789"/> <polygon fill="black" stroke="black" points="52.5001,-208.93 56,-218.93 59.5001,-208.93 52.5001,-208.93"/> </g> <!-- ci345cde --> <g id="node3" class="node"><title>ci345cde</title> <ellipse fill="#ffffcc" stroke="black" cx="62" cy="-359" rx="42.7926" ry="42.7926"/> <text text-anchor="middle" x="62" y="-355.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">345cde</text> </g> <!-- pr101->ci345cde --> <g id="edge2" class="edge"><title>pr101->ci345cde</title> <path fill="none" stroke="black" d="M56.8597,-255.193C57.5237,-268.473 58.4796,-287.592 59.3874,-305.748"/> <polygon fill="black" stroke="black" points="55.904,-306.17 59.8991,-315.982 62.8953,-305.82 55.904,-306.17"/> </g> <!-- pr102 --> <g id="node4" class="node"><title>pr102</title> <polygon fill="#ccccff" stroke="black" points="129,-474 17,-474 17,-438 129,-438 129,-474"/> <text text-anchor="middle" x="73" y="-452.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #102</text> </g> <!-- ci345cde->pr102 --> <g id="edge3" class="edge"><title>ci345cde->pr102</title> <path fill="none" stroke="black" d="M66.8248,-401.668C67.8523,-410.542 68.9117,-419.692 69.8567,-427.853"/> <polygon fill="black" stroke="black" points="66.3936,-428.375 71.0207,-437.906 73.3472,-427.57 66.3936,-428.375"/> </g> <!-- pr100 --> <g id="node10" class="node"><title>pr100</title> <polygon fill="#ccccff" stroke="black" points="174,-546 62,-546 62,-510 174,-510 174,-546"/> <text text-anchor="middle" x="118" y="-524.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #100</text> </g> <!-- pr102->pr100 --> <g id="edge6" class="edge"><title>pr102->pr100</title> <path fill="none" stroke="black" d="M84.1236,-474.303C89.355,-482.441 95.6999,-492.311 101.478,-501.299"/> <polygon fill="black" stroke="black" points="98.6526,-503.377 107.004,-509.896 104.541,-499.591 98.6526,-503.377"/> </g> <!-- version_commit --> <g id="node5" class="node"><title>version_commit</title> <ellipse fill="#ccffcc" stroke="black" cx="173" cy="-237" rx="42.7926" ry="42.7926"/> <text text-anchor="middle" x="173" y="-233.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">678fed</text> </g> <!-- dev_commit --> <g id="node6" class="node"><title>dev_commit</title> <ellipse fill="#ffffcc" stroke="black" cx="169" cy="-359" rx="42.7926" ry="42.7926"/> <text text-anchor="middle" x="169" y="-355.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">456dcb</text> </g> <!-- version_commit->dev_commit --> <g id="edge4" class="edge"><title>version_commit->dev_commit</title> <path fill="none" stroke="black" d="M171.601,-279.97C171.322,-288.326 171.027,-297.195 170.739,-305.844"/> <polygon fill="black" stroke="black" points="167.24,-305.74 170.405,-315.851 174.236,-305.973 167.24,-305.74"/> </g> <!-- dev_commit->pr100 --> <g id="edge8" class="edge"><title>dev_commit->pr100</title> <path fill="none" stroke="black" d="M158.36,-400.568C152.438,-422.433 144.719,-449.815 137,-474 134.253,-482.606 131.013,-491.906 127.994,-500.278"/> <polygon fill="black" stroke="black" points="124.616,-499.325 124.47,-509.919 131.191,-501.729 124.616,-499.325"/> </g> <!-- release_info --> <g id="node7" class="node"><title>release_info</title> <text text-anchor="middle" x="304" y="-240.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">pkg/version/base.go:</text> <text text-anchor="middle" x="304" y="-225.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">gitVersion = "v0.5";</text> </g> <!-- dev_info --> <g id="node8" class="node"><title>dev_info</title> <text text-anchor="middle" x="309" y="-362.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">pkg/version/base.go:</text> <text text-anchor="middle" x="309" y="-347.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">gitVersion = "v0.5-dev";</text> </g> <!-- pr99 --> <g id="node9" class="node"><title>pr99</title> <polygon fill="#ccccff" stroke="black" points="143,-36 39,-36 39,-0 143,-0 143,-36"/> <text text-anchor="middle" x="91" y="-14.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #99</text> </g> <!-- pr99->ci012abc --> <g id="edge5" class="edge"><title>pr99->ci012abc</title> <path fill="none" stroke="black" d="M84.5805,-36.4245C81.5586,-44.6267 77.7879,-54.8615 73.9865,-65.1795"/> <polygon fill="black" stroke="black" points="70.6804,-64.0292 70.5075,-74.6226 77.2488,-66.4492 70.6804,-64.0292"/> </g> <!-- pr99->version_commit --> <g id="edge7" class="edge"><title>pr99->version_commit</title> <path fill="none" stroke="black" d="M97.4344,-36.0276C109.585,-68.1826 136.317,-138.924 154.498,-187.038"/> <polygon fill="black" stroke="black" points="151.318,-188.523 158.127,-196.64 157.866,-186.048 151.318,-188.523"/> </g> <!-- tag --> <g id="node11" class="node"><title>tag</title> <ellipse fill="#ffcccc" stroke="black" cx="226" cy="-115" rx="71.4873" ry="18"/> <text text-anchor="middle" x="226" y="-111.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">$ git tag -a v0.5</text> </g> <!-- tag->version_commit --> <g id="edge9" class="edge"><title>tag->version_commit</title> <path fill="none" stroke="black" d="M218.519,-132.939C212.168,-147.318 202.736,-168.673 194.103,-188.22"/> <polygon fill="black" stroke="black" points="190.784,-187.071 189.946,-197.632 197.188,-189.899 190.784,-187.071"/> </g> </g> </svg> </ins>


Question:
On Fedora20 I have the following error: [fedora ~]$ sudo kube-proxy.service - Kubernetes Kube-Proxy Server Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled) Active: active (running) since mer 2014-10-29 15:15:21 UTC; 24ms ago Docs: Main PID: (kube-proxy) CGroup: /usr/bin/kube-proxy --logtostderr=true --v=0 --etcdservers=http://fed-master:4001 ott 29 15:15:21 fed-minion systemd[1]: Started Kubernetes Kube-Proxy Server. ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: get registry/services/specs...:4001] ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: Connecting to etcd: attempt...=false ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: http://fed- GET ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: recv.response.fromhttp://fe...=false ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: Hint: Some lines were ellipsized, use -l to show in full. kubelet.service - Kubernetes Kubelet Server Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled) Active: activating (auto-restart) (Result: exit-code) since mer 2014-10-29 15:15:21 UTC; 29ms ago Docs: Main PID: (code=exited, status=2) ott 29 15:15:21 fed-minion systemd[1]: kubelet.service: main process exited, code=exited, status=2/INVALIDARGUMENT ott 29 15:15:21 fed-minion systemd[1]: Unit kubelet.service entered failed state. docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled) Active: active (running) since mer 2014-10-29 15:15:21 UTC; 28ms ago Docs: Main PID: (docker) CGroup: /usr/bin/docker -d -H fd:// --selinux-enabled ott 29 15:15:21 fed-minion docker[]: 2014/10/29 15:15:21 docker daemon: 1.3.0 /1.3.0; execdriver: native; graphdriver: ott 29 15:15:21 fed-minion docker[]: [] +job serveapi(fd://) ott 29 15:15:21 fed-minion docker[]: [info] Listening for HTTP on fd () ott 29 15:15:21 fed-minion docker[]: [] +job initnetworkdriver() ott 29 15:15:21 fed-minion docker[]: [] -job init_networkdriver() = OK (0) ott 29 15:15:21 fed-minion docker[]: [info] Loading containers: ott 29 15:15:21 fed-minion docker[]: [info] : done. ott 29 15:15:21 fed-minion docker[]: [] +job acceptconnections() ott 29 15:15:21 fed-minion docker[]: [] -job acceptconnections() = OK (0) ott 29 15:15:21 fed-minion systemd[1]: Started Docker Application Container Engine. When I run netstat -tulnp (cadvisor)" I don't see anything in output.
Exit code of 2 implies glog.Fatalf to me. What are the contents of the logs for kubelet and kube-proxy? What was the command line of kubelet?
Please reopen if you have more details.

<ins> # Maintainers Eric Paris <eparis@redhat.com> </ins>


Question:
On Fedora20 I have the following error: [fedora ~]$ sudo kube-proxy.service - Kubernetes Kube-Proxy Server Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled) Active: active (running) since mer 2014-10-29 15:15:21 UTC; 24ms ago Docs: Main PID: (kube-proxy) CGroup: /usr/bin/kube-proxy --logtostderr=true --v=0 --etcdservers=http://fed-master:4001 ott 29 15:15:21 fed-minion systemd[1]: Started Kubernetes Kube-Proxy Server. ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: get registry/services/specs...:4001] ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: Connecting to etcd: attempt...=false ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: http://fed- GET ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: recv.response.fromhttp://fe...=false ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: Hint: Some lines were ellipsized, use -l to show in full. kubelet.service - Kubernetes Kubelet Server Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled) Active: activating (auto-restart) (Result: exit-code) since mer 2014-10-29 15:15:21 UTC; 29ms ago Docs: Main PID: (code=exited, status=2) ott 29 15:15:21 fed-minion systemd[1]: kubelet.service: main process exited, code=exited, status=2/INVALIDARGUMENT ott 29 15:15:21 fed-minion systemd[1]: Unit kubelet.service entered failed state. docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled) Active: active (running) since mer 2014-10-29 15:15:21 UTC; 28ms ago Docs: Main PID: (docker) CGroup: /usr/bin/docker -d -H fd:// --selinux-enabled ott 29 15:15:21 fed-minion docker[]: 2014/10/29 15:15:21 docker daemon: 1.3.0 /1.3.0; execdriver: native; graphdriver: ott 29 15:15:21 fed-minion docker[]: [] +job serveapi(fd://) ott 29 15:15:21 fed-minion docker[]: [info] Listening for HTTP on fd () ott 29 15:15:21 fed-minion docker[]: [] +job initnetworkdriver() ott 29 15:15:21 fed-minion docker[]: [] -job init_networkdriver() = OK (0) ott 29 15:15:21 fed-minion docker[]: [info] Loading containers: ott 29 15:15:21 fed-minion docker[]: [info] : done. ott 29 15:15:21 fed-minion docker[]: [] +job acceptconnections() ott 29 15:15:21 fed-minion docker[]: [] -job acceptconnections() = OK (0) ott 29 15:15:21 fed-minion systemd[1]: Started Docker Application Container Engine. When I run netstat -tulnp (cadvisor)" I don't see anything in output.
Exit code of 2 implies glog.Fatalf to me. What are the contents of the logs for kubelet and kube-proxy? What was the command line of kubelet?
Please reopen if you have more details.

<ins> #!bash # # bash completion file for core kubecfg commands # # This script provides completion of non replication controller options # # To enable the completions either: #  - place this file in /etc/bash_completion.d #  or #  - copy this file and add the line below to your .bashrc after #    bash completion features are loaded #     . kubecfg # # Note: # Currently, the completions will not work if the apiserver daemon is not # running on localhost on the standard port 8080 __contains_word () { local w word=$1; shift for w in "$@"; do [[ $w = "$word" ]] && return done return 1 } # This should be provided by the bash-completions, but give a really simple # stoopid version just in case.  It works most of the time. if ! declare -F _get_comp_words_by_ref >/dev/null 2>&1; then _get_comp_words_by_ref () { while [ $# -gt 0 ]; do case "$1" in cur) cur=${COMP_WORDS[COMP_CWORD]} ;; prev) prev=${COMP_WORDS[COMP_CWORD-1]} ;; words) words=("${COMP_WORDS[@]}") ;; cword) cword=$COMP_CWORD ;; -n) shift # we don't handle excludes ;; esac shift done } fi __has_service() { local i for ((i=0; i < cword; i++)); do local word=${words[i]} # strip everything after a / so things like pods/[id] match word=${word%%/*} if __contains_word "${word}" "${services[@]}" && ! __contains_word "${words[i-1]}" "${opts[@]}"; then return 0 fi done return 1 } # call kubecfg list $1, # exclude blank lines # skip the header stuff kubecfg prints on the first 2 lines # append $1/ to the first column and use that in compgen __kubecfg_parse_list() { local kubecfg_output if kubecfg_output=$(kubecfg list "$1" 2>/dev/null); then out=($(echo "${kubecfg_output}" | awk -v prefix="$1" '/^$/ {next} NR > 2 {print prefix"/"$1}')) COMPREPLY=( $( compgen -W "${out[*]}" -- "$cur" ) ) fi } _kubecfg_specific_service_match() { case "$cur" in pods/*) __kubecfg_parse_list pods ;; minions/*) __kubecfg_parse_list minions ;; replicationControllers/*) __kubecfg_parse_list replicationControllers ;; services/*) __kubecfg_parse_list services ;; *) if __has_service; then return 0 fi compopt -o nospace COMPREPLY=( $( compgen -S / -W "${services[*]}" -- "$cur" ) ) ;; esac } _kubecfg_service_match() { if __has_service; then return 0 fi COMPREPLY=( $( compgen -W "${services[*]}" -- "$cur" ) ) } _kubecfg() { local opts=( -h -c ) local create_services=(pods replicationControllers services) local update_services=(replicationControllers) local all_services=(pods replicationControllers services minions) local services=("${all_services[@]}") local json_commands=(create update) local all_commands=(create update get list delete stop rm rollingupdate resize) local commands=("${all_commands[@]}") COMPREPLY=() local command local cur prev words cword _get_comp_words_by_ref -n : cur prev words cword if __contains_word "$prev" "${opts[@]}"; then case $prev in -c) _filedir '@(json|yml|yaml)' return 0 ;; -h) return 0 ;; esac fi if [[ "$cur" = -* ]]; then COMPREPLY=( $(compgen -W "${opts[*]}" -- "$cur") ) return 0 fi # if you passed -c, you are limited to create or update if __contains_word "-c" "${words[@]}"; then services=("${create_services[@]}" "${update_services[@]}") commands=("${json_commands[@]}") fi # figure out which command they are running, remembering that arguments to # options don't count as the command!  So a hostname named 'create' won't # trip things up local i for ((i=0; i < cword; i++)); do if __contains_word "${words[i]}" "${commands[@]}" && ! __contains_word "${words[i-1]}" "${opts[@]}"; then command=${words[i]} break fi done # tell the list of possible commands if [[ -z ${command} ]]; then COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) ) return 0 fi # remove services which you can't update given your command if [[ ${command} == "create" ]]; then services=("${create_services[@]}") elif [[ ${command} == "update" ]]; then services=("${update_services[@]}") fi case $command in create | list) _kubecfg_service_match ;; update | get | delete) _kubecfg_specific_service_match ;; *) ;; esac return 0 } complete -F _kubecfg kubecfg # ex: ts=4 sw=4 et filetype=sh </ins>


Question:
On Fedora20 I have the following error: [fedora ~]$ sudo kube-proxy.service - Kubernetes Kube-Proxy Server Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled) Active: active (running) since mer 2014-10-29 15:15:21 UTC; 24ms ago Docs: Main PID: (kube-proxy) CGroup: /usr/bin/kube-proxy --logtostderr=true --v=0 --etcdservers=http://fed-master:4001 ott 29 15:15:21 fed-minion systemd[1]: Started Kubernetes Kube-Proxy Server. ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: get registry/services/specs...:4001] ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: Connecting to etcd: attempt...=false ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: http://fed- GET ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: recv.response.fromhttp://fe...=false ott 29 15:15:21 fed-minion kube-proxy[]: I1029 15:15:21. ] etcd DEBUG: Hint: Some lines were ellipsized, use -l to show in full. kubelet.service - Kubernetes Kubelet Server Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled) Active: activating (auto-restart) (Result: exit-code) since mer 2014-10-29 15:15:21 UTC; 29ms ago Docs: Main PID: (code=exited, status=2) ott 29 15:15:21 fed-minion systemd[1]: kubelet.service: main process exited, code=exited, status=2/INVALIDARGUMENT ott 29 15:15:21 fed-minion systemd[1]: Unit kubelet.service entered failed state. docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled) Active: active (running) since mer 2014-10-29 15:15:21 UTC; 28ms ago Docs: Main PID: (docker) CGroup: /usr/bin/docker -d -H fd:// --selinux-enabled ott 29 15:15:21 fed-minion docker[]: 2014/10/29 15:15:21 docker daemon: 1.3.0 /1.3.0; execdriver: native; graphdriver: ott 29 15:15:21 fed-minion docker[]: [] +job serveapi(fd://) ott 29 15:15:21 fed-minion docker[]: [info] Listening for HTTP on fd () ott 29 15:15:21 fed-minion docker[]: [] +job initnetworkdriver() ott 29 15:15:21 fed-minion docker[]: [] -job init_networkdriver() = OK (0) ott 29 15:15:21 fed-minion docker[]: [info] Loading containers: ott 29 15:15:21 fed-minion docker[]: [info] : done. ott 29 15:15:21 fed-minion docker[]: [] +job acceptconnections() ott 29 15:15:21 fed-minion docker[]: [] -job acceptconnections() = OK (0) ott 29 15:15:21 fed-minion systemd[1]: Started Docker Application Container Engine. When I run netstat -tulnp (cadvisor)" I don't see anything in output.
Exit code of 2 implies glog.Fatalf to me. What are the contents of the logs for kubelet and kube-proxy? What was the command line of kubelet?
Please reopen if you have more details.

<del> #!bash # # bash completion file for core kubecfg commands # # This script provides completion of non replication controller options # # To enable the completions either: #  - place this file in /etc/bash_completion.d #  or #  - copy this file and add the line below to your .bashrc after #    bash completion features are loaded #     . kubecfg # # Note: # Currently, the completions will not work if the apiserver daemon is not # running on localhost on the standard port 8080 __contains_word () { local w word=$1; shift for w in "$@"; do [[ $w = "$word" ]] && return done return 1 } # This should be provided by the bash-completions, but give a really simple # stoopid version just in case.  It works most of the time. if ! declare -F _get_comp_words_by_ref >/dev/null 2>&1; then _get_comp_words_by_ref () { while [ $# -gt 0 ]; do case "$1" in cur) cur=${COMP_WORDS[COMP_CWORD]} ;; prev) prev=${COMP_WORDS[COMP_CWORD-1]} ;; words) words=("${COMP_WORDS[@]}") ;; cword) cword=$COMP_CWORD ;; -n) shift # we don't handle excludes ;; esac shift done } fi __has_service() { local i for ((i=0; i < cword; i++)); do local word=${words[i]} # strip everything after a / so things like pods/[id] match word=${word%%/*} if __contains_word "${word}" "${services[@]}" && ! __contains_word "${words[i-1]}" "${opts[@]}"; then return 0 fi done return 1 } # call kubecfg list $1, # exclude blank lines # skip the header stuff kubecfg prints on the first 2 lines # append $1/ to the first column and use that in compgen __kubecfg_parse_list() { local kubecfg_output if kubecfg_output=$(kubecfg list "$1" 2>/dev/null); then out=($(echo "${kubecfg_output}" | awk -v prefix="$1" '/^$/ {next} NR > 2 {print prefix"/"$1}')) COMPREPLY=( $( compgen -W "${out[*]}" -- "$cur" ) ) fi } _kubecfg_specific_service_match() { case "$cur" in pods/*) __kubecfg_parse_list pods ;; minions/*) __kubecfg_parse_list minions ;; replicationControllers/*) __kubecfg_parse_list replicationControllers ;; services/*) __kubecfg_parse_list services ;; *) if __has_service; then return 0 fi compopt -o nospace COMPREPLY=( $( compgen -S / -W "${services[*]}" -- "$cur" ) ) ;; esac } _kubecfg_service_match() { if __has_service; then return 0 fi COMPREPLY=( $( compgen -W "${services[*]}" -- "$cur" ) ) } _kubecfg() { local opts=( -h -c ) local create_services=(pods replicationControllers services) local update_services=(replicationControllers) local all_services=(pods replicationControllers services minions) local services=("${all_services[@]}") local json_commands=(create update) local all_commands=(create update get list delete stop rm rollingupdate resize) local commands=("${all_commands[@]}") COMPREPLY=() local command local cur prev words cword _get_comp_words_by_ref -n : cur prev words cword if __contains_word "$prev" "${opts[@]}"; then case $prev in -c) _filedir '@(json|yml|yaml)' return 0 ;; -h) return 0 ;; esac fi if [[ "$cur" = -* ]]; then COMPREPLY=( $(compgen -W "${opts[*]}" -- "$cur") ) return 0 fi # if you passed -c, you are limited to create or update if __contains_word "-c" "${words[@]}"; then services=("${create_services[@]}" "${update_services[@]}") commands=("${json_commands[@]}") fi # figure out which command they are running, remembering that arguments to # options don't count as the command!  So a hostname named 'create' won't # trip things up local i for ((i=0; i < cword; i++)); do if __contains_word "${words[i]}" "${commands[@]}" && ! __contains_word "${words[i-1]}" "${opts[@]}"; then command=${words[i]} break fi done # tell the list of possible commands if [[ -z ${command} ]]; then COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) ) return 0 fi # remove services which you can't update given your command if [[ ${command} == "create" ]]; then services=("${create_services[@]}") elif [[ ${command} == "update" ]]; then services=("${update_services[@]}") fi case $command in create | list) _kubecfg_service_match ;; update | get | delete) _kubecfg_specific_service_match ;; *) ;; esac return 0 } complete -F _kubecfg kubecfg # ex: ts=4 sw=4 et filetype=sh </del>


Question:
introduces PullPolicies (PullAlways, PullNever, PullIfNotPresent), but by default, the policy is PullAlways, which bite docker repository again today. The related #google-containers IRC conversation can be found at: We should Introduce a new policy to cap the number of attempts. Maybe getting rid of PullAlways completely? cc/
I would like to change the default one to PullIfNoPresent for now. We can continue discussing the right policies.
Suggestions: PullIfNotPresent should be our default? PullAlways needs to be nerfed. Suggestion: max N pull attempts per pod config change, with <attempt#*10 second delay between each pull attempt, where N is a small number like 3 or 5.
I think both PullIfNotPresent and PullAlways (if we decide to still have it) should be nerfed.
Agreed, when "not present", we should behave the same as PullAlways, which should be nerfed.
Not finding another issue, so recording here for posterity: we really need a TTL on images to make PullIfNotPresent maximally useful.
So if a registry is down and the cluster restarts, pods will eventually give up (if we're nerfing pull always)? Seems like we should drastically reduce pull frequency after failures, but giving up means you can get into weird various scenarios. Esp if the registry is hosted in the cluster. Why not just back off?
I vote for some exponential back off. , Clayton Coleman wrote:
We think we do have backoff now, check and close it if so.

{% set auth_path = "-auth_path=/var/lib/kubelet/kubernetes_auth" -%} <ins> {% set registry_qps = "-registry_qps=0.1" %} </ins> DAEMON_ARGS="{{daemon_args}} {{etcd_servers}} {{apiservers}} {{auth_path}} {{hostname_override}} {{address}} {{config}} --allow_privileged={{pillar['allow_privileged']}}"


Question:
From kubelet logs: E1117 23:17:17. ] Sleeping: Unable to write event: event "e2e-test-etune-minion-1.c.seventh-circle-487.internal." is invalid: involvedObject.namespace: invalid value '' The kubelet code says: I'll update when I figure out where the record.Event is coming from. The code is:
Event in question is: I1117 23:25:42. ] Event(api.ObjectReference{Kind:"Minion", Namespace:"", Name:"e2e-test-etune-minion-1.c.seventh-circle-487.internal", UID:"e2e-test-etune-minion-1.c.seventh-circle-487.internal", APIVersion:"", ResourceVersion:"", FieldPath:""}): status: '', reason: 'starting' Starting kubelet.
I think the namespace needs to be kubelet. Note that one bad event blocks other good events from getting sent. Probably better to change that.
I'll file a new one for bad event blocking good.
I have a similar issue related to event. When generating event for image, what kind of namespace I should use? Image is not APIObject kubernetes managed, and it could shared crossing several containers even belongs to different pod. Currently I plan to just use default when creating ObjectReference for image related event. If we use kubelet for event related to minion, are we going to make it reserved one?
what events are you planning to add for images? "default" seems like a good enough namespace for minion events for now.

// TODO: get the real minion object of ourself, // and use the real minion name and UID. ref := &api.ObjectReference{ <del> Kind: "Minion", Name: kl.hostname, UID:  kl.hostname, </del> <ins> Kind:      "Minion", Name:      kl.hostname, UID:       kl.hostname, Namespace: api.NamespaceDefault, </ins> } record.Eventf(ref, "", "starting", "Starting kubelet.") }


Question:
For example, creating a service with id 'redis-master10012345678999' fails and the returned message contains the following: "message": "service "redis-master10012345678999" is invalid: name: invalid value 'redis-master10012345678999' 'invalid value' is too general, users should understand exactly what is wrong with the input. for example - is it too long? does it contain characters that are not allowed?
Agree - there is now a place to stick the explanations in the ValidationError - we need to link together validation logic and that string. , abonas wrote:
I checked and it seems there are only two errors that do not have detailed explanation. I'll send a PR to fix this.
This is one:
Thanks for the pointer Brian! I'll send second PR to fix this one.
Still second PR to be sent.

if !util.IsQualifiedName(strings.ToLower(k)) { allErrs = append(allErrs, errs.NewFieldInvalid(field, k, qualifiedNameErrorMsg)) } <del> if !util.IsValidAnnotationValue(v) { allErrs = append(allErrs, errs.NewFieldInvalid(field, k, "")) } </del> totalSize += (int64)(len(k)) + (int64)(len(v)) } if totalSize > (int64)(totalAnnotationSizeLimitB) { <del> allErrs = append(allErrs, errs.NewFieldTooLong("annotations", "")) </del> <ins> allErrs = append(allErrs, errs.NewFieldTooLong(field, "", totalAnnotationSizeLimitB)) </ins> } return allErrs }


Question:
For example, creating a service with id 'redis-master10012345678999' fails and the returned message contains the following: "message": "service "redis-master10012345678999" is invalid: name: invalid value 'redis-master10012345678999' 'invalid value' is too general, users should understand exactly what is wrong with the input. for example - is it too long? does it contain characters that are not allowed?
Agree - there is now a place to stick the explanations in the ValidationError - we need to link together validation logic and that string. , abonas wrote:
I checked and it seems there are only two errors that do not have detailed explanation. I'll send a PR to fix this.
This is one:
Thanks for the pointer Brian! I'll send second PR to fix this one.
Still second PR to be sent.

func (v *ValidationError) Error() string { var s string switch v.Type { <del> case ValidationErrorTypeRequired: </del> <ins> case ValidationErrorTypeRequired, ValidationErrorTypeTooLong: </ins> s = spew.Sprintf("%s: %s", v.Field, v.Type) default: s = spew.Sprintf("%s: %s '%+v'", v.Field, v.Type, v.BadValue)


Question:
For example, creating a service with id 'redis-master10012345678999' fails and the returned message contains the following: "message": "service "redis-master10012345678999" is invalid: name: invalid value 'redis-master10012345678999' 'invalid value' is too general, users should understand exactly what is wrong with the input. for example - is it too long? does it contain characters that are not allowed?
Agree - there is now a place to stick the explanations in the ValidationError - we need to link together validation logic and that string. , abonas wrote:
I checked and it seems there are only two errors that do not have detailed explanation. I'll send a PR to fix this.
This is one:
Thanks for the pointer Brian! I'll send second PR to fix this one.
Still second PR to be sent.

return &ValidationError{ValidationErrorTypeNotFound, field, value, ""} } <del> func NewFieldTooLong(field string, value interface{}) *ValidationError { return &ValidationError{ValidationErrorTypeTooLong, field, value, ""} </del> <ins> func NewFieldTooLong(field string, value interface{}, maxLength int) *ValidationError { return &ValidationError{ValidationErrorTypeTooLong, field, value, fmt.Sprintf("must have at most %d characters", maxLength)} </ins> } type ValidationErrorList []error


Question:
For example, creating a service with id 'redis-master10012345678999' fails and the returned message contains the following: "message": "service "redis-master10012345678999" is invalid: name: invalid value 'redis-master10012345678999' 'invalid value' is too general, users should understand exactly what is wrong with the input. for example - is it too long? does it contain characters that are not allowed?
Agree - there is now a place to stick the explanations in the ValidationError - we need to link together validation logic and that string. , abonas wrote:
I checked and it seems there are only two errors that do not have detailed explanation. I'll send a PR to fix this.
This is one:
Thanks for the pointer Brian! I'll send second PR to fix this one.
Still second PR to be sent.

return (len(value) <= LabelValueMaxLength && labelValueRegexp.MatchString(value)) } <del> // Annotation values are opaque. func IsValidAnnotationValue(value string) bool { return true } </del> const QualifiedNameFmt string = "(" + qnameTokenFmt + "/)?" + qnameTokenFmt const QualifiedNameMaxLength int = 253


Question:
For example, creating a service with id 'redis-master10012345678999' fails and the returned message contains the following: "message": "service "redis-master10012345678999" is invalid: name: invalid value 'redis-master10012345678999' 'invalid value' is too general, users should understand exactly what is wrong with the input. for example - is it too long? does it contain characters that are not allowed?
Agree - there is now a place to stick the explanations in the ValidationError - we need to link together validation logic and that string. , abonas wrote:
I checked and it seems there are only two errors that do not have detailed explanation. I'll send a PR to fix this.
This is one:
Thanks for the pointer Brian! I'll send second PR to fix this one.
Still second PR to be sent.

} pod.Spec.Containers = newContainers if !api.Semantic.DeepEqual(pod.Spec, oldPod.Spec) { <del> // TODO: a better error would include all immutable fields explicitly. allErrs = append(allErrs, errs.NewFieldInvalid("spec.containers", newPod.Spec.Containers, "some fields are immutable")) </del> <ins> allErrs = append(allErrs, errs.NewFieldInvalid("spec", newPod.Spec, "may not update fields other than container.image")) </ins> } newPod.Status = oldPod.Status


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

if err != nil { glog.Errorf("Unable to get pod with name %q and uid %q info, health checks may be invalid", podFullName, uid) } <del> netInfo, found := podStatus.Info[dockertools.PodInfraContainerName] if found { podStatus.PodIP = netInfo.PodIP } </del> for _, container := range pod.Spec.Containers { expectedHash := dockertools.HashContainer(&container)


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

return nil, false } <ins> // getPhase returns the phase of a pod given its container info. func getPhase(spec *api.PodSpec, info api.PodInfo) api.PodPhase { if info == nil { return api.PodPending } running := 0 waiting := 0 stopped := 0 failed := 0 succeeded := 0 unknown := 0 for _, container := range spec.Containers { if containerStatus, ok := info[container.Name]; ok { if containerStatus.State.Running != nil { running++ } else if containerStatus.State.Termination != nil { stopped++ if containerStatus.State.Termination.ExitCode == 0 { succeeded++ } else { failed++ } } else if containerStatus.State.Waiting != nil { waiting++ } else { unknown++ } } else { unknown++ } } switch { case waiting > 0: // One or more containers has not been started return api.PodPending case running > 0 && unknown == 0: // All containers have been started, and at least // one container is running return api.PodRunning case running == 0 && stopped > 0 && unknown == 0: // All containers are terminated if spec.RestartPolicy.Always != nil { // All containers are in the process of restarting return api.PodRunning } if stopped == succeeded { // RestartPolicy is not Always, and all // containers are terminated in success return api.PodSucceeded } if spec.RestartPolicy.Never != nil { // RestartPolicy is Never, and all containers are // terminated with at least one in failure return api.PodFailed } // RestartPolicy is OnFailure, and at least one in failure // and in the process of restarting return api.PodRunning default: return api.PodPending } } </ins> // GetPodStatus returns information from Docker about the containers in a pod func (kl *Kubelet) GetPodStatus(podFullName string, uid types.UID) (api.PodStatus, error) { <del> var manifest api.PodSpec </del> <ins> var spec api.PodSpec </ins> for _, pod := range kl.pods { if GetPodFullName(&pod) == podFullName { <del> manifest = pod.Spec </del> <ins> spec = pod.Spec </ins> break } } <del> info, err := dockertools.GetDockerPodInfo(kl.dockerClient, manifest, podFullName, uid) </del> <ins> info, err := dockertools.GetDockerPodInfo(kl.dockerClient, spec, podFullName, uid) </ins> <del> // TODO(dchen1107): Determine PodPhase here </del> var podStatus api.PodStatus <ins> podStatus.Phase = getPhase(&spec, info) netContainerInfo, found := info[dockertools.PodInfraContainerName] if found { podStatus.PodIP = netContainerInfo.PodIP } // TODO(dchen1107): Change Info to list from map </ins> podStatus.Info = info return podStatus, err


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

} } } <ins> func TestPodPhaseWithRestartAlways(t *testing.T) { desiredState := api.PodSpec{ Containers: []api.Container{ {Name: "containerA"}, {Name: "containerB"}, }, RestartPolicy: api.RestartPolicy{Always: &api.RestartPolicyAlways{}}, } currentState := api.PodStatus{ Host: "machine", } runningState := api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, } stoppedState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{}, }, } tests := []struct { pod    *api.Pod status api.PodPhase test   string }{ {&api.Pod{Spec: desiredState, Status: currentState}, api.PodPending, "waiting"}, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": runningState, }, Host: "machine", }, }, api.PodRunning, "all running", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": stoppedState, "containerB": stoppedState, }, Host: "machine", }, }, api.PodRunning, "all stopped with restart always", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": stoppedState, }, Host: "machine", }, }, api.PodRunning, "mixed state #1 with restart always", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, }, Host: "machine", }, }, api.PodPending, "mixed state #2 with restart always", }, } for _, test := range tests { if status := getPhase(&test.pod.Spec, test.pod.Status.Info); status != test.status { t.Errorf("In test %s, expected %v, got %v", test.test, test.status, status) } } } func TestPodPhaseWithRestartNever(t *testing.T) { desiredState := api.PodSpec{ Containers: []api.Container{ {Name: "containerA"}, {Name: "containerB"}, }, RestartPolicy: api.RestartPolicy{Never: &api.RestartPolicyNever{}}, } currentState := api.PodStatus{ Host: "machine", } runningState := api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, } succeededState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: 0, }, }, } failedState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: -1, }, }, } tests := []struct { pod    *api.Pod status api.PodPhase test   string }{ {&api.Pod{Spec: desiredState, Status: currentState}, api.PodPending, "waiting"}, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": runningState, }, Host: "machine", }, }, api.PodRunning, "all running with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": succeededState, "containerB": succeededState, }, Host: "machine", }, }, api.PodSucceeded, "all succeeded with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": failedState, "containerB": failedState, }, Host: "machine", }, }, api.PodFailed, "all failed with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": succeededState, }, Host: "machine", }, }, api.PodRunning, "mixed state #1 with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, }, Host: "machine", }, }, api.PodPending, "mixed state #2 with restart never", }, } for _, test := range tests { if status := getPhase(&test.pod.Spec, test.pod.Status.Info); status != test.status { t.Errorf("In test %s, expected %v, got %v", test.test, test.status, status) } } } func TestPodPhaseWithRestartOnFailure(t *testing.T) { desiredState := api.PodSpec{ Containers: []api.Container{ {Name: "containerA"}, {Name: "containerB"}, }, RestartPolicy: api.RestartPolicy{OnFailure: &api.RestartPolicyOnFailure{}}, } currentState := api.PodStatus{ Host: "machine", } runningState := api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, } succeededState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: 0, }, }, } failedState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: -1, }, }, } tests := []struct { pod    *api.Pod status api.PodPhase test   string }{ {&api.Pod{Spec: desiredState, Status: currentState}, api.PodPending, "waiting"}, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": runningState, }, Host: "machine", }, }, api.PodRunning, "all running with restart onfailure", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": succeededState, "containerB": succeededState, }, Host: "machine", }, }, api.PodSucceeded, "all succeeded with restart onfailure", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": failedState, "containerB": failedState, }, Host: "machine", }, }, api.PodRunning, "all failed with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": succeededState, }, Host: "machine", }, }, api.PodRunning, "mixed state #1 with restart onfailure", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, }, Host: "machine", }, }, api.PodPending, "mixed state #2 with restart onfailure", }, } for _, test := range tests { if status := getPhase(&test.pod.Spec, test.pod.Status.Info); status != test.status { t.Errorf("In test %s, expected %v, got %v", test.test, test.status, status) } } } </ins>


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

"github.com/GoogleCloudPlatform/kubernetes/pkg/api" "github.com/GoogleCloudPlatform/kubernetes/pkg/client" <del> "github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/leaky" </del> "github.com/GoogleCloudPlatform/kubernetes/pkg/labels" "github.com/GoogleCloudPlatform/kubernetes/pkg/registry/pod"


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

newStatus.Phase = api.PodUnknown } else { newStatus.Info = result.Status.Info <del> newStatus.Phase = getPhase(&pod.Spec, newStatus.Info) if netContainerInfo, ok := newStatus.Info[leaky.PodInfraContainerName]; ok { if netContainerInfo.PodIP != "" { newStatus.PodIP = netContainerInfo.PodIP } </del> <ins> newStatus.PodIP = result.Status.PodIP if newStatus.Info == nil { // There is a small race window that kubelet couldn't // propulated the status yet. This should go away once // we removed boundPods newStatus.Phase = api.PodPending } else { newStatus.Phase = result.Status.Phase </ins> } } return newStatus, err


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

} wg.Wait() } <del> // getPhase returns the phase of a pod given its container info. // TODO(dchen1107): push this all the way down into kubelet. func getPhase(spec *api.PodSpec, info api.PodInfo) api.PodPhase { if info == nil { return api.PodPending } running := 0 waiting := 0 stopped := 0 failed := 0 succeeded := 0 unknown := 0 for _, container := range spec.Containers { if containerStatus, ok := info[container.Name]; ok { if containerStatus.State.Running != nil { running++ } else if containerStatus.State.Termination != nil { stopped++ if containerStatus.State.Termination.ExitCode == 0 { succeeded++ } else { failed++ } } else if containerStatus.State.Waiting != nil { waiting++ } else { unknown++ } } else { unknown++ } } switch { case waiting > 0: // One or more containers has not been started return api.PodPending case running > 0 && unknown == 0: // All containers have been started, and at least // one container is running return api.PodRunning case running == 0 && stopped > 0 && unknown == 0: // All containers are terminated if spec.RestartPolicy.Always != nil { // All containers are in the process of restarting return api.PodRunning } if stopped == succeeded { // RestartPolicy is not Always, and all // containers are terminated in success return api.PodSucceeded } if spec.RestartPolicy.Never != nil { // RestartPolicy is Never, and all containers are // terminated with at least one in failure return api.PodFailed } // RestartPolicy is OnFailure, and at least one in failure // and in the process of restarting return api.PodRunning default: return api.PodPending } } </del>


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

"reflect" "sync" "testing" <del> "time" </del> "github.com/GoogleCloudPlatform/kubernetes/pkg/api" "github.com/GoogleCloudPlatform/kubernetes/pkg/client" "github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/leaky" "github.com/GoogleCloudPlatform/kubernetes/pkg/registry/registrytest" <del> "github.com/GoogleCloudPlatform/kubernetes/pkg/util" </del> ) type podInfoCall struct {


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

if status == nil { t.Errorf("Unexpected non-status.") } <del> expected := &api.PodStatus{ Phase:  "Pending", Host:   "machine", HostIP: "1.2.3.5", Info: api.PodInfo{ "bar": api.ContainerStatus{}, }, } if !reflect.DeepEqual(status, expected) { t.Errorf("expected:n%#vngot:n%#vn", expected, status) } </del> } type podCacheTestConfig struct {


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

} } <del> func TestFillPodStatus(t *testing.T) { pod := makePod(api.NamespaceDefault, "foo", "machine", "bar") expectedIP := "1.2.3.4" expectedTime, _ := time.Parse("2013-Feb-03", "2013-Feb-03") config := podCacheTestConfig{ kubeletContainerInfo: api.PodStatus{ Phase:  api.PodPending, Host:   "machine", HostIP: "ip of machine", PodIP:  expectedIP, Info: api.PodInfo{ leaky.PodInfraContainerName: { State: api.ContainerState{ Running: &api.ContainerStateRunning{ StartedAt: util.NewTime(expectedTime), }, }, RestartCount: 1, PodIP:        expectedIP, }, }, }, nodes: []api.Node{*makeHealthyNode("machine", "ip of machine")}, pods:  []api.Pod{*pod}, } cache := config.Construct() err := cache.updatePodStatus(&config.pods[0]) if err != nil { t.Fatalf("Unexpected error: %+v", err) } status, err := cache.GetPodStatus(pod.Namespace, pod.Name) if e, a := &config.kubeletContainerInfo, status; !reflect.DeepEqual(e, a) { t.Errorf("Expected: %+v, Got %+v", e, a) } } </del> func TestFillPodInfoNoData(t *testing.T) { pod := makePod(api.NamespaceDefault, "foo", "machine", "bar") expectedIP := ""


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

} } <del> func TestPodPhaseWithRestartAlways(t *testing.T) { desiredState := api.PodSpec{ Containers: []api.Container{ {Name: "containerA"}, {Name: "containerB"}, }, RestartPolicy: api.RestartPolicy{Always: &api.RestartPolicyAlways{}}, } currentState := api.PodStatus{ Host: "machine", } runningState := api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, } stoppedState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{}, }, } tests := []struct { pod    *api.Pod status api.PodPhase test   string }{ {&api.Pod{Spec: desiredState, Status: currentState}, api.PodPending, "waiting"}, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": runningState, }, Host: "machine", }, }, api.PodRunning, "all running", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": stoppedState, "containerB": stoppedState, }, Host: "machine", }, }, api.PodRunning, "all stopped with restart always", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": stoppedState, }, Host: "machine", }, }, api.PodRunning, "mixed state #1 with restart always", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, }, Host: "machine", }, }, api.PodPending, "mixed state #2 with restart always", }, } for _, test := range tests { if status := getPhase(&test.pod.Spec, test.pod.Status.Info); status != test.status { t.Errorf("In test %s, expected %v, got %v", test.test, test.status, status) } } } func TestPodPhaseWithRestartNever(t *testing.T) { desiredState := api.PodSpec{ Containers: []api.Container{ {Name: "containerA"}, {Name: "containerB"}, }, RestartPolicy: api.RestartPolicy{Never: &api.RestartPolicyNever{}}, } currentState := api.PodStatus{ Host: "machine", } runningState := api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, } succeededState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: 0, }, }, } failedState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: -1, }, }, } tests := []struct { pod    *api.Pod status api.PodPhase test   string }{ {&api.Pod{Spec: desiredState, Status: currentState}, api.PodPending, "waiting"}, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": runningState, }, Host: "machine", }, }, api.PodRunning, "all running with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": succeededState, "containerB": succeededState, }, Host: "machine", }, }, api.PodSucceeded, "all succeeded with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": failedState, "containerB": failedState, }, Host: "machine", }, }, api.PodFailed, "all failed with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": succeededState, }, Host: "machine", }, }, api.PodRunning, "mixed state #1 with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, }, Host: "machine", }, }, api.PodPending, "mixed state #2 with restart never", }, } for _, test := range tests { if status := getPhase(&test.pod.Spec, test.pod.Status.Info); status != test.status { t.Errorf("In test %s, expected %v, got %v", test.test, test.status, status) } } } func TestPodPhaseWithRestartOnFailure(t *testing.T) { desiredState := api.PodSpec{ Containers: []api.Container{ {Name: "containerA"}, {Name: "containerB"}, }, RestartPolicy: api.RestartPolicy{OnFailure: &api.RestartPolicyOnFailure{}}, } currentState := api.PodStatus{ Host: "machine", } runningState := api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, } succeededState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: 0, }, }, } failedState := api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: -1, }, }, } tests := []struct { pod    *api.Pod status api.PodPhase test   string }{ {&api.Pod{Spec: desiredState, Status: currentState}, api.PodPending, "waiting"}, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": runningState, }, Host: "machine", }, }, api.PodRunning, "all running with restart onfailure", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": succeededState, "containerB": succeededState, }, Host: "machine", }, }, api.PodSucceeded, "all succeeded with restart onfailure", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": failedState, "containerB": failedState, }, Host: "machine", }, }, api.PodRunning, "all failed with restart never", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, "containerB": succeededState, }, Host: "machine", }, }, api.PodRunning, "mixed state #1 with restart onfailure", }, { &api.Pod{ Spec: desiredState, Status: api.PodStatus{ Info: map[string]api.ContainerStatus{ "containerA": runningState, }, Host: "machine", }, }, api.PodPending, "mixed state #2 with restart onfailure", }, } for _, test := range tests { if status := getPhase(&test.pod.Spec, test.pod.Status.Info); status != test.status { t.Errorf("In test %s, expected %v, got %v", test.test, test.status, status) } } } </del> func TestGarbageCollection(t *testing.T) { pod1 := makePod(api.NamespaceDefault, "foo", "machine", "bar") pod2 := makePod(api.NamespaceDefault, "baz", "machine", "qux")


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

m := make(api.PodInfo) for k, v := range r.Status.Info { v.Ready = true <del> v.PodIP = "1.2.3.4" </del> m[k] = v } r.Status.Info = m


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

return func() (bool, error) { endpoints, err := c.Endpoints(serviceNamespace).Get(serviceID) if err != nil { <ins> glog.Infof("Error on creating endpoints: %v", err) </ins> return false, nil } <ins> glog.Infof("endpoints: %v", endpoints.Endpoints) </ins> return len(endpoints.Endpoints) == endpointCount, nil } }


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

return &containerStatus, nil } <del> // GetDockerPodInfo returns docker info for all containers in the pod/manifest. </del> <ins> // GetDockerPodInfo returns docker info for all containers in the pod/manifest and // infrastructure container </ins> func GetDockerPodInfo(client DockerInterface, manifest api.PodSpec, podFullName string, uid types.UID) (api.PodInfo, error) { info := api.PodInfo{} expectedContainers := make(map[string]api.Container)


Question:
Kubelet now only reports ContainerStatus. PodStatus / PodCondition is observed at master side based on a set of ContainerStatus. There is no other way to get PodIP except net container for master side. This is very confusion to the end user, , , ... Now we are finalizing on v1beta3 API, including PodStatus & ContainerStatus. We can make net container completely invisible to the end users by finishing the following steps: 1) Expose network related errors at API level 2) Generate PodStaus directly from Kubelet
3) once PodStatus generated by kubelet, PodIP can be removed from ContainerStatus.
I think reporting PodStatus rather than just container status is most important. How hard is that?
Used to be hard since there are many dependencies when I first report Status back. Now it is totally feasible. I already started to write PRs for this. :-)
One more PR required to close this one.

// Assume info is ready to process podStatus.Phase = getPhase(spec, info) <ins> podStatus.Info = api.PodInfo{} </ins> for _, c := range spec.Containers { containerStatus := info[c.Name] containerStatus.Ready = kl.readiness.IsReady(containerStatus) <del> info[c.Name] = containerStatus </del> <ins> podStatus.Info[c.Name] = containerStatus </ins> } <del> podStatus.Conditions = append(podStatus.Conditions, getPodReadyCondition(spec, info)...) </del> <ins> podStatus.Conditions = append(podStatus.Conditions, getPodReadyCondition(spec, podStatus.Info)...) </ins> netContainerInfo, found := info[dockertools.PodInfraContainerName] if found { podStatus.PodIP = netContainerInfo.PodIP } <del> // TODO(dchen1107): Change Info to list from map podStatus.Info = info </del> return podStatus, nil }


Question:
In case it isn't clear why this happened, it's because gcloud ships with an older version of kubectl (currently version 0.7.0). We keep it as up to date as we can, but it will always lag a release or so behind head.
Fixed by the revert.

} // Takes a list of strings and compiles them into a list of regular expressions <del> func CompileRegexps(regexpStrings StringList) ([]*regexp.Regexp, error) { </del> <ins> func CompileRegexps(regexpStrings []string) ([]*regexp.Regexp, error) { </ins> regexps := []*regexp.Regexp{} for _, regexpStr := range regexpStrings { r, err := regexp.Compile(regexpStr)


Question:
In case it isn't clear why this happened, it's because gcloud ships with an older version of kubectl (currently version 0.7.0). We keep it as up to date as we can, but it will always lag a release or so behind head.
Fixed by the revert.

t.Errorf("diff returned %v", diff) } } <ins> func TestCompileRegex(t *testing.T) { uncompiledRegexes := []string{"endsWithMe$", "^startingWithMe"} regexes, err := CompileRegexps(uncompiledRegexes) if err != nil { t.Errorf("Failed to compile legal regexes: '%v': %v", uncompiledRegexes, err) } if len(regexes) != len(uncompiledRegexes) { t.Errorf("Wrong number of regexes returned: '%v': %v", uncompiledRegexes, regexes) } if !regexes[0].MatchString("Something that endsWithMe") { t.Errorf("Wrong regex returned: '%v': %v", uncompiledRegexes[0], regexes[0]) } if regexes[0].MatchString("Something that doesn't endsWithMe.") { t.Errorf("Wrong regex returned: '%v': %v", uncompiledRegexes[0], regexes[0]) } if !regexes[1].MatchString("startingWithMe is very important") { t.Errorf("Wrong regex returned: '%v': %v", uncompiledRegexes[1], regexes[1]) } if regexes[1].MatchString("not startingWithMe should fail") { t.Errorf("Wrong regex returned: '%v': %v", uncompiledRegexes[1], regexes[1]) } } </ins>


Question:
In case it isn't clear why this happened, it's because gcloud ships with an older version of kubectl (currently version 0.7.0). We keep it as up to date as we can, but it will always lag a release or so behind head.
Fixed by the revert.

// kubectl command (begining with a space). func kubectlArgs() string { if *checkVersionSkew { <del> return " --match-server-version --v=4" </del> <ins> return " --match-server-version" </ins> } <del> return " --v=4" </del> <ins> return "" </ins> } func bashWrap(cmd string) string {


Question:
I'm trying containervm with kubelet built from head and this YAML file: The YAML file is v1beta2 and doesn't set a name to the Pod. When starting kubelet, it doesn't start the "nc" echo server and complains about this on the logs: Chatting with on IM she mentioned it might be due to a change from that now requires non-empty pod names (replaces random generation with validation for non-emptiness) but that means all existing YAML files (even the ones with an older version) will stop working... Is that expected? Thanks! Filipe
Yes, the latest kubelet call validation on all sources: etcd, file, http, etc. now, and PodName is a required field. One potential fix for this is also invoke conversion code before call validation, like what we have done at apiserver. Thoughts?
Just so I get unblocked... Do you have an example YAML file that will set the pod name and will work? Right now this is what I'm using: Should I bump up version to v1beta3 and include a new field somewhere? Thanks! Filipe
The type of that yaml is not pod, and the kubelet code doesn't know how to read pods from a file. So, I think change needs to be reverted before you can make the next ContainerVM release. The v1beta2 vs v1beta3 should not have anything to do with it, unless I'm misunderstanding.
Or will it work to do this:
Still, not sure if you want to release a containerVM version that doesn't accept the old format?
Thanks I made it work with the YAML above! I still have errors in the logs: But the docker containers are being brought up... It's weird that is the YAML field that is missing, considering the source seems to say it's a deprecated field that will be removed at some point... I agree with your point that this needs to be fixed at head before the next containervm release. I opened to increase test coverage of standalone kubelet since we have been seeing many changes break that model recently. Cheers, Filipe
id in v1beta2 maps to name in v1beta3 Tim's change made name required. The internal representation is like v1beta3. But v1beta3 is not officially stable so we don't want to use it in the GKE or containerVM docs.
The issue is caused by , and affect both v0.8 and v0.9 release for standalone kubelet to support old format ContainerManifests.
Ugh, I thought ID was really required. I spoke with Dawn a bit, and here's my thinking. 1) update all our docs and examples to include ID 2) in file and HTTP config sources, check for Name = "" and apply a default, maybe log a warning that this is deprecated 3) wait for the ContainerManifest -Pod conversion to really make it required

pod.UID = types.UID(hex.EncodeToString(hasher.Sum(nil)[0:])) glog.V(5).Infof("Generated UID %q for pod %q from file %s", pod.UID, pod.Name, filename) } <ins> // This is required for backward compatibility, and should be removed once we // completely deprecate ContainerManifest. if len(pod.Name) == 0 { pod.Name = string(pod.UID) glog.V(5).Infof("Generated Name %q for UID %q from file %s", pod.Name, pod.UID, filename) } </ins> if len(pod.Namespace) == 0 { hasher := adler32.New() fmt.Fprint(hasher, filename)


Question:
I'm trying containervm with kubelet built from head and this YAML file: The YAML file is v1beta2 and doesn't set a name to the Pod. When starting kubelet, it doesn't start the "nc" echo server and complains about this on the logs: Chatting with on IM she mentioned it might be due to a change from that now requires non-empty pod names (replaces random generation with validation for non-emptiness) but that means all existing YAML files (even the ones with an older version) will stop working... Is that expected? Thanks! Filipe
Yes, the latest kubelet call validation on all sources: etcd, file, http, etc. now, and PodName is a required field. One potential fix for this is also invoke conversion code before call validation, like what we have done at apiserver. Thoughts?
Just so I get unblocked... Do you have an example YAML file that will set the pod name and will work? Right now this is what I'm using: Should I bump up version to v1beta3 and include a new field somewhere? Thanks! Filipe
The type of that yaml is not pod, and the kubelet code doesn't know how to read pods from a file. So, I think change needs to be reverted before you can make the next ContainerVM release. The v1beta2 vs v1beta3 should not have anything to do with it, unless I'm misunderstanding.
Or will it work to do this:
Still, not sure if you want to release a containerVM version that doesn't accept the old format?
Thanks I made it work with the YAML above! I still have errors in the logs: But the docker containers are being brought up... It's weird that is the YAML field that is missing, considering the source seems to say it's a deprecated field that will be removed at some point... I agree with your point that this needs to be fixed at head before the next containervm release. I opened to increase test coverage of standalone kubelet since we have been seeing many changes break that model recently. Cheers, Filipe
id in v1beta2 maps to name in v1beta3 Tim's change made name required. The internal representation is like v1beta3. But v1beta3 is not officially stable so we don't want to use it in the GKE or containerVM docs.
The issue is caused by , and affect both v0.8 and v0.9 release for standalone kubelet to support old format ContainerManifests.
Ugh, I thought ID was really required. I spoke with Dawn a bit, and here's my thinking. 1) update all our docs and examples to include ID 2) in file and HTTP config sources, check for Name = "" and apply a default, maybe log a warning that this is deprecated 3) wait for the ContainerManifest -Pod conversion to really make it required

pod.UID = types.UID(hex.EncodeToString(hasher.Sum(nil)[0:])) glog.V(5).Infof("Generated UID %q for pod %q from URL %s", pod.UID, pod.Name, url) } <ins> // This is required for backward compatibility, and should be removed once we // completely deprecate ContainerManifest. if len(pod.Name) == 0 { pod.Name = string(pod.UID) glog.V(5).Infof("Generate Name %q from UID %q from URL %s", pod.Name, pod.UID, url) } </ins> if len(pod.Namespace) == 0 { hasher := adler32.New() fmt.Fprint(hasher, url)


Question:
I'm trying containervm with kubelet built from head and this YAML file: The YAML file is v1beta2 and doesn't set a name to the Pod. When starting kubelet, it doesn't start the "nc" echo server and complains about this on the logs: Chatting with on IM she mentioned it might be due to a change from that now requires non-empty pod names (replaces random generation with validation for non-emptiness) but that means all existing YAML files (even the ones with an older version) will stop working... Is that expected? Thanks! Filipe
Yes, the latest kubelet call validation on all sources: etcd, file, http, etc. now, and PodName is a required field. One potential fix for this is also invoke conversion code before call validation, like what we have done at apiserver. Thoughts?
Just so I get unblocked... Do you have an example YAML file that will set the pod name and will work? Right now this is what I'm using: Should I bump up version to v1beta3 and include a new field somewhere? Thanks! Filipe
The type of that yaml is not pod, and the kubelet code doesn't know how to read pods from a file. So, I think change needs to be reverted before you can make the next ContainerVM release. The v1beta2 vs v1beta3 should not have anything to do with it, unless I'm misunderstanding.
Or will it work to do this:
Still, not sure if you want to release a containerVM version that doesn't accept the old format?
Thanks I made it work with the YAML above! I still have errors in the logs: But the docker containers are being brought up... It's weird that is the YAML field that is missing, considering the source seems to say it's a deprecated field that will be removed at some point... I agree with your point that this needs to be fixed at head before the next containervm release. I opened to increase test coverage of standalone kubelet since we have been seeing many changes break that model recently. Cheers, Filipe
id in v1beta2 maps to name in v1beta3 Tim's change made name required. The internal representation is like v1beta3. But v1beta3 is not officially stable so we don't want to use it in the GKE or containerVM docs.
The issue is caused by , and affect both v0.8 and v0.9 release for standalone kubelet to support old format ContainerManifests.
Ugh, I thought ID was really required. I spoke with Dawn a bit, and here's my thinking. 1) update all our docs and examples to include ID 2) in file and HTTP config sources, check for Name = "" and apply a default, maybe log a warning that this is deprecated 3) wait for the ContainerManifest -Pod conversion to really make it required

}), }, { <ins> desc:      "Single manifest without ID", manifests: api.ContainerManifest{Version: "v1beta1", UUID: "111"}, expected: CreatePodUpdate(kubelet.SET, kubelet.HTTPSource, api.BoundPod{ ObjectMeta: api.ObjectMeta{ UID:       "111", Name:      "111", Namespace: "foobar", }, Spec: api.PodSpec{ RestartPolicy: api.RestartPolicy{Always: &api.RestartPolicyAlways{}}, DNSPolicy:     api.DNSClusterFirst, }, }), }, { </ins> desc: "Multiple manifests", manifests: []api.ContainerManifest{ {Version: "v1beta1", ID: "foo", UUID: "111", Containers: []api.Container{{Name: "1", Image: "foo"}}},