 --- title: kube-apiserver content_type: tool-reference weight: 30 auto_generated: true ---  ## {{% heading "synopsis" %}} The Kubernetes API server validates and configures data for the api objects which include pods, services, replicationcontrollers, and others. The API Server services REST operations and provides the frontend to the cluster's shared state through which all other components interact. ``` kube-apiserver [flags] ``` ## {{% heading "options" %}}
 <colgroup> <col span="1" style="width: 10px;" /> <col span="1" /> </colgroup>

 --add-dir-header

If true, adds the file directory to the header of the log messages

 --admission-control-config-file string

File with admission control configuration.

 --advertise-address string

The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.

 --allow-metric-labels stringToString     Default: []

The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.

 --allow-privileged

If true, allow privileged containers. [default=false]

 --alsologtostderr

log to standard error as well as files

 --anonymous-auth     Default: true

Enables anonymous requests to the secure port of the API server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated.

 --api-audiences strings

Identifiers of the API. The service account token authenticator will validate that tokens used against the API are bound to at least one of these audiences. If the --service-account-issuer flag is configured and this flag is not, this field defaults to a single element list containing the issuer URL.

 --apiserver-count int     Default: 1

The number of apiservers running in the cluster, must be a positive number. (In use when --endpoint-reconciler-type=master-count is enabled.)

 --audit-log-batch-buffer-size int     Default: 10000

The size of the buffer to store events before batching and writing. Only used in batch mode.

 --audit-log-batch-max-size int     Default: 1

The maximum size of a batch. Only used in batch mode.

 --audit-log-batch-max-wait duration

The amount of time to wait before force writing the batch that hadn't reached the max size. Only used in batch mode.

 --audit-log-batch-throttle-burst int

Maximum number of requests sent at the same moment if ThrottleQPS was not utilized before. Only used in batch mode.

 --audit-log-batch-throttle-enable

Whether batching throttling is enabled. Only used in batch mode.

 --audit-log-batch-throttle-qps float

Maximum average number of batches per second. Only used in batch mode.

 --audit-log-compress

If set, the rotated log files will be compressed using gzip.

 --audit-log-format string     Default: "json"

Format of saved audits. "legacy" indicates 1-line text format for each event. "json" indicates structured json format. Known formats are legacy,json.

 --audit-log-maxage int

The maximum number of days to retain old audit log files based on the timestamp encoded in their filename.

 --audit-log-maxbackup int

The maximum number of old audit log files to retain.

 --audit-log-maxsize int

The maximum size in megabytes of the audit log file before it gets rotated.

 --audit-log-mode string     Default: "blocking"

Strategy for sending audit events. Blocking indicates sending events should block server responses. Batch causes the backend to buffer and write events asynchronously. Known modes are batch,blocking,blocking-strict.

 --audit-log-path string

If set, all requests coming to the apiserver will be logged to this file.  '-' means standard out.

 --audit-log-truncate-enabled

Whether event and batch truncating is enabled.

 --audit-log-truncate-max-batch-size int     Default: 10485760

Maximum size of the batch sent to the underlying backend. Actual serialized size can be several hundreds of bytes greater. If a batch exceeds this limit, it is split into several batches of smaller size.

 --audit-log-truncate-max-event-size int     Default: 102400

Maximum size of the audit event sent to the underlying backend. If the size of an event is greater than this number, first request and response are removed, and if this doesn't reduce the size enough, event is discarded.

 --audit-log-version string     Default: "audit.k8s.io/v1"

API group and version used for serializing audit events written to log.

 --audit-policy-file string

Path to the file that defines the audit policy configuration.

 --audit-webhook-batch-buffer-size int     Default: 10000

The size of the buffer to store events before batching and writing. Only used in batch mode.

 --audit-webhook-batch-max-size int     Default: 400

The maximum size of a batch. Only used in batch mode.

 --audit-webhook-batch-max-wait duration     Default: 30s

The amount of time to wait before force writing the batch that hadn't reached the max size. Only used in batch mode.

 --audit-webhook-batch-throttle-burst int     Default: 15

Maximum number of requests sent at the same moment if ThrottleQPS was not utilized before. Only used in batch mode.

 --audit-webhook-batch-throttle-enable     Default: true

Whether batching throttling is enabled. Only used in batch mode.

 --audit-webhook-batch-throttle-qps float     Default: 10

Maximum average number of batches per second. Only used in batch mode.

 --audit-webhook-config-file string

Path to a kubeconfig formatted file that defines the audit webhook configuration.

 --audit-webhook-initial-backoff duration     Default: 10s

The amount of time to wait before retrying the first failed request.

 --audit-webhook-mode string     Default: "batch"

Strategy for sending audit events. Blocking indicates sending events should block server responses. Batch causes the backend to buffer and write events asynchronously. Known modes are batch,blocking,blocking-strict.

 --audit-webhook-truncate-enabled

Whether event and batch truncating is enabled.

 --audit-webhook-truncate-max-batch-size int     Default: 10485760

Maximum size of the batch sent to the underlying backend. Actual serialized size can be several hundreds of bytes greater. If a batch exceeds this limit, it is split into several batches of smaller size.

 --audit-webhook-truncate-max-event-size int     Default: 102400

Maximum size of the audit event sent to the underlying backend. If the size of an event is greater than this number, first request and response are removed, and if this doesn't reduce the size enough, event is discarded.

 --audit-webhook-version string     Default: "audit.k8s.io/v1"

API group and version used for serializing audit events written to webhook.

 --authentication-token-webhook-cache-ttl duration     Default: 2m0s

The duration to cache responses from the webhook token authenticator.

 --authentication-token-webhook-config-file string

File with webhook configuration for token authentication in kubeconfig format. The API server will query the remote service to determine authentication for bearer tokens.

 --authentication-token-webhook-version string     Default: "v1beta1"

The API version of the authentication.k8s.io TokenReview to send to and expect from the webhook.

 --authorization-mode strings     Default: "AlwaysAllow"

Ordered list of plug-ins to do authorization on secure port. Comma-delimited list of: AlwaysAllow,AlwaysDeny,ABAC,Webhook,RBAC,Node.

 --authorization-policy-file string

File with authorization policy in json line by line format, used with --authorization-mode=ABAC, on the secure port.

 --authorization-webhook-cache-authorized-ttl duration     Default: 5m0s

The duration to cache 'authorized' responses from the webhook authorizer.

 --authorization-webhook-cache-unauthorized-ttl duration     Default: 30s

The duration to cache 'unauthorized' responses from the webhook authorizer.

 --authorization-webhook-config-file string

File with webhook configuration in kubeconfig format, used with --authorization-mode=Webhook. The API server will query the remote service to determine access on the API server's secure port.

 --authorization-webhook-version string     Default: "v1beta1"

The API version of the authorization.k8s.io SubjectAccessReview to send to and expect from the webhook.

 --azure-container-registry-config string

Path to the file containing Azure container registry configuration information.

 --bind-address string     Default: 0.0.0.0

The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces will be used.

 --cert-dir string     Default: "/var/run/kubernetes"

The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.

 --client-ca-file string

If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.

 --cloud-config string

The path to the cloud provider configuration file. Empty string for no configuration file.

 --cloud-provider string

The provider for cloud services. Empty string for no provider.

 --cloud-provider-gce-l7lb-src-cidrs cidrs     Default: 130.211.0.0/22,35.191.0.0/16

CIDRs opened in GCE firewall for L7 LB traffic proxy & health checks

 --contention-profiling

Enable lock contention profiling, if profiling is enabled

 --cors-allowed-origins strings

List of allowed origins for CORS, comma separated.  An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled.

 --default-not-ready-toleration-seconds int     Default: 300

Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.

 --default-unreachable-toleration-seconds int     Default: 300

Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.

 --default-watch-cache-size int     Default: 100

Default watch cache size. If zero, watch cache will be disabled for resources that do not have a default watch size set.

 --delete-collection-workers int     Default: 1

Number of workers spawned for DeleteCollection call. These are used to speed up namespace cleanup.

 --disable-admission-plugins strings

admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

 --disabled-metrics strings

This flag provides an escape hatch for misbehaving metrics. You must provide the fully qualified metric name in order to disable it. Disclaimer: disabling metrics is higher in precedence than showing hidden metrics.

 --egress-selector-config-file string

File with apiserver egress selector configuration.

 --enable-admission-plugins strings

admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

 --enable-aggregator-routing

Turns on aggregator routing requests to endpoints IP rather than cluster IP.

 --enable-bootstrap-token-auth

Enable to allow secrets of type 'bootstrap.kubernetes.io/token' in the 'kube-system' namespace to be used for TLS bootstrapping authentication.

 --enable-garbage-collector     Default: true

Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-controller-manager.

 --enable-priority-and-fairness     Default: true

If true and the APIPriorityAndFairness feature gate is enabled, replace the max-in-flight handler with an enhanced one that queues and dispatches with priority and fairness

 --encryption-provider-config string

The file containing configuration for encryption providers to be used for storing secrets in etcd

 --endpoint-reconciler-type string     Default: "lease"

Use an endpoint reconciler (master-count, lease, none)

 --etcd-cafile string

SSL Certificate Authority file used to secure etcd communication.

 --etcd-certfile string

SSL certification file used to secure etcd communication.

 --etcd-compaction-interval duration     Default: 5m0s

The interval of compaction requests. If 0, the compaction request from apiserver is disabled.

 --etcd-count-metric-poll-period duration     Default: 1m0s

Frequency of polling etcd for number of resources per type. 0 disables the metric collection.

 --etcd-db-metric-poll-interval duration     Default: 30s

The interval of requests to poll etcd and update metric. 0 disables the metric collection

 --etcd-healthcheck-timeout duration     Default: 2s

The timeout to use when checking etcd health.

 --etcd-keyfile string

SSL key file used to secure etcd communication.

 --etcd-prefix string     Default: "/registry"

The prefix to prepend to all resource paths in etcd.

 --etcd-servers strings

List of etcd servers to connect with (scheme://ip:port), comma separated.

 --etcd-servers-overrides strings

Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are URLs, semicolon separated. Note that this applies only to resources compiled into this server binary.

 --event-ttl duration     Default: 1h0m0s

Amount of time to retain events.

 --experimental-logging-sanitization

[Experimental] When enabled prevents logging of fields tagged as sensitive (passwords, keys, tokens).
Runtime log sanitization may introduce significant computation overhead and therefore should not be enabled in production.

 --external-hostname string

The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs or OpenID Discovery).

 --feature-gates <comma-separated 'key=True|False' pairs>

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIListChunking=true|false (BETA - default=true)
APIPriorityAndFairness=true|false (BETA - default=true)
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (ALPHA - default=false)
APIServerTracing=true|false (ALPHA - default=false)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnyVolumeDataSource=true|false (ALPHA - default=false)
AppArmor=true|false (BETA - default=true)
CPUManager=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (ALPHA - default=false)
CSIInlineVolume=true|false (BETA - default=true)
CSIMigration=true|false (BETA - default=true)
CSIMigrationAWS=true|false (BETA - default=false)
CSIMigrationAzureDisk=true|false (BETA - default=false)
CSIMigrationAzureFile=true|false (BETA - default=false)
CSIMigrationGCE=true|false (BETA - default=false)
CSIMigrationOpenStack=true|false (BETA - default=true)
CSIMigrationvSphere=true|false (BETA - default=false)
CSIStorageCapacity=true|false (BETA - default=true)
CSIVolumeFSGroupPolicy=true|false (BETA - default=true)
CSIVolumeHealth=true|false (ALPHA - default=false)
CSRDuration=true|false (BETA - default=true)
ConfigurableFSGroupPolicy=true|false (BETA - default=true)
ControllerManagerLeaderMigration=true|false (BETA - default=true)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DaemonSetUpdateSurge=true|false (BETA - default=true)
DefaultPodTopologySpread=true|false (BETA - default=true)
DelegateFSGroupToCSIDriver=true|false (ALPHA - default=false)
DevicePlugins=true|false (BETA - default=true)
DisableAcceleratorUsageMetrics=true|false (BETA - default=true)
DisableCloudProviders=true|false (ALPHA - default=false)
DownwardAPIHugePages=true|false (BETA - default=false)
EfficientWatchResumption=true|false (BETA - default=true)
EndpointSliceTerminatingCondition=true|false (BETA - default=true)
EphemeralContainers=true|false (ALPHA - default=false)
ExpandCSIVolumes=true|false (BETA - default=true)
ExpandInUsePersistentVolumes=true|false (BETA - default=true)
ExpandPersistentVolumes=true|false (BETA - default=true)
ExpandedDNSConfig=true|false (ALPHA - default=false)
ExperimentalHostUserNamespaceDefaulting=true|false (BETA - default=false)
GenericEphemeralVolume=true|false (BETA - default=true)
GracefulNodeShutdown=true|false (BETA - default=true)
HPAContainerMetrics=true|false (ALPHA - default=false)
HPAScaleToZero=true|false (ALPHA - default=false)
IPv6DualStack=true|false (BETA - default=true)
InTreePluginAWSUnregister=true|false (ALPHA - default=false)
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
InTreePluginGCEUnregister=true|false (ALPHA - default=false)
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
IndexedJob=true|false (BETA - default=true)
IngressClassNamespacedParams=true|false (BETA - default=true)
JobTrackingWithFinalizers=true|false (ALPHA - default=false)
KubeletCredentialProviders=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResources=true|false (BETA - default=true)
KubeletPodResourcesGetAllocatable=true|false (ALPHA - default=false)
LocalStorageCapacityIsolation=true|false (BETA - default=true)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
LogarithmicScaleDown=true|false (BETA - default=true)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MixedProtocolLBService=true|false (ALPHA - default=false)
NetworkPolicyEndPort=true|false (BETA - default=true)
NodeSwap=true|false (ALPHA - default=false)
NonPreemptingPriority=true|false (BETA - default=true)
PodAffinityNamespaceSelector=true|false (BETA - default=true)
PodDeletionCost=true|false (BETA - default=true)
PodOverhead=true|false (BETA - default=true)
PodSecurity=true|false (ALPHA - default=false)
PreferNominatedNode=true|false (BETA - default=true)
ProbeTerminationGracePeriod=true|false (BETA - default=false)
ProcMountType=true|false (ALPHA - default=false)
ProxyTerminatingEndpoints=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
ReadWriteOncePod=true|false (ALPHA - default=false)
RemainingItemCount=true|false (BETA - default=true)
RemoveSelfLink=true|false (BETA - default=true)
RotateKubeletServerCertificate=true|false (BETA - default=true)
SeccompDefault=true|false (ALPHA - default=false)
ServiceInternalTrafficPolicy=true|false (BETA - default=true)
ServiceLBNodePortControl=true|false (BETA - default=true)
ServiceLoadBalancerClass=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StatefulSetMinReadySeconds=true|false (ALPHA - default=false)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
SuspendJob=true|false (BETA - default=true)
TTLAfterFinished=true|false (BETA - default=true)
TopologyAwareHints=true|false (ALPHA - default=false)
TopologyManager=true|false (BETA - default=true)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostProcessContainers=true|false (ALPHA - default=false)

 --goaway-chance float

To prevent HTTP/2 clients from getting stuck on a single apiserver, randomly close a connection (GOAWAY). The client's other in-flight requests won't be affected, and the client will reconnect, likely landing on a different apiserver after going through the load balancer again. This argument sets the fraction of requests that will be sent a GOAWAY. Clusters with single apiservers, or which don't use a load balancer, should NOT enable this. Min is 0 (off), Max is .02 (1/50 requests); .001 (1/1000) is a recommended starting point.

 -h, --help

help for kube-apiserver

 --http2-max-streams-per-connection int

The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default.

 --identity-lease-duration-seconds int     Default: 3600

The duration of kube-apiserver lease in seconds, must be a positive number. (In use when the APIServerIdentity feature gate is enabled.)

 --identity-lease-renew-interval-seconds int     Default: 10

The interval of kube-apiserver renewing its lease in seconds, must be a positive number. (In use when the APIServerIdentity feature gate is enabled.)

 --kubelet-certificate-authority string

Path to a cert file for the certificate authority.

 --kubelet-client-certificate string

Path to a client cert file for TLS.

 --kubelet-client-key string

Path to a client key file for TLS.

 --kubelet-preferred-address-types strings     Default: "Hostname,InternalDNS,InternalIP,ExternalDNS,ExternalIP"

List of the preferred NodeAddressTypes to use for kubelet connections.

 --kubelet-timeout duration     Default: 5s

Timeout for kubelet operations.

 --kubernetes-service-node-port int

If non-zero, the Kubernetes master service (which apiserver creates/maintains) will be of type NodePort, using this as the value of the port. If zero, the Kubernetes master service will be of type ClusterIP.

 --lease-reuse-duration-seconds int     Default: 60

The time in seconds that each lease is reused. A lower value could avoid large number of objects reusing the same lease. Notice that a too small value may cause performance problems at storage layer.

 --livez-grace-period duration

This option represents the maximum amount of time it should take for apiserver to complete its startup sequence and become live. From apiserver's start time to when this amount of time has elapsed, /livez will assume that unfinished post-start hooks will complete successfully and therefore return true.

 --log-backtrace-at <a string in the form 'file:N'>     Default: :0

when logging hits line file:N, emit a stack trace

 --log-dir string

If non-empty, write log files in this directory

 --log-file string

If non-empty, use this log file

 --log-file-max-size uint     Default: 1800

Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited.

 --log-flush-frequency duration     Default: 5s

Maximum number of seconds between log flushes

 --logging-format string     Default: "text"

Sets the log format. Permitted formats: "text".
Non-default formats don't honor these flags: --add-dir-header, --alsologtostderr, --log-backtrace-at, --log-dir, --log-file, --log-file-max-size, --logtostderr, --one-output, --skip-headers, --skip-log-headers, --stderrthreshold, --vmodule, --log-flush-frequency.
Non-default choices are currently alpha and subject to change without warning.

 --logtostderr     Default: true

log to standard error instead of files

 --master-service-namespace string     Default: "default"

DEPRECATED: the namespace from which the Kubernetes master services should be injected into pods.

 --max-connection-bytes-per-sec int

If non-zero, throttle each user connection to this number of bytes/sec. Currently only applies to long-running requests.

 --max-mutating-requests-inflight int     Default: 200

This and --max-requests-inflight are summed to determine the server's total concurrency limit (which must be positive) if --enable-priority-and-fairness is true. Otherwise, this flag limits the maximum number of mutating requests in flight, or a zero value disables the limit completely.

 --max-requests-inflight int     Default: 400

This and --max-mutating-requests-inflight are summed to determine the server's total concurrency limit (which must be positive) if --enable-priority-and-fairness is true. Otherwise, this flag limits the maximum number of non-mutating requests in flight, or a zero value disables the limit completely.

 --min-request-timeout int     Default: 1800

An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load.

 --oidc-ca-file string

If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used.

 --oidc-client-id string

The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set.

 --oidc-groups-claim string

If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be a string or array of strings. This flag is experimental, please see the authentication documentation for further details.

 --oidc-groups-prefix string

If provided, all groups will be prefixed with this value to prevent conflicts with other authentication strategies.

 --oidc-issuer-url string

The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT).

 --oidc-required-claim <comma-separated 'key=value' pairs>

A key=value pair that describes a required claim in the ID Token. If set, the claim is verified to be present in the ID Token with a matching value. Repeat this flag to specify multiple claims.

 --oidc-signing-algs strings     Default: "RS256"

Comma-separated list of allowed JOSE asymmetric signing algorithms. JWTs with a 'alg' header value not in this list will be rejected. Values are defined by RFC 7518 https://tools.ietf.org/html/rfc7518#section-3.1.

 --oidc-username-claim string     Default: "sub"

The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details.

 --oidc-username-prefix string

If provided, all usernames will be prefixed with this value. If not provided, username claims other than 'email' are prefixed by the issuer URL to avoid clashes. To skip any prefixing, provide the value '-'.

 --one-output

If true, only write logs to their native severity level (vs also writing to each lower severity level)

 --permit-address-sharing

If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false]

 --permit-port-sharing

If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false]

 --profiling     Default: true

Enable profiling via web interface host:port/debug/pprof/

 --proxy-client-cert-file string

Client certificate used to prove the identity of the aggregator or kube-apiserver when it must call out during a request. This includes proxying requests to a user api-server and calling out to webhook admission plugins. It is expected that this cert includes a signature from the CA in the --requestheader-client-ca-file flag. That CA is published in the 'extension-apiserver-authentication' configmap in the kube-system namespace. Components receiving calls from kube-aggregator should use that CA to perform their half of the mutual TLS verification.

 --proxy-client-key-file string

Private key for the client certificate used to prove the identity of the aggregator or kube-apiserver when it must call out during a request. This includes proxying requests to a user api-server and calling out to webhook admission plugins.

 --request-timeout duration     Default: 1m0s

An optional field indicating the duration a handler must keep a request open before timing it out. This is the default request timeout for requests but may be overridden by flags such as --min-request-timeout for specific types of requests.

 --requestheader-allowed-names strings

List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.

 --requestheader-client-ca-file string

Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests.

 --requestheader-extra-headers-prefix strings

List of request header prefixes to inspect. X-Remote-Extra- is suggested.

 --requestheader-group-headers strings

List of request headers to inspect for groups. X-Remote-Group is suggested.

 --requestheader-username-headers strings

List of request headers to inspect for usernames. X-Remote-User is common.

 --runtime-config <comma-separated 'key=value' pairs>

A set of key=value pairs that enable or disable built-in APIs. Supported options are:
v1=true|false for the core API group
<group>/<version>=true|false for a specific API group and version (e.g. apps/v1=true)
api/all=true|false controls all API versions
api/ga=true|false controls all API versions of the form v[0-9]+
api/beta=true|false controls all API versions of the form v[0-9]+beta[0-9]+
api/alpha=true|false controls all API versions of the form v[0-9]+alpha[0-9]+
api/legacy is deprecated, and will be removed in a future version

 --secure-port int     Default: 6443

The port on which to serve HTTPS with authentication and authorization. It cannot be switched off with 0.

 --service-account-extend-token-expiration     Default: true

Turns on projected service account expiration extension during token generation, which helps safe transition from legacy token to bound service account token feature. If this flag is enabled, admission injected tokens would be extended up to 1 year to prevent unexpected failure during transition, ignoring value of service-account-max-token-expiration.

 --service-account-issuer strings

Identifier of the service account token issuer. The issuer will assert this identifier in "iss" claim of issued tokens. This value is a string or URI. If this option is not a valid URI per the OpenID Discovery 1.0 spec, the ServiceAccountIssuerDiscovery feature will remain disabled, even if the feature gate is set to true. It is highly recommended that this value comply with the OpenID spec: https://openid.net/specs/openid-connect-discovery-1_0.html. In practice, this means that service-account-issuer must be an https URL. It is also highly recommended that this URL be capable of serving OpenID discovery documents at {service-account-issuer}/.well-known/openid-configuration. When this flag is specified multiple times, the first is used to generate tokens and all are used to determine which issuers are accepted.

 --service-account-jwks-uri string

Overrides the URI for the JSON Web Key Set in the discovery doc served at /.well-known/openid-configuration. This flag is useful if the discovery docand key set are served to relying parties from a URL other than the API server's external (as auto-detected or overridden with external-hostname). Only valid if the ServiceAccountIssuerDiscovery feature gate is enabled.

 --service-account-key-file strings

File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. The specified file can contain multiple keys, and the flag can be specified multiple times with different files. If unspecified, --tls-private-key-file is used. Must be specified when --service-account-signing-key is provided

 --service-account-lookup     Default: true

If true, validate ServiceAccount tokens exist in etcd as part of authentication.

 --service-account-max-token-expiration duration

The maximum validity duration of a token created by the service account token issuer. If an otherwise valid TokenRequest with a validity duration larger than this value is requested, a token will be issued with a validity duration of this value.

 --service-account-signing-key-file string

Path to the file that contains the current private key of the service account token issuer. The issuer will sign issued ID tokens with this private key.

 --service-cluster-ip-range string

A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes or pods. Max of two dual-stack CIDRs is allowed.

 --service-node-port-range <a string in the form 'N1-N2'>     Default: 30000-32767

A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range.

 --show-hidden-metrics-for-version string

The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.

 --shutdown-delay-duration duration

Time to delay the termination. During that time the server keeps serving requests normally. The endpoints /healthz and /livez will return success, but /readyz immediately returns failure. Graceful termination starts after this delay has elapsed. This can be used to allow load balancer to stop sending traffic to this server.

 --skip-headers

If true, avoid header prefixes in the log messages

 --skip-log-headers

If true, avoid headers when opening log files

 --stderrthreshold int     Default: 2

logs at or above this threshold go to stderr

 --storage-backend string

The storage backend for persistence. Options: 'etcd3' (default).

 --storage-media-type string     Default: "application/vnd.kubernetes.protobuf"

The media type to use to store objects in storage. Some resources or storage backends may only support a specific media type and will ignore this setting.

 --strict-transport-security-directives strings

List of directives for HSTS, comma separated. If this list is empty, then HSTS directives will not be added. Example: 'max-age=31536000,includeSubDomains,preload'

 --tls-cert-file string

File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir.

 --tls-cipher-suites strings

Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.
Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.

 --tls-min-version string

Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13

 --tls-private-key-file string

File containing the default x509 private key matching --tls-cert-file.

 --tls-sni-cert-key string

A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com".

 --token-auth-file string

If set, the file that will be used to secure the secure port of the API server via token authentication.

 --tracing-config-file string

File with apiserver tracing configuration.

 -v, --v int

number for the log level verbosity

 --version version[=true]

Print version information and quit

 --vmodule <comma-separated 'pattern=N' settings>

comma-separated list of pattern=N settings for file-filtered logging

 --watch-cache     Default: true

Enable watch caching in the apiserver

 --watch-cache-sizes strings

Watch cache size settings for some resources (pods, nodes, etc.), comma separated. The individual setting format: resource[.group]#size, where resource is lowercase plural (no version), group is omitted for resources of apiVersion v1 (the legacy core API) and included for others, and size is a number. It takes effect when watch-cache is enabled. Some resources (replicationcontrollers, endpoints, nodes, pods, services, apiservices.apiregistration.k8s.io) have system defaults set by heuristics, others default to default-watch-cache-size


 --- title: kube-apiserver Audit Configuration (v1) content_type: tool-reference package: audit.k8s.io/v1 auto_generated: true --- ## Resource Types - [Event](#audit-k8s-io-v1-Event) - [EventList](#audit-k8s-io-v1-EventList) - [Policy](#audit-k8s-io-v1-Policy) - [PolicyList](#audit-k8s-io-v1-PolicyList) ## `Event`     {#audit-k8s-io-v1-Event} **Appears in:** - [EventList](#audit-k8s-io-v1-EventList) Event captures all the information that can be included in an API audit log.

Field	Description

apiVersion
string	audit.k8s.io/v1

kind
string	Event

level [Required]
 Level 	  AuditLevel at which event was generated

auditID [Required]
 k8s.io/apimachinery/pkg/types.UID 	  Unique audit ID, generated for each request.

stage [Required]
 Stage 	  Stage of the request handling when this event instance was generated.

requestURI [Required]
 string 	  RequestURI is the request URI as sent by the client to a server.

verb [Required]
 string 	  Verb is the kubernetes verb associated with the request. For non-resource requests, this is the lower-cased HTTP method.

user [Required]
 authentication/v1.UserInfo 	  Authenticated user information.

impersonatedUser
 authentication/v1.UserInfo 	  Impersonated user information.

sourceIPs
 []string 	  Source IPs, from where the request originated and intermediate proxies.

userAgent
 string 	  UserAgent records the user agent string reported by the client. Note that the UserAgent is provided by the client, and must not be trusted.

objectRef
 ObjectReference 	  Object reference this request is targeted at. Does not apply for List-type requests, or non-resource requests.

responseStatus
 meta/v1.Status 	  The response status, populated even when the ResponseObject is not a Status type. For successful responses, this will only include the Code and StatusSuccess. For non-status type error responses, this will be auto-populated with the error Message.

requestObject
 k8s.io/apimachinery/pkg/runtime.Unknown 	  API object from the request, in JSON format. The RequestObject is recorded as-is in the request (possibly re-encoded as JSON), prior to version conversion, defaulting, admission or merging. It is an external versioned object type, and may not be a valid object on its own. Omitted for non-resource requests.  Only logged at Request Level and higher.

responseObject
 k8s.io/apimachinery/pkg/runtime.Unknown 	  API object returned in the response, in JSON. The ResponseObject is recorded after conversion to the external type, and serialized as JSON.  Omitted for non-resource requests.  Only logged at Response Level.

requestReceivedTimestamp
 meta/v1.MicroTime 	  Time the request reached the apiserver.

stageTimestamp
 meta/v1.MicroTime 	  Time the request reached current audit stage.

annotations
 map[string]string 	  Annotations is an unstructured key value map stored with an audit event that may be set by plugins invoked in the request serving chain, including authentication, authorization and admission plugins. Note that these annotations are for the audit event, and do not correspond to the metadata.annotations of the submitted object. Keys should uniquely identify the informing component to avoid name collisions (e.g. podsecuritypolicy.admission.k8s.io/policy). Values should be short. Annotations are included in the Metadata level.

 ## `EventList`     {#audit-k8s-io-v1-EventList} EventList is a list of audit Events.

Field	Description

apiVersion
string	audit.k8s.io/v1

kind
string	EventList

metadata
 meta/v1.ListMeta 	  No description provided.

items [Required]
 []Event 	  No description provided.

 ## `Policy`     {#audit-k8s-io-v1-Policy} **Appears in:** - [PolicyList](#audit-k8s-io-v1-PolicyList) Policy defines the configuration of audit logging, and the rules for how different request categories are logged.

Field	Description

apiVersion
string	audit.k8s.io/v1

kind
string	Policy

metadata
 meta/v1.ObjectMeta 	  ObjectMeta is included for interoperability with API infrastructure.Refer to the Kubernetes API documentation for the fields of the metadata field.

rules [Required]
 []PolicyRule 	  Rules specify the audit Level a request should be recorded at. A request may match multiple rules, in which case the FIRST matching rule is used. The default audit level is None, but can be overridden by a catch-all rule at the end of the list. PolicyRules are strictly ordered.

omitStages
 []Stage 	  OmitStages is a list of stages for which no events are created. Note that this can also be specified per rule in which case the union of both are omitted.

 ## `PolicyList`     {#audit-k8s-io-v1-PolicyList} PolicyList is a list of audit Policies.

Field	Description

apiVersion
string	audit.k8s.io/v1

kind
string	PolicyList

metadata
 meta/v1.ListMeta 	  No description provided.

items [Required]
 []Policy 	  No description provided.

 ## `GroupResources`     {#audit-k8s-io-v1-GroupResources} **Appears in:** - [PolicyRule](#audit-k8s-io-v1-PolicyRule) GroupResources represents resource kinds in an API group.

Field	Description

group
 string 	  Group is the name of the API group that contains the resources. The empty string represents the core API group.

resources
 []string 	  Resources is a list of resources this rule applies to. For example: 'pods' matches pods. 'pods/log' matches the log subresource of pods. '∗' matches all resources and their subresources. 'pods/∗' matches all subresources of pods. '∗/scale' matches all scale subresources. If wildcard is present, the validation rule will ensure resources do not overlap with each other. An empty list implies all resources and subresources in this API groups apply.

resourceNames
 []string 	  ResourceNames is a list of resource instance names that the policy matches. Using this field requires Resources to be specified. An empty list implies that every instance of the resource is matched.

 ## `Level`     {#audit-k8s-io-v1-Level} (Alias of `string`) **Appears in:** - [Event](#audit-k8s-io-v1-Event) - [PolicyRule](#audit-k8s-io-v1-PolicyRule) Level defines the amount of information logged during auditing ## `ObjectReference`     {#audit-k8s-io-v1-ObjectReference} **Appears in:** - [Event](#audit-k8s-io-v1-Event) ObjectReference contains enough information to let you inspect or modify the referred object.

Field	Description

resource
 string 	  No description provided.

namespace
 string 	  No description provided.

name
 string 	  No description provided.

uid
 k8s.io/apimachinery/pkg/types.UID 	  No description provided.

apiGroup
 string 	  APIGroup is the name of the API group that contains the referred object. The empty string represents the core API group.

apiVersion
 string 	  APIVersion is the version of the API group that contains the referred object.

resourceVersion
 string 	  No description provided.

subresource
 string 	  No description provided.

 ## `PolicyRule`     {#audit-k8s-io-v1-PolicyRule} **Appears in:** - [Policy](#audit-k8s-io-v1-Policy) PolicyRule maps requests based off metadata to an audit Level. Requests must match the rules of every field (an intersection of rules).

Field	Description

level [Required]
 Level 	  The Level that requests matching this rule are recorded at.

users
 []string 	  The users (by authenticated user name) this rule applies to. An empty list implies every user.

userGroups
 []string 	  The user groups this rule applies to. A user is considered matching if it is a member of any of the UserGroups. An empty list implies every user group.

verbs
 []string 	  The verbs that match this rule. An empty list implies every verb.

resources
 []GroupResources 	  Resources that this rule matches. An empty list implies all kinds in all API groups.

namespaces
 []string 	  Namespaces that this rule matches. The empty string "" matches non-namespaced resources. An empty list implies every namespace.

nonResourceURLs
 []string 	  NonResourceURLs is a set of URL paths that should be audited. ∗s are allowed, but only as the full, final step in the path. Examples: "/metrics" - Log requests for apiserver metrics "/healthz∗" - Log all health checks

omitStages
 []Stage 	  OmitStages is a list of stages for which no events are created. Note that this can also be specified policy wide in which case the union of both are omitted. An empty list means no restrictions will apply.

 ## `Stage`     {#audit-k8s-io-v1-Stage} (Alias of `string`) **Appears in:** - [Event](#audit-k8s-io-v1-Event) - [Policy](#audit-k8s-io-v1-Policy) - [PolicyRule](#audit-k8s-io-v1-PolicyRule) Stage defines the stages in request handling that audit events may be generated.


<ins> --- content_type: concept reviewers: - ptux title: 監査 --- <!-- overview --> Kubernetesの監査はクラスター内の一連の行動を記録するセキュリティに関連した時系列の記録を提供します。 クラスターはユーザー、Kubernetes APIを使用するアプリケーション、 およびコントロールプレーン自体によって生成されたアクティビティなどを監査します。 監査により、クラスター管理者は以下の質問に答えることができます: - 何が起きたのか？ - いつ起こったのか？ - 誰がそれを始めたのか？ - 何のために起こったのか？ - それはどこで観察されましたか？ - それはどこから始まったのか？ - それはどこへ向かっていたのか？ <!-- body --> 監査記録のライフサイクルは[kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/)コンポーネントの中で始まります。 各リクエストは、その実行の各段階でその実行の各段階で、監査イベントが生成されます。 ポリシーに従って前処理され、バックエンドに書き込まれます。 ポリシーが何を記録するかを決定しを決定し、 バックエンドがその記録を永続化します。現在のバックエンドの実装はログファイルやWebhookなどがあります。 各リクエストは関連する _stage_ で記録されます。 定義されたステージは以下の通りです: - `RequestReceived` - 監査ハンドラーがリクエストを受信すると同時に生成されるイベントのステージ。 つまり、ハンドラーチェーンに委譲される前に生成されるイベントのステージです。 - `ResponseStarted` - レスポンスヘッダーが送信された後、レスポンスボディが送信される前のステージです。 このステージは長時間実行されるリクエスト（watchなど）でのみ発生します。 - `ResponseComplete` - レスポンスボディの送信が完了して、それ以上のバイトは送信されません。 - `Panic` - パニックが起きたときに発生するイベント。 {{< note >}} [Audit Event configuration](/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Event)の設定は[Event](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#event-v1-core)APIオブジェクトとは異なります。 {{< /note >}} 監査ログ機能は、リクエストごとに監査に必要なコンテキストが保存されるため、APIサーバーのメモリ消費量が増加します。 メモリの消費量は、監査ログ機能の設定によって異なります。 ## Audit policy 監査ポリシーはどのようなイベントを記録し、どのようなデータを含むべきかについてのルールを定義します。 監査ポリシーのオブジェクト構造は、[audit.k8s.io` API group](/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Policy)で定義されています。 イベントが処理されると、そのイベントは順番にルールのリストと比較されます。 最初のマッチングルールは、イベントの監査レベルを設定します。 定義されている監査レベルは: - `None` - ルールに一致するイベントを記録しません。 - `Metadata` - lリクエストのメタデータ（リクエストしたユーザー、タイムスタンプ、リソース、動作など）を記録しますが、リクエストやレスポンスのボディは記録しません。 - `Request` - ログイベントのメタデータとリクエストボディは表示されますが、レスポンスボディは表示されません。 これは非リソースリクエストには適用されません。 - `RequestResponse` - イベントのメタデータ、リクエストとレスポンスのボディを記録しますが、 非リソースリクエストには適用されません。 You can pass a file with the policy to `kube-apiserver` using the `--audit-policy-file` flag. If the flag is omitted, no events are logged. Note that the `rules` field __must__ be provided in the audit policy file. A policy with no (0) rules is treated as illegal. `audit-policy-file`フラグを使って、ポリシーを記述したファイルを `kube-apiserver`に渡すことができます。 このフラグが省略された場合イベントは記録されません。 監査ポリシーファイルでは、`rules`フィールドが必ず指定されることに注意してください。 ルールがない(0)ポリシーは不当なものとして扱われます。 以下は監査ポリシーファイルの例: {{< codenew file="audit/audit-policy.yaml" >}} 最小限の監査ポリシーファイルを使用して、すべてのリクエストを `Metadata`レベルで記録することができます。 ```yaml # Log all requests at the Metadata level. apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata ``` 独自の監査プロファイルを作成する場合は、Google Container-Optimized OSの監査プロファイルを出発点として使用できます。 監査ポリシーファイルを生成する[configure-helper.sh](https://github.com/kubernetes/kubernetes/blob/master/cluster/gce/gci/configure-helper.sh)スクリプトを確認することができます。 スクリプトを直視することで、監査ポリシーファイルのほとんどを見ることができます。 また、定義されているフィールドの詳細については、[Policy` configuration reference](/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Policy)を参照できます。 ## 監査バックエンド 監査バックエンドは監査イベントを外部ストレージに永続化します。 kube-apiserverには2つのバックエンドが用意されています。 - イベントをファイルシステムに書き込むログバックエンド - 外部のHTTP APIにイベントを送信するWebhookバックエンド いずれの場合も、監査イベントはKubernetes API[`audit.k8s.io` API group](/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Event)で定義されている構造に従います。 {{< note >}} パッチの場合、リクエストボディはパッチ操作を含むJSON配列であり、適切なKubernetes APIオブジェクトを含むJSONオブジェクトではありません。 例えば、以下のリクエストボディは`/apis/batch/v1/namespaces/some-namespace/jobs/some-job-name`に対する有効なパッチリクエストです。 ```json [ { "op": "replace", "path": "/spec/parallelism", "value": 0 }, { "op": "remove", "path": "/spec/template/spec/containers/0/terminationMessagePolicy" } ] ``` {{< /note >}} ### ログバックエンド ログバックエンドは監査イベントを[JSONlines](https://jsonlines.org/)形式のファイルに書き込みます。 以下の `kube-apiserver` フラグを使ってログ監査バックエンドを設定できます。 - `--audit-log-path` は、ログバックエンドが監査イベントを書き込む際に使用するログファイルのパスを指定します。 このフラグを指定しないと、ログバックエンドは無効になります。`-` は標準出力を意味します。 - `--audit-log-maxage` は、古い監査ログファイルを保持する最大日数を定義します。 - `audit-log-maxbackup`は、保持する監査ログファイルの最大数を定義します。 - `--audit-log-maxsize` は、監査ログファイルがローテーションされるまでの最大サイズをメガバイト単位で定義します。 クラスターのコントロールプレーンでkube-apiserverをPodとして動作させている場合は、監査記録が永久化されるように、ポリシーファイルとログファイルの場所に`hostPath`をマウントすることを忘れないでください。 例えば: ```shell --audit-policy-file=/etc/kubernetes/audit-policy.yaml  --audit-log-path=/var/log/audit.log ``` それからボリュームをマウントします: ```yaml ... volumeMounts: - mountPath: /etc/kubernetes/audit-policy.yaml name: audit readOnly: true - mountPath: /var/log/audit.log name: audit-log readOnly: false ``` 最後に`hostPath`を設定します: ```yaml ... - name: audit hostPath: path: /etc/kubernetes/audit-policy.yaml type: File - name: audit-log hostPath: path: /var/log/audit.log type: FileOrCreate ``` ### Webhook バックエンド Webhook監査バックエンドは、監査イベントをリモートのWeb APIに送信しますが、 これは認証手段を含むKubernetes APIの形式であると想定されます。 Webhook監査バックエンドを設定するには、以下のkube-apiserverフラグを使用します。 - `--audit-webhook-config-file` は、Webhookの設定ファイルのパスを指定します。 webhookの設定は、事実上特化した [kubeconfig](/docs/tasks/access-application-cluster/configure-access-multiple-clusters) です。 - `--audit-webhook-initial-backoff` は、最初に失敗したリクエストの後、再試行するまでに待つ時間を指定します。 それ以降のリクエストは、指数関数的なバックオフで再試行されます。 Webhookの設定ファイルは、kubeconfig 形式でサービスのリモートアドレスと接続に使用する認証情報を指定します。 ## イベントバッチ {#batching} ログバックエンドとwebhookバックエンドの両方がバッチ処理をサポートしています。 webhookを例に、利用可能なフラグの一覧を示します。 ログバックエンドで同じフラグを取得するには、フラグ名の`webhook`を`log`に置き換えてください。 デフォルトでは、バッチングは`webhook`では有効で、`log`では無効です。 同様に、デフォルトではスロットリングは `webhook` で有効で、`log`では無効です。 - `--audit-webhook-mode` は、バッファリング戦略を定義します。以下のいずれかとなります。 - `batch` - イベントをバッファリングして、非同期にバッチ処理します。これがデフォルトです。 - `blocking` - 個々のイベントを処理する際に、APIサーバーの応答をブロックします。 - `blocking-strict` - blockingと同じですが、RequestReceivedステージでの監査ログに失敗した場合は RequestReceivedステージで監査ログに失敗すると、kube-apiserverへのリクエスト全体が失敗します。 以下のフラグは `batch` モードでのみ使用されます: - `--audit-webhook-batch-buffer-size`は、バッチ処理を行う前にバッファリングするイベントの数を定義します。 入力イベントの割合がバッファをオーバーフローすると、イベントはドロップされます。 - `--audit-webhook-batch-max-size`は、1つのバッチに入れるイベントの最大数を定義します。 - `--audit-webhook-batch-max-wait`は、キュー内のイベントを無条件にバッチ処理するまでの最大待機時間を定義します。 - `--audit-webhook-batch-throttle-qps`は、1秒あたりに生成されるバッチの最大平均数を定義します。 - `--audit-webhook-batch-throttle-burst`は、許可された QPS が低い場合に、同じ瞬間に生成されるバッチの最大数を定義します。 ## パラメーターチューニング パラメーターは、APIサーバーの負荷に合わせて設定してください。 例えば、kube-apiserverが毎秒100件のリクエストを受け取り、それぞれのリクエストが`ResponseStarted`と`ResponseComplete`の段階でのみ監査されるとします。毎秒≅200の監査イベントが発生すると考えてください。 1 つのバッチに最大 100 個のイベントがあるの場合、スロットリングレベルを少なくとも2クエリ/秒に設定する必要があります。 バックエンドがイベントを書き込むのに最大で5秒かかる場合、5秒分のイベントを保持するようにバッファサイズを設定する必要があります。 10バッチ、または1000イベントとなります。 しかし、ほとんどの場合デフォルトのパラメーターで十分であり、手動で設定する必要はありません。 kube-apiserverが公開している以下のPrometheusメトリクスや、ログを見て監査サブシステムの状態を監視することができます。 - `apiserver_audit_event_total` メトリックには、エクスポートされた監査イベントの合計数が含まれます。 - `apiserver_audit_error_total` メトリックには、エクスポート中にエラーが発生してドロップされたイベントの総数が含まれます。 ### ログエントリー・トランケーション {#truncate} logバックエンドとwebhookバックエンドは、ログに記録されるイベントのサイズを制限することをサポートしています。 例として、logバックエンドで利用可能なフラグの一覧を以下に示します - `audit-log-truncate-enabled` イベントとバッチの切り捨てを有効にするかどうかです。 - `audit-log-truncate-max-batch-size` バックエンドに送信されるバッチのバイト単位の最大サイズ。 - `audit-log-truncate-max-event-size` バックエンドに送信される監査イベントのバイト単位の最大サイズです。 デフォルトでは、`webhook`と`log`の両方で切り捨ては無効になっていますが、クラスター管理者は `audit-log-truncate-enabled`または`audit-webhook-truncate-enabled`を設定して、この機能を有効にする必要があります。 ## {{% heading "whatsnext" %}} * [Mutating webhook auditing annotations](/docs/reference/access-authn-authz/extensible-admission-controllers/#mutating-webhook-auditing-annotations). * [`Event`](/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Event) * [`Policy`](/docs/reference/config-api/apiserver-audit.v1/#audit-k8s-io-v1-Policy) </ins>


### Reverting the Registry Name in kubelet <del> The image used by kubelet for the pod sandbox (`pause`) can be overridden by setting the `--pod-infra-container-image` flag. For example: </del> <ins> The image used by kubelet for the pod sandbox (`pause`) can be overridden by configuring your container runtime or by setting the `--pod-infra-container-image` flag depending on the version of Kubernetes you are using. Other runtimes: [containerd](/docs/setup/production-environment/container-runtimes/#override-pause-image-containerd), [CRI-O](/docs/setup/production-environment/container-runtimes/#override-pause-image-cri-o), [cri-dockerd](/docs/setup/production-environment/container-runtimes/#override-pause-image-cri-dockerd-mcr). When using dockershim before v1.23: </ins> ``` kubelet --pod-infra-container-image=k8s.gcr.io/pause:3.5


Your container runtime must support at least v1alpha2 of the container runtime interface. <del> Kubernetes {{< skew currentVersion >}}  defaults to using v1 of the CRI API. If a container runtime does not support the v1 API, the kubelet falls back to </del> <ins> Kubernetes [starting v1.26](/blog/2022/11/18/upcoming-changes-in-kubernetes-1-26/#cri-api-removal) _only works_ with v1 of the CRI API. Earlier versions default to v1 version, however if a container runtime does not support the v1 API, the kubelet falls back to </ins> using the (deprecated) v1alpha2 API instead. ## Container runtimes


You might need to restart `containerd` as well once you've updated the config file: `systemctl restart containerd`. <ins> Please note, that it is a best practice for kubelet to declare the matching `pod-infra-container-image`. If not configured, kubelet may attempt to garbage collect the `pause` image. There is ongoing work in [containerd to pin the pause image](https://github.com/containerd/containerd/issues/6352) and not require this setting on kubelet any longer. </ins> ### CRI-O This section contains the necessary steps to install CRI-O as a container runtime.


If you use Docker Engine via dockershim as your container runtime and wish to upgrade to v1.24, it is recommended that you either migrate to another runtime or find an alternative means to obtain Docker Engine support. Check out the [container runtimes](/docs/setup/production-environment/container-runtimes/) <del> section to know your options. Make sure to </del> <ins> section to know your options. The version of Kubernetes with dockershim (1.23) is out of support and the v1.24 will run out of support [soon](/releases/#release-v1-24). Make sure to </ins> [report issues](https://github.com/kubernetes/kubernetes/issues) you encountered with the migration so the issues can be fixed in a timely manner and your cluster would be <del> ready for dockershim removal. </del> <ins> ready for dockershim removal. After v1.24 running out of support, you will need to contact your Kubernetes provider for support or upgrade multiple versions at a time if there are critical issues affecting your cluster. </ins> Your cluster might have more than one kind of node, although this is not a common configuration.


* Check out [container runtimes](/docs/setup/production-environment/container-runtimes/) to understand your options for an alternative. <del> * There is a [GitHub issue](https://github.com/kubernetes/kubernetes/issues/106917) to track the discussion about the deprecation and removal of dockershim. </del> * If you find a defect or other technical concern relating to migrating away from dockershim, you can [report an issue](https://github.com/kubernetes/kubernetes/issues/new/choose) to the Kubernetes project.


Kubernetes. They needed to be pushed to some registry to allow them to be used by Kubernetes. <ins> ## Known issues ### Some filesystem metrics are missing and the metrics format is different The Kubelet `/metrics/cadvisor` endpoint provides Prometheus metrics, as documented in [Metrics for Kubernetes system components](/docs/concepts/cluster-administration/system-metrics/). If you install a metrics collector that depends on that endpoint, you might see the following issues: - The metrics format on the Docker node is `k8s_<container-name>_<pod-name>_<namespace>_<pod-uid>_<restart-count>` but the format on other runtime is different. For example, on containerd node it is `<container-id>`. - Some filesystem metrics are missing, as follows: ``` container_fs_inodes_free container_fs_inodes_total container_fs_io_current container_fs_io_time_seconds_total container_fs_io_time_weighted_seconds_total container_fs_limit_bytes container_fs_read_seconds_total container_fs_reads_merged_total container_fs_sector_reads_total container_fs_sector_writes_total container_fs_usage_bytes container_fs_write_seconds_total container_fs_writes_merged_total ``` #### Workaround You can mitigate this issue by using [cAdvisor](https://github.com/google/cadvisor) as a standalone daemonset. 1. Find the latest [cAdvisor release](https://github.com/google/cadvisor/releases) with the name pattern `vX.Y.Z-containerd-cri` (for example, `v0.42.0-containerd-cri`). 2. Follow the steps in [cAdvisor Kubernetes Daemonset](https://github.com/google/cadvisor/tree/master/deploy/kubernetes) to create the daemonset. 3. Point the installed metrics collector to use the cAdvisor `/metrics` endpoint which provides the full set of [Prometheus container metrics](https://github.com/google/cadvisor/blob/master/docs/storage/prometheus.md). Alternatives: - Use alternative third party metrics collection solution. - Collect metrics from the Kubelet summary API that is served at `/stats/summary`. </ins> ## {{% heading "whatsnext" %}} - Read [Migrating from dockershim](/docs/tasks/administer-cluster/migrating-from-dockershim/) to understand your next steps


<!-- overview --> <del> If you want to control traffic flow at the IP address or port level (OSI layer 3 or 4), then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster.  NetworkPolicies are an application-centric construct which allow you to specify how a {{< glossary_tooltip text="pod" term_id="pod">}} is allowed to communicate with various network "entities" (we use the word "entity" here to avoid overloading the more common terms such as "endpoints" and "services", which have specific Kubernetes connotations) over the network. </del> <ins> If you want to control traffic flow at the IP address or port level (OSI layer 3 or 4), then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster.  NetworkPolicies are an application-centric construct which allow you to specify how a {{< glossary_tooltip text="pod" term_id="pod">}} is allowed to communicate with various network "entities" (we use the word "entity" here to avoid overloading the more common terms such as "endpoints" and "services", which have specific Kubernetes connotations) over the network. NetworkPolicies apply to a connection with a pod on one or both ends, and are not relevant to other connections. </ins> The entities that a Pod can communicate with are identified through a combination of the following 3 identifiers:


Network policies are implemented by the [network plugin](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/). To use network policies, you must be using a networking solution which supports NetworkPolicy. Creating a NetworkPolicy resource without a controller that implements it will have no effect. <del> ## Isolated and Non-isolated Pods </del> <ins> ## The Two Sorts of Pod Isolation </ins> <del> By default, pods are non-isolated; they accept traffic from any source. </del> <ins> There are two sorts of isolation for a pod: isolation for egress, and isolation for ingress.  They concern what connections may be established.  "Isolation" here is not absolute, rather it means "some restrictions apply".  The alternative, "non-isolated for $direction", means that no restrictions apply in the stated direction.  The two sorts of isolation (or not) are declared independently, and are both relevant for a connection from one pod to another. </ins> <del> Pods become isolated by having a NetworkPolicy that selects them. Once there is any NetworkPolicy in a namespace selecting a particular pod, that pod will reject any connections that are not allowed by any NetworkPolicy. (Other pods in the namespace that are not selected by any NetworkPolicy will continue to accept all traffic.) </del> <ins> By default, a pod is non-isolated for egress; all outbound connections are allowed.  A pod is isolated for egress if there is any NetworkPolicy that both selects the pod and has "Egress" in its `policyTypes`; we say that such a policy applies to the pod for egress.  When a pod is isolated for egress, the only allowed connections from the pod are those allowed by the `egress` list of some NetworkPolicy that applies to the pod for egress.  The effects of those `egress` lists combine additively. </ins> <del> Network policies do not conflict; they are additive. If any policy or policies select a pod, the pod is restricted to what is allowed by the union of those policies' ingress/egress rules. Thus, order of evaluation does not affect the policy result. </del> <ins> By default, a pod is non-isolated for ingress; all inbound connections are allowed.  A pod is isolated for ingress if there is any NetworkPolicy that both selects the pod and has "Ingress" in its `policyTypes`; we say that such a policy applies to the pod for ingress.  When a pod is isolated for ingress, the only allowed connections into the pod are those from the pod's node and those allowed by the `ingress` list of some NetworkPolicy that applies to the pod for ingress.  The effects of those `ingress` lists combine additively. </ins> <del> For a network flow between two pods to be allowed, both the egress policy on the source pod and the ingress policy on the destination pod need to allow the traffic. If either the egress policy on the source, or the ingress policy on the destination denies the traffic, the traffic will be denied. </del> <ins> Network policies do not conflict; they are additive. If any policy or policies apply to a given pod for a given direction, the connections allowed in that direction from that pod is the union of what the applicable policies allow. Thus, order of evaluation does not affect the policy result. For a connection from a source pod to a destination pod to be allowed, both the egress policy on the source pod and the ingress policy on the destination pod need to allow the connection. If either side does not allow the connection, it will not happen. </ins> ## The NetworkPolicy resource {#networkpolicy-resource}


### Default deny all ingress traffic <del> You can create a "default" isolation policy for a namespace by creating a NetworkPolicy that selects all pods but does not allow any ingress traffic to those pods. </del> <ins> You can create a "default" ingress isolation policy for a namespace by creating a NetworkPolicy that selects all pods but does not allow any ingress traffic to those pods. </ins> {{< codenew file="service/networking/network-policy-default-deny-ingress.yaml" >}} <del> This ensures that even pods that aren't selected by any other NetworkPolicy will still be isolated. This policy does not change the default egress isolation behavior. </del> <ins> This ensures that even pods that aren't selected by any other NetworkPolicy will still be isolated for ingress. This policy does not affect isolation for egress from any pod. </ins> <del> ### Default allow all ingress traffic </del> <ins> ### Allow all ingress traffic </ins> <del> If you want to allow all traffic to all pods in a namespace (even if policies are added that cause some pods to be treated as "isolated"), you can create a policy that explicitly allows all traffic in that namespace. </del> <ins> If you want to allow all incoming connections to all pods in a namespace, you can create a policy that explicitly allows that. </ins> {{< codenew file="service/networking/network-policy-allow-all-ingress.yaml" >}} <ins> With this policy in place, no additional policy or policies can cause any incoming connection to those pods to be denied.  This policy has no effect on isolation for egress from any pod. </ins> ### Default deny all egress traffic You can create a "default" egress isolation policy for a namespace by creating a NetworkPolicy that selects all pods but does not allow any egress traffic from those pods.


{{< codenew file="service/networking/network-policy-default-deny-egress.yaml" >}} This ensures that even pods that aren't selected by any other NetworkPolicy will not be allowed egress traffic. This policy does not <del> change the default ingress isolation behavior. </del> <ins> change the ingress isolation behavior of any pod. </ins> <del> ### Default allow all egress traffic </del> <ins> ### Allow all egress traffic </ins> <del> If you want to allow all traffic from all pods in a namespace (even if policies are added that cause some pods to be treated as "isolated"), you can create a policy that explicitly allows all egress traffic in that namespace. </del> <ins> If you want to allow all connections from all pods in a namespace, you can create a policy that explicitly allows all outgoing connections from pods in that namespace. </ins> {{< codenew file="service/networking/network-policy-allow-all-egress.yaml" >}} <ins> With this policy in place, no additional policy or policies can cause any outgoing connection from those pods to be denied.  This policy has no effect on isolation for ingress to any pod. </ins> ### Default deny all ingress and all egress traffic You can create a "default" policy for a namespace which prevents all ingress AND egress traffic by creating the following NetworkPolicy in that namespace.


If you don't specify a registry hostname, Kubernetes assumes that you mean the Docker public registry. <del> After the image name part you can add a _tag_ (as also using with commands such as `docker` and `podman`). </del> <ins> After the image name part you can add a _tag_ (in the same way you would when using with commands like `docker` or `podman`). </ins> Tags let you identify different versions of the same series of images. Image tags consist of lowercase and uppercase letters, digits, underscores (`_`),


replace `<image-name>:<tag>` with `<image-name>@<digest>` (for example, `image@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2`). <del> When using image tags, if the image registry were to change the code that the tag on that image represents, you might end up with a mix of Pods running the old and new code. An image digest uniquely identifies a specific version of the image, so Kubernetes runs the same code every time it starts a container with that image name and digest specified. Specifying an image fixes the code that you run so that a change at the registry cannot lead to that mix of versions. </del> <ins> When using image tags, if the image registry were to change the code that the tag on that image represents, you might end up with a mix of Pods running the old and new code. An image digest uniquely identifies a specific version of the image, so Kubernetes runs the same code every time it starts a container with that image name and digest specified. Specifying an image by digest fixes the code that you run so that a change at the registry cannot lead to that mix of versions. </ins> There are third-party [admission controllers](/docs/reference/access-authn-authz/admission-controllers/) that mutate Pods (and pod templates) when they are created, so that the


### Configuring nodes to authenticate to a private registry <del> If you run Docker on your nodes, you can configure the Docker container runtime to authenticate to a private container registry. </del> <ins> Specific instructions for setting credentials depends on the container runtime and registry you chose to use. You should refer to your solution's documentation for the most accurate information. </ins> <del> This approach is suitable if you can control node configuration. {{< note >}} Default Kubernetes only supports the `auths` and `HttpHeaders` section in Docker configuration. Docker credential helpers (`credHelpers` or `credsStore`) are not supported. {{< /note >}} Docker stores keys for private registries in the `$HOME/.dockercfg` or `$HOME/.docker/config.json` file.  If you put the same file in the search paths list below, kubelet uses it as the credential provider when pulling images. * `{--root-dir:-/var/lib/kubelet}/config.json` * `{cwd of kubelet}/config.json` * `${HOME}/.docker/config.json` * `/.docker/config.json` * `{--root-dir:-/var/lib/kubelet}/.dockercfg` * `{cwd of kubelet}/.dockercfg` * `${HOME}/.dockercfg` * `/.dockercfg` {{< note >}} You may have to set `HOME=/root` explicitly in the environment of the kubelet process. {{< /note >}} Here are the recommended steps to configuring your nodes to use a private registry.  In this example, run these on your desktop/laptop: 1. Run `docker login [server]` for each set of credentials you want to use.  This updates `$HOME/.docker/config.json` on your PC. 1. View `$HOME/.docker/config.json` in an editor to ensure it contains only the credentials you want to use. 1. Get a list of your nodes; for example: - if you want the names: `nodes=$( kubectl get nodes -o jsonpath='{range.items[*].metadata}{.name} {end}' )` - if you want to get the IP addresses: `nodes=$( kubectl get nodes -o jsonpath='{range .items[*].status.addresses[?(@.type=="ExternalIP")]}{.address} {end}' )` 1. Copy your local `.docker/config.json` to one of the search paths list above. - for example, to test this out: `for n in $nodes; do scp ~/.docker/config.json root@"$n":/var/lib/kubelet/config.json; done` {{< note >}} For production clusters, use a configuration management tool so that you can apply this setting to all the nodes where you need it. {{< /note >}} Verify by creating a Pod that uses a private image; for example: ```shell kubectl apply -f - <<EOF apiVersion: v1 kind: Pod metadata: name: private-image-test-1 spec: containers: - name: uses-private-image image: $PRIVATE_IMAGE_NAME imagePullPolicy: Always command: [ "echo", "SUCCESS" ] EOF ``` ``` pod/private-image-test-1 created ``` If everything is working, then, after a few moments, you can run: ```shell kubectl logs private-image-test-1 ``` and see that the command outputs: ``` SUCCESS ``` If you suspect that the command failed, you can run: ```shell kubectl describe pods/private-image-test-1 | grep 'Failed' ``` In case of failure, the output is similar to: ``` Fri, 26 Jun 2015 15:36:13 -0700    Fri, 26 Jun 2015 15:39:13 -0700    19    {kubelet node-i2hq}    spec.containers{uses-private-image}    failed        Failed to pull image "user/privaterepo:v1": Error: image user/privaterepo:v1 not found ``` You must ensure all nodes in the cluster have the same `.docker/config.json`.  Otherwise, pods will run on some nodes and fail to run on others.  For example, if you use node autoscaling, then each instance template needs to include the `.docker/config.json` or mount a drive that contains it. All pods will have read access to images in any private registry once private registry keys are added to the `.docker/config.json`. </del> <ins> For an example of configuring a private container image registry, see the [Pull an Image from a Private Registry](/docs/tasks/configure-pod-container/pull-image-private-registry) task. That example uses a private registry in Docker Hub. </ins> ### Interpretation of config.json {#config-json}


to be pulled, then the kubelet will try to download them from both authentication sources if one of them fails. ### Pre-pulled images {{< note >}}


#### Creating a Secret with a Docker config <ins> You need to know the username, registry password and client email address for authenticating to the registry, as well as its hostname. </ins> Run the following command, substituting the appropriate uppercase values: ```shell


common use cases and suggested solutions. 1. Cluster running only non-proprietary (e.g. open-source) images.  No need to hide images. <del> - Use public images on the Docker hub. </del> <ins> - Use public images from a public registry </ins> - No configuration required. - Some cloud providers automatically cache or mirror public images, which improves availability and reduces the time to pull images. 1. Cluster running some proprietary images which should be hidden to those outside the company, but visible to all cluster users. <del> - Use a hosted private [Docker registry](https://docs.docker.com/registry/). - It may be hosted on the [Docker Hub](https://hub.docker.com/signup), or elsewhere. - Manually configure .docker/config.json on each node as described above. </del> <ins> - Use a hosted private registry - Manual configuration may be required on the nodes that need to access to private registry </ins> - Or, run an internal private registry behind your firewall with open read access. - No Kubernetes configuration is required. - Use a hosted container image registry service that controls image access


If you need access to multiple registries, you can create one secret for each registry. <del> Kubelet will merge any `imagePullSecrets` into a single virtual `.docker/config.json` </del> ## {{% heading "whatsnext" %}}


[the design doc](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/metrics-server.md). ### Summary API Source <del> The [Kubelet](/docs/reference/command-line-tools-reference/kubelet/) gathers stats at node, volume, pod and container level, and omits them in the [Summary API](https://github.com/kubernetes/kubernetes/blob/7d309e0104fedb57280b261e5677d919cb2a0e2d/staging/src/k8s.io/kubelet/pkg/apis/stats/v1alpha1/types.go) </del> <ins> The [Kubelet](/docs/reference/command-line-tools-reference/kubelet/) gathers stats at node, volume, pod and container level, and emits their statistics in the [Summary API](https://github.com/kubernetes/kubernetes/blob/7d309e0104fedb57280b261e5677d919cb2a0e2d/staging/src/k8s.io/kubelet/pkg/apis/stats/v1alpha1/types.go) </ins> for consumers to read. Pre-1.23, these resources have been primarily gathered from [cAdvisor](https://github.com/google/cadvisor). However, in 1.23 with the


A partir de Kubernetes 1.8, el servidor de métricas se despliega por defecto como un objeto de tipo [Deployment](https://github.com/docs/concepts/workloads/controllers/deployment/) en clústeres creados con el script `kube-up.sh`. Si usas otro mecanismo de configuración de Kubernetes, puedes desplegarlo <del> usando los [yamls de despliegue](https://github.com/kubernetes-incubator/metrics-server/tree/master/deploy) </del> <ins> usando los [yamls de despliegue](https://github.com/kubernetes-sigs/metrics-server/releases) </ins> proporcionados. Está soportado a partir de Kubernetes 1.7 (más detalles al final). El servidor reune métricas de la Summary API, que es expuesta por el [Kubelet](/docs/admin/kubelet/) en cada nodo.


1. The kubelet on a node self-registers to the control plane 2. You (or another human user) manually add a Node object <del> After you create a Node object, or the kubelet on a node self-registers, the </del> <ins> After you create a Node {{< glossary_tooltip text="object" term_id="object" >}}, or the kubelet on a node self-registers, the </ins> control plane checks whether the new Node object is valid. For example, if you try to create a Node from the following JSON manifest:


} } <del> body > footer { width: 100vw; } </del> /* FOOTER */ footer { background-color: #303030;


margin-top: 25px; } <del> footer { float: left !important; } </del> p { font-weight: 300 !important; }


.resources { width: 100%; <del> margin-top; 5%; </del> <ins> margin-top: 5%; </ins> margin-bottom: 3%; float: left; }


font-weight: bold; } <del> /* .fullbutton { padding: 3% !important; } */ </del> } @media (max-width:1000px) {


.resources { width: 100%; <del> margin-top; 8%; </del> <ins> margin-top: 8%; </ins> margin-bottom: 8%; float: left; }


margin-left: 5%; } .contributor { float: left; width: 100%;


font-size: 0.7em !important; } }  No newline at end of file


<ins> --- title: 拡張リソースのリソースビンパッキング content_type: concept weight: 80 --- <!-- overview --> {{< feature-state for_k8s_version="v1.16" state="alpha" >}} kube-schedulerでは、優先度関数`RequestedToCapacityRatioResourceAllocation`を使用した、 拡張リソースを含むリソースのビンパッキングを有効化できます。優先度関数はそれぞれのニーズに応じて、kube-schedulerを微調整するために使用できます。 <!-- body --> ## `RequestedToCapacityRatioResourceAllocation`を使用したビンパッキングの有効化 Kubernetesでは、キャパシティー比率への要求に基づいたNodeのスコアリングをするために、各リソースの重みと共にリソースを指定することができます。これにより、ユーザーは適切なパラメーターを使用することで拡張リソースをビンパックすることができ、大規模クラスターにおける希少なリソースを有効活用できるようになります。優先度関数`RequestedToCapacityRatioResourceAllocation`の動作は`RequestedToCapacityRatioArgs`と呼ばれる設定オプションによって変わります。この引数は`shape`と`resources`パラメーターによって構成されます。`shape`パラメーターは`utilization`と`score`の値に基づいて、最も要求が多い場合か最も要求が少ない場合の関数をチューニングできます。`resources`パラメーターは、スコアリングの際に考慮されるリソース名の`name`と、各リソースの重みを指定する`weight`で構成されます。 以下は、拡張リソース`intel.com/foo`と`intel.com/bar`のビンパッキングに`requestedToCapacityRatioArguments`を設定する例になります。 ```yaml apiVersion: kubescheduler.config.k8s.io/v1beta1 kind: KubeSchedulerConfiguration profiles: # ... pluginConfig: - name: RequestedToCapacityRatio args: shape: - utilization: 0 score: 10 - utilization: 100 score: 0 resources: - name: intel.com/foo weight: 3 - name: intel.com/bar weight: 5 ``` スケジューラーには、kube-schedulerフラグ`--config=/path/to/config/file`を使用して`KubeSchedulerConfiguration`のファイルを指定することで渡すことができます。 **この機能はデフォルトで無効化されています** ### 優先度関数のチューニング `shape`は`RequestedToCapacityRatioPriority`関数の動作を指定するために使用されます。 ```yaml shape: - utilization: 0 score: 0 - utilization: 100 score: 10 ``` 上記の引数は、`utilization`が0%の場合は0、`utilization`が100%の場合は10という`score`をNodeに与え、ビンパッキングの動作を有効にしています。最小要求を有効にするには、次のようにスコアを反転させる必要があります。 ```yaml shape: - utilization: 0 score: 10 - utilization: 100 score: 0 ``` `resources`はオプションパラメーターで、デフォルトでは以下の通りです。 ``` yaml resources: - name: cpu weight: 1 - name: memory weight: 1 ``` 以下のように拡張リソースの追加に利用できます。 ```yaml resources: - name: intel.com/foo weight: 5 - name: cpu weight: 3 - name: memory weight: 1 ``` `weight`はオプションパラメーターで、指定されてない場合1が設定されます。また、マイナスの値は設定できません。 ### キャパシティ割り当てのためのNodeスコアリング このセクションは、本機能の内部詳細について理解したい方を対象としています。以下は、与えられた値に対してNodeのスコアがどのように計算されるかの例です。 要求されたリソース: ``` intel.com/foo : 2 memory: 256MB cpu: 2 ``` リソースの重み: ``` intel.com/foo : 5 memory: 1 cpu: 3 ``` `shape`の値 {{0, 0}, {100, 10}} Node1のスペック: ``` Available: intel.com/foo: 4 memory: 1 GB cpu: 8 Used: intel.com/foo: 1 memory: 256MB cpu: 1 ``` Nodeのスコア: ``` intel.com/foo  = resourceScoringFunction((2+1),4) = (100 - ((4-3)*100/4) = (100 - 25) = 75                       # requested + used = 75% * available = rawScoringFunction(75) = 7                        # floor(75/10) memory         = resourceScoringFunction((256+256),1024) = (100 -((1024-512)*100/1024)) = 50                       # requested + used = 50% * available = rawScoringFunction(50) = 5                        # floor(50/10) cpu            = resourceScoringFunction((2+1),8) = (100 -((8-3)*100/8)) = 37.5                     # requested + used = 37.5% * available = rawScoringFunction(37.5) = 3                        # floor(37.5/10) NodeScore   =  (7 * 5) + (5 * 1) + (3 * 3) / (5 + 1 + 3) =  5 ``` Node2のスペック: ``` Available: intel.com/foo: 8 memory: 1GB cpu: 8 Used: intel.com/foo: 2 memory: 512MB cpu: 6 ``` Nodeのスコア: ``` intel.com/foo  = resourceScoringFunction((2+2),8) =  (100 - ((8-4)*100/8) =  (100 - 50) =  50 =  rawScoringFunction(50) = 5 memory         = resourceScoringFunction((256+512),1024) = (100 -((1024-768)*100/1024)) = 75 = rawScoringFunction(75) = 7 cpu            = resourceScoringFunction((2+6),8) = (100 -((8-8)*100/8)) = 100 = rawScoringFunction(100) = 10 NodeScore   =  (5 * 5) + (7 * 1) + (10 * 3) / (5 + 1 + 3) =  7 ``` ## {{% heading "whatsnext" %}} - [スケジューリングフレームワーク](/ja/docs/concepts/scheduling-eviction/scheduling-framework/)について更に読む - [スケジューラーの設定](/docs/reference/scheduling/config/)について更に読む </ins>


## Targeting a Namespace by its name <del> {{< feature-state state="beta" for_k8s_version="1.21" >}} </del> <ins> {{< feature-state for_k8s_version="1.22" state="stable" >}} </ins> The Kubernetes control plane sets an immutable label `kubernetes.io/metadata.name` on all namespaces, provided that the `NamespaceDefaultLabelName`