# Deserializing Debug Proto Representations

How to log debugging information in Protocol Buffers.

From version 30.x, Protobuf `DebugString` APIs (`Message::DebugString`, `Message::ShortDebugString`, `Message::Utf8DebugString`), additional Protobuf APIs (`proto2::ShortFormat`, `proto2::Utf8Format`), Abseil string functions (such as `absl::StrCat`, `absl::StrFormat`, `absl::StrAppend`, and `absl::Substitute`), and Abseil logging API will begin to automatically convert proto arguments into a new debugging format . See the related announcement [here](https://protobuf.dev/news/2024-12-04/).

Unlike the Protobuf DebugString output format, the new debugging format automatically redacts sensitive fields by replacing their values with the string “[REDACTED]” (without the quotation marks). In addition, to ensure that this new output format cannot be deserialized by Protobuf TextFormat parsers, regardless of whether the underlying proto contains SPII fields, we add a set of randomized links pointing to this article and a randomized-length whitespace sequence. The new debugging format looks as follows:

    goo.gle/debugstr
    spii_field: [REDACTED]
    normal_field: "value"

Note that the new debugging format is only different from the output format of DebugString format in two ways:

  * The URL prefix
  * The values of SPII fields are replaced by “[REDACTED]” (without the quotes)

The new debugging format never removes any field names; it only replaces the value with “[REDACTED]” if the field is considered sensitive. **If you don’t see certain fields in the output, it is because those fields are not set in the proto.**

**Tip:** If you see only the URL and nothing else, your proto is empty!

## Why is this URL here?

We want to make sure nobody deserializes human-readable representations of a protobuf message intended for humans debugging a system. Historically, `.DebugString()` and `TextFormat` were interchangeable, and existing systems use DebugString to transport and store data.

We want to make sure sensitive data does not accidentally end up in logs. Therefore, we are transparently redacting some field values from protobuf messages before turning them into a string ("[REDACTED]"). This reduces the security & privacy risk of accidental logging, but risks data loss if other systems deserialize your message. To address this risk, we are intentionally splitting the machine-readable TextFormat from the human-readable debug format to be used in log messages.

### Why are there links in my web page? Why is my code producing this new “debug representation”?

This is intentional, to make the “debug representation” of your protos (produced, for example, by logging) incompatible with TextFormat. We want to prevent anyone from depending on debugging mechanisms to transport data between programs. Historically, the debug format (generated by the DebugString APIs) and TextFormat have been incorrectly used in a interchangeable fashion. We hope this intentional effort will prevent that going forward.

We intentionally picked a link over less visible format changes to get an opportunity to provide context. This might stand out in UIs, such as if you display status information on a table in a webpage. You may use `TextFormat::PrintToString` instead, which will not redact any information and preserves formatting. However, use this API cautiously – there are no built in protections. As a rule of thumb, if you are writing data to debug logs, or producing status messages, you should continue to use the Debug Format with the link. Even if you are currently not handling sensitive data, keep in mind that systems can change and code gets re-used.

### I tried converting this message into TextFormat, but I noticed the format changes every time my process restarts.

This is intentional. Don’t attempt to parse the output of this debug format. We reserve the right to change the syntax without notice. The debug format syntax randomly changes per process to prevent inadvertent dependencies. If a syntactic change in the debug format would break your system, chances are you should be using a TextFormat API rather than using the debug representation of a proto.

## FAQ

### Can I Just Use TextFormat Everywhere?

Don’t use TextFormat for producing log messages. This will bypass all built-in protections, and you risk accidentally logging sensitive information. Even if your systems are currently not handling any sensitive data, this can change in the future.

Distinguish logs from information that’s meant for further processing by other systems by using either the debug representation or TextFormat as appropriate.

### I Want to Write Configuration Files That Need to Be Both Human-Readable And Machine-Readable

For this use case, you can use TextFormat explicitly. You are responsible for making sure your configuration files don’t contain any PII.

### I Am Writing a Unit Test, and Want to Compare Debugstring in a Test Assertion

If you want to compare protobuf values, use `MessageDifferencer` like in the following:

    using google::protobuf::util::MessageDifferencer;
    ...
    MessageDifferencer diff;
    ...
    diff.Compare(foo, bar);

Besides ignoring formatting and field order differences, you will also get better error messages.


# Protocol Buffers Well-Known Types

API documentation for the google.protobuf package.

## Index

  * `Any` (message)
  * `Api` (message)
  * `BoolValue` (message)
  * `BytesValue` (message)
  * `DoubleValue` (message)
  * `Duration` (message)
  * `Empty` (message)
  * `Enum` (message)
  * `EnumValue` (message)
  * `Field` (message)
  * `Field.Cardinality` (enum)
  * `Field.Kind` (enum)
  * `FieldMask` (message)
  * `FloatValue` (message)
  * `Int32Value` (message)
  * `Int64Value` (message)
  * `ListValue` (message)
  * `Method` (message)
  * `Mixin` (message)
  * `NullValue` (enum)
  * `Option` (message)
  * `SourceContext` (message)
  * `StringValue` (message)
  * `Struct` (message)
  * `Syntax` (enum)
  * `Timestamp` (message)
  * `Type` (message)
  * `UInt32Value` (message)
  * `UInt64Value` (message)
  * `Value` (message)

Well-Known Types that end in “`Value`” are wrapper messages for other types, such as `BoolValue` and `EnumValue`. These are now obsolete. The only reasons to use wrappers today would be:

  * Wire compatibility with messages that already use them.
  * If you want to put a scalar value into an `Any` message.

In most cases, there are better options:

  * For new messages, it’s better to use regular explicit-presence fields (`optional` in proto2/proto3, regular field in edition >= 2023).
  * Extensions are generally a better option than `Any` fields.

## Any

`Any` contains an arbitrary serialized message along with a URL that describes the type of the serialized message.

#### JSON

The JSON representation of an `Any` value uses the regular representation of the deserialized, embedded message, with an additional field `@type` which contains the type URL. Example:

    package google.profile;
    message Person {
      string first_name = 1;
      string last_name = 2;

      "@type": "type.googleapis.com/google.profile.Person",
      "firstName": <string>,
      "lastName": <string>

If the embedded message type is well-known and has a custom JSON representation, that representation will be embedded adding a field `value` which holds the custom JSON in addition to the `@type` field. Example (for message `google.protobuf.Duration`):

      "@type": "type.googleapis.com/google.protobuf.Duration",
      "value": "1.212s"

Field name| Type| Description
---|---|---
`type_url`| `string`| A URL/resource name whose content describes the type of the serialized message.For URLs which use the schema `http`, `https`, or no schema, the following restrictions and interpretations apply:

  * If no schema is provided, `https` is assumed.
  * The last segment of the URL's path must represent the fully qualified name of the type (as in `path/google.protobuf.Duration`).
  * An HTTP GET on the URL must yield a `google.protobuf.Type` value in binary format, or produce an error.
  * Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)

Schemas other than `http`, `https` (or the empty schema) might be used with implementation specific semantics.
`value`| `bytes`| Must be valid serialized data of the above specified type.

## Api

Api is a light-weight descriptor for a protocol buffer service.

Field name| Type| Description
---|---|---
`name`| `string`| The fully qualified name of this api, including package name followed by the api's simple name.
`methods`| `Method`| The methods of this api, in unspecified order.
`options`| `Option`| Any metadata attached to the API.
`version`| `string`| A version string for this api. If specified, must have the form `major-version.minor-version`, as in `1.10`. If the minor version is omitted, it defaults to zero. If the entire version field is empty, the major version is derived from the package name, as outlined below. If the field is not empty, the version in the package name will be verified to be consistent with what is provided here.The versioning schema uses [semantic versioning](http://semver.org) where the major version number indicates a breaking change and the minor version an additive, non-breaking change. Both version numbers are signals to users what to expect from different versions, and should be carefully chosen based on the product plan.The major version is also reflected in the package name of the API, which must end in `v<major-version>`, as in `google.feature.v1`. For major versions 0 and 1, the suffix can be omitted. Zero major versions must only be used for experimental, none-GA apis.
`source_context`| ``SourceContext``| Source context for the protocol buffer service represented by this message.
`mixins`| ``Mixin``| Included APIs. See `Mixin`.
`syntax`| `Syntax`| The source syntax of the service.

## BoolValue

Wrapper message for `bool`.

The JSON representation for `BoolValue` is JSON `true` and `false`.

Field name| Type| Description
---|---|---
`value`| `bool`| The bool value.

## BytesValue

Wrapper message for `bytes`.

The JSON representation for `BytesValue` is JSON string.

Field name| Type| Description
---|---|---
`value`| `bytes`| The bytes value.

## DoubleValue

Wrapper message for `double`.

The JSON representation for `DoubleValue` is JSON number.

Field name| Type| Description
---|---|---
`value`| `double`| The double value.

## Duration

A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

Example 1: Compute Duration from two Timestamps in pseudo code.

    Timestamp start = ...;
    Timestamp end = ...;
    Duration duration = ...;

    duration.seconds = end.seconds - start.seconds;
    duration.nanos = end.nanos - start.nanos;

    if (duration.seconds < 0 && duration.nanos > 0) {
      duration.seconds += 1;
      duration.nanos -= 1000000000;
    } else if (duration.seconds > 0 && duration.nanos < 0) {
      duration.seconds -= 1;
      duration.nanos += 1000000000;

Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

    Timestamp start = ...;
    Duration duration = ...;
    Timestamp end = ...;

    end.seconds = start.seconds + duration.seconds;
    end.nanos = start.nanos + duration.nanos;

    if (end.nanos < 0) {
      end.seconds -= 1;
      end.nanos += 1000000000;
    } else if (end.nanos >= 1000000000) {
      end.seconds += 1;
      end.nanos -= 1000000000;

The JSON representation for `Duration` is a `String` that ends in `s` to indicate seconds and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds.

Field name| Type| Description
---|---|---
`seconds`| `int64`| Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive.
`nanos`| `int32`| Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 `seconds` field and a positive or negative `nanos` field. For durations of one second or more, a non-zero value for the `nanos` field must be of the same sign as the `seconds` field. Must be from -999,999,999 to +999,999,999 inclusive.

## Empty

A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance:

    service Foo {
      rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);

The JSON representation for `Empty` is empty JSON object `{}`.

## Enum

Enum type definition

Field name| Type| Description
---|---|---
`name`| `string`| Enum type name.
`enumvalue`| `EnumValue`| Enum value definitions.
`options`| `Option`| Protocol buffer options.
`source_context`| `SourceContext`| The source context.
`syntax`| `Syntax`| The source syntax.
`edition`| `string`| The source edition if `syntax` is `SYNTAX_EDITIONS`.

## EnumValue

Enum value definition.

Field name| Type| Description
---|---|---
`name`| `string`| Enum value name.
`number`| `int32`| Enum value number.
`options`| `Option`| Protocol buffer options.

## Field

A single field of a message type.

Field name| Type| Description
---|---|---
`kind`| `Kind`| The field type.
`cardinality`| `Cardinality`| The field cardinality.
`number`| `int32`| The field number.
`name`| `string`| The field name.
`type_url`| `string`| The field type URL, without the scheme, for message or enumeration types. Example: `"type.googleapis.com/google.protobuf.Timestamp"`.
`oneof_index`| `int32`| The index of the field type in `Type.oneofs`, for message or enumeration types. The first type has index 1; zero means the type is not in the list.
`packed`| `bool`| Whether to use alternative packed wire representation.
`options`| `Option`| The protocol buffer options.
`json_name`| `string`| The field JSON name.
`default_value`| `string`| The string value of the default value of this field. Proto2 syntax only.

## Cardinality

Whether a field is optional, required, or repeated.

Enum value| Description
`CARDINALITY_UNKNOWN`| For fields with unknown cardinality.
`CARDINALITY_OPTIONAL`| For optional fields.
`CARDINALITY_REQUIRED`| For required fields. Proto2 syntax only.
`CARDINALITY_REPEATED`| For repeated fields.

## Kind

Basic field types.

Enum value| Description
`TYPE_UNKNOWN`| Field type unknown.
`TYPE_DOUBLE`| Field type double.
`TYPE_FLOAT`| Field type float.
`TYPE_INT64`| Field type int64.
`TYPE_UINT64`| Field type uint64.
`TYPE_INT32`| Field type int32.
`TYPE_FIXED64`| Field type fixed64.
`TYPE_FIXED32`| Field type fixed32.
`TYPE_BOOL`| Field type bool.
`TYPE_STRING`| Field type string.
`TYPE_GROUP`| Field type group. Proto2 syntax only, and deprecated.
`TYPE_MESSAGE`| Field type message.
`TYPE_BYTES`| Field type bytes.
`TYPE_UINT32`| Field type uint32.
`TYPE_ENUM`| Field type enum.
`TYPE_SFIXED32`| Field type sfixed32.
`TYPE_SFIXED64`| Field type sfixed64.
`TYPE_SINT32`| Field type sint32.
`TYPE_SINT64`| Field type sint64.

## FieldMask

`FieldMask` represents a set of symbolic field paths, for example:

    paths: "f.a"
    paths: "f.b.d"

Here `f` represents a field in some root message, `a` and `b` fields in the message found in `f`, and `d` a field found in the message in `f.b`.

Field masks are used to specify a subset of fields that should be returned by a get operation (a _projection_), or modified by an update operation. Field masks also have a custom JSON encoding (see below).

#### Field Masks in Projections

When a `FieldMask` specifies a _projection_ , the API will filter the response message (or sub-message) to contain only those fields specified in the mask. For example, consider this "pre-masking" response message:

    f {
      a : 22
      b {
        d : 1
        x : 2
      y : 13
    z: 8

After applying the mask in the previous example, the API response will not contain specific values for fields x, y, or z (their value will be set to the default, and omitted in proto text output):

    f {
      a : 22
      b {
        d : 1

A repeated field is not allowed except at the last position of a field mask.

If a `FieldMask` object is not present in a get operation, the operation applies to all fields (as if a FieldMask of all fields had been specified).

Note that a field mask does not necessarily apply to the top-level response message. In case of a REST get operation, the field mask applies directly to the response, but in case of a REST list operation, the mask instead applies to each individual message in the returned resource list. In case of a REST custom method, other definitions may be used. Where the mask applies will be clearly documented together with its declaration in the API. In any case, the effect on the returned resource/resources is required behavior for APIs.

#### Field Masks in Update Operations

A field mask in update operations specifies which fields of the targeted resource are going to be updated. The API is required to only change the values of the fields as specified in the mask and leave the others untouched. If a resource is passed in to describe the updated values, the API ignores the values of all fields not covered by the mask.

In order to reset a field’s value to the default, the field must be in the mask and set to the default value in the provided resource. Hence, in order to reset all fields of a resource, provide a default instance of the resource and set all fields in the mask, or do not provide a mask as described below.

If a field mask is not present on update, the operation applies to all fields (as if a field mask of all fields has been specified). Note that in the presence of schema evolution, this may mean that fields the client does not know and has therefore not filled into the request will be reset to their default. If this is unwanted behavior, a specific service may require a client to always specify a field mask, producing an error if not.

As with get operations, the location of the resource which describes the updated values in the request message depends on the operation kind. In any case, the effect of the field mask is required to be honored by the API.

##### Considerations for HTTP REST

The HTTP kind of an update operation which uses a field mask must be set to PATCH instead of PUT in order to satisfy HTTP semantics (PUT must only be used for full updates).

#### JSON Encoding of Field Masks

In JSON, a field mask is encoded as a single string where paths are separated by a comma. Fields name in each path are converted to/from lower-camel naming conventions.

As an example, consider the following message declarations:

    message Profile {
      User user = 1;
      Photo photo = 2;
    message User {
      string display_name = 1;
      string address = 2;

In proto a field mask for `Profile` may look as such:

    mask {
      paths: "user.display_name"
      paths: "photo"

In JSON, the same mask is represented as below:

      mask: "user.displayName,photo"

Field name| Type| Description
---|---|---
`paths`| `string`| The set of field mask paths.

## FloatValue

Wrapper message for `float`.

The JSON representation for `FloatValue` is JSON number.

Field name| Type| Description
---|---|---
`value`| `float`| The float value.

## Int32Value

Wrapper message for `int32`.

The JSON representation for `Int32Value` is JSON number.

Field name| Type| Description
---|---|---
`value`| `int32`| The int32 value.

## Int64Value

Wrapper message for `int64`.

The JSON representation for `Int64Value` is JSON string.

Field name| Type| Description
---|---|---
`value`| `int64`| The int64 value.

## ListValue

`ListValue` is a wrapper around a repeated field of values.

The JSON representation for `ListValue` is JSON array.

Field name| Type| Description
---|---|---
`values`| `Value`| Repeated field of dynamically typed values.

## Method

Method represents a method of an api.

Field name| Type| Description
---|---|---
`name`| `string`| The simple name of this method.
`request_type_url`| `string`| A URL of the input message type.
`request_streaming`| `bool`| If true, the request is streamed.
`response_type_url`| `string`| The URL of the output message type.
`response_streaming`| `bool`| If true, the response is streamed.
`options`| `Option`| Any metadata attached to the method.
`syntax`| `Syntax`| The source syntax of this method.

## Mixin

Declares an API to be included in this API. The including API must redeclare all the methods from the included API, but documentation and options are inherited as follows:

  * If after comment and whitespace stripping, the documentation string of the redeclared method is empty, it will be inherited from the original method.

  * Each annotation belonging to the service config (http, visibility) which is not set in the redeclared method will be inherited.

  * If an http annotation is inherited, the path pattern will be modified as follows. Any version prefix will be replaced by the version of the including API plus the `root` path if specified.

Example of a simple mixin:

    package google.acl.v1;
    service AccessControl {
      // Get the underlying ACL object.
      rpc GetAcl(GetAclRequest) returns (Acl) {
        option (google.api.http).get = "/v1/{resource=**}:getAcl";

    package google.storage.v2;
    service Storage {
      //       rpc GetAcl(GetAclRequest) returns (Acl);

      // Get a data record.
      rpc GetData(GetDataRequest) returns (Data) {
        option (google.api.http).get = "/v2/{resource=**}";

Example of a mixin configuration:

    apis:
    - name: google.storage.v2.Storage
      mixins:
      - name: google.acl.v1.AccessControl

The mixin construct implies that all methods in `AccessControl` are also declared with same name and request/response types in `Storage`. A documentation generator or annotation processor will see the effective `Storage.GetAcl` method after inheriting documentation and annotations as follows:

    service Storage {
      // Get the underlying ACL object.
      rpc GetAcl(GetAclRequest) returns (Acl) {
        option (google.api.http).get = "/v2/{resource=**}:getAcl";
      ...

Note how the version in the path pattern changed from `v1` to `v2`.

If the `root` field in the mixin is specified, it should be a relative path under which inherited HTTP paths are placed. Example:

    apis:
    - name: google.storage.v2.Storage
      mixins:
      - name: google.acl.v1.AccessControl
        root: acls

This implies the following inherited HTTP annotation:

    service Storage {
      // Get the underlying ACL object.
      rpc GetAcl(GetAclRequest) returns (Acl) {
        option (google.api.http).get = "/v2/acls/{resource=**}:getAcl";
      ...

Field name| Type| Description
---|---|---
`name`| `string`| The fully qualified name of the API which is included.
`root`| `string`| If non-empty specifies a path under which inherited HTTP paths are rooted.

## NullValue

`NullValue` is a singleton enumeration to represent the null value for the `Value` type union.

The JSON representation for `NullValue` is JSON `null`.

Enum value| Description
`NULL_VALUE`| Null value.

## Option

A protocol buffer option, which can be attached to a message, field, enumeration, etc.

Field name| Type| Description
---|---|---
`name`| `string`| The option's name. For example, `"java_package"`.
`value`| `Any`| The option's value. For example, `"com.google.protobuf"`.

## SourceContext

`SourceContext` represents information about the source of a protobuf element, like the file in which it is defined.

Field name| Type| Description
---|---|---
`file_name`| `string`| The path-qualified name of the .proto file that contained the associated protobuf element. For example: `"google/protobuf/source.proto"`.

## StringValue

Wrapper message for `string`.

The JSON representation for `StringValue` is JSON string.

Field name| Type| Description
---|---|---
`value`| `string`| The string value.

## Struct

`Struct` represents a structured data value, consisting of fields which map to dynamically typed values. In some languages, `Struct` might be supported by a native representation. For example, in scripting languages like JS a struct is represented as an object. The details of that representation are described together with the proto support for the language.

The JSON representation for `Struct` is JSON object.

Field name| Type| Description
---|---|---
`fields`| `map<string, Value>`| Map of dynamically typed values.

## Syntax

The syntax in which a protocol buffer element is defined.

Enum value| Description
`SYNTAX_PROTO2`| Syntax `proto2`.
`SYNTAX_PROTO3`| Syntax `proto3`.
`SYNTAX_EDITIONS`| Syntax uses the `edition` construct.

## Timestamp

A Timestamp represents a point in time independent of any time zone or calendar, represented as seconds and fractions of seconds at nanosecond resolution in UTC Epoch time. It is encoded using the Proleptic Gregorian Calendar which extends the Gregorian calendar backwards to year one. It is encoded assuming all minutes are 60 seconds long, i.e. leap seconds are “smeared” so that no leap second table is needed for interpretation. Range is from 0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting to that range, we ensure that we can convert to and from RFC 3339 date strings. See <https://www.ietf.org/rfc/rfc3339.txt>.

The Timestamp type is encoded as a string in the RFC 3339 format: “`{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z`”, where `{year}` is always expressed using four digits while `{month}`, `{day}`, `{hour}`, `{min}`, and `{sec}` are zero-padded to two digits each. The fractional seconds, which can go up to 9 digits (that is, up to 1 nanosecond resolution), are optional. The “Z” suffix indicates the timezone (“UTC”); the timezone is required. A proto3 JSON serializer should always use UTC (as indicated by “Z”) when printing the Timestamp type and a proto3 JSON parser should be able to accept both UTC and other timezones (as indicated by an offset).

Example 1: Compute Timestamp from POSIX `time()`.

    Timestamp timestamp;
    timestamp.set_seconds(time(NULL));
    timestamp.set_nanos(0);

Example 2: Compute Timestamp from POSIX `gettimeofday()`.

    struct timeval tv;
    gettimeofday(&tv, NULL);

    Timestamp timestamp;
    timestamp.set_seconds(tv.tv_sec);
    timestamp.set_nanos(tv.tv_usec * 1000);

Example 3: Compute Timestamp from Win32 `GetSystemTimeAsFileTime()`.

    FILETIME ft;
    GetSystemTimeAsFileTime(&ft);
    UINT64 ticks = (((UINT64)ft.dwHighDateTime) << 32) | ft.dwLowDateTime;

    // A Windows tick is 100 nanoseconds. Windows epoch 1601-01-01T00:00:00Z
    // is 11644473600 seconds before Unix epoch 1970-01-01T00:00:00Z.
    Timestamp timestamp;
    timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL));
    timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));

Example 4: Compute Timestamp from Java `System.currentTimeMillis()`.

    long millis = System.currentTimeMillis();

    Timestamp timestamp = Timestamp.newBuilder().setSeconds(millis / 1000)
        .setNanos((int) ((millis % 1000) * 1000000)).build();

Example 5: Compute Timestamp from current time in Python.

    now = time.time()
    seconds = int(now)
    nanos = int((now - seconds) * 10**9)
    timestamp = Timestamp(seconds=seconds, nanos=nanos)

Field name| Type| Description
---|---|---
`seconds`| `int64`| Represents seconds of UTC time since Unix epoch 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to 9999-12-31T23:59:59Z inclusive.
`nanos`| `int32`| Non-negative fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values that count forward in time. Must be from 0 to 999,999,999 inclusive.

## Type

A protocol buffer message type.

Field name| Type| Description
---|---|---
`name`| `string`| The fully qualified message name.
`fields`| `Field`| The list of fields.
`oneofs`| `string`| The list of types appearing in `oneof` definitions in this type.
`options`| `Option`| The protocol buffer options.
`source_context`| `SourceContext`| The source context.
`syntax`| `Syntax`| The source syntax.

## UInt32Value

Wrapper message for `uint32`.

The JSON representation for `UInt32Value` is JSON number.

Field name| Type| Description
---|---|---
`value`| `uint32`| The uint32 value.

## UInt64Value

Wrapper message for `uint64`.

The JSON representation for `UInt64Value` is JSON string.

Field name| Type| Description
---|---|---
`value`| `uint64`| The uint64 value.

## Value

`Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of that variants, absence of any variant indicates an error.

The JSON representation for `Value` is JSON value.

Field name| Type| Description
---|---|---
Union field, only one of the following:
`null_value`| `NullValue`| Represents a null value.
`number_value`| `double`| Represents a double value. Note that attempting to serialize NaN or Infinity results in error. (We can't serialize these as string "NaN" or "Infinity" values like we do for regular fields, because they would parse as string_value, not number_value).
`string_value`| `string`| Represents a string value.
`bool_value`| `bool`| Represents a boolean value.
`struct_value`| `Struct`| Represents a structured value.
`list_value`| `ListValue`| Represents a repeated `Value`.


# Go FAQ

A list of frequently asked questions about implementing protocol buffers in Go, with answer for each.

## Versions

### What’s the difference between `github.com/golang/protobuf` and `google.golang.org/protobuf`?

The [`github.com/golang/protobuf`](https://pkg.go.dev/github.com/golang/protobuf?tab=overview) module is the original Go protocol buffer API.

The [`google.golang.org/protobuf`](https://pkg.go.dev/google.golang.org/protobuf?tab=overview) module is an updated version of this API designed for simplicity, ease of use, and safety. The flagship features of the updated API are support for reflection and a separation of the user-facing API from the underlying implementation.

We recommend that you use `google.golang.org/protobuf` in new code.

Version `v1.4.0` and higher of `github.com/golang/protobuf` wrap the new implementation and permit programs to adopt the new API incrementally. For example, the well-known types defined in `github.com/golang/protobuf/ptypes` are simply aliases of those defined in the newer module. Thus, [`google.golang.org/protobuf/types/known/emptypb`](https://pkg.go.dev/google.golang.org/protobuf/types/known/emptypb) and [`github.com/golang/protobuf/ptypes/empty`](https://pkg.go.dev/github.com/golang/protobuf/ptypes/empty) may be used interchangeably.

### What are `proto1`, `proto2`, `proto3`, and editions?

These are revisions of the protocol buffer _language_. It is different from the Go _implementation_ of protobufs.

  * **Editions** are the newest and recommended way of writing Protocol Buffers. New features will be released as part of new editions. For more information, see [Protocol Buffer Editions](https://protobuf.dev/editions).

  * `proto3` is a legacy version of the language. We encourage new code to use editions.

  * `proto2` is a legacy version of the language. Despite being superseded by proto3 and editions, proto2 is still fully supported.

  * `proto1` is an obsolete version of the language. It was never released as open source.

### There are several different `Message` types. Which should I use?

  * [`"google.golang.org/protobuf/proto".Message`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Message) is an interface type implemented by all messages generated by the current version of the protocol buffer compiler. Functions that operate on arbitrary messages, such as [`proto.Marshal`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Marshal) or [`proto.Clone`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Clone), accept or return this type.

  * [`"google.golang.org/protobuf/reflect/protoreflect".Message`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message) is an interface type describing a reflection view of a message.

Call the `ProtoReflect` method on a `proto.Message` to get a `protoreflect.Message`.

  * [`"google.golang.org/protobuf/reflect/protoreflect".ProtoMessage`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#ProtoMessage) is an alias of `"google.golang.org/protobuf/proto".Message`. The two types are interchangeable.

  * [`"github.com/golang/protobuf/proto".Message`](https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#Message) is an interface type defined by the legacy Go protocol buffer API. All generated message types implement this interface, but the interface does not describe the behavior expected from these messages. New code should avoid using this type.

## Common problems

### “`go install`”: `working directory is not part of a module`

On Go 1.15 and below, you have set the environment variable `GO111MODULE=on` and are running the `go install` command outside of a module directory. Set `GO111MODULE=auto`, or unset the environment variable.

On Go 1.16 and above, `go install` can be invoked outside of a module by specifying an explicit version: `go install google.golang.org/protobuf/cmd/protoc-gen-go@latest`

### `constant -1 overflows protoimpl.EnforceVersion`

You are using a generated `.pb.go` file which requires a newer version of the `"google.golang.org/protobuf"` module.

Update to a newer version with:

    go get -u google.golang.org/protobuf/proto

### `undefined: "github.com/golang/protobuf/proto".ProtoPackageIsVersion4`

You are using a generated `.pb.go` file which requires a newer version of the `"github.com/golang/protobuf"` module.

Update to a newer version with:

    go get -u github.com/golang/protobuf/proto

### What is a protocol buffer namespace conflict?

All protocol buffers declarations linked into a Go binary are inserted into a global registry.

Every protobuf declaration (for example, enums, enum values, or messages) has an absolute name, which is the concatenation of the [package name](https://protobuf.dev/programming-guides/proto2#packages) with the relative name of the declaration in the `.proto` source file (for example, `my.proto.package.MyMessage.NestedMessage`). The protobuf language assumes that all declarations are universally unique.

If two protobuf declarations linked into a Go binary have the same name, then this leads to a namespace conflict, and it is impossible for the registry to properly resolve that declaration by name. Depending on which version of Go protobufs is being used, this will either panic at init-time or silently drop the conflict and lead to a potential bug later during runtime.

### How do I fix a protocol buffer namespace conflict?

The way to best fix a namespace conflict depends on the reason why a conflict is occurring.

Common ways that namespace conflicts occur:

  * **Vendored .proto files.** When a single `.proto` file is generated into two or more Go packages and linked into the same Go binary, it conflicts on every protobuf declaration in the generated Go packages. This typically occurs when a `.proto` file is vendored and a Go package is generated from it, or the generated Go package itself is vendored. Users should avoid vendoring and instead depend on a centralized Go package for that `.proto` file.

    * If a `.proto` file is owned by an external party and is lacking a `go_package` option, then you should coordinate with the owner of that `.proto` file to specify a centralized Go package that a plurality of users can all depend on.
  * **Missing or generic proto package names.** If a `.proto` file does not specify a package name or uses an overly generic package name (for example, “my_service”), then there is a high probability that declarations within that file will conflict with other declarations elsewhere in the universe. We recommend that every `.proto` file have a package name that is deliberately chosen to be universally unique (for example, prefixed with the name of a company).

#### Warning

Retroactively changing the package name on a `.proto` file is not backwards compatible for types used as extension fields, stored in `google.protobuf.Any`, or for gRPC Service definitions.

Starting with v1.26.0 of the `google.golang.org/protobuf` module, a hard error will be reported when a Go program starts up that has multiple conflicting protobuf names linked into it. While it is preferable that the source of the conflict be fixed, the fatal error can be immediately worked around in one of two ways:

  1. **At compile time.** The default behavior for handling conflicts can be specified at compile time with a linker-initialized variable: `go build -ldflags "-X google.golang.org/protobuf/reflect/protoregistry.conflictPolicy=warn"`

  2. **At program execution.** The behavior for handling conflicts when executing a particular Go binary can be set with an environment variable: `GOLANG_PROTOBUF_REGISTRATION_CONFLICT=warn ./main`

### How do I use protocol buffer editions?

To use a protobuf edition, you must specify the edition in your `.proto` file. For example, to use the 2023 edition, add the following to the top of your `.proto` file:

    edition = "2023";

The protocol buffer compiler will then generate Go code that is compatible with the specified edition. With editions, you can also enable or disable specific features for your `.proto` file. For more information, see [Protocol Buffer Editions](https://protobuf.dev/editions/overview).

### How do I control the behavior of my generated Go code?

With editions, you can control the behavior of the generated Go code by enabling or disabling specific features in your `.proto` file. For example, to set the API behavior for your implementation, you would add the following to your `.proto` file:

    edition = "2023";

    option features.(pb.go).api_level = API_OPAQUE;

When `api_level` is set to `API_OPAQUE`, the Go code generated by the protocol buffer compiler hides struct fields so they can no longer be directly accessed. Instead, new accessor methods are created for getting, setting, or clearing a field.

For a complete list of available features and their descriptions, see [Features for Editions](https://protobuf.dev/editions/features).

### Why does `reflect.DeepEqual` behave unexpectedly with protobuf messages?

Generated protocol buffer message types include internal state which can vary even between equivalent messages.

In addition, the `reflect.DeepEqual` function is not aware of the semantics of protocol buffer messages, and can report differences where none exist. For example, a map field containing a `nil` map and one containing a zero-length, non-`nil` map are semantically equivalent, but will be reported as unequal by `reflect.DeepEqual`.

Use the [`proto.Equal`](https://pkg.go.dev/google.golang.org/protobuf/proto#Equal) function to compare message values.

In tests, you can also use the [`"github.com/google/go-cmp/cmp"`](https://pkg.go.dev/github.com/google/go-cmp/cmp?tab=doc) package with the [`protocmp.Transform()`](https://pkg.go.dev/google.golang.org/protobuf/testing/protocmp#Transform) option. The `cmp` package can compare arbitrary data structures, and [`cmp.Diff`](https://pkg.go.dev/github.com/google/go-cmp/cmp#Diff) produces human-readable reports of the differences between values.

    if diff := cmp.Diff(a, b, protocmp.Transform()); diff != "" {
      t.Errorf("unexpected difference:\n%v", diff)

## Hyrum’s Law

### What is Hyrum’s Law, and why is it in this FAQ?

[Hyrum’s Law](https://www.hyrumslaw.com/) states:

> With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

A design goal of the latest version of the Go protocol buffer API is to avoid, where possible, providing observable behaviors that we cannot promise to keep stable in the future. It is our philosophy that deliberate instability in areas where we make no promises is better than giving the illusion of stability, only for that to change in the future after a project has potentially been long depending on that false assumption.

### Why does the text of errors keep changing?

Tests depending on the exact text of errors are brittle and break often when that text changes. To discourage unsafe use of error text in tests, the text of errors produced by this module is deliberately unstable.

If you need to identify whether an error is produced by the [`protobuf`](https://pkg.go.dev/mod/google.golang.org/protobuf) module, we guarantee that all errors will match [`proto.Error`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Error) according to [`errors.Is`](https://pkg.go.dev/errors?tab=doc#Is).

### Why does the output of [`protojson`](https://pkg.go.dev/google.golang.org/protobuf/encoding/protojson) keep changing?

We make no promises about the long-term stability of Go’s implementation of the [JSON format for protocol buffers](https://protobuf.dev/programming-guides/proto3#json). The specification only specifies what is valid JSON, but provides no specification for a _canonical_ format for how a marshaler ought to _exactly_ format a given message. To avoid giving the illusion that the output is stable, we deliberately introduce minor differences so that byte-for-byte comparisons are likely to fail.

To gain some degree of output stability, we recommend running the output through a JSON formatter.

### Why does the output of [`prototext`](https://pkg.go.dev/google.golang.org/protobuf/encoding/prototext) keep changing?

We make no promises about the long-term stability of Go’s implementation of the text format. There is no canonical specification of the protobuf text format, and we would like to preserve the ability to make improvements in the `prototext` package output in the future. Since we don’t promise stability in the package’s output, we’ve deliberately introduced instability to discourage users from depending on it.

To obtain some degree of stability, we recommend passing the output of `prototext` through the [`txtpbfmt`](https://github.com/protocolbuffers/txtpbfmt) program. The formatter can be directly invoked in Go using [`parser.Format`](https://pkg.go.dev/github.com/protocolbuffers/txtpbfmt/parser?tab=doc#Format).

## Miscellaneous

### How do I use a protocol buffer message as a hash key?

You need canonical serialization, where the marshaled output of a protocol buffer message is guaranteed to be stable over time. Unfortunately, no specification for canonical serialization exists at this time. You’ll need to write your own or find a way to avoid needing one.

### Can I add a new feature to the Go protocol buffer implementation?

Maybe. We always like suggestions, but we’re very cautious about adding new things.

The Go implementation of protocol buffers strives to be consistent with the other language implementations. As such, we tend to shy away from feature that are overly specialized to just Go. Go-specific features hinder the goal of protocol buffers being a language-neutral data interchange format.

Unless your idea is specific to the Go implementation, you should join the [protobuf discussion group](http://groups.google.com/group/protobuf) and suggest it there.

If you have an idea for the Go implementation, file an issue on our issue tracker: <https://github.com/golang/protobuf/issues>

### Can I add an option to `Marshal` or `Unmarshal` to customize it?

Only if that option exists in other implementations (e.g., C++, Java). The encoding of protocol buffers (binary, JSON, and text) must be consistent across implementations, so a program written in one language is able to read messages written by another one.

We will not add any options to the Go implementation that affect the data output by `Marshal` functions or read by `Unmarshal` functions unless an equivalent option exist in at least one other supported implementation.

### Can I customize the code generated by `protoc-gen-go`?

In general, no. Protocol buffers are intended to be a language-agnostic data interchange format, and implementation-specific customizations run counter to that intent.


# Rust Proto Design Decisions

Explains some of the design choices that the Rust Proto implementation makes.

As with any library, Rust Protobuf is designed considering the needs of both Google’s first-party usage of Rust as well that of external users. Choosing a path in that design space means that some choices made will not be optimal for some users in some cases, even if it is the right choice for the implementation overall.

This page covers some of the larger design decisions that the Rust Protobuf implementation makes and the considerations which led to those decisions.

## Designed to Be ‘Backed’ by Other Protobuf Implementations, Including C++ Protobuf

Protobuf Rust is not a pure Rust implementation of protobuf, but a safe Rust API implemented on top of existing protobuf implementations, or as we call these implementations: kernels.

The biggest factor that goes into this decision was to enable zero-cost of adding Rust to a preexisting binary which already uses non-Rust Protobuf. By enabling the implementation to be ABI-compatible with the C++ Protobuf generated code, it is possible to share Protobuf messages across the language boundary (FFI) as plain pointers, avoiding the need to serialize in one language, pass the byte array across the boundary, and deserialize in the other language. This also reduces binary size for these use cases by avoiding having redundant schema information embedded in the binary for the same messages for each language.

Google sees Rust as an opportunity to incrementally get memory safety to key portions of preexisting brownfield C++ servers; the cost of serialization at the language boundaries would prevent adoption of Rust to replace C++ in many of these important and performance-sensitive cases. If we pursued a greenfield Rust Protobuf implementation that did not have this support, it would end up blocking Rust adoption and require that these important cases stay on C++ instead.

Protobuf Rust currently supports three kernels:

  * C++ kernel - the generated code is backed by C++ Protocol Buffers (the “full” implementation, typically used for servers). This kernel offers in-memory interoperability with C++ code that uses the C++ runtime. This is the default for servers within Google.
  * C++ Lite kernel - the generated code is backed by C++ Lite Protocol Buffers (typically used for mobile). This kernel offers in-memory interoperability with C++ code that uses the C++ Lite runtime. This is the default for for mobile apps within Google.
  * upb kernel - the generated code is backed by [upb](https://github.com/protocolbuffers/protobuf/tree/main/upb), a highly performant and small-binary-size Protobuf library written in C. upb is designed to be used as an implementation detail by Protobuf runtimes in other languages. This is the default in open source builds where we expect static linking with code already using C++ Protobuf to be more rare.

Rust Protobuf is designed to support multiple alternate implementations (including multiple different memory layouts) while exposing exactly the same API, allowing for the same application code to be recompiled targeting being backed by a different implementation. This design constraint significantly influences our public API decisions, including the types used on getters (discussed later in this document).

### No Pure Rust Kernel

Given that we designed the API to be implementable by multiple backing implementations, a natural question is why the only supported kernels are written in the memory unsafe languages of C and C++ today.

While Rust being a memory-safe language can significantly reduce exposure to critical security issues, no language is immune to security issues. The Protobuf implementations that we support as kernels have been scrutinized and fuzzed to the extent that Google is comfortable using those implementations to perform unsandboxed parsing of untrusted inputs in our own servers and apps.

A greenfield binary parser written in Rust at this time would be understood to be much more likely to contain critical vulnerabilities than our preexisting C++ Protobuf or upb parsers, which have been extensively fuzzed, tested, and reviewed.

There are legitimate arguments for supporting a pure Rust kernel implementation long-term, including the ability for developers to avoid needing to have Clang available to compile C code at build time.

We expect that Google will support a pure Rust implementation with the same exposed API at some later date, but we have no concrete roadmap for it at this time. A second official Rust Protobuf implementation that has a ‘better’ API by avoiding the constraints that come from being backed by C++ Proto and upb is not planned, as we wouldn’t want to fragment Google’s own Protobuf usage.

## View/Mut Proxy Types

The Rust Proto API is designed with opaque “Proxy” types. For a `.proto` file that defines `message SomeMsg {}`, we generate the Rust types `SomeMsg`, `SomeMsgView<'_>` and `SomeMsgMut<'_>`. The simple rule of thumb is that we expect the View and Mut types to stand in for `&SomeMsg` and `&mut SomeMsg` in all usages by default, while still getting all of the borrow checking/Send/etc. behavior that you would expect from those types.

### Another Lens to Understand These Types

To better understand the nuances of these types, it may be useful to think of these types as follows:

    struct SomeMsg(Box<cpp::SomeMsg>);
    struct SomeMsgView<'a>(&'a cpp::SomeMsg);
    struct SomeMsgMut<'a>(&'a mut cpp::SomeMsg);

Under this lens you can see that:

  * Given a `&SomeMsg` it is possible to get a `SomeMsgView` (similar to how given a `&Box<T>` you can get a `&T`)
  * Given a `SomeMsgView` it in _not_ possible to get a `&SomeMsg` (similar to how given a `&T` you couldn’t get a `&Box<T>`).

Just like with the `&Box` example, this means that on function arguments, it is generally better to default to use `SomeMsgView<'a>` rather than a `&'a SomeMsg`, as it will allow a superset of callers to use the function.

### Why

There are two main reasons for this design: to unlock possible optimization benefits, and as an inherent outcome of the kernel design.

#### Optimization Opportunity Benefit

Protobuf being such a core and widespread technology makes it unusually both prone to all possible observable behaviors being depended on by someone, as well as relatively small optimizations having unusually major net impact at scale. We have found that more opaqueness of types gives unusually high amount of leverage: they permit us to be more deliberate about exactly what behaviors are exposed, and give us more room to optimize the implementation.

A `SomeMsgMut<'_>` provides those opportunities where a `&mut SomeMsg` would not: namely that we can construct them lazily and with an implementation detail which is not the same as the owned message representation. It also inherently allows us to control certain behaviors that we couldn’t otherwise limit or control: for example, any `&mut` can be used with `std::mem::swap()`, which is a behavior that would place strong limits on what invariants you are able to maintain between a parent and child struct if `&mut SomeChild` is given to callers.

#### Inherent to Kernel Design

The other reason for the proxy types is more of an inherent limitation to our kernel design; when you have a `&T` there must be a real Rust `T` type in memory somewhere.

Our C++ kernel design allows you to parse a message which contains nested messages, and create only a small Rust stack-allocated object to representing the root message, with all other memory being stored on the C++ Heap. When you later access a child message, there will be no already-allocated Rust object which corresponds to that child, and so there’s no Rust instance to borrow at that moment.

By using proxy types, we’re able to on-demand create the Rust proxy types that semantically acting as borrows, without there being any eagerly allocated Rust memory for those instances ahead of time.

## Non-Std Types

### Simple Types Which May Have a Directly Corresponding Std Type

In some cases the Rust Protobuf API may choose to create our own types where a corresponding std type exists with the same name, where the current implementation may even simply wrap the std type, for example `protobuf::UTF8Error`.

Using these types rather than std types gives us more flexibility in optimizing the implementation in the future. While our current implementation uses the Rust std UTF-8 validation today, by creating our own `protobuf::Utf8Error` type it enables us to change the implementation to use the highly optimized C++ implementation of UTF-8 validation that we use from C++ Protobuf which is faster than Rust’s std UTF-8 validation.

### ProtoString

Rust’s `str` and `std::string::String` types maintain a strict invariant that they only contain valid UTF-8, but C++’s `std::string` type does not enforce any such guarantee. `string` typed Protobuf fields are intended to only ever contain valid UTF-8, and C++ Protobuf does use a correct and highly optimized UTF8 validator. However, C++ Protobuf’s API surface is not set up to strictly enforce as a runtime invariant that its `string` fields always contain valid UTF-8, instead, in some cases it allows setting of non-UTF8 data into a `string` field and validation will only occur at a later time when serialization is happening.

To enable integrating Rust into preexisting codebases that use C++ Protobuf while allowing for zero-cost boundary crossings with no risk of undefined behavior in Rust, we unfortunately have to avoid the `str`/`String` types for `string` field getters. Instead, the types `ProtoStr` and `ProtoString` are used, which are equivalent types, except that they may contain invalid UTF-8 in rare situations. Those types let the application code choose if they wish to perform the validation on-demand to observe the fields as a `Result<&str>`, or operate on the raw bytes to avoid any runtime validation. All of the setter paths are still designed to allow you to pass `&str` or `String` types.

We are aware that vocabulary types like `str` are very important to idiomatic usage, and intend to keep an eye on if this decision is the right one as usage details of Rust evolves.


# Java Reference

Reference documentation for working with protocol buffer classes in Java.

##### [Java Generated Code Guide](https://protobuf.dev/reference/java/java-generated/)

Describes exactly what Java code the protocol buffer compiler generates for any given protocol definition.

##### [Java Proto Names](https://protobuf.dev/reference/java/java-proto-names/)

Names that are generated by the Java protoc plugin.

##### [Java API](https://protobuf.dev/reference/java/api-docs/overview-summary.html)


# ProtoJSON Format

Describes the spec details of the canonical JSON representation for Protobuf messages.

Protobuf supports a canonical encoding in JSON, making it easier to share data with systems that do not support the standard protobuf binary wire format.

This page specifies the format, but a number of additional edge cases which define a conformant ProtoJSON parser are covered in the Protobuf Conformance Test Suite and are not exhaustively detailed here.

## Non-goals of the Format

### Cannot Represent Some JSON schemas

The ProtoJSON format is designed to be a JSON representation of schemas which are expressible in the Protobuf schema language.

It may be possible to represent many pre-existing JSON schemas as a Protobuf schema and parse it using ProtoJSON, but it is not designed to be able to represent arbitrary JSON schemas.

For example, there is no way to express in Protobuf schema to write types that may be common in JSON schemas like `number[][]` or `number|string`.

It is possible to use `google.protobuf.Struct` and `google.protobuf.Value` types to allow arbitrary JSON to be parsed into a Protobuf schema, but these only allow you to capture the values as schemaless unordered key-value maps.

### Not as efficient as the binary wire format

ProtoJSON Format is not as efficient as binary wire format and never will be.

The converter uses more CPU to encode and decode messages and (except in rare cases) encoded messages consume more space.

### Does not have as good schema-evolution guarantees as binary wire format

ProtoJSON format does not support unknown fields, and it puts field and enum value names into encoded messages which makes it much harder to change those names later. Removing fields is a breaking change that will trigger a parsing error.

See JSON Wire Safety below for more details.

## Format Description

### Representation of each type

The following table shows how data is represented in JSON files.

Protobuf type| JSON| JSON example| Notes
---|---|---|---
message| object| `{"fooBar": v, "g": null, ...}`| Generates JSON objects.Keys are serialized as lowerCamelCase of field name. See Field Names for more special cases regarding mapping of field names to object keys.Well-known types have special representations, as described in the Well-known types table.`null` is valid for any field and leaves the field unset. See Null Values for clarification about the semantic behavior of null values.
enum| string| `"FOO_BAR"`| The name of the enum value as specified in proto is used. Parsers accept both enum names and integer values.
map<K,V>| object| `{"k": v, ...}`| All keys are converted to strings (object keys in JSON can only be strings).
repeated V| array| `[v, ...]`|
bool| true, false| `true, false`|
string| string| `"Hello World!"`|
bytes| base64 string| `"YWJjMTIzIT8kKiYoKSctPUB+"`| JSON value will be the data encoded as a string using standard base64 encoding with paddings. Either standard or URL-safe base64 encoding with/without paddings are accepted.
int32, fixed32, uint32| number| `1, -10, 0`| JSON value will be a number. Either numbers or strings are accepted. Empty strings are invalid. Exponent notation (such as `1e2`) is accepted in both quoted and unquoted forms.
int64, fixed64, uint64| string| `"1", "-10"`| JSON value will be a decimal string. Either numbers or strings are accepted. Empty strings are invalid. Exponent notation (such as `1e2`) is accepted in both quoted and unquoted forms. See Strings for int64s for the explanation why strings are used for int64s.
float, double| number| `1.1, -10.0, 0, "NaN", "Infinity"`| JSON value will be a number or one of the special string values "NaN", "Infinity", and "-Infinity". Either numbers or strings are accepted. Empty strings are invalid. Exponent notation is also accepted.

### Well-Known Types

Some messages in the `google.protobuf` package have a special representation when represented in JSON.

No message type outside of the `google.protobuf` package has a special ProtoJSON handling; for example, types in `google.types` package are represented with the neutral representation.Message type| JSON| JSON example| Notes
---|---|---|---
Any| `object`| `{"@type": "url", "f": v, ... }`| See Any
Timestamp| string| `"1972-01-01T10:00:20.021Z"`| Uses RFC 3339 (see clarification). Generated output will always be Z-normalized with 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted.
Duration| string| `"1.000340012s", "1s"`| Generated output always contains 0, 3, 6, or 9 fractional digits, depending on required precision, followed by the suffix "s". Accepted are any fractional digits (also none) as long as they fit into nanoseconds precision and the suffix "s" is required. This is **not** RFC 3339 'duration' format (see Durations for clarification).
Struct| `object`| `{ ... }`| Any JSON object. See `struct.proto`.
Wrapper types| various types| `2, "2", "foo", true, "true", null, 0, ...`| Wrappers use the same representation in JSON as the wrapped primitive type, except that `null` is allowed and preserved during data conversion and transfer.
FieldMask| string| `"f.fooBar,h"`| See `field_mask.proto`.
ListValue| array| `[foo, bar, ...]`|
Value| value| | Any JSON value. Check [google.protobuf.Value](https://protobuf.dev/reference/protobuf/google.protobuf#value) for details.
NullValue| null| | JSON null. Special case of the null parsing behavior.
Empty| object| `{}` **(not special cased)**| An empty JSON object

### Field names as JSON keys

Message field names are mapped to lowerCamelCase to be used as JSON object keys. If the `json_name` field option is specified, the specified value will be used as the key instead.

Parsers accept both the lowerCamelCase name (or the one specified by the `json_name` option) and the original proto field name. This allows for a serializer option to choose to print using the original field name (see JSON Options) and have the resulting output still be parsed back by all spec parsers.

`\0 (nul)` is not allowed within a `json_name` value. For more on why, see [Stricter validation for json_name](https://protobuf.dev/news/2023-04-28#json-name). Note that `\0` is still considered a legal character within the value of a `string` field.

### Presence and default-values

When generating JSON-encoded output from a protocol buffer, if a field supports presence, serializers must emit the field value if and only if the corresponding hazzer would return true.

If the field doesn’t support field presence and has the default value (for example any empty repeated field) serializers should omit it from the output. An implementation may provide options to include fields with default values in the output.

### Null values

Serializers should not emit `null` values.

Parsers accept `null` as a legal value for any field, with the following behavior:

  * Any key validity checking should still occur (disallowing unknown fields).
  * The field should remain unset, as though it was not present in the input at all (hazzers should still return false where applicable).

The implication of this is that a `null` value for an implicit presence field will behave the identically to the behavior to the default value of that field, since there are no hazzers for those fields. For example, a value of `null` or `[]` for a repeated field will cause key-validation checks, but both will otherwise behave the same as if the field was not present in the JSON at all.

`null` values are not allowed within repeated fields.

`google.protobuf.NullValue` is a special exception to this behavior: `null` is handled as a sentinel-present value for this type, and so a field of this type must be handled by serializers and parsers under the standard presence behavior. This behavior correspondingly allows `google.protobuf.Struct` and `google.protobuf.Value` to losslessly round trip arbitrary JSON.

### Duplicate values

Serializers must never serialize the same field multiple times, nor multiple different cases in the same oneof in the same JSON object.

Parsers should accept the same field being duplicated, and the last value provided should be retained. This also applies to “alternate spellings” of the same field name.

If implementations cannot maintain the necessary information about field order it is preferred to reject inputs with duplicate keys rather than have an arbitrary value win. In some implementations maintaining field order of objects may be impractical or infeasible, so it is strongly recommended that systems avoid relying on specific behavior for duplicate fields in ProtoJSON where possible.

### Out of range numeric values

When parsing a numeric value, if the number that is is parsed from the wire doesn’t fit in the corresponding type, the parser should fail to parse.

This includes any negative number for `uint32`, and numbers less than `INT_MIN` or larger than `INT_MAX` for `int32`.

Values with nonzero fractional portions are not allowed for integer-typed fields. Zero fractional portions are accepted. For example `1.0` is valid for an int32 field, but `1.5` is not.

### Strings for int64

Unfortunately, the [json.org](https://json.org) spec does not speak to the intended precision limits of numbers. Many implementations follow the original JS behavior that JSON was derived from and interpret all numbers as binary64 (double precision) and are silently lossy if a number is an integer larger than 2**53. Other implementations may support unlimited precision bigints, int64s, or even bigfloats with unlimited fractional precision.

This creates a situation where if the JSON contains a number that is not exactly representable by double precision, different parsers will behave differently, including silent precision loss in many languages.

To avoid these problems, ProtoJSON serializers emit int64s as strings to ensure no precision loss will occur on large int64s by any implementation.

When parsing a bare number when expecting an int64, the implementation should coerce that value to double-precision even if the corresponding language’s built-in JSON parser supports parsing of JSON numbers as bigints. This ensures a consistent interpretation of the same data, regardless of language used.

This design follows established best practices in how to handle large numbers in JSON when prioritizing interoperability, including:

  * [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6) includes a note that software that intends good interoperability should only presume double precision on all numbers.

  * [OpenAPI int64](https://spec.openapis.org/registry/format/int64) documentation recommends using a JSON string instead of a number when precision beyond 2**53 is desired.

## Any

### Normal messages

For any message that is not a well-known type with a special JSON representation, the message contained inside the `Any` is turned into a JSON object with an additional `"@type"` field inserted that contains the `type_url` that was set on the `Any`.

For example, if you have this message definition:

    package x;
    message Child { int32 x = 1; string y = 2; }

When an instance of Child is packed into an `Any`, the JSON representation is:

      "@type": "type.googleapis.com/x.Child",
      "x": 1,
      "y": "hello world"

### Special-cased well-known types

If the `Any` contains a well-known type that has a special JSON mapping, the message is converted into the special representation and set as a field with key “value”.

For example, a `google.protobuf.Duration` that represents 3.1 seconds will be represented by the string `"3.1s"` in the special case handling. When that `Duration` is packed into an `Any` it will be serialized as:

      "@type": "type.googleapis.com/google.protobuf.Duration",
      "value": "3.1s"

Message types with special JSON encodings include:

  * `google.protobuf.Any`
  * `google.protobuf.BoolValue`
  * `google.protobuf.BytesValue`
  * `google.protobuf.DoubleValue`
  * `google.protobuf.Duration`
  * `google.protobuf.FieldMask`
  * `google.protobuf.FloatValue`
  * `google.protobuf.Int32Value`
  * `google.protobuf.Int64Value`
  * `google.protobuf.ListValue`
  * `google.protobuf.StringValue`
  * `google.protobuf.Struct`
  * `google.protobuf.Timestamp`
  * `google.protobuf.UInt32Value`
  * `google.protobuf.UInt64Value`
  * `google.protobuf.Value`

Note that `google.protobuf.Empty` is not considered to have any special JSON mapping; it is simply a normal message that has zero fields. This means the expected representation of an `Empty` packed into an `Any` is `{"@type": "type.googleapis.com/google.protobuf.Empty"}` and not `{"@type": "type.googleapis.com/google.protobuf.Empty", "value": {}}`.

## ProtoJSON Wire Safety

When using ProtoJSON, only some schema changes are safe to make in a distributed system. This contrasts with the same concepts applied to the [the binary wire format](https://protobuf.dev/programming-guides/editions#updating).

### JSON Wire-unsafe Changes

Wire-unsafe changes are schema changes that will break if you parse data that was serialized using the old schema with a parser that is using the new schema (or vice versa). You should almost never do this shape of schema change.

  * Changing a field to or from an extension of same number and type is not safe.
  * Changing a field between `string` and `bytes` is not safe.
  * Changing a field between a message type and `bytes` is not safe.
  * Changing any field from `optional` to `repeated` is not safe.
  * Changing a field between a `map<K, V>` and the corresponding `repeated` message field is not safe.
  * Moving fields into an existing `oneof` is not safe.

### JSON Wire-safe Changes

Wire-safe changes are ones where it is fully safe to evolve the schema in this way without risk of data loss or new parse failures.

Note that nearly all wire-safe changes may be a breaking change to application code. For example, adding a value to a preexisting enum would be a compilation break for any code with an exhaustive switch on that enum. For that reason, Google may avoid making some of these types of changes on public messages. The AIPs contain guidance for which of these changes are safe to make there.

  * Changing a single `optional` field into a member of a **new** `oneof` is safe.
  * Changing a `oneof` which contains only one field to an `optional` field is safe.
  * Changing a field between any of `int32`, `sint32`, `sfixed32`, `fixed32` is safe.
  * Changing a field between any of `int64`, `sint64`, `sfixed64`, `fixed64` is safe.
  * Changing a field number is safe (as the field numbers are not used in the ProtoJSON format), but still strongly discouraged since it is very unsafe in the binary wire format.
  * Adding values to an enum is safe if the “Emit enum values as integers” is set on all relevant clients (see options)

### JSON Wire-compatible Changes (Conditionally safe)

Unlike wire-safe changes, wire-compatible means that the same data can be parsed both before and after a given change. However, a client that reads it will get lossy data under this shape of change. For example, changing an int32 to an int64 is a compatible change, but if a value larger than INT32_MAX is written, a client that reads it as an int32 will discard the high order bits.

You can make compatible changes to your schema only if you manage the roll out to your system carefully. For example, you may change an int32 to an int64 but ensure you continue to only write legal int32 values until the new schema is deployed to all endpoints, and then start writing larger values after that.

#### Compatible But With Unknown Field Handling Problems

Unlike the binary wire format, ProtoJSON implementations generally do not propagate unknown fields. This means that adding to schemas is generally compatible but will result in parse failures if a client using the old schema observes the new content.

This means you can add to your schema, but you cannot safely start writing them until you know the schema has been deployed to the relevant client or server (or that the relevant clients set an Ignore Unknown Fields flag, discussed below).

  * Adding and removing fields is considered compatible with this caveat.
  * Removing enum values is considered compatible with this caveat.

#### Compatible But Potentially Lossy

  * Changing between any of the 32-bit integers (`int32`, `uint32`, `sint32`, `sfixed32`, `fixed32`) and any of the 64-bit integers ( `int64`, `uint64`, `sint64`, `sfixed32`) is a compatible change.
    * If a number is parsed from the wire that doesn’t fit in the corresponding type, a parse failure will occur.
    * Unlike binary wire format, `bool` is not compatible with integers.
    * Note that the int64 and uint64 types are quoted by default to avoid precision loss when handled as a double or JavaScript number, and the 32 bit types are unquoted by default. Conformant parsers will accept either quoted or unquoted for all integer types, but nonconformant implementations may mishandle this case and not handle quoted-int32s or unquoted-int64s, so caution should be taken.
  * `enum` may be conditionally compatible with `string`
    * If “enums-as-ints” flag is used by any client, then enums will instead be compatible with the integer types instead.

## RFC 3339 Clarifications

### Timestamps

ProtoJSON timestamps use the RFC 3339 timestamp format. Unfortunately, some ambiguity in the RFC 3339 spec has created a few edge cases where various other RFC 3339 implementations do not agree on whether or not the format is legal.

[RFC 3339](https://www.rfc-editor.org/rfc/rfc3339) intends to declare a strict subset of ISO-8601 format, and some additional ambiguity was created since RFC 3339 was published in 2002 and then ISO-8601 was subsequently revised without any corresponding revisions of RFC 3339.

Most notably, ISO-8601-1988 contains this note:

> In date and time representations lower case characters may be used when upper case characters are not available.

It is ambiguous whether this note is suggesting that parsers should accept lowercase letters in general, or if it is only suggesting that lowercase letters may be used as a substitute in environments where uppercase cannot be technically used. RFC 3339 contains a note that intends to clarify the interpretation to be that lowercase letters should be accepted in general.

ISO-8601-2019 does not contain the corresponding note and is unambiguous that lowercase letters are not allowed.

This created some confusion for all libraries that declare they support RFC 3339: today RFC 3339 declares it is a profile of ISO-8601 but contains a clarifying note referencing text that is not present in the latest ISO-8601 spec.

ProtoJSON spec takes the decision that the timestamp format is the stricter definition of “RFC 3339 as a profile of ISO-8601-2019”. Some Protobuf implementations may be non-conformant by using a timestamp parsing implementation that is implemented as “RFC 3339 as a profile of ISO-8601-1988,” which will accept a few additional edge cases.

For consistent interoperability, parsers should only accept the stricter subset format where possible. When using a non-conformant implementation that accepts the laxer definition, strongly avoid relying on the additional edge cases being accepted.

### Durations

RFC 3339 also defines a duration format, but unfortunately the RFC 3339 duration format does not have any way to express sub-second resolution.

The ProtoJSON duration encoding is directly inspired by RFC 3339 `dur-seconds` representation, but it is able to encode nanosecond precision. For integer number of seconds the two representations may match (like `10s`), but the ProtoJSON durations accept fractional values and conformant implementations must precisely represent nanosecond precision (like `10.500000001s`).

## JSON Options

A conformant protobuf JSON implementation may provide the following options:

  * **Always emit fields without presence**: Fields that don’t support presence and that have their default value are omitted by default in JSON output (for example, an implicit presence integer with a 0 value, implicit presence string fields that are empty strings, and empty repeated and map fields). An implementation may provide an option to override this behavior and output fields with their default values.

As of v25.x, the C++, Java, and Python implementations are nonconformant, as this flag affects proto2 `optional` fields but not proto3 `optional` fields. A fix is planned for a future release.

  * **Ignore unknown fields**: The protobuf JSON parser should reject unknown fields by default but may provide an option to ignore unknown fields in parsing.

  * **Use proto field name instead of lowerCamelCase name**: By default the protobuf JSON printer should convert the field name to lowerCamelCase and use that as the JSON name. An implementation may provide an option to use proto field name as the JSON name instead. Protobuf JSON parsers are required to accept both the converted lowerCamelCase name and the proto field name.

  * **Emit enum values as integers instead of strings**: The name of an enum value is used by default in JSON output. An option may be provided to use the numeric value of the enum value instead.


# Style Guide

Provides direction for how best to structure your proto definitions.

This document provides a style guide for `.proto` files. By following these conventions, you’ll make your protocol buffer message definitions and their corresponding classes consistent and easy to read.

Enforcement of the following style guidelines is controlled via [enforce_naming_style](https://protobuf.dev/editions/features#enforce-naming).

## Standard File Formatting

  * Keep the line length to 80 characters.
  * Use an indent of 2 spaces.
  * Prefer the use of double quotes for strings.

## File Structure

Files should be named `lower_snake_case.proto`.

All files should be ordered in the following manner:

  1. License header (if applicable)
  2. File overview
  3. Syntax or edition
  4. Package
  5. Imports (sorted)
  6. File options
  7. Everything else

## Identifier naming styles

Protobuf identifiers use one of the following naming styles:

  1. TitleCase
     * Contains uppercase letters, lowercase letters, and numbers
     * The initial character is an uppercase letter
     * The initial letter of each word is capitalized
  2. lower_snake_case
     * Contains lowercase letters, underscores, and numbers
     * Words are separated by a single underscore
  3. UPPER_SNAKE_CASE
     * Contains uppercase letters, underscores, and numbers
     * Words are separated by a single underscore
  4. camelCase
     * Contains uppercase letters, lowercase letters, and numbers
     * The initial character is an lowercase letter
     * The initial letter of each subsequent word is capitalized
     * **Note:** The style guide below does not use camelCase for any identifier in .proto files; the terminology is only clarified here since some language’s generated code may transform identifiers into this style.

In all cases, treat abbreviations as though they are single words: use `GetDnsRequest` rather than `GetDNSRequest`, `dns_request` rather than `d_n_s_request`.

#### Underscores in Identifiers

Don’t use underscores as the initial or final character of a name. Any underscore should always be followed by a letter (not a number or a second underscore).

The motivation for this rule is that each protobuf language implementation may convert identifiers into the local language style: a name of `song_id` in a .proto file may end up having accessors for the field which are capitalized as `SongId`, `songId` or `song_id` depending on the language.

By using underscores only before letters, it avoids situations where names may be distinct in one style, but would collide after they are transformed into one of the other styles.

For example, both `DNS2` and `DNS_2` would both transform into TitleCase as `Dns2`. Allowing either of those names can be lead to painful situations when a message is used only in some languages where the generated code keeps the original UPPER_SNAKE_CASE style, becomes widely established, and then is only later used in a language where names are transformed to TitleCase where they collide.

When applied, this style rule means that you should use `XYZ2` or `XYZ_V2` rather than `XYZ_2` or `XYZ_2V`.

## Packages

Packages names should be a dot-delimited sequence of lower_snake_case names. They should not contain uppercase letters.

Multi-word package names may be lower_snake_case or dot.delimited (dot-delimited package names are emitted as nested packages/namespaces in most languages).

Package names should attempt to be a short but unique name based on the project name. The package should not be coupled with the directory path, especially when the files are in a deeply nested path.

Package names should not be Java packages (`com.company.x.y`); instead use `x.y` as the package and use the `java_package` option.

## Message Names

Use TitleCase for message names.

    message SongRequest {

## Field Names

Use snake_case for field names, including extensions.

Use pluralized names for repeated fields.

    string song_name = 1;
    repeated Song songs = 2;

## Oneof Names

Use lower_snake_case for oneof names.

    oneof song_id {
      string song_human_readable_id = 1;
      int64 song_machine_id = 2;

## Enums

Use TitleCase for enum type names.

Use UPPER_SNAKE_CASE for enum value names.

    enum FooBar {
      FOO_BAR_UNSPECIFIED = 0;
      FOO_BAR_FIRST_VALUE = 1;
      FOO_BAR_SECOND_VALUE = 2;

The first listed value should be a zero value enum and have the suffix of either `_UNSPECIFIED` or `_UNKNOWN`. This value may be used as an unknown/default value and should be distinct from any of the semantic values you expect to be explicitly set. For more information on the unspecified enum value, see [the Proto Best Practices page](https://protobuf.dev/best-practices/dos-donts#unspecified-enum).

Assign the number used for each value densely and sequentially increasing. Gaps in numbering should only occur when a previously used enum value is removed. Avoid using negative values.

#### Enum Value Prefixing

Enum values are semantically considered to not be scoped by their containing enum name, so the same name in two sibling enums is not allowed. For example, the following would be rejected by protoc since the `SET` value defined in the two enums are considered to be in the same scope:

    enum CollectionType {
      COLLECTION_TYPE_UNSPECIFIED = 0;
      SET = 1;
      MAP = 2;
      ARRAY = 3;

    // Won't compile - `SET` enum name will clash
    // with the one defined in `CollectionType` enum.
    enum TennisVictoryType {
      TENNIS_VICTORY_TYPE_UNSPECIFIED = 0;
      GAME = 1;
      SET = 2;
      MATCH = 3;

Name collisions are a high risk when enums are defined at the top level of a file (not nested inside a message definition); in that case the siblings include enums defined in other files that set the same package, where protoc may not be able to detect the collision has occurred at code generation time.

To avoid these risks, it is strongly recommended to do one of:

  * Prefix every value with the enum name (converted to UPPER_SNAKE_CASE)
  * Nest the enum inside a containing message

Either option is enough to mitigate collision risks, but prefer top-level enums with prefixed values over creating a message simply to mitigate the issue. Since some languages don’t support an enum being defined inside a “struct” type, preferring prefixed values ensures a consistent approach across binding languages.

When prefixing enum values, the remainder of the name with the prefix stripped should still be a legal and style-conformant enum name. For example, avoid the following:

    enum DeviceTier {
      DEVICE_TIER_UNKNOWN = 0;
      DEVICE_TIER_1 = 1;
      DEVICE_TIER_2 = 2;

Instead, use a value name like `DEVICE_TIER_TIER1`, where the `DEVICE_TIER_` portion is viewed as scoping the enum value rather than as part of the individual enum value name. Some Protobuf implementations automatically strip the prefix that matches the containing enum name where it is safe to do so, but could not in this example since a bare `1` is not a legal enum value name.

A future Edition will add support for scoped enums, which will eliminate the need to manually prefix each enum value and enable this to be written succinctly as `TIER1 = 1`.

## Services

Use TitleCase for service names and method names.

    service FooService {
      rpc GetSomething(GetSomethingRequest) returns (GetSomethingResponse);
      rpc ListSomething(ListSomethingRequest) returns (ListSomethingResponse);

## Things to Avoid

### Avoid confusing relative packages on field types

The .proto syntax allows you to reference types that are defined in a different package in a relative manner. Resolution will search up the parent packages to resolve a relative name. For example:

    // File: a.proto
    package a.b;
    message C {}

    // File: x.proto
    package a.x;
    message SomeMsg {
      // Since we are in a.x.SomeMsg, this name is searched for in this order:
      //  .a.x.SomeMsg.b.C
      //  .a.x.b.C
      //  .a.b.C
      //  .b.C
      b.C field3 = 1;

Relying on the “relative to a parent package” functionality when referencing types defined in a different package is supported but can be confusing: in this example the spelling `a.b.C` should be used.

Naming other types in the current package (for example, sibling messages) without writing the package is idiomatic.

You can write a fully qualified name by using a leading period to avoid any ambiguity concerns, but is commonly omitted in most cases even if the package is fully written out.

### Required Fields

Required fields are a way to enforce that a given field must be set when parsing wire bytes, and otherwise refuse to parse the message. The required invariant is generally not enforced on messages constructed in memory. Required fields were removed in proto3. Proto2 `required` fields that have been migrated to editions 2023 can use the `field_presence` feature set to `LEGACY_REQUIRED` to accommodate.

While enforcement of required fields at the schema level is intuitively desirable, one of the primary design goals of protobuf is to support long term schema evolution. No matter how obviously required a given field seems to be today, there is a plausible future where the field should no longer be set (e.g. an `int64 user_id` may need to migrate to a `UserId user_id` in the future).

Especially in the case of middleware servers that may forward messages that they don’t really need to process, the semantics of `required` has proven too harmful for those long-term evolution goals, and so is now very strongly discouraged.

See [Required is Strongly Deprecated](https://protobuf.dev/programming-guides/proto2#required-deprecated).

### Groups

Groups is an alternate syntax and wire format for nested messages. Groups are considered deprecated in proto2, were removed from proto3, and are converted to a delimited representation in edition 2023. You can use a nested message definition and field of that type instead of using the group syntax, using the [`message_encoding`](https://protobuf.dev/editions/features#message_encoding) feature for wire-compatibility.

See [groups](https://protobuf.dev/programming-guides/proto2#groups).