# Go Generated Code Guide (Opaque)

Describes exactly what Go code the protocol buffer compiler generates for any given protocol definition.

Any differences between proto2 and proto3 generated code are highlighted - note that these differences are in the generated code as described in this document, not the base API, which are the same in both versions. You should read the [proto2 language guide](https://protobuf.dev/programming-guides/proto2) and/or the [proto3 language guide](https://protobuf.dev/programming-guides/proto3) before reading this document.

#### Note

You are looking at documentation for the Opaque API, which is the current version. If you are working with .proto files that use the older Open Struct API (you can tell by the API level setting in the respective .proto files), see [Go Generated Code (Open)](https://protobuf.dev/reference/go/go-generated) for the corresponding documentation. See [Go Protobuf: The new Opaque API](https://go.dev/blog/protobuf-opaque) for the introduction of the Opaque API.

## Compiler Invocation

The protocol buffer compiler requires a plugin to generate Go code. Install it using Go 1.16 or higher by running:

    go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

This will install a `protoc-gen-go` binary in `$GOBIN`. Set the `$GOBIN` environment variable to change the installation location. It must be in your `$PATH` for the protocol buffer compiler to find it.

The protocol buffer compiler produces Go output when invoked with the `go_out` flag. The argument to the `go_out` flag is the directory where you want the compiler to write your Go output. The compiler creates a single source file for each `.proto` file input. The name of the output file is created by replacing the `.proto` extension with `.pb.go`.

Where in the output directory the generated `.pb.go` file is placed depends on the compiler flags. There are several output modes:

  * If the `paths=import` flag is specified, the output file is placed in a directory named after the Go package’s import path (such as one provided by the `go_package` option within the `.proto` file). For example, an input file `protos/buzz.proto` with a Go import path of `example.com/project/protos/fizz` results in an output file at `example.com/project/protos/fizz/buzz.pb.go`. This is the default output mode if a `paths` flag is not specified.
  * If the `module=$PREFIX` flag is specified, the output file is placed in a directory named after the Go package’s import path (such as one provided by the `go_package` option within the `.proto` file), but with the specified directory prefix removed from the output filename. For example, an input file `protos/buzz.proto` with a Go import path of `example.com/project/protos/fizz` and `example.com/project` specified as the `module` prefix results in an output file at `protos/fizz/buzz.pb.go`. Generating any Go packages outside the module path results in an error. This mode is useful for outputting generated files directly into a Go module.
  * If the `paths=source_relative` flag is specified, the output file is placed in the same relative directory as the input file. For example, an input file `protos/buzz.proto` results in an output file at `protos/buzz.pb.go`.

Flags specific to `protoc-gen-go` are provided by passing a `go_opt` flag when invoking `protoc`. Multiple `go_opt` flags may be passed. For example, when running:

    protoc --proto_path=src --go_out=out --go_opt=paths=source_relative foo.proto bar/baz.proto

the compiler will read input files `foo.proto` and `bar/baz.proto` from within the `src` directory, and write output files `foo.pb.go` and `bar/baz.pb.go` to the `out` directory. The compiler automatically creates nested output sub-directories if necessary, but will not create the output directory itself.

## Packages

In order to generate Go code, the Go package’s import path must be provided for every `.proto` file (including those transitively depended upon by the `.proto` files being generated). There are two ways to specify the Go import path:

  * by declaring it within the `.proto` file, or
  * by declaring it on the command line when invoking `protoc`.

We recommend declaring it within the `.proto` file so that the Go packages for `.proto` files can be centrally identified with the `.proto` files themselves and to simplify the set of flags passed when invoking `protoc`. If the Go import path for a given `.proto` file is provided by both the `.proto` file itself and on the command line, then the latter takes precedence over the former.

The Go import path is locally specified in a `.proto` file by declaring a `go_package` option with the full import path of the Go package. Example usage:

    option go_package = "example.com/project/protos/fizz";

The Go import path may be specified on the command line when invoking the compiler, by passing one or more `M${PROTO_FILE}=${GO_IMPORT_PATH}` flags. Example usage:

    protoc --proto_path=src \
      --go_opt=Mprotos/buzz.proto=example.com/project/protos/fizz \
      --go_opt=Mprotos/bar.proto=example.com/project/protos/foo \
      protos/buzz.proto protos/bar.proto

Since the mapping of all `.proto` files to their Go import paths can be quite large, this mode of specifying the Go import paths is generally performed by some build tool (e.g., [Bazel](https://bazel.build/)) that has control over the entire dependency tree. If there are duplicate entries for a given `.proto` file, then the last one specified takes precedence.

For both the `go_package` option and the `M` flag, the value may include an explicit package name separated from the import path by a semicolon. For example: `"example.com/protos/foo;package_name"`. This usage is discouraged since the package name will be derived by default from the import path in a reasonable manner.

The import path is used to determine which import statements must be generated when one `.proto` file imports another `.proto` file. For example, if `a.proto` imports `b.proto`, then the generated `a.pb.go` file needs to import the Go package which contains the generated `b.pb.go` file (unless both files are in the same package). The import path is also used to construct output filenames. See the "Compiler Invocation" section above for details.

There is no correlation between the Go import path and the [`package` specifier](https://protobuf.dev/programming-guides/proto3#packages) in the `.proto` file. The latter is only relevant to the protobuf namespace, while the former is only relevant to the Go namespace. Also, there is no correlation between the Go import path and the `.proto` import path.

## API level

The generated code either uses the Open Struct API or the Opaque API. See the [Go Protobuf: The new Opaque API](https://go.dev/blog/protobuf-opaque) blog post for an introduction.

Depending on the syntax your `.proto` file uses, here is which API will be used:

`.proto` syntax| API level
proto2| Open Struct API
proto3| Open Struct API
edition 2023| Open Struct API
edition 2024+| Opaque API

You can select the API by setting the `api_level` editions feature in your `.proto` file. This can be set per file or per message:

    edition = "2023";

    package log;

    import "google/protobuf/go_features.proto";
    option features.(pb.go).api_level = API_OPAQUE;

    message LogEntry { … }

For your convenience, you can also override the default API level with a `protoc` command-line flag:

    protoc […] --go_opt=default_api_level=API_HYBRID

To override the default API level for a specific file (instead of all files), use the `apilevelM` mapping flag (similar to the `M` flag for import paths):

    protoc […] --go_opt=apilevelMhello.proto=API_HYBRID

The command-line flags also work for `.proto` files still using proto2 or proto3 syntax, but if you want to select the API level from within the `.proto` file, you need to migrate said file to editions first.

## Messages

Given a simple message declaration:

    message Artist {}

the protocol buffer compiler generates a struct called `Artist`. An `*Artist` implements the [`proto.Message`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Message) interface.

The [`proto` package](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc) provides functions which operate on messages, including conversion to and from binary format.

The `proto.Message` interface defines a `ProtoReflect` method. This method returns a [`protoreflect.Message`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message) which provides a reflection-based view of the message.

The `optimize_for` option does not affect the output of the Go code generator.

When multiple goroutines concurrently access the same message, the following rules apply:

  * Accessing (reading) fields concurrently is safe, with one exception:
    * Accessing a [lazy field](https://github.com/protocolbuffers/protobuf/blob/cacb096002994000f8ccc6d9b8e1b5b0783ee561/src/google/protobuf/descriptor.proto#L609) for the first time is a modification.
  * Modifying different fields in the same message is safe.
  * Modifying a field concurrently is not safe.
  * Modifying a message in any way concurrently with functions of the [`proto` package](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc), such as [`proto.Marshal`](https://pkg.go.dev/google.golang.org/protobuf/proto#Marshal) or [`proto.Size`](https://pkg.go.dev/google.golang.org/protobuf/proto#Size) is not safe.

### Nested Types

A message can be declared inside another message. For example:

    message Artist {
      message Name {

In this case, the compiler generates two structs: `Artist` and `Artist_Name`.

## Fields

The protocol buffer compiler generates accessor methods (setters and getters) for each field defined within a message.

Note that the generated Go accessor methods always use camel-case naming, even if the field name in the `.proto` file uses lower-case with underscores ([as it should](https://protobuf.dev/programming-guides/style)). The case-conversion works as follows:

  1. The first letter is capitalized for export. If the first character is an underscore, it is removed and a capital X is prepended.
  2. If an interior underscore is followed by a lower-case letter, the underscore is removed, and the following letter is capitalized.

Thus, you can access the proto field `birth_year` using the `GetBirthYear()` method in Go, and `_birth_year_2` using `GetXBirthYear_2()`.

### Singular Fields

For this field definition:

    // proto2 and proto3
    message Artist {
      optional int32 birth_year = 1;

    // editions
    message Artist {
      int32 birth_year = 1 [features.field_presence = EXPLICIT];

the compiler generates a Go struct with the following accessor methods:

    func (m *Artist) GetBirthYear() int32
    func (m *Artist) SetBirthYear(v int32)

With implicit presence, the getter returns the `int32` value in `birth_year` or the [zero value](https://golang.org/ref/spec#The_zero_value) of that type if the field is unset (`0` for numbers, the empty string for strings). With explicit presence, the getter returns the `int32` value in `birth_year` or the default value if the field is unset. If the default is not explicitly set, the zero value is used instead.

For other scalar field types (including `bool`, `bytes`, and `string`), `int32` is replaced with the corresponding Go type according to the [scalar value types table](https://protobuf.dev/programming-guides/proto3#scalar).

In fields with explicit presence, you can also use these methods:

    func (m *Artist) HasBirthYear() bool
    func (m *Artist) ClearBirthYear()

### Singular Message Fields

Given the message type:

    message Band {}

For a message with a `Band` field:

    // proto2
    message Concert {
      optional Band headliner = 1;
      // The generated code is the same result if required instead of optional.

    // proto3 and editions
    message Concert {
      Band headliner = 1;

The compiler will generate a Go struct with the following accessor methods:

    type Concert struct { ... }

    func (m *Concert) GetHeadliner() *Band { ... }
    func (m *Concert) SetHeadliner(v *Band) { ... }
    func (m *Concert) HasHeadliner() bool { ... }
    func (m *Concert) ClearHeadliner() { ... }

The `GetHeadliner()` accessor method is safe to call even if `m` is nil. This makes it possible to chain get calls without intermediate `nil` checks:

    var m *Concert // defaults to nil
    log.Infof("GetFoundingYear() = %d (no panic!)", m.GetHeadliner().GetFoundingYear())

If the field is unset, the getter will return the default value of the field. For messages, the default value is a nil pointer.

Contrary to getters, setters do not perform nil checks for you. Therefore, you cannot safely call setters on possibly-nil messages.

### Repeated Fields

For repeated fields, the accessor methods use a slice type. For this message with a repeated field:

    message Concert {
      // Best practice: use pluralized names for repeated fields:
      // /programming-guides/style#repeated-fields
      repeated Band support_acts = 1;

the compiler generates a Go struct with the following accessor methods:

    type Concert struct { ... }

    func (m *Concert) GetSupportActs() []*Band { ... }
    func (m *Concert) SetSupportActs(v []*Band) { ... }

Likewise, for the field definition `repeated bytes band_promo_images = 1;` the compiler will generate accessors working with the `[][]byte` type. For a repeated enumeration `repeated MusicGenre genres = 2;`, the compiler generates accessors working with the `[]MusicGenre` type.

The following example shows how to construct a `Concert` message using a builder.

    concert := Concert_builder{
      SupportActs: []*Band{
        {}, // First element.
        {}, // Second element.
      },
    }.Build()

Alternatively, you can use setters:

    concert := &Concert{}
    concert.SetSupportActs([]*Band{
        {}, // First element.
        {}, // Second element.
    })

To access the field, you can do the following:

    support := concert.GetSupportActs() // support type is []*Band.
    b1 := support[0] // b1 type is *Band, the first element in support_acts.

### Map Fields

Each map field generates accessors working with type `map[TKey]TValue` where `TKey` is the field’s key type and `TValue` is the field’s value type. For this message with a map field:

    message MerchItem {}

    message MerchBooth {
      // items maps from merchandise item name ("Signed T-Shirt") to
      // a MerchItem message with more details about the item.
      map<string, MerchItem> items = 1;

the compiler generates a Go struct with the following accessor methods:

    type MerchBooth struct { ... }

    func (m *MerchBooth) GetItems() map[string]*MerchItem { ... }
    func (m *MerchBooth) SetItems(v map[string]*MerchItem) { ... }

### Oneof Fields

For a oneof field, the protobuf compiler generates accessors for each of the singular fields within the oneof.

For this message with a oneof field:

    package account;
    message Profile {
      oneof avatar {
        string image_url = 1;
        bytes image_data = 2;

the compiler generates a Go struct with the following accessor methods:

    type Profile struct { ... }

    func (m *Profile) WhichAvatar() case_Profile_Avatar { ... }
    func (m *Profile) GetImageUrl() string { ... }
    func (m *Profile) GetImageData() []byte { ... }

    func (m *Profile) SetImageUrl(v string) { ... }
    func (m *Profile) SetImageData(v []byte) { ... }

    func (m *Profile) HasAvatar() bool { ... }
    func (m *Profile) HasImageUrl() bool { ... }
    func (m *Profile) HasImageData() bool { ... }

    func (m *Profile) ClearAvatar() { ... }
    func (m *Profile) ClearImageUrl() { ... }
    func (m *Profile) ClearImageData() { ... }

The following example shows how to set the field using a builder:

    p1 := accountpb.Profile_builder{
      ImageUrl: proto.String("https://example.com/image.png"),
    }.Build()

…or, equivalently, using a setter:

    // imageData is []byte
    imageData := getImageData()
    p2 := &accountpb.Profile{}
    p2.SetImageData(imageData)

To access the field, you can use a switch statement on the `WhichAvatar()` result:

    switch m.WhichAvatar() {
    case accountpb.Profile_ImageUrl_case:
        // Load profile image based on URL
        // using m.GetImageUrl()

    case accountpb.Profile_ImageData_case:
        // Load profile image based on bytes
        // using m.GetImageData()

    case accountpb.Profile_Avatar_not_set_case:
        // The field is not set.

    default:
        return fmt.Errorf("Profile.Avatar has an unexpected new oneof field %v", x)

### Builders

Builders are a convenient way to construct and initialize a message within a single expression, especially when working with nested messages like unit tests.

Contrary to builders in other languages (like Java), Go protobuf builders are not meant to be passed around between functions. Instead, call `Build()` immediately and pass the resulting proto message instead, using setters to modify fields.

Here is an example for using builders to create a `Band` message, which is the only repeated field inside the enclosing `Concert` message:

    concert := Concert_builder{
      SupportActs: []*Band{
        Band_builder{
          Name: proto.String("Varint and the Marshals"),
        }.Build()
      },
    }.Build()

## Enumerations

Given an enumeration like:

    message Venue {
      enum Kind {
        KIND_UNSPECIFIED = 0;
        KIND_CONCERT_HALL = 1;
        KIND_STADIUM = 2;
        KIND_BAR = 3;
        KIND_OPEN_AIR_FESTIVAL = 4;
      Kind kind = 1;
      // ...

the protocol buffer compiler generates a type and a series of constants with that type:

    type Venue_Kind int32

    const (
        Venue_KIND_UNSPECIFIED       Venue_Kind = 0
        Venue_KIND_CONCERT_HALL      Venue_Kind = 1
        Venue_KIND_STADIUM           Venue_Kind = 2
        Venue_KIND_BAR               Venue_Kind = 3
        Venue_KIND_OPEN_AIR_FESTIVAL Venue_Kind = 4
    )

For enums within a message (like the one above), the type name begins with the message name:

    type Venue_Kind int32

For a package-level enum:

    enum Genre {
      GENRE_UNSPECIFIED = 0;
      GENRE_ROCK = 1;
      GENRE_INDIE = 2;
      GENRE_DRUM_AND_BASS = 3;
      // ...

the Go type name is unmodified from the proto enum name:

    type Genre int32

This type has a `String()` method that returns the name of a given value.

The `Enum()` method initializes freshly allocated memory with a given value and returns the corresponding pointer:

    func (Genre) Enum() *Genre

The protocol buffer compiler generates a constant for each value in the enum. For enums within a message, the constants begin with the enclosing message’s name:

    const (
        Venue_KIND_UNSPECIFIED       Venue_Kind = 0
        Venue_KIND_CONCERT_HALL      Venue_Kind = 1
        Venue_KIND_STADIUM           Venue_Kind = 2
        Venue_KIND_BAR               Venue_Kind = 3
        Venue_KIND_OPEN_AIR_FESTIVAL Venue_Kind = 4
    )

For a package-level enum, the constants begin with the enum name instead:

    const (
        Genre_GENRE_UNSPECIFIED   Genre = 0
        Genre_GENRE_ROCK          Genre = 1
        Genre_GENRE_INDIE         Genre = 2
        Genre_GENRE_DRUM_AND_BASS Genre = 3
    )

The protobuf compiler also generates a map from integer values to the string names and a map from the names to the values:

    var Genre_name = map[int32]string{
        0: "GENRE_UNSPECIFIED",
        1: "GENRE_ROCK",
        2: "GENRE_INDIE",
        3: "GENRE_DRUM_AND_BASS",
    var Genre_value = map[string]int32{
        "GENRE_UNSPECIFIED":   0,
        "GENRE_ROCK":          1,
        "GENRE_INDIE":         2,
        "GENRE_DRUM_AND_BASS": 3,

Note that the `.proto` language allows multiple enum symbols to have the same numeric value. Symbols with the same numeric value are synonyms. These are represented in Go in exactly the same way, with multiple names corresponding to the same numeric value. The reverse mapping contains a single entry for the numeric value to the name which appears first in the .proto file.

## Extensions (proto2)

Given an extension definition:

    extend Concert {
      optional int32 promo_id = 123;

The protocol buffer compiler will generate a [`protoreflect.ExtensionType`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#ExtensionType) value named `E_Promo_id`. This value may be used with the [`proto.GetExtension`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#GetExtension), [`proto.SetExtension`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#SetExtension), [`proto.HasExtension`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#HasExtension), and [`proto.ClearExtension`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#ClearExtension) functions to access an extension in a message. The `GetExtension` function and `SetExtension` functions respectively return and accept an `interface{}` value containing the extension value type.

For singular scalar extension fields, the extension value type is the corresponding Go type from the [scalar value types table](https://protobuf.dev/programming-guides/proto3#scalar).

For singular embedded message extension fields, the extension value type is `*M`, where `M` is the field message type.

For repeated extension fields, the extension value type is a slice of the singular type.

For example, given the following definition:

    extend Concert {
      optional int32 singular_int32 = 1;
      repeated bytes repeated_strings = 2;
      optional Band singular_message = 3;

Extension values may be accessed as:

    m := &somepb.Concert{}
    proto.SetExtension(m, extpb.E_SingularInt32, int32(1))
    proto.SetExtension(m, extpb.E_RepeatedString, []string{"a", "b", "c"})
    proto.SetExtension(m, extpb.E_SingularMessage, &extpb.Band{})

    v1 := proto.GetExtension(m, extpb.E_SingularInt32).(int32)
    v2 := proto.GetExtension(m, extpb.E_RepeatedString).([][]byte)
    v3 := proto.GetExtension(m, extpb.E_SingularMessage).(*extpb.Band)

Extensions can be declared nested inside of another type. For example, a common pattern is to do something like this:

    message Promo {
      extend Concert {
        optional int32 promo_id = 124;

In this case, the `ExtensionType` value is named `E_Promo_Concert`.

## Services

The Go code generator does not produce output for services by default. If you enable the [gRPC](https://www.grpc.io/) plugin (see the [gRPC Go Quickstart guide](https://github.com/grpc/grpc-go/tree/master/examples)) then code will be generated to support gRPC.


# C++ Arena Allocation Guide

Arena allocation is a C++-only feature that helps you optimize your memory usage and improve performance when working with protocol buffers.

This page describes exactly what C++ code the protocol buffer compiler generates in addition to the code described in the [C++ Generated Code Guide](https://protobuf.dev/reference/cpp/cpp-generated) when arena allocation is enabled. It assumes that you are familiar with the material in the [language guide](https://protobuf.dev/programming-guides/editions) and the [C++ Generated Code Guide](https://protobuf.dev/reference/cpp/cpp-generated).

## Why Use Arena Allocation?

Memory allocation and deallocation constitutes a significant fraction of CPU time spent in protocol buffers code. By default, protocol buffers performs heap allocations for each message object, each of its subobjects, and several field types, such as strings. These allocations occur in bulk when parsing a message and when building new messages in memory, and associated deallocations happen when messages and their subobject trees are freed.

Arena-based allocation has been designed to reduce this performance cost. With arena allocation, new objects are allocated out of a large piece of preallocated memory called the arena. Objects can all be freed at once by discarding the entire arena, ideally without running destructors of any contained object (though an arena can still maintain a “destructor list” when required). This makes object allocation faster by reducing it to a simple pointer increment, and makes deallocation almost free. Arena allocation also provides greater cache efficiency: when messages are parsed, they are more likely to be allocated in continuous memory, which makes traversing messages more likely to hit hot cache lines.

To get these benefits you’ll need to be aware of object lifetimes and find a suitable granularity at which to use arenas (for servers, this is often per-request). You can find out more about how to get the most from arena allocation in Usage patterns and best practices.

This table summarizes the typical performance advantages and disadvantages of using arenas:

Operation| Heap-allocated proto messages| Arena-allocated proto messages
---|---|---
 _Message allocation_|  Slower on average| Faster on average
 _Message destruction_|  Slower on average| Faster on average
 _Message moves_|  Always a move (equivalent to a [shallow copy](https://en.wikipedia.org/wiki/Object_copying#Shallow_copy) in cost)| Sometimes a [deep copy](https://en.wikipedia.org/wiki/Object_copying#Deep_copy)

## Getting Started

The protocol buffer compiler generates code for arena allocation for the messages in your file, as used in the following example.

    #include <google/protobuf/arena.h>
      google::protobuf::Arena arena;
      MyMessage* message = google::protobuf::Arena::Create<MyMessage>(&arena);
      // ...

The message object created by `Create()` exists for as long as `arena` exists, and you should not `delete` the returned message pointer. All of the message object’s internal storage (with a few exceptions1) and submessages (for example, submessages in a repeated field within `MyMessage`) are allocated on the arena as well.

For the most part, the rest of your code will be the same as if you weren’t using arena allocation.

We’ll look at the arena API in more detail in the following sections, and you can see a more extensive example at the end of the document.

## Arena Class API

You create message objects on the arena using the [`google::protobuf::Arena`](https://protobuf.dev/reference/cpp/api-docs/google.protobuf.arena) class. This class implements the following public methods.

### Constructors

  * `Arena()`: Creates a new arena with default parameters, tuned for average use cases.
  * `Arena(const ArenaOptions& options)`: Creates a new arena that uses the specified allocation options. The options available in `ArenaOptions` include the ability to use an initial block of user-provided memory for allocations before resorting to the system allocator, control over the initial and maximum request sizes for blocks of memory, and allowing you to pass in custom block allocation and deallocation function pointers to build freelists and others on top of the blocks.

### Allocation Methods

  * `template<typename T> static T* Create(Arena* arena)` or `template<typename T> static T* Create(Arena* arena, args...)`

    * If `T` is fully compatible2, then the method creates a new protocol buffer object of type `T` and its subobjects on the arena.

If `arena` is not NULL, the returned object is allocated on the arena, its internal storage and sub-types (if any) will be allocated on the same arena, and its lifetime is the same as that of the arena. The object must not be deleted/freed manually: the arena owns the object for lifetime purposes.

If `arena` is NULL, the returned object is allocated on the heap, and the caller owns the object upon return.

    * If `T` is a user-type, the method lets you create an object but not the subobjects on the arena. For example, let’s say you have this C++ class:

          class MyCustomClass {
              MyCustomClass(int arg1, int arg2);
              // ...
          };

…you can create an instance of it on the arena like this:

          void func() {
              // ...
              google::protobuf::Arena arena;
              MyCustomClass* c = google::protobuf::Arena::Create<MyCustomClass>(&arena, constructor_arg1, constructor_arg2);
              // ...

  * `template<typename T> static T* CreateArray(Arena* arena, size_t n)`: If `arena` is not NULL, this method allocates raw storage for `n` elements of type `T` and returns it. The arena owns the returned memory and will free it on its own destruction. If `arena` is NULL, this method allocates storage on the heap and the caller receives ownership.

`T` must have a trivial constructor: constructors are not called when the array is created on the arena.

### “Owned list” Methods

The following methods let you specify that particular objects or destructors are “owned” by the arena, ensuring that they are deleted or called when the arena itself is deleted

  * `template<typename T> void Own(T* object)`: Adds `object` to the arena’s list of owned heap objects. When the arena is destroyed, it traverses this list and frees each object using operator delete, i.e., the system memory allocator. This method is useful in cases when an object’s lifetime should be tied to the arena but, for whatever reason, the object itself cannot be or was not already allocated on the arena.
  * `template<typename T> void OwnDestructor(T* object)`: Adds the destructor of `object` to the arena’s list of destructors to call. When the arena is destroyed, it traverses this list and calls each destructor in turn. It does not attempt to free the underlying memory of object. This method is useful when an object is embedded in arena-allocated storage but its destructor will not otherwise be called, for example because its containing class is a protobuf message whose destructor won’t be called, or because it was manually constructed in a block allocated by `AllocateArray()`.

### Other Methods

  * `uint64 SpaceUsed() const`: Returns the total size of the arena, which is the sum of the sizes of the underlying blocks. This method is thread-safe; however, if there are concurrent allocations from multiple threads this method’s return value may not include the sizes of those new blocks.
  * `uint64 Reset()`: Destroys the arena’s storage, first calling all registered destructors and freeing all registered heap objects and then discarding all arena blocks. This teardown procedure is equivalent to that which occurs when the arena’s destructor runs, except the arena is reusable for new allocations after this method returns. Returns the total size used by the arena: this information is useful for tuning performance.
  * `template<typename T> Arena* GetArena()`: Returns a pointer to this arena. Not directly very useful but allows `Arena` to be used in template instantiations that expect `GetArena()` methods to be present.

### Thread Safety

`google::protobuf::Arena`’s allocation methods are thread-safe, and the underlying implementation goes to some length to make multithreaded allocation fast. The `Reset()` method is _not_ thread-safe: the thread performing the arena reset must synchronize with all threads performing allocations or using objects allocated from that arena first.

## Generated Message Class

The following message class members are changed or added when you enable arena allocation.

### Message Class Methods

  * `Message(Message&& other)`: If the source message is not on arena, the move constructor efficiently _moves_ all fields from one message to another without making copies or heap allocations (the time complexity of this operation is `O(number-of-declared-fields)`). However, if the source message is on arena, it performs a _deep copy_ of the underlying data. In both cases the source message is left in a valid but unspecified state.
  * `Message& operator=(Message&& other)`: If both messages are not on arena or are on the _same_ arena, the move-assignment operator efficiently _moves_ all fields from one message to another without making copies or heap allocations (the time complexity of this operation is `O(number-of-declared-fields)`). However, if only one message is on arena, or the messages are on different arenas, it performs a _deep copy_ of the underlying data. In both cases the source message is left in a valid but unspecified state.
  * `void Swap(Message* other)`: If both messages to be swapped are not on arenas or are on the _same_ arena, [`Swap()`](https://protobuf.dev/reference/cpp/cpp-generated#message) behaves as it does without having arena allocation enabled: it efficiently swaps the message objects’ contents, almost exclusively through cheap pointer swaps, avoiding copies. However, if only one message is on an arena, or the messages are on different arenas, `Swap()` performs _deep copies_ of the underlying data. This new behavior is necessary because otherwise the swapped sub-objects could have differing lifetimes, leading potentially to use-after-free bugs.
  * `Message* New(Arena* arena)`: An alternate override for the standard `New()` method. It allows a new message object of this type to be created on the given arena. Its semantics are identical to `Arena::Create<T>(arena)` if the concrete message type on which it is called is generated with arena allocation enabled. If the message type is not generated with arena allocation enabled, then it is equivalent to an ordinary allocation followed by `arena->Own(message)` if `arena` is not NULL.
  * `Arena* GetArena()`: Returns the arena on which this message object was allocated, if any.
  * `void UnsafeArenaSwap(Message* other)`: Identical to `Swap()`, except it assumes both objects are on the same arena (or not on arenas at all) and always uses the efficient pointer-swapping implementation of this operation. Using this method can improve performance as, unlike `Swap()`, it doesn’t need to check which messages live on which arena before performing the swap. As the `Unsafe` prefix suggests, you should only use this method if you are sure the messages you want to swap aren’t on different arenas; otherwise this method could have unpredictable results.

### Embedded Message Fields

When you allocate a message object on an arena, its embedded message field objects (submessages) are automatically owned by the arena as well. How these message objects are allocated depends on where they are defined:

  * If the message type is also defined in a `.proto` file with arena allocation enabled, the object is allocated on the arena directly.
  * If the message type is from another `.proto` without arena allocation enabled, the object is heap-allocated but is “owned” by the parent message’s arena. This means that when the arena is destroyed, the object will be freed along with the objects on the arena itself.

For the field definition:

    Bar foo = 1;

The following methods are added or have some special behavior when arena allocation is enabled. Otherwise, accessor methods just use the [default behavior](https://protobuf.dev/reference/cpp/cpp-generated#embeddedmessage).

  * `Bar* mutable_foo()`: Returns a mutable pointer to the submessage instance. If the parent object is on an arena then the returned object will be as well.
  * `void set_allocated_foo(Bar* bar)`: Takes a new object and adopts it as the new value for the field. Arena support adds additional copying semantics to maintain proper ownership when objects cross arena/arena or arena/heap boundaries:
    * If the parent object is on the heap and `bar` is on the heap, or if the parent and message are on the same arena, this method’s behavior is unchanged.
    * If the parent is on an arena and `bar` is on the heap, the parent message adds `bar` to its arena’s ownership list with `arena->Own()`.
    * If the parent is on an arena and `bar` is on a different arena, this method makes a copy of message and takes the copy as the new field value.
  * `Bar* release_foo()`: Returns the existing submessage instance of the field, if set, or a NULL pointer if not set, releasing ownership of this instance to the caller and clearing the parent message’s field. Arena support adds additional copying semantics to maintain the contract that the returned object is always _heap-allocated_ :
    * If the parent message is on an arena, this method will make a copy of the submessage on the heap, clear the field value, and return the copy.
    * If the parent message is on the heap, the method behavior is unchanged.
  * `void unsafe_arena_set_allocated_foo(Bar* bar)`: Identical to `set_allocated_foo`, but assumes both parent and submessage are on the same arena. Using this version of the method can improve performance as it doesn’t need to check whether the messages are on a particular arena or the heap. See allocated/release patterns for details on safe ways to use this.
  * `Bar* unsafe_arena_release_foo()`: Similar to `release_foo()`, but skips all ownership checking. See allocated/release patterns for details on safe ways to use this.

### String Fields

String fields store their data on the heap even when their parent message is on the arena. Because of this, string accessor methods use the [default behavior](https://protobuf.dev/reference/cpp/cpp-generated#string) even when arena allocation is enabled.

### Repeated Fields

Repeated fields allocate their internal array storage on the arena when the containing message is arena-allocated, and also allocate their elements on the arena when these elements are separate objects retained by pointer (messages or strings). At the message-class level, generated methods for repeated fields do not change. However, the `RepeatedField` and `RepeatedPtrField` objects that are returned by accessors do have new methods and modified semantics when arena support is enabled.

#### Repeated Numeric Fields

`RepeatedField` objects that contain primitive types have the following new/changed methods when arena allocation is enabled:

  * `void UnsafeArenaSwap(RepeatedField* other)`: Performs a swap of `RepeatedField` contents without validating that this repeated field and other are on the same arena. If they are not, the two repeated field objects must be on arenas with equivalent lifetimes. The case where one is on an arena and one is on the heap is checked and disallowed.
  * `void Swap(RepeatedField* other)`: Checks each repeated field object’s arena, and if one is on an arena while one is on the heap or if both are on arenas but on different ones, the underlying arrays are copied before the swap occurs. This means that after the swap, each repeated field object holds an array on its own arena or heap, as appropriate.

#### Repeated Embedded Message Fields

`RepeatedPtrField` objects that contain messages have the following new/changed methods when arena allocation is enabled.

  * `void UnsafeArenaSwap(RepeatedPtrField* other)`: Performs a swap of `RepeatedPtrField` contents without validating that this repeated field and other have the same arena pointer. If they do not, the two repeated field objects must have arena pointers with equivalent lifetimes. The case where one has a non-NULL arena pointer and one has a NULL arena pointer is checked and disallowed.

  * `void Swap(RepeatedPtrField* other)`: Checks each repeated field object’s arena pointer, and if one is non-NULL (contents on arena) while one is NULL (contents on heap) or if both are non-NULL but have different values, the underlying arrays and their pointed-to objects are copied before the swap occurs. This means that after the swap, each repeated field object holds an array on its own arena or on the heap, as appropriate.

  * `void AddAllocated(SubMessageType* value)`: Checks that the provided message object is on the same arena as the repeated field’s arena pointer.

    * The source and destination are both arena-allocated and on the same arena: the object pointer is added directly to the underlying array.
    * The source and destination are both arena-allocated and on different arenas: a copy is made, the original is freed if it was heap-allocated, and the copy is placed on the array.
    * The source is heap-allocated and the destination is arena-allocated: No copy is made.
    * The source is arena-allocated and the destination is heap-allocated: A copy is made and placed on the array.
    * Both source and destination are heap allocated: The object pointer is added directly to the underlying array.

This maintains the invariant that all objects pointed to by a repeated field are in the same ownership domain (heap or specific arena) as indicated by the repeated field’s arena pointer.

  * `SubMessageType* ReleaseLast()`: Returns a heap-allocated message equivalent to the last message in the repeated field, removing it from the repeated field. If the repeated field itself has a NULL arena pointer (and thus, all of its pointed-to messages are heap-allocated), then this method simply returns a pointer to the original object. Otherwise, if the repeated field has a non-NULL arena pointer, this method makes a copy that is heap-allocated and returns that copy. In both cases, the caller receives ownership of a heap-allocated object and is responsible for deleting the object.

  * `void UnsafeArenaAddAllocated(SubMessageType* value)`: Like `AddAllocated()`, but does not perform heap/arena checks or any message copies. It adds the provided pointer directly to the internal array of pointers for this repeated field. See allocated/release patterns for details on safe ways to use this.

  * `SubMessageType* UnsafeArenaReleaseLast()`: Like `ReleaseLast()` but performs no copies, even if the repeated field has a non-NULL arena pointer. Instead, it directly returns the pointer to the object as it was in the repeated field. See allocated/release patterns for details on safe ways to use this.

  * `void ExtractSubrange(int start, int num, SubMessageType** elements)`: Removes `num` elements from the repeated field, starting from index `start`, and returns them in `elements` if it is not NULL. If the repeated field is on an arena, and elements are being returned, the elements are copied to the heap first. In both cases (arena or no arena), the caller owns the returned objects on the heap.

  * `void UnsafeArenaExtractSubrange(int start, int num, SubMessageType** elements)`: Removes `num` elements from the repeated field, starting from index `start`, and returns them in `elements` if it is not NULL. Unlike `ExtractSubrange()`, this method never copies the extracted elements. See allocated/release patterns for details on safe ways to use this.

#### Repeated String Fields

Repeated fields of strings have the same new methods and modified semantics as repeated fields of messages, because they also maintain their underlying objects (namely, strings) by pointer reference.

## Usage Patterns and Best Practices

When using arena-allocated messages, several usage patterns can result in unintended copies or other negative performance effects. You should be aware of the following common patterns that may need to be altered when adapting code for arenas. (Note that we have taken care in the API design to ensure that correct behavior still occurs — but higher-performance solutions may require some reworking.)

### Unintended Copies

Several methods that never create object copies when not using arena allocation may end up doing so when arena support is enabled. These unwanted copies can be avoided if you make sure that your objects are allocated appropriately and/or use provided arena-specific method versions, as described in more detail below.

#### Set Allocated/Add Allocated/Release

By default, the `release_field()` and `set_allocated_field()` methods (for singular message fields), and the `ReleaseLast()` and `AddAllocated()` methods (for repeated message fields) allow user code to directly attach and detach submessages, passing ownership of pointers without copying any data.

However, when the parent message is on an arena, these methods now sometimes need to copy the passed in or returned object to maintain compatibility with existing ownership contracts. More specifically, methods that take ownership (`set_allocated_field()` and `AddAllocated()`) may copy data if the parent is on an arena and the new subobject is not, or vice versa, or they are on different arenas. Methods that release ownership (`release_field()` and `ReleaseLast()`) may copy data if the parent is on the arena, because the returned object must be on the heap, by contract.

To avoid such copies, we have added corresponding “unsafe arena” versions of these methods where copies are **never performed**: `unsafe_arena_set_allocated_field()`, `unsafe_arena_release_field()`, `UnsafeArenaAddAllocated()`, and `UnsafeArenaRelease()` for singular and repeated fields, respectively. These methods should be used only when you know they are safe to do so. There are two common patterns for these methods:

  * Moving messages trees between parts of the same arena. Note that the messages must be on the same arena for this case to be safe.
  * Temporarily loaning an owned message to a tree to avoid copies. Pairing an unsafe _add_ /_set_ method with an unsafe _release_ method performs the loan in the cheapest way possible regardless of how either message is owned (this pattern works when they are on the same arena, different arena, or no arena at all). Note that between the unsafe _add_ /_set_ and its corresponding _release_ , the borrower must not be swapped, moved, cleared or destroyed; the loaned message must not be swapped or moved; the loaned message must not be cleared or released by the borrower; and the loaned message must not be destroyed.

Here’s an example of how you can avoid unnecessary copies with these methods. Let’s say you have created the following messages on an arena.

    Arena* arena = new google::protobuf::Arena();
    MyFeatureMessage* arena_message_1 =
      google::protobuf::Arena::Create<MyFeatureMessage>(arena);
    arena_message_1->mutable_nested_message()->set_feature_id(11);

    MyFeatureMessage* arena_message_2 =
      google::protobuf::Arena::Create<MyFeatureMessage>(arena);

The following code makes inefficient usage of the `release_...()` API:

    arena_message_2->set_allocated_nested_message(arena_message_1->release_nested_message());

    arena_message_1->release_message(); // returns a copy of the underlying nested_message and deletes underlying pointer

Using the “unsafe arena” version instead avoids the copy:

    arena_message_2->unsafe_arena_set_allocated_nested_message(
       arena_message_1->unsafe_arena_release_nested_message());

You can find out more about these methods in the Embedded message fields section above.

#### Swap

When two messages’ contents are swapped with `Swap()`, the underlying subobjects may be copied if the two messages live on different arenas, or if one is on the arena and the other is on the heap. If you want to avoid this copy and either (i) know that the two messages are on the same arena or different arenas but the arenas have equivalent lifetimes, or (ii) know that the two messages are on the heap, you can use a new method, `UnsafeArenaSwap()`. This method both avoids the overhead of performing the arena check and avoids the copy if one would have occurred.

For example, the following code incurs a copy in the `Swap()` call:

    MyFeatureMessage* message_1 =
      google::protobuf::Arena::Create<MyFeatureMessage>(arena);
    message_1->mutable_nested_message()->set_feature_id(11);

    MyFeatureMessage* message_2 = new MyFeatureMessage;
    message_2->mutable_nested_message()->set_feature_id(22);

    message_1->Swap(message_2); // Inefficient swap!

To avoid the copy in this code, you allocate `message_2` on the same arena as `message_1`:

    MyFeatureMessage* message_2 =
       google::protobuf::Arena::Create<MyFeatureMessage>(arena);

### Granularity

We have found in most application server use cases that an “arena-per-request” model works well. You may be tempted to divide arena use further, either to reduce heap overhead (by destroying smaller arenas more often) or to reduce perceived thread-contention issues. However, the use of more fine-grained arenas may lead to unintended message copying, as we describe above. We have also spent effort to optimize the `Arena` implementation for the multithreaded use-case, so a single arena should be appropriate for use throughout a request lifetime even if multiple threads process that request.

## Example

Here’s a simple complete example demonstrating some of the features of the arena allocation API.

    // my_feature.proto
    edition = "2023";

    import "nested_message.proto";

    package feature_package;

    // NEXT Tag to use: 4
    message MyFeatureMessage {
      string feature_name = 1;
      repeated int32 feature_data = 2;
      NestedMessage nested_message = 3;
    };

    // nested_message.proto
    edition = "2023";

    package feature_package;

    // NEXT Tag to use: 2
    message NestedMessage {
      int32 feature_id = 1;
    };

Message construction and deallocation:

    #include <google/protobuf/arena.h>

    Arena arena;

    MyFeatureMessage* arena_message =
       google::protobuf::Arena::Create<MyFeatureMessage>(&arena);

    arena_message->set_feature_name("Editions Arena");
    arena_message->mutable_feature_data()->Add(2);
    arena_message->mutable_feature_data()->Add(4);
    arena_message->mutable_nested_message()->set_feature_id(247);

  1. Currently, string fields store their data on the heap even when the containing message is on the arena. Unknown fields are also heap-allocated. ↩

  2. What it takes to be a “fully compatible” type is internal to the protobuf library, and should not be assumed to be reliable. ↩


# Proto Serialization Is Not Canonical

Explains how serialization works and why it is not canonical.

Many people want a serialized proto to canonically represent the contents of that proto. Use cases include:

  * using a serialized proto as a key in a hash table
  * taking a fingerprint or checksum of a serialized proto
  * comparing serialized payloads as a way of checking message equality

Unfortunately, _protobuf serialization is not (and cannot be) canonical_. There are a few notable exceptions, such as MapReduce, but in general you should generally think of proto serialization as unstable. This page explains why.

## Deterministic is not Canonical

Deterministic serialization is not canonical. The serializer can generate different output for many reasons, including but not limited to the following variations:

  1. The protobuf schema changes in any way.
  2. The application being built changes in any way.
  3. The binary is built with different flags (eg. opt vs. debug).
  4. The protobuf library is updated.

This means that hashes of serialized protos are fragile and not stable across time or space.

There are many reasons why the serialized output can change. The above list is not exhaustive. Some of them are inherent difficulties in the problem space that would make it inefficient or impossible to guarantee canonical serialization even if we wanted to. Others are things we intentionally leave undefined to allow for optimization opportunities.

## Inherent Barriers to Stable Serialization

Protobuf objects preserve unknown fields to provide forward and backward compatibility. The handling of unknown fields is a primary obstacle to canonical serialization.

In the wire format, bytes fields and nested sub-messages use the same wire type. This ambiguity makes it impossible to correctly canonicalize messages stored in the unknown field set. Since the exact same contents may be either one, it is impossible to know whether to treat it as a message and recurse down or not.

For efficiency, implementations typically serialize unknown fields after known fields. Canonical serialization, however, would require interleaving unknown fields with known fields according to field number. This would impose significant efficiency and code size costs on all users, even those not requiring this feature.

## Things Intentionally Left Undefined

Even if canonical serialization was feasible (that is, if we could solve the unknown field problem), we intentionally leave serialization order undefined to allow for more optimization opportunities:

  1. If we can prove a field is never used in a binary, we can remove it from the schema completely and process it as an unknown field. This saves substantial code size and CPU cycles.
  2. There may be opportunities to optimize by serializing vectors of the same field together, even though this would break field number order.

To leave room for optimizations like this, we want to intentionally scramble field order in some configurations, so that applications do not inappropriately depend on field order.


# No Nullable Setters/Getters Support

Covers why Protobuf doesn’t support nullable setters and getters

We have heard feedback that some folks would like protobuf to support nullable getters/setters in their null-friendly language of choice (particularly Kotlin, C#, and Rust). While this does seem to be a helpful feature for folks using those languages, the design choice has tradeoffs which have led to the Protobuf team choosing not to implement them.

Explicit presence is not a concept that directly maps to the traditional notion of nullability. It is subtle, but explicit presence philosophy is closer to “the fields are not nullable, but you can detect if the field was explicitly assigned a value or not. Normal access will see some default value if it is not assigned, but you can check if the field was actively written to or not, when needed.”

The biggest reason not to have nullable fields is the intended behavior of default values specified in a `.proto` file. By design, calling a getter on an unset field will return the default value of that field.

**Note:** C# does treat _message_ fields as nullable. This inconsistency with other languages stems from the lack of immutable messages, which makes it impossible to create shared immutable default instances. Because message fields can’t have defaults, there’s no functional problem with this.

As an example, consider this `.proto` file:

    message Msg { Child child = 1; }
    message Child { Grandchild grandchild = 1; }
    message Grandchild { int32 foo = 1 [default = 72]; }

and corresponding Kotlin getters:

    // With our API where getters are always non-nullable:
    msg.child.grandchild.foo == 72

    // With nullable submessages the ?. operator fails to get the default value:
    msg?.child?.grandchild?.foo == null

    // Or verbosely duplicating the default value at the usage site:
    (msg?.child?.grandchild?.foo ?: 72)

and corresponding Rust getters:

    // With our API:
    msg.child().grandchild().foo()   // == 72

    // Where every getter is an Option<T>, verbose and no default observed
    msg.child().map(|c| c.grandchild()).map(|gc| gc.foo()) // == Option::None

    // For the rare situations where code may want to observe both the presence and
    // value of a field, the _opt() accessor which returns a custom Optional type
    // can also be used here (the Optional type is similar to Option except can also
    // be aware of the default value):
    msg.child().grandchild().foo_opt() // Optional::Unset(72)

If a nullable getter existed, it would necessarily ignore the user-specified defaults (to return null instead) which would lead to surprising and inconsistent behavior. If users of nullable getters want to access the default value of the field, they would have to write their own custom handling to use the default if null is returned, which removes the supposed benefit of cleaner/easier code with null getters.

Similarly, we do not provide nullable setters as the behavior would be unintuitive. Performing a set and then get would not always give the same value back, and calling a set would only sometimes affect the has-bit for the field.

Note that message-typed fields are always explicit presence fields (with hazzers). Proto3 defaults to scalar fields having implicit presence (without hazzers) unless they are explicitly marked `optional`, while Proto2 does not support implicit presence. With [Editions](https://protobuf.dev/editions/features#field_presence), explicit presence is the default behavior unless an implicit presence feature is used. With the forward expectation that almost all fields will have explicit presence, the ergonomic concerns that come with nullable getters are expected to be more of a concern than they may have been for Proto3 users.

Due to these issues, nullable setters/getters would radically change the way default values can be used. While we understand the possible utility, we have decided it’s not worth the inconsistencies and difficulty it introduces.


# News

Get the latest news about Protocol Buffers.

News topics provide information about past events and changes with Protocol Buffers, and plans for upcoming changes. The information is available both chronologically and per-release. Note that not everything is included in the per-release topics, as some content is not tied to a version.

New news topics will also be published to the [protobuf@](https://groups.google.com/g/protobuf) mailing list under the subject [Announcement].

## Chronological

The following news topics provide information in the reverse order in which it was released.

  * [March 13, 2026](https://protobuf.dev/news/2026-03-13) \- Breaking changes in Bazel for the upcoming 34.x release
  * [January 16, 2026](https://protobuf.dev/news/2026-01-16) \- Prebuilt proto compiler (protoc) for Bazel
  * [September 19, 2025](https://protobuf.dev/news/2025-09-19) \- Breaking changes in the upcoming 34.x release
  * [July 16, 2025](https://protobuf.dev/news/2025-07-16) \- Resuming support for Bazel with MSVC
  * [July 14, 2025](https://protobuf.dev/news/2025-07-14) \- Deprecating FieldDescriptor labels
  * [June 27, 2025](https://protobuf.dev/news/2025-06-27) \- Edition 2024
  * [March 18, 2025](https://protobuf.dev/news/2025-03-18) \- Dropping support for Ruby 3.0
  * [January 23, 2025](https://protobuf.dev/news/2025-01-23) \- Poison Java gencode
  * [December 18, 2024](https://protobuf.dev/news/2024-12-18) \- Go Protobuf: The new Opaque API
  * [December 13, 2024](https://protobuf.dev/news/2024-12-13) \- Removing `MutableRepeatedFieldRef<T>::Reserve()` in v30
  * [December 4, 2024](https://protobuf.dev/news/2024-12-04) \- `DebugString` replaced
  * [November 7, 2024](https://protobuf.dev/news/2024-11-07) \- More breaking changes in the upcoming 30.x release
  * [October 2, 2024](https://protobuf.dev/news/2024-10-02) \- Breaking changes in the upcoming 30.x release
  * [October 1, 2024](https://protobuf.dev/news/2024-10-01) \- Changes to Bazel builds
  * [June 26, 2024](https://protobuf.dev/news/2024-06-26) \- Dropping support for building Protobuf Java from source with Maven
  * [February 27, 2024](https://protobuf.dev/news/2024-02-27) \- Dropping support for older versions of Ruby
  * [February 5, 2024](https://protobuf.dev/news/2024-02-05) \- Breaking changes in Java, C++, and Python in the 26.x line.
  * [January 31, 2024](https://protobuf.dev/news/2024-01-31) \- Breaking changes in the 26.x line for Python
  * [January 5, 2024](https://protobuf.dev/news/2024-01-05) \- Breaking changes in the 26.x line for Ruby and Python
  * [December 27, 2023](https://protobuf.dev/news/2023-12-27) \- Breaking changes in the 26.x line for Ruby, PHP, Python, and upb
  * [December 13, 2023](https://protobuf.dev/news/2023-12-13) \- Breaking Python and C++ changes in the 26.x line
  * [December 5, 2023](https://protobuf.dev/news/2023-12-05) \- Breaking Java changes in the 26.x line
  * [October 10, 2023](https://protobuf.dev/news/2023-10-10) \- Documentation for Protobuf Editions features is published
  * [September 15, 2023](https://protobuf.dev/news/2023-09-15) \- μpb Moving to the Protobuf GitHub repository
  * [August 15, 2023](https://protobuf.dev/news/2023-08-15) \- Breaking Python change with the replacement of `message.UnknownFields()`
  * [August 9, 2023](https://protobuf.dev/news/2023-08-09) \- Support policy for .NET
  * [July 17, 2023](https://protobuf.dev/news/2023-07-17) \- Dropping support for older versions of Bazel
  * [July 6, 2023](https://protobuf.dev/news/2023-07-06) \- Dropping support for older versions of PHP, Ruby, and Python
  * [June 29, 2023](https://protobuf.dev/news/2023-06-29) \- Protobuf Editions announcement
  * [April 28, 2023](https://protobuf.dev/news/2023-04-28) \- Null no longer allowed in `json_name` field options
  * [April 20, 2023](https://protobuf.dev/news/2023-04-20) \- Update to Ruby code generator
  * [April 11, 2023](https://protobuf.dev/news/2023-04-11) \- Removing syntax reflection, releasing C++ CORD support for singular `bytes` fields to OSS, option retention, and dropping support for Bazel <5.3
  * [August 3, 2022](https://protobuf.dev/news/2022-08-03) \- Platform Support Changes and Upcoming Changes in the C++ 22.x Line
  * [July 6, 2022](https://protobuf.dev/news/2022-07-06) \- Library Breaking Change Policy
  * [May 6, 2022](https://protobuf.dev/news/2022-05-06) \- Versioning, Python Updates, and JavaScript Support

## Per Release

The following new topics provide per-release information. All News entries appear in the chronological listing in the previous section, but only entries that are specific to a particular version appear in the pages listed in this section.

These pages do not replace the [release notes](https://github.com/protocolbuffers/protobuf/releases), as the release notes will be more complete. Also, not everything from the chronological listing will be in these topics, as some content is not specific to a particular release.

  * [Version 34.x](https://protobuf.dev/news/v34)
  * [Version 32.x](https://protobuf.dev/news/v32)
  * [Version 31.x](https://protobuf.dev/news/v31)
  * [Version 30.x](https://protobuf.dev/news/v30)
  * [Version 29.x](https://protobuf.dev/news/v29)
  * [Version 26.x](https://protobuf.dev/news/v26)
  * [Version 25.x](https://protobuf.dev/news/v25)
  * [Version 24.x](https://protobuf.dev/news/v24)
  * [Version 23.x](https://protobuf.dev/news/v23)
  * [Version 22.x](https://protobuf.dev/news/v22)
  * [Version 21.x](https://protobuf.dev/news/v21)


# Protocol Buffer Compiler Installation

How to install the protocol buffer compiler.

The protocol buffer compiler, `protoc`, is used to compile `.proto` files, which contain service and message definitions. Choose one of the methods given below to install `protoc`.

### Install Pre-compiled Binaries (Any OS)

To install the latest release of the protocol compiler from pre-compiled binaries, follow these instructions:

  1. From <https://github.com/google/protobuf/releases>, manually download the zip file corresponding to your operating system and computer architecture (`protoc-<version>-<os>-<arch>.zip`), or fetch the file using commands such as the following:

         PB_REL="https://github.com/protocolbuffers/protobuf/releases"
         curl -LO $PB_REL/download/v30.2/protoc-30.2-linux-x86_64.zip

  2. Unzip the file under `$HOME/.local` or a directory of your choice. For example:

         unzip protoc-30.2-linux-x86_64.zip -d $HOME/.local

  3. Update your environment’s path variable to include the path to the `protoc` executable. For example:

         export PATH="$PATH:$HOME/.local/bin"

### Install Using a Package Manager

#### Warning

Run `protoc --version` to check the version of `protoc` after using a package manager for installation to ensure that it is sufficiently recent. The versions of `protoc` installed by some package managers can be quite dated. See the [Version Support page](https://protobuf.dev/support/version-support) to compare the output of the version check to the minor version number of the supported version of the language(s) you are using.

You can install the protocol compiler, `protoc`, with a package manager under Linux, macOS, or Windows using the following commands.

  * Linux, using `apt` or `apt-get`, for example:

        apt install -y protobuf-compiler
        protoc --version  # Ensure compiler version is 3+

  * MacOS, using [Homebrew](https://brew.sh):

        brew install protobuf
        protoc --version  # Ensure compiler version is 3+

  * Windows, using [Winget](https://learn.microsoft.com/en-us/windows/package-manager/winget/)

        > winget install protobuf
        > protoc --version # Ensure compiler version is 3+

### Other Installation Options

If you’d like to build the protocol compiler from sources, or access older versions of the pre-compiled binaries, see [Download Protocol Buffers](https://protobuf.dev/downloads).


# Tutorials

Each tutorial in this section shows you how to implement a simple application using protocol buffers in your favourite language, introducing you to the language’s protocol buffer API as well as showing you the basics of creating and using .proto files.

The complete sample code for each application is also provided.

The tutorials don’t assume that you know anything about protocol buffers, but do assume that you are comfortable writing code in your chosen language, including using file I/O.

##### [Protocol Buffer Basics: C++](https://protobuf.dev/getting-started/cpptutorial/)

A basic C++ programmers introduction to working with protocol buffers.

##### [Protocol Buffer Basics: C#](https://protobuf.dev/getting-started/csharptutorial/)

A basic C# programmers introduction to working with protocol buffers.

##### [Protocol Buffer Basics: Dart](https://protobuf.dev/getting-started/darttutorial/)

A basic Dart programmers introduction to working with protocol buffers.

##### [Protocol Buffer Basics: Go](https://protobuf.dev/getting-started/gotutorial/)

A basic Go programmers introduction to working with protocol buffers.

##### [Protocol Buffer Basics: Java](https://protobuf.dev/getting-started/javatutorial/)

A basic Java programmers introduction to working with protocol buffers.

##### [Protocol Buffer Basics: Kotlin](https://protobuf.dev/getting-started/kotlintutorial/)

A basic Kotlin programmers introduction to working with protocol buffers.

##### [Protocol Buffer Basics: Python](https://protobuf.dev/getting-started/pythontutorial/)

A basic Python programmers introduction to working with protocol buffers.


# History

A brief history behind the creation of protocol buffers.

Understanding why protobuf was created and the decisions that changed it over time can help you to better use the features of the tool.

## Why Did You Release Protocol Buffers?

There are several reasons that we released Protocol Buffers.

Protocol buffers are used by many projects inside Google. We had other projects we wanted to release as open source that use protocol buffers, so to do this, we needed to release protocol buffers first. In fact, bits of the technology had already found their way into the open; if you dig into the code for Google AppEngine, you might find some of it.

We wanted to provide public APIs that accept protocol buffers as well as XML, both because it is more efficient and because we convert that XML to protocol buffers on our end, anyway.

We thought that people outside Google might find protocol buffers useful. Getting protocol buffers into a form we were happy to release was a fun side project.

## Why Is the First Release Version 2? What Happened to Version 1?

The initial version of protocol buffers (“Proto1”) was developed starting in early 2001 and evolved over the course of many years, sprouting new features whenever someone needed them and was willing to do the work to create them. Like anything created in such a way, it was a bit of a mess. We came to the conclusion that it would not be feasible to release the code as it was.

Version 2 (“Proto2”) was a complete rewrite, though it kept most of the design and used many of the implementation ideas from Proto1. Some features were added, some removed. Most importantly, though, the code was cleaned up and did not have any dependencies on Google libraries that were not yet open-sourced.

## Why the Name “Protocol Buffers”?

The name originates from the early days of the format, before we had the protocol buffer compiler to generate classes for us. At the time, there was a class called `ProtocolBuffer` that actually acted as a buffer for an individual method. Users would add tag/value pairs to this buffer individually by calling methods like `AddValue(tag, value)`. The raw bytes were stored in a buffer that could then be written out once the message had been constructed.

Since that time, the “buffers” part of the name has lost its meaning, but it is still the name we use. Today, people usually use the term “protocol message” to refer to a message in an abstract sense, “protocol buffer” to refer to a serialized copy of a message, and “protocol message object” to refer to an in-memory object representing the parsed message.

## Does Google Have Any Patents on Protocol Buffers?

Google currently has no issued patents on protocol buffers, and we are happy to address any concerns around protocol buffers and patents that people may have.

## How Do Protocol Buffers Differ from ASN.1, COM, CORBA, and Thrift?

We think all of these systems have strengths and weaknesses. Google relies on protocol buffers internally and they are a vital component of our success, but that doesn’t mean they are the ideal solution for every problem. You should evaluate each alternative in the context of your own project.

It is worth noting, though, that several of these technologies define both an interchange format and an RPC (remote procedure call) protocol. Protocol buffers are just an interchange format. They could easily be used for RPC—and, indeed, they do have limited support for defining RPC services—but they are not tied to any one RPC implementation or protocol.