Engineering
9 min read

Type safe frontend APIs

Johan Brandhorst-Satzkorn headshot

Johan Brandhorst-Satzkorn

Software Engineer

At Oblique, we use Protobuf, Connect and gRPC to define and implement the APIs between the frontend and backend. In this post, we will explore how to set up a basic project leveraging Protobuf and Connect for type safe frontend APIs to the backend.

The benefits of schema-driven development have recently become more widely known in the backend. Cross-language RPCs, performance enhancements, compatibility guarantees, and a single source of truth for the API definitions are only some of the benefits. While Protobuf and gRPC provide the means for service-to-service communications, it remains difficult to accomplish this for frontend to backend communications. gRPC is itself not possible to use in the browser, because of its reliance on the obscure Trailer HTTP feature. OpenAPI has been trying to provide an IDL for RESTful services, but support is fragmented (OpenAPIv2 vs OpenAPIv3), the quality of language implementations varies widely and the spec is incredibly broad.

A number of projects have popped up that aim to bridge the gap to the frontend using Protobuf as the IDL, such as:

Out of all of them, the most interesting one is Connect. Connect was created by Buf, but was donated to the CNFC and now lives alongside gRPC as a community supported open-source project, making it more compelling to companies and independent projects alike. Connect is built on the ES2017 standard and so is compatible with all modern frameworks, with examples in major frameworks such as React, Next.js, Vue, Svelte, Angular, Astro and many more.

Connect also integrates nicely with Tanstack Query, a popular framework-agnostic state management solution. At Oblique, we use the Connect API generators with our React frontend and a Connect-compatible backend using Vanguard, another Buf project. Vanguard isn't necessary, and isn't part of this post. Vanguard also allows us to expose a REST-like JSON API, but that will be a topic of another blog post.

The backend

What does it look like to use? As with any Protobuf API, it starts with a .proto file. For this example, we’ll assume a Go backend, which is what we use at Oblique. Let’s define a basic service:

edition = "2023";

package mycorp.mybackend.v1;

// Important for Go generation, not necessary in general
option go_package = "github.com/mycorp/myproject/gen/mycorp/mybackend/v1;mybackendv1";

service BackendService {
	rpc GetCurrentUser(GetCurrentUserRequest) returns (GetCurrentUserResponse);
}

message GetCurrentUserRequest {}

message GetCurrentUserResponse {
	string user_id = 1;
	string full_name = 2;
	string email = 3;
	string avatar_url = 4;
}

An RPC to get information about the currently logged in user is a very common part of any frontend API. This document shows what it might look like when expressed as a Protobuf RPC.

Lets save this file as mycorp/mybackend/v1/backend.proto. Note how the Protobuf package matches the folder structure we picked. We will use the buf Protobuf compiler to generate our client and server libraries. Install buf using the method of your choice. We start by creating a buf.yaml file to define lint rules and other configuration options:

version: v2
lint:
	use:
    - STANDARD
breaking:
  use:
    - FILE

We can now try running buf build to see that everything is set up correctly:

$ buf build

If nothing happens, everything is working! Running buf build gathers all of our Protobuf files and “compiles” them, ensuring there are no missing imports or syntax errors. To generate client and server libraries, we create a new file: buf.gen.yaml, and define and install the Go plugins we want to use:

version: v2
plugins:
  - local: protoc-gen-go
    out: gen
    opt:
      - paths=source_relative
  - local: protoc-gen-connect-go
    out: gen
    opt:
      - paths=source_relative

We use the paths=source_relative option to generate the Go files in the same folder structure as our Protobuf files use. We can now try running buf generate to see that everything is working:

$ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
$ go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest
$ buf generate

We should see two new folders created:

gen/mycorp/mybackend/v1/backend.pb.go
gen/mycorp/mybackend/v1/mybackendv1connect/backend.connect.go

They’re the generated Go files containing the type and service definitions for the services and messages in our Protobuf package. At this point, we will need to make sure we have a go.mod file in place. Using the module name we foreshadowed in our go_package option above, run:

$ go mod init github.com/mycorp/myproject
$ go mod tidy

Now we are ready to implement the backend logic. Create a main.go:

package main

import (
	"context"
	"log"
	"net/http"

	"connectrpc.com/connect"
	mybackendv1 "github.com/mycorp/myproject/gen/mycorp/mybackend/v1"
	"github.com/mycorp/myproject/gen/mycorp/mybackend/v1/mybackendv1connect"
	"google.golang.org/protobuf/proto"
)

type backendService struct {
}

func (s *backendService) GetCurrentUser(ctx context.Context, _ *connect.Request[mybackendv1.GetCurrentUserRequest]) (*connect.Response[mybackendv1.GetCurrentUserResponse], error) {
	return connect.NewResponse(&mybackendv1.GetCurrentUserResponse{
		UserId:    proto.String("12345"),
		FullName:  proto.String("John Doe"),
		Email:     proto.String("john.doe@example.com"),
		AvatarUrl: proto.String("https://example.com/avatar.jpg"),
	}), nil
}

func main() {
	mux := http.NewServeMux()
	mux.Handle(mybackendv1connect.NewBackendServiceHandler(&backendService{}))
	log.Println("Serving on http://localhost:8080")
	if err := http.ListenAndServe("localhost:8080", mux); err != nil && err != http.ErrServerClosed {
		log.Fatalf("failed to start server: %v", err)
	}
}

This is of course just a mockup of the real thing. In a real scenario we’d probably have the backend inspect a user’s cookie and look up the authenticated user in a database of some kind, or reject the request with an error if the cookie isn’t authenticated. The server can be run using:

$ go run main.go
Serving on http://localhost:8080

If we wanted to verify that the server is running, we can just use cURL, since Connect supports regular HTTP/1.1 and the JSON content-type:

$ curl \
	-d '{}' \
	-H 'Content-Type: application/json' \
	-X POST \
	http://localhost:8080/mycorp.mybackend.v1.BackendService/GetCurrentUser 
{"userId":"12345", "fullName":"John Doe", "email":"john.doe@example.com", "avatarUrl":"https://example.com/avatar.jpg"}

Note how the method is always POST, and the path is a combination of Protobuf package, service name and RPC name. Now we are ready to implement the frontend!

The frontend

As mentioned before, Connect has examples for several different frontend frameworks, but we’ll use React for this example, which is also what we use at Oblique. Let’s start with a basic frontend skeleton:

app/
	node_modules/
		...
	dist/
		index.html
	index.tsx
	package.json
	package-lock.json

Use a package manager of your choice to initialize package.json and package-lock.json. Here is dist/index.html:

<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta http-equiv="X-UA-Compatible" content="ie=edge" />
    <title>My app</title>
    <script src="index.js" defer></script>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

And here is index.tsx:

import React from "react";
import ReactDOM from "react-dom/client";

const App: React.FC = () => {
  return <div>Hello world!</div>;
};

ReactDOM.createRoot(document.getElementById("root") as HTMLElement).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
);

To turn our TypeScript file into an index.js that the browser can interpret and execute, we’ll use the esbuild bundler. You can use any bundler of your choice for this step, esbuild is not mandatory.

$ go install github.com/evanw/esbuild/cmd/esbuild@latest
$ cd app && esbuild --bundle --outdir=dist index.tsx

This will produce our dist/index.js. With this dist folder setup, we can serve the app using any HTTP file server, but we’ll use our Go server to serve the app because it’s easy, and avoids the need for CORS. One way to do that is to drop an app.go file into the app folder and then embed the dist folder:

package app

import "embed"

//go:embed dist
var Dist embed.FS

This can be imported as a Go package with the Dist variable being an in-memory embedding of the contents of the dist folder. Lets modify main.go to import this file and serve it when not serving API requests:

package main

import (
	"context"
	"log"
	"net/http"

	"connectrpc.com/connect"
	"github.com/mycorp/myproject/app"
	mybackendv1 "github.com/mycorp/myproject/gen/mycorp/mybackend/v1"
	"github.com/mycorp/myproject/gen/mycorp/mybackend/v1/mybackendv1connect"
	"google.golang.org/protobuf/proto"
)

// backend service definitions, omitted for brevity

func main() {
	mux := http.NewServeMux()
	mux.Handle(mybackendv1connect.NewBackendServiceHandler(&backendService{}))
	distFS, err := fs.Sub(app.Dist, "dist")
	if err != nil {
		log.Fatalf("failed to sub dist from embedded filesystem: %v", err)
	}
	mux.Handle("/", http.FileServerFS(distFS))
	log.Println("Serving on http://localhost:8080")
	if err := http.ListenAndServe("localhost:8080", mux); err != nil && err != http.ErrServerClosed {
		log.Fatalf("failed to start server: %v", err)
	}
}

Now if we serve the backend and visit http://localhost:8080 we are greeted with our rendered React app.

An image showing the rendered frontend

Not the prettiest web app ever made, but it gets the job done. Now that we’ve hosted the frontend, lets try generating the client to let it talk to the backend over Connect. The Protobuf plugin we are looking for is called protoc-gen-es, and there are a few different ways to use it. The easiest way for open-source or non-sensitive code is to use the Buf Schema Registry’s remote plugin. Note that this will send your protobuf files to Buf’s servers to allow them to generate your code for you, so don’t do this for sensitive or private code unless you know that this is OK. For this example, we will use a local plugin installed into our node_modules folder, but we should understand that managing local Protobuf plugins can introduce hard-to-debug issues for local users, such as when the wrong version of a plugin is being used.

Install the npm package @bufbuild/protoc-gen-es using your favorite package manager. This will create a file that we can execute using a shell, node_modules/.bin/protoc-gen-es. We can use this file as a protobuf plugin through our buf.yaml file:

version: v2
plugins:
  - local: protoc-gen-go
    out: gen
    opt:
      - paths=source_relative
  - local: protoc-gen-connect-go
    out: gen
    opt:
      - paths=source_relative
  - local: ./app/node_modules/.bin/protoc-gen-es
    out: app/gen/
    opt:
      - target=ts

Use the option target=ts to generate a TypeScript file. Regenerate the files:

$ buf generate

This will create a new file, app/gen/mycorp/mybackend/v1/backend_pb.ts. Again, note how the file path is based on the Protobuf file structure. The generated file exposes some types and values but is mostly useless on its own. To make use of it, we need to add some more npm packages:

  • @bufbuild/protobuf
  • @connectrcp/connect
  • @connectrcp/connect-web

Now we can update our minimal frontend app to make use of the backend over Connect:

import { createClient } from "@connectrpc/connect";
import { createConnectTransport } from "@connectrpc/connect-web";
import React, { useState } from "react";
import ReactDOM from "react-dom/client";
import {
  BackendService,
  GetCurrentUserResponse,
} from "./gen/mycorp/mybackend/v1/backend_pb";

const client = createClient(
  BackendService,
  createConnectTransport({
    baseUrl: ".",
  }),
);

const App: React.FC = () => {
  const [user, setUser] = useState<GetCurrentUserResponse | null>(null);

  const onClick = async () => {
    const response = await client.getCurrentUser({});
    setUser(response);
  };

  return (
    <div>
      <h1>User Dashboard</h1>
      <button onClick={onClick}>Get Current User</button>
      {user && (
        <div
          style={{
            marginTop: "20px",
            border: "1px solid #ccc",
            padding: "10px",
          }}
        >
          <h3>User Details</h3>
          <p>
            <strong>ID:</strong> {user.userId}
          </p>
          <p>
            <strong>Name:</strong> {user.fullName}
          </p>
          <p>
            <strong>Email:</strong> {user.email}
          </p>
          <p>
            <strong>Avatar URL:</strong> {user.avatarUrl}
          </p>
        </div>
      )}
    </div>
  );
};

ReactDOM.createRoot(document.getElementById("root") as HTMLElement).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
);

There’s a lot going on here, so lets break it down. First, we create a client:

const client = createClient(
  BackendService,
  createConnectTransport({
    baseUrl: ".",
  }),
);

This client uses the generated BackendService value and some TypeScript magic to forward the type information. The baseUrl is . since we’re using the Go backend both for hosting the app and the API.

const [user, setUser] = useState<GetCurrentUserResponse | null>(null);

const onClick = async () => {
  const response = await client.getCurrentUser({});
  setUser(response);
};

This is where we make the call to the backend. The {} parameter to getCurrentUser is because there are currently no fields on the input request to getCurrentUser. Naturally, the function is async, so we use await to get the response, so that we can set it on useState.

return (
  <div>
    <h1>User Dashboard</h1>
    <button onClick={onClick}>Get Current User</button>
  </div>
);

We hook up the onClick handler to a button to trigger the call to the backend.

{user && (
  <div
    style={{
      marginTop: "20px",
      border: "1px solid #ccc",
      padding: "10px",
    }}
  >
    <h3>User Details</h3>
    <p>
      <strong>ID:</strong> {user.userId}
    </p>
    <p>
      <strong>Name:</strong> {user.fullName}
    </p>
    <p>
      <strong>Email:</strong> {user.email}
    </p>
    <p>
      <strong>Avatar URL:</strong> {user.avatarUrl}
    </p>
  </div>
)}

We conditionally render the user information when the user is available (once the backend function call has returned). Re-bundle:

$ cd app && esbuild --bundle --outdir=dist index.tsx

The final website look:

Recording of an interaction with the UI

Making changes

Whew, that was a lot! As we mentioned at the start, there is quite a bit of boilerplate to get everything set up, but now that all that work is done, making changes, or even adding new APIs becomes so much easier that it’ll all have been worth it. Lets try adding a new input parameter to the GetCurrentUser RPC:

// Rest of Protobuf file omitted for brevity
message GetCurrentUserRequest {
	uint32 avatar_option = 1;
}

We then regenerate the files:

$ buf generate

If we look at our frontend, we can see that we now have the option to specify an avatarOption field in getCurrentUser:

const onClick = async () => {
  const response = await client.getCurrentUser({
    avatarOption: 1,
  });
  setUser(response);
};

The TypeScript compiler helpfully tells us that the type of this field is number, so we can’t accidentally send a string or something else nonsensical. Don’t forget to re-bundle the frontend:

$ cd app && esbuild --bundle --outdir=dist index.tsx

On the backend side, we implement support for a few different avatar options:

func (s *backendService) GetCurrentUser(ctx context.Context, req *connect.Request[mybackendv1.GetCurrentUserRequest]) (*connect.Response[mybackendv1.GetCurrentUserResponse], error) {
	resp := &mybackendv1.GetCurrentUserResponse{
		UserId:    proto.String("12345"),
		FullName:  proto.String("John Doe"),
		Email:     proto.String("john.doe@example.com"),
		AvatarUrl: proto.String("https://example.com/avatar.jpg"),
	}
	switch req.Msg.GetAvatarOption() {
	case 0:
		// Default avatar
	case 1:
		resp.AvatarUrl = proto.String("https://example.com/avatar1.jpg")
	case 2:
		resp.AvatarUrl = proto.String("https://example.com/avatar2.jpg")
	default:
		return nil, connect.NewError(connect.CodeInvalidArgument, nil)
	}
	return connect.NewResponse(resp), nil
}

Now we can run the website again and see that the backend will return the avatar corresponding to option 1!

Adding new RPCs and even new services is similarly simple, and with a single source-of-truth for your API definitions, there is never a question of whether the API is exposed by the backend yet, or what the request format looks like. Connect has changed the game by thinking through and following through on a frontend developer experience that finally gets frontend developers onboard and excited about typed API development.

Further reading

If you want to get started with type safe frontend APIs in your own company or project, the Connect documentation has a lot of information to get you started in the language of your choice.

For generic Protobuf API design, the Google AIPs are a great source of advice and rules that you should read. There is a linter you can use to ensure you are conformant.

Likewise, the Buf style guide and buf lint command help you get started and stay conformat with Protobuf and their best practices recommendations.

Signup image

Ready to simplify access management?

Experience the joy of fewer IT tickets

We’d love to help you get to more maintainable access controls

Book a demo