What, Why, and How of gRPC

Introduction

When developers start a new software project, they frequently give a lot of consideration to the choice of programming language or languages, frameworks, and persistence tools; however, they often give little thought to how the services or endpoints will communicate. Projects will automatically select REST, or something REST-like for this communication.  There are other options available, each with their own strengths and weaknesses, and one of the more interesting ones is gRPC.

What is gRPC?

Google developed gRPC as an open-source framework designed to facilitate efficient, scalable communication between micro or distributed services.  They announced the 1.0.0 release in August 2016, and there have been consistent minor releases since then.  It is widely used inside Google and other companies such as Cisco and Netflix.

gRPC is a Remote Procedure Call (RPC) framework that supports a wide variety of programming languages and environments.  This includes not only many server environments and languages, but can also run on iOS and Android.

gRPC leverages HTTP/2 for transport, and Protocol Buffers (Protobuf) for message serialization. HTTP/2 provides streaming capabilities for real-time data flow.  A Protobuf message typically requires less bandwidth than json text messages, and provides very efficient marshaling.

gRPC has a custom Interface Definition Language (IDL) for defining the data structures and procedure APIs.  This IDL enforces statically typed messages for procedure parameters and return values.  In addition, this facilitates gRPC’s service discovery capabilities.

Why use gRPC?

gRPC provides a number of interesting and powerful features which make it a viable option for a lot of scenarios.  Some of these strengths include:

Performance:

Two main factors help to make gRPC a fast efficient framework for remote procedure calls.  As stated above, gRPC uses HTTP/2 and Protocol Buffers. HTTP/2 has some performance benefits over HTTP/1.1.  Protocol Buffer messages are in a condensed binary format that typically take up less bandwidth, and can be marshaled faster than an equivalent JSON message.

Strongly typed API contract:

gRPC uses an IDL and Protocol Buffers to define a strongly typed contract.  This can be very helpful in reducing ambiguity when working with an external API.

Service discovery:

The ability to list all available remote procedures, and describe the structures of both parameters and responses  can be a great assistance when working with an external service.

Streaming Data:

In addition to conventional request/response calls, gRPC supports server streaming, client streaming, and bidirectional streaming.  These streaming options are built in features of HTTP/2.  If a service needs to provide continuous updates, like a chat server or stock ticker, this can be a great option instead of web sockets.

How to get started with gRPC?

Because gRPC supports a wide variety of languages, platforms, and even procedure call lifecycles, this is not meant to be a comprehensive guide.  This is an example of how to get a simple inventory management service, written in Go, that can handle a couple of different requests.

Prerequisites:

• Go. Installation instructions for the current version.
• Protocol Buffer compiler. Version 3
• Go plugins for Protocol Buffer compiler.  Run the following commands to install:

$ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

• grpcurl (optional).  Installation instructions Used to test the service at the end.

Create the Project Structure:

From the root of your project, create the following directory structure:

`-- app
    |-- cmd          // contains the main.go to run the service
    |-- protobuf     // contains the .proto 
    `-- service      // contains the service go source code

 

Create the Go project

From the app directory, run the following command:

$ go mod init grpc_inventory

This will create the go.mod file. Edit the file, and add the grpc library dependency. The result file should look similar to the following:

module grpc_inventory

go 1.21.9

require google.golang.org/grpc v1.66.0

Create the .proto file:

In the protobuf directory, create the following file:

// inventory.proto
syntax = "proto3";

package inventory;

// the relative package for the generated go code
option go_package = "./pb";

// A basic item that is tracked by the inventory system
message Item {
    string id = 1;
    string sku = 2;
    string name = 3;
}

// An item plus the storage location and quantity
message ItemInventory {
    Item item = 1;
    string location = 2;
    uint32 quantity = 3;
}

// The parameter for the FindInventoryByName procedure call
message FindInventoryByNameRequest {
    string name = 1;
}

// The response for the FindInventoryByName procedure call
message FindInventoryResponse {
    repeated ItemInventory inventory_items = 1;
}

// The parameters for transfering inventory from one location to another
message TransferInventoryRequest {
    string item_id = 1;
    uint32 quantity = 2;
    string from_location = 3;
    string to_location = 4;
}

// The response for transfer inventory procedure call contains the update quantities
message TransferInventoryResponse {
    uint32 from_location_quantity = 1;
    uint32 to_location_quantity = 2;
}

// The actual service and two procedure definitions
service InventoryService {
    rpc FindInventoryByName(FindInventoryByNameRequest) returns (FindInventoryResponse);
    rpc TransferInventory(TransferInventoryRequest) returns (TransferInventoryResponse);
}

Generate the Go protobuf files:

From the root directory, run the following command:

$ protoc \
    --go_out=./app/service \
    --go-grpc_out=./app/service \
    ./app/protobuf/*.proto

This will create inventory.pb.go and inventory_grpc.pb.go in the app/service/pb directory.

 

Create the Go service:

In the app/service directory, create the following go source file:

// inventory.go
package service

import (
	"context"
	"fmt"
	"grpc_inventory/service/pb"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"
)

// the struct that contains the server for handling the inventory procedure calls
type inventoryService struct {
	pb.UnimplementedInventoryServiceServer
}

// the function for initializing the inventory server. This would typically set up 
// the service or data access layer called by the grpc server.
func NewInventoryServer() *grpc.Server {
	grpcServer := grpc.NewServer()
	invService := inventoryService{}

	pb.RegisterInventoryServiceServer(grpcServer, &invService)
	return grpcServer
}

// Implementation of the FindInventoryByName grpc procedure.
// This is stubbed out with both success and failure responses.
// Note the status and codes are from grpc libraries.
func (svc *inventoryService) FindInventoryByName(ctx context.Context, req *pb.FindInventoryByNameRequest) (*pb.FindInventoryResponse, error) {
	if req.Name == "bogus" {
		return nil, status.Errorf(codes.NotFound, fmt.Sprintf("No items found with name %s", req.Name))
	}

	inv1 := pb.ItemInventory{
		Item: &pb.Item{
			Id:   "001",
			Sku:  "sku-1",
			Name: req.Name,
		},
		Quantity: 10,
		Location: "warehouse-1",
	}

	inv2 := pb.ItemInventory{
		Item: &pb.Item{
			Id:   "002",
			Sku:  "sku-2",
			Name: req.Name,
		},
		Quantity: 15,
		Location: "warehouse-2",
	}

	resp := pb.FindInventoryResponse{
		InventoryItems: []*pb.ItemInventory{&inv1, &inv2},
	}

	return &resp, nil
}

// Implementation of the TransferInventory grpc procedure.
// This also has stubbed out with both success and failure responses.
func (svc *inventoryService) TransferInventory(ctx context.Context, req *pb.TransferInventoryRequest) (*pb.TransferInventoryResponse, error) {
	if req.ItemId == "invalid" {
		return nil, status.Errorf(codes.InvalidArgument, fmt.Sprintf("ItemId %s does not exist", req.ItemId))
	}
	return &pb.TransferInventoryResponse{FromLocationQuantity: 20, ToLocationQuantity: 15}, nil
}

 

Create the main.go startup file:

In the app/cmd directory, create the following file:

//main.go
package main

import (
	"grpc_inventory/service"
	"log"
	"net"

	"google.golang.org/grpc/reflection"
)

func main() {
	port := "8080"

	address := "0.0.0.0:" + port

	// Create the tcp listener
	listener, err := net.Listen("tcp", address)
	if err != nil {
		log.Fatalf("Error starting listener on %s: %v", port, err)
	}

	log.Printf("Server started. Listening on %s", port)

	// Create the grpc server
	server := service.NewInventoryServer()
	reflection.Register(server)

	// Set the grpc server to 'serve' requests from the tcp listener
	err = server.Serve(listener)
	if err != nil {
		log.Fatalf("Failed to initialize server: %v", err)
	}
}

Run the service:

From the app directory, run the following command:

$ go run ./cmd/main.go

This will log a message to the console with Server started. Listening on 8080.

Test the service:

In a separate terminal, you can run grpcurl to test the service. The following shows a few examples of how to list the procedures, view the details, and execute procedure calls:

$ grpcurl --plaintext localhost:8080 list inventory.InventoryService
inventory.InventoryService.FindInventoryByName
inventory.InventoryService.TransferInventory

$ grpcurl --plaintext localhost:8080 describe inventory.InventoryService.FindInventoryByName
inventory.InventoryService.FindInventoryByName is a method:
rpc FindInventoryByName ( .inventory.FindInventoryByNameRequest ) returns ( .inventory.FindInventoryResponse );

$ grpcurl --plaintext localhost:8080 describe inventory.FindInventoryByNameRequest
inventory.FindInventoryByNameRequest is a message:
message FindInventoryByNameRequest {
  string name = 1;
}

$ grpcurl --plaintext localhost:8080 describe inventory.FindInventoryResponse
inventory.FindInventoryResponse is a message:
message FindInventoryResponse {
  repeated .inventory.ItemInventory inventory_items = 1;
}

$ grpcurl --plaintext localhost:8080 describe inventory.ItemInventory
inventory.ItemInventory is a message:
message ItemInventory {
  .inventory.Item item = 1;
  string location = 2;
  uint32 quantity = 3;
}

$ grpcurl -d '{"name": "widget"}' -plaintext localhost:8080 inventory.InventoryService.FindInventoryByName
{
  "inventoryItems": [
    {
      "item": {
        "id": "001",
        "sku": "sku-1",
        "name": "widget"
      },
      "location": "warehouse-1",
      "quantity": 10
    },
    {
      "item": {
        "id": "002",
        "sku": "sku-2",
        "name": "widget"
      },
      "location": "warehouse-2",
      "quantity": 15
    }
  ]
}

$ grpcurl -d '{"name": "bogus"}' -plaintext localhost:8080 inventory.InventoryService.FindInventoryByName
ERROR:
  Code: NotFound
  Message: No items found with name bogus

Conclusion

gRPC offers many interesting features for programmers working with distributed systems, especially in environments where performance, scalability, and cross-language communication are critical. gRPC’s efficient use of resources, strong contract enforcement, and streaming capabilities, make it a serious candidate for any micro service architecture project.