Protocol Buffers and gRPC Best Practices

Overview

This project's architecture is built on Protocol Buffers (protobuf) and gRPC as the foundation for design-driven development. The design is expressed through schema and RPC definitions in .proto files, and everything else flows downstream from this single source of truth.

Design-Driven Development Flow

┌─────────────────────────────────────────────────────────────────┐
│                      Proto Definitions                           │
│                   (Single Source of Truth)                       │
│  • Messages (data schemas)                                       │
│  • Services (RPC methods)                                        │
│  • Enums (with custom annotations)                               │
│  • Comments (API documentation)                                  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ├─────────────────────────────────────┐
                              │                                     │
                              ▼                                     ▼
              ┌───────────────────────────┐         ┌───────────────────────────┐
              │   Backend (Java)          │         │   Frontend (TypeScript)   │
              │                           │         │                           │
              │ • gRPC Service Stubs      │         │ • gRPC-Web Clients        │
              │ • Message Classes         │         │ • TypeScript Interfaces   │
              │ • Enum Types              │         │ • Enum Types              │
              │ • JSON Serialization      │         │ • JSON Serialization      │
              └───────────────────────────┘         └───────────────────────────┘
                              │                                     │
                              └──────────────┬──────────────────────┘
                                             │
                              ┌──────────────▼──────────────────┐
                              │   REST API (gRPC-Gateway)       │
                              │   • HTTP/JSON transcoding       │
                              │   • OpenAPI specs               │
                              │   • Automatic routing           │
                              └──────────────┬──────────────────┘
                                             │
                              ┌──────────────▼──────────────────┐
                              │   MCP Tools (GitHub, etc.)      │
                              │   • Type-safe integrations      │
                              │   • Consistent data models      │
                              └─────────────────────────────────┘

Benefits of Proto-First Design

1. ✅ Single Source of Truth: Schema, documentation, and API contracts defined once
2. ✅ Type Safety: Compile-time checks across Java, TypeScript, and Go
3. ✅ Automatic Code Generation: No manual DTO/model duplication
4. ✅ Backward Compatibility: Proto evolution rules ensure safe updates
5. ✅ Multi-Protocol Support: gRPC (binary), REST (JSON), and MCP (JSON-RPC)
6. ✅ Self-Documenting: Comments in proto files become API documentation
7. ✅ Efficient Serialization: Binary protobuf for gRPC, JSON for REST/MCP

---

Proto File Organization

Current Structure

src/main/proto/
├── api.proto                                    # Main API (Phase 1)
├── rbac.proto                                   # Access control
├── task.proto                                   # Background tasks
└── code/                                        # Building codes (Phase 2+)
    ├── common.proto                             # Shared custom options
    ├── icc/                                     # ICC (International Code Council)
    │   ├── ibc/                                 # IBC (International Building Code)
    │   │   ├── ibc_common.proto                 # IBC-specific options
    │   │   ├── ibc_occupancy.proto              # Occupancy enums and messages
    │   │   ├── ibc_construction.proto           # Construction type enums
    │   │   ├── ibc_fire_protection.proto        # Fire protection enums
    │   │   └── ibc_height_area.proto            # Height/area messages
    │   └── irc/                                 # IRC (International Residential Code)
    │       └── irc_dwelling.proto
    └── nfpa/                                    # NFPA (National Fire Protection Association)
        └── nfpa_101.proto                       # Life Safety Code

Package Naming Convention

// Main API
package org.codetricks.construction.code.assistant.service;

// Code-specific (by supplier hierarchy)
package org.codetricks.construction.code.icc.ibc.occupancy;
package org.codetricks.construction.code.icc.ibc.construction;
package org.codetricks.construction.code.nfpa.lifesafety;

Rationale: Granular packages prevent enum value collisions and mirror industry structure (ICC publishes IBC/IRC, NFPA publishes various standards).

---

Enum Best Practices

Using Custom Annotations for Rich Metadata

Protocol Buffers allows extending EnumValueOptions to embed metadata directly into enum definitions. This provides a single source of truth for display names, code references, and documentation.

Defining Custom Options

// code/common.proto or code/icc/ibc/ibc_common.proto
syntax = "proto3";
package org.codetricks.construction.code.icc.ibc;

import "google/protobuf/descriptor.proto";

// Custom options for IBC enum metadata
extend google.protobuf.EnumValueOptions {
  string ibc_code = 50010;          // IBC code (e.g., "A-1", "R-2")
  string ibc_display_name = 50011;  // Human-readable name
  string ibc_section = 50012;       // IBC section reference (e.g., "303.2")
  string ibc_chapter = 50013;       // IBC chapter reference (e.g., "Chapter 3")
}

Using Annotations in Enums

// code/icc/ibc/ibc_occupancy.proto
syntax = "proto3";
package org.codetricks.construction.code.icc.ibc.occupancy;

import "code/icc/ibc/ibc_common.proto";

// IBC Occupancy Groups (IBC Chapter 3, Section 302.1)
enum IbcOccupancyGroup {
  UNKNOWN = 0 [
    (ibc_code) = "UNKNOWN",
    (ibc_display_name) = "Unknown Occupancy",
    (ibc_section) = "",
    (ibc_chapter) = ""
  ];
  
  // Assembly Groups
  A_1 = 1 [
    (ibc_code) = "A-1",
    (ibc_display_name) = "Assembly - Theaters, concert halls (fixed seating)",
    (ibc_section) = "303.2",
    (ibc_chapter) = "Chapter 3"
  ];
  
  R_2 = 21 [
    (ibc_code) = "R-2",
    (ibc_display_name) = "Residential - Apartments, dormitories (>2 units)",
    (ibc_section) = "310.3",
    (ibc_chapter) = "Chapter 3"
  ];
  
  // ... more values
}

Key Principles:

• ✅ No code prefixes in display names - Client can combine ibc_code + ibc_display_name as needed
• ✅ Clean enum values - Use R_2 instead of OCCUPANCY_R_2 (package provides namespace)
• ✅ Descriptive display names - Focus on human readability
• ✅ Complete references - Include section and chapter for traceability

Accessing Enum Annotations in Java

Create utility classes to extract annotation metadata:

// src/main/java/org/codetricks/construction/code/icc/ibc/occupancy/IbcOccupancyGroupUtils.java
package org.codetricks.construction.code.icc.ibc.occupancy;

import com.google.protobuf.Descriptors;
import org.codetricks.construction.code.icc.ibc.occupancy.IbcOccupancyGroup;
import org.codetricks.construction.code.icc.ibc.IbcCommonProto;

/**
 * Utility class for working with IbcOccupancyGroup enum.
 * Provides access to proto annotations for display names and IBC section references.
 * Similar to PlanIngestionStepUtils.
 */
public class IbcOccupancyGroupUtils {
    
    /**
     * Gets the IBC code (e.g., "A-1", "R-2") from proto annotations.
     */
    public static String getIbcCode(IbcOccupancyGroup occupancy) {
        try {
            Descriptors.EnumValueDescriptor descriptor = occupancy.getValueDescriptor();
            if (descriptor.getOptions().hasExtension(IbcCommonProto.ibcCode)) {
                return descriptor.getOptions().getExtension(IbcCommonProto.ibcCode);
            }
        } catch (Exception e) {
            System.err.println("Warning: Could not read ibc_code annotation: " + e.getMessage());
        }
        return occupancy.name();
    }
    
    /**
     * Gets the human-readable display name from proto annotations.
     * Returns: "Assembly - Theaters, concert halls (fixed seating)"
     */
    public static String getDisplayName(IbcOccupancyGroup occupancy) {
        try {
            Descriptors.EnumValueDescriptor descriptor = occupancy.getValueDescriptor();
            if (descriptor.getOptions().hasExtension(IbcCommonProto.ibcDisplayName)) {
                return descriptor.getOptions().getExtension(IbcCommonProto.ibcDisplayName);
            }
        } catch (Exception e) {
            System.err.println("Warning: Could not read ibc_display_name annotation: " + e.getMessage());
        }
        return getIbcCode(occupancy);
    }
    
    /**
     * Gets the IBC section reference (e.g., "303.2") from proto annotations.
     */
    public static String getIbcSection(IbcOccupancyGroup occupancy) {
        try {
            Descriptors.EnumValueDescriptor descriptor = occupancy.getValueDescriptor();
            if (descriptor.getOptions().hasExtension(IbcCommonProto.ibcSection)) {
                return descriptor.getOptions().getExtension(IbcCommonProto.ibcSection);
            }
        } catch (Exception e) {
            System.err.println("Warning: Could not read ibc_section annotation: " + e.getMessage());
        }
        return "";
    }
    
    /**
     * Gets the full IBC reference string.
     * Returns: "IBC Chapter 3, Section 303.2"
     */
    public static String getFullReference(IbcOccupancyGroup occupancy) {
        String chapter = getIbcChapter(occupancy);
        String section = getIbcSection(occupancy);
        if (!chapter.isEmpty() && !section.isEmpty()) {
            return "IBC " + chapter + ", Section " + section;
        }
        return "";
    }
    
    /**
     * Parses an IBC code string (e.g., "R-2") to the corresponding enum value.
     */
    public static IbcOccupancyGroup fromIbcCode(String ibcCode) {
        for (IbcOccupancyGroup occupancy : IbcOccupancyGroup.values()) {
            if (getIbcCode(occupancy).equalsIgnoreCase(ibcCode)) {
                return occupancy;
            }
        }
        return IbcOccupancyGroup.UNKNOWN;
    }
}

Usage Example:

// Clean syntax - no prefixes!
IbcOccupancyGroup occupancy = IbcOccupancyGroup.R_2;

// Get metadata
String code = IbcOccupancyGroupUtils.getIbcCode(occupancy);           // "R-2"
String displayName = IbcOccupancyGroupUtils.getDisplayName(occupancy); // "Residential - Apartments..."
String section = IbcOccupancyGroupUtils.getIbcSection(occupancy);      // "310.3"
String fullRef = IbcOccupancyGroupUtils.getFullReference(occupancy);   // "IBC Chapter 3, Section 310.3"

// Client can combine as needed
String fullLabel = code + ": " + displayName;  // "R-2: Residential - Apartments..."

Accessing Enum Annotations in TypeScript

For TypeScript/Angular, create a service that mirrors the Java utility:

// src/app/services/ibc-occupancy-group.service.ts
import { IbcOccupancyGroup } from '../../generated.commonjs/ibc_occupancy_pb';

export class IbcOccupancyGroupService {
  // Metadata map generated from proto annotations (can be auto-generated from Java utility)
  private static readonly METADATA: Record<IbcOccupancyGroup, {
    ibcCode: string;
    displayName: string;
    section: string;
    chapter: string;
  }> = {
    [IbcOccupancyGroup.R_2]: {
      ibcCode: 'R-2',
      displayName: 'Residential - Apartments, dormitories (>2 units)',
      section: '310.3',
      chapter: 'Chapter 3'
    },
    // ... other mappings (auto-generate from proto)
  };
  
  static getIbcCode(occupancy: IbcOccupancyGroup): string {
    return this.METADATA[occupancy]?.ibcCode || occupancy.toString();
  }
  
  static getDisplayName(occupancy: IbcOccupancyGroup): string {
    return this.METADATA[occupancy]?.displayName || this.getIbcCode(occupancy);
  }
  
  static getFullReference(occupancy: IbcOccupancyGroup): string {
    const metadata = this.METADATA[occupancy];
    if (metadata && metadata.chapter && metadata.section) {
      return `IBC ${metadata.chapter}, Section ${metadata.section}`;
    }
    return '';
  }
}

// Usage in component
const occupancy = IbcOccupancyGroup.R_2;  // ✅ Clean syntax!
const label = IbcOccupancyGroupService.getDisplayName(occupancy);

---

JSON Serialization Best Practices

Field Naming Convention: camelCase vs snake_case

📋 Open Discussion: Issue #226 - Establish JSON field naming convention

The codebase currently has mixed usage of camelCase and snake_case in JSON files. See Issue #226 for analysis and discussion on establishing a consistent convention.

Available Options:

Method	Output Format	Example
Proto.toJson()	camelCase	{"pageNumber": 1, "projectName": "..."}
Proto.toJsonSnakeCase()	snake_case	{"page_number": 1, "project_name": "..."}
Proto.loadJson()	Accepts BOTH	Flexible parser

Current State:

• Proto definitions: snake_case (account_id, project_name)
• gRPC-Web generated code: camelCase (accountId, projectName)
• Existing metadata files: Mixed (some camelCase, some snake_case)
• Legacy string formatting: snake_case

Choose based on your use case until Issue #226 is resolved.

Existing Utility Classes

The project provides utility classes for proto serialization:

• Proto.java - JSON serialization utilities

  • Location: src/main/java/org/codetricks/construction/code/assistant/io/Proto.java
  • Methods: toJson(), loadJson()
  • Features: Generic type support, error handling, ignoringUnknownFields()

• TextProtoLoader.java - Text proto parsing utilities

  • Location: src/main/java/org/codetricks/construction/code/assistant/io/TextProtoLoader.java
  • Methods: get(Class, InputStream), get(Class, String)
  • Features: Stream and string support, generic type support

✅ Recently Enhanced: Issue #224 - Added Reader/Writer methods and snake_case serialization options.

Proto to JSON (Java)

Protocol Buffers provides built-in JSON serialization via JsonFormat:

import com.google.protobuf.util.JsonFormat;
import org.codetricks.construction.code.assistant.proto.ProjectMetadata;

// Serialize proto message to JSON
ProjectMetadata metadata = ProjectMetadata.newBuilder()
    .setProjectName("Multi-Family Residential Building")
    .setProjectDescription("5-story apartment complex")
    .build();

String json = JsonFormat.printer()
    .includingDefaultValueFields()  // Include fields with default values
    .preservingProtoFieldNames()    // Use proto field names (snake_case)
    .print(metadata);

// Deserialize JSON to proto message
ProjectMetadata.Builder builder = ProjectMetadata.newBuilder();
JsonFormat.parser()
    .ignoringUnknownFields()  // Ignore fields not in proto (for backward compatibility)
    .merge(json, builder);
ProjectMetadata parsedMetadata = builder.build();

Key Options:

• includingDefaultValueFields(): Include fields with default values (0, false, empty string)
• preservingProtoFieldNames(): Use project_name instead of projectName in JSON
• ignoringUnknownFields(): Ignore extra fields in JSON (useful for backward compatibility)

Enum Serialization

Enums serialize to their string name by default:

IbcOccupancyGroup occupancy = IbcOccupancyGroup.R_2;

// JSON output: "R_2" (enum name, not the ibc_code annotation)
String json = JsonFormat.printer().print(
    IbcOccupancyClassification.newBuilder()
        .setPrimaryGroup(occupancy)
        .build()
);
// Result: { "primary_group": "R_2" }

To serialize with custom code (e.g., "R-2" instead of "R_2"), manually convert:

// Custom serialization with IBC code
String ibcCode = IbcOccupancyGroupUtils.getIbcCode(occupancy);  // "R-2"
// Store in a string field or custom JSON object

Writing Proto Messages to Files

Using Proto.java with FileSystemHandler (Recommended)

Current Implementation:

import org.codetricks.construction.code.assistant.io.Proto;
import org.codetricks.construction.code.assistant.data.rag.corpus.FileSystemHandler;

// Write proto to file
ProjectMetadata metadata = ProjectMetadata.newBuilder()
    .setProjectName("My Project")
    .build();

String json = Proto.toJson(metadata);
fileSystemHandler.writeFile(metadataPath, json);

// Read proto from file
String jsonContent = fileSystemHandler.readFile(metadataPath);
ProjectMetadata loaded = Proto.loadJson(ProjectMetadata.class, jsonContent);

Enhanced Implementation (Issue #224 ✅ Complete):

import org.codetricks.construction.code.assistant.io.Proto;

// Write with snake_case field names (ideal for metadata files)
String json = Proto.toJsonWithDefaults(metadata);
fileSystemHandler.writeFile(metadataPath, json);

// Read (existing method works fine)
String jsonContent = fileSystemHandler.readFile(metadataPath);
ProjectMetadata loaded = Proto.loadJson(ProjectMetadata.class, jsonContent);

Using Reader/Writer Pattern (current in InputFileMetadataService):

// Write with Writer
try (Writer writer = fileSystemHandler.getWriter(metadataPath)) {
    String jsonString = Proto.toJson(metadata);
    writer.write(jsonString);
}

// Read with Reader
try (Reader reader = fileSystemHandler.getReader(metadataPath)) {
    StringBuilder jsonContent = new StringBuilder();
    char[] buffer = new char[1024];
    int length;
    while ((length = reader.read(buffer)) != -1) {
        jsonContent.append(buffer, 0, length);
    }
    return Proto.loadJson(InputFileMetadata.class, jsonContent.toString());
}

Enhanced Reader/Writer (Issue #224 ✅ Complete):

// Simplified Reader usage
try (Reader reader = fileSystemHandler.getReader(metadataPath)) {
    InputFileMetadata metadata = Proto.loadJsonFromReader(InputFileMetadata.class, reader);
}

// Simplified Writer usage (with snake_case field names)
try (Writer writer = fileSystemHandler.getWriter(metadataPath)) {
    Proto.writeJsonToWriter(metadata, writer);
}

Real-World Examples from Codebase

Example 1: ArchitecturalPlanPageLoader (line 68):

// Simple pattern - readFile returns String
return Proto.loadJson(ArchitecturalPlanPage.class, fileSystemHandler.readFile(planPageMetadataFilePath));

Example 2: ArchitecturalPlanReviewer (line 777):

// Simple pattern - writeFile takes String
fileSystemHandler.writeFile(planFilePath, Proto.toJson(plan));

Example 3: InputFileMetadataService (lines 158-165):

// Writer pattern for more control
public void saveMetadataToFile(InputFileMetadata metadata, String metadataPath) throws Exception {
    try (Writer writer = fileSystemHandler.getWriter(metadataPath)) {
        String jsonString = Proto.toJson(metadata);
        writer.write(jsonString);
        logger.info("Saved metadata to: " + metadataPath);
    }
}

Example 4: IccSearchClient (lines 65-70):

// Custom pretty-printing
public void saveToFile(SearchResult searchResult, String filePath) throws IOException {
    String jsonContent = Proto.toJson(searchResult);
    String prettyJsonContent = Json.prettyPrintJson(jsonContent);  // Custom pretty printer
    fileSystemHandler.writeFile(filePath, prettyJsonContent);
}

Current Implementation (String Formatting - Legacy)

Some existing code uses string formatting for JSON (found in ArchitecturalPlanWriteServiceImpl):

// ⚠️ Legacy approach - avoid for new code
private void createProjectMetadataFile(String projectName, String projectDescription) {
    String json = String.format("""
        {
          "project_name": "%s",
          "project_description": "%s"
        }
        """, projectName, projectDescription);
    
    fileSystemHandler.writeFile(metadataPath, json.getBytes(StandardCharsets.UTF_8));
}

Recommendation: Migrate to JsonFormat.printer() for type safety and consistency.

---

Code Generation Workflow

Backend (Java)

Protobuf compilation is integrated into the Maven build:

# Regenerate Java classes from proto files
mvn clean protobuf:compile protobuf:compile-custom

Maven Plugin Configuration (in pom.xml):

<plugin>
    <groupId>org.xolstice.maven.plugins</groupId>
    <artifactId>protobuf-maven-plugin</artifactId>
    <version>0.6.1</version>
    <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.21.12:exe:${os.detected.classifier}</protocArtifact>
        <pluginId>grpc-java</pluginId>
        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.51.0:exe:${os.detected.classifier}</pluginArtifact>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
                <goal>compile-custom</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Generated Files Location: target/generated-sources/protobuf/

Frontend (TypeScript/Angular)

gRPC-Web client sources are generated using a helper script:

# From project root
./cli/sdlc/utils/generate-grpc-web-sources.sh

What it does:

1. Checks for protoc availability
2. Generates TypeScript interfaces and gRPC-Web clients
3. Outputs to web-ng-m3/src/generated.commonjs/

Manual Command (if script unavailable):

protoc -I=src/main/proto \
  -I=env/dependencies/googleapis \
  --js_out=import_style=commonjs:web-ng-m3/src/generated.commonjs \
  --grpc-web_out=import_style=typescript,mode=grpcwebtext:web-ng-m3/src/generated.commonjs \
  src/main/proto/*.proto \
  env/dependencies/googleapis/google/type/date.proto \
  env/dependencies/googleapis/google/api/*.proto

When to Regenerate:

• ✅ After modifying .proto files
• ✅ After adding new services or methods
• ✅ After changing message definitions
• ✅ When frontend compilation fails with missing types

---

Testing gRPC APIs

Using grpcurl (Command Line)

grpcurl is the curl equivalent for gRPC. It supports server reflection for API discovery.

Installation

# macOS
brew install grpcurl

# Ubuntu/Debian
sudo apt-get install grpcurl

# Go install
go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest

API Discovery

# List all services (using server reflection)
grpcurl -plaintext localhost:8080 list

# Describe a specific service
grpcurl -plaintext localhost:8080 describe \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService

# Describe a specific method
grpcurl -plaintext localhost:8080 describe \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService.GetProjectMetadata

# List all methods in a service
grpcurl -plaintext localhost:8080 list \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService

Making RPC Calls

With explicit proto files (recommended for CI/CD):

grpcurl -plaintext \
  -import-path src/main/proto \
  -import-path env/dependencies/googleapis \
  -proto src/main/proto/api.proto \
  -d '{"architectural_plan_id": "R2024.0091-2024-10-14"}' \
  localhost:8080 \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService/GetArchitecturalPlan

With server reflection (faster for development):

grpcurl -plaintext \
  -d '{"architectural_plan_id": "R2024.0091-2024-10-14"}' \
  localhost:8080 \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService/GetArchitecturalPlan

With authentication (Cloud Run):

# Get ID token for Cloud Run
TOKEN=$(gcloud auth print-identity-token)

# Make authenticated request
grpcurl \
  -import-path src/main/proto \
  -proto src/main/proto/api.proto \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{"architectural_plan_id": "R2024.0091-2024-10-14"}' \
  construction-code-expert-dev-856365345080.us-central1.run.app:443 \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService/GetArchitecturalPlan

With large messages (e.g., PDF responses):

grpcurl -plaintext \
  -max-msg-sz $((10 * 1024 * 1024)) \
  -d '{"architectural_plan_id": "R2024.0091-2024-10-14", "page_number": 1}' \
  localhost:8080 \
  org.codetricks.construction.code.assistant.service.ArchitecturalPlanService/GetArchitecturalPlanPagePdf

Testing with JSON Files

For complex requests, use JSON files:

# Create request JSON
cat > request.json <<EOF
{
  "project_name": "Multi-Family Residential",
  "project_description": "5-story apartment complex",
  "project_address": {
    "street": "123 Main St",
    "city": "San Francisco",
    "state": "CA",
    "postal_code": "94102"
  }
}
EOF

# Make request
grpcurl -plaintext \
  -import-path src/main/proto \
  -proto src/main/proto/api.proto \
  -d @ \
  localhost:8080 \
  org.codetricks.construction.code.assistant.service.ProjectMetadataService/CreateProjectMetadata \
  < request.json

Testing REST API (via gRPC-Gateway)

When using gRPC-Gateway or ESPv2 for HTTP/JSON transcoding:

# List architectural plans (GET)
curl http://localhost:8082/v1/architectural-plans

# Get specific plan (GET)
curl http://localhost:8082/v1/architectural-plans/R2024.0091-2024-10-14

# Create project metadata (POST)
curl -X POST http://localhost:8082/v1/projects/metadata \
  -H "Content-Type: application/json" \
  -d '{
    "project_name": "Multi-Family Residential",
    "project_description": "5-story apartment complex"
  }'

# Update project metadata (PUT/PATCH)
curl -X PATCH http://localhost:8082/v1/projects/R2024.0091-2024-10-14/metadata \
  -H "Content-Type: application/json" \
  -d '{
    "project_name": "Updated Project Name"
  }'

# Search ICC codes (POST)
curl -X POST http://localhost:8082/v1/icc-books/2217/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Cooling towers",
    "max_results": 3
  }'

Automated Testing Scripts

Create reusable test scripts for common workflows:

#!/bin/bash
# test-project-metadata.sh

set -e

GRPC_HOST="${GRPC_HOST:-localhost:8080}"
PLAN_ID="${PLAN_ID:-R2024.0091-2024-10-14}"

echo "Testing Project Metadata API..."

# Test 1: Create project metadata
echo "1. Creating project metadata..."
grpcurl -plaintext \
  -import-path src/main/proto \
  -proto src/main/proto/api.proto \
  -d "{
    \"architectural_plan_id\": \"${PLAN_ID}\",
    \"project_name\": \"Test Project\",
    \"project_description\": \"Automated test\"
  }" \
  ${GRPC_HOST} \
  org.codetricks.construction.code.assistant.service.ProjectMetadataService/CreateProjectMetadata

# Test 2: Get project metadata
echo "2. Retrieving project metadata..."
grpcurl -plaintext \
  -import-path src/main/proto \
  -proto src/main/proto/api.proto \
  -d "{\"architectural_plan_id\": \"${PLAN_ID}\"}" \
  ${GRPC_HOST} \
  org.codetricks.construction.code.assistant.service.ProjectMetadataService/GetProjectMetadata

# Test 3: Update project metadata
echo "3. Updating project metadata..."
grpcurl -plaintext \
  -import-path src/main/proto \
  -proto src/main/proto/api.proto \
  -d "{
    \"architectural_plan_id\": \"${PLAN_ID}\",
    \"project_name\": \"Updated Test Project\"
  }" \
  ${GRPC_HOST} \
  org.codetricks.construction.code.assistant.service.ProjectMetadataService/UpdateProjectMetadata

echo "✅ All tests passed!"

Usage:

# Test local server
./test-project-metadata.sh

# Test Cloud Run deployment
GRPC_HOST=construction-code-expert-dev-856365345080.us-central1.run.app:443 \
  ./test-project-metadata.sh

---

Proto Evolution and Backward Compatibility

Safe Changes (Non-Breaking)

✅ Adding new fields (with new field numbers):

message ProjectMetadata {
  string project_name = 1;
  string project_description = 2;
  ProjectAddress project_address = 3;  // ✅ New field - safe
}

✅ Adding new enum values:

enum IbcOccupancyGroup {
  UNKNOWN = 0;
  A_1 = 1;
  R_2 = 21;
  S_3 = 26;  // ✅ New value - safe
}

✅ Adding new RPC methods:

service ProjectMetadataService {
  rpc GetProjectMetadata(GetProjectMetadataRequest) returns (GetProjectMetadataResponse);
  rpc CreateProjectMetadata(CreateProjectMetadataRequest) returns (CreateProjectMetadataResponse);  // ✅ New method - safe
}

✅ Adding new services:

// ✅ New service - safe
service ProjectSettingsService {
  rpc GetProjectSettings(GetProjectSettingsRequest) returns (GetProjectSettingsResponse);
}

Breaking Changes (Avoid)

❌ Changing field numbers:

message ProjectMetadata {
  string project_name = 2;  // ❌ Was 1 - BREAKING!
}

❌ Changing field types:

message ProjectMetadata {
  int32 project_name = 1;  // ❌ Was string - BREAKING!
}

❌ Removing fields (use reserved instead):

message ProjectMetadata {
  reserved 2;  // ✅ Mark as reserved instead of deleting
  reserved "old_field_name";
  
  string project_name = 1;
  // string project_description = 2;  // Removed - now reserved
}

❌ Renaming fields (changes JSON representation):

message ProjectMetadata {
  string project_title = 1;  // ❌ Was project_name - BREAKING for JSON clients!
}

Migration Strategy

When you need to make breaking changes:

1. Add new field with new number
2. Deprecate old field with comment
3. Populate both fields in code (dual-write)
4. Migrate clients to use new field
5. Remove old field after migration complete (mark as reserved)

message ProjectMetadata {
  // Deprecated: Use project_title instead
  string project_name = 1 [deprecated = true];
  
  string project_title = 10;  // New field
}

---

Common Patterns

Request/Response Pairs

Always create dedicated request/response messages (even if empty):

// ✅ Good - explicit request/response
message GetProjectMetadataRequest {
  string architectural_plan_id = 1;
}

message GetProjectMetadataResponse {
  ProjectMetadata metadata = 1;
}

service ProjectMetadataService {
  rpc GetProjectMetadata(GetProjectMetadataRequest) returns (GetProjectMetadataResponse);
}

// ❌ Bad - using message directly
service ProjectMetadataService {
  rpc GetProjectMetadata(ProjectMetadata) returns (ProjectMetadata);  // Hard to evolve!
}

Wrapping vs. Flattening

Wrap when you want to return a reusable message:

// ✅ Good - reusable ProjectMetadata
message GetProjectMetadataResponse {
  ProjectMetadata metadata = 1;  // Can be used in other RPCs
}

message ProjectMetadata {
  string project_name = 1;
  string project_description = 2;
}

Flatten when fields are RPC-specific:

// ✅ Also good - RPC-specific fields
message GetProjectMetadataResponse {
  string project_name = 1;
  string project_description = 2;
  bool is_legacy = 3;  // RPC-specific metadata
}

Pagination

Use consistent pagination patterns:

message ListProjectsRequest {
  string account_id = 1;
  int32 page_size = 2;   // Max items per page
  string page_token = 3; // Opaque token from previous response
}

message ListProjectsResponse {
  repeated ProjectMetadata projects = 1;
  string next_page_token = 2;  // Empty if no more pages
  int32 total_count = 3;       // Optional: total count
}

Error Handling

Use google.rpc.Status for rich error details:

import "google/rpc/status.proto";

message CreateProjectMetadataResponse {
  oneof result {
    ProjectMetadata metadata = 1;
    google.rpc.Status error = 2;
  }
}

---

Related Documentation

• Developer Playbook: Build and deployment workflows
• Local gRPC Server: Running gRPC server locally
• gRPC-Gateway: HTTP/JSON transcoding setup
• Project Metadata TDD: Detailed proto design example

---

Utility Classes Reference

Proto.java

Location: src/main/java/org/codetricks/construction/code/assistant/io/Proto.java

Current Methods:

// JSON serialization
String json = Proto.toJson(message);
MyMessage msg = Proto.loadJson(MyMessage.class, jsonString);

// Timestamp conversion
Timestamp ts = Proto.toProtoTimestamp(Instant.now());

New Methods (Issue #224 ✅ Complete):

// Snake_case serialization (ideal for metadata files)
String json = Proto.toJsonWithDefaults(message);  // snake_case field names
fileSystemHandler.writeFile(path, json);

// Or alternative name (same functionality)
String json = Proto.toJsonPreservingFieldNames(message);

// Reader/Writer helpers
try (Reader reader = fileSystemHandler.getReader(path)) {
    MyMessage msg = Proto.loadJsonFromReader(MyMessage.class, reader);
}

try (Writer writer = fileSystemHandler.getWriter(path)) {
    Proto.writeJsonToWriter(message, writer);  // Uses snake_case
}

TextProtoLoader.java

Location: src/main/java/org/codetricks/construction/code/assistant/io/TextProtoLoader.java

Current Methods:

// Parse text proto from stream
MyMessage msg = TextProtoLoader.get(MyMessage.class, inputStream);

// Parse text proto from string
MyMessage msg = TextProtoLoader.get(MyMessage.class, textProtoString);

Coming Soon (Issue #224):

// Serialization (for use with FileSystemHandler)
String textProto = TextProtoLoader.toTextProto(message);
fileSystemHandler.writeFile(path, textProto);

// Reader/Writer helpers
try (Reader reader = fileSystemHandler.getReader(path)) {
    MyMessage msg = TextProtoLoader.readFromReader(MyMessage.class, reader);
}

---

Quick Reference

Common Commands

# Regenerate Java classes
mvn clean protobuf:compile protobuf:compile-custom

# Regenerate TypeScript clients
./cli/sdlc/utils/generate-grpc-web-sources.sh

# List gRPC services
grpcurl -plaintext localhost:8080 list

# Describe service
grpcurl -plaintext localhost:8080 describe SERVICE_NAME

# Make gRPC call
grpcurl -plaintext -d '{"field": "value"}' localhost:8080 SERVICE/METHOD

# Test REST API
curl http://localhost:8082/v1/endpoint

File Locations

• Proto files: src/main/proto/
• Generated Java: target/generated-sources/protobuf/
• Generated TypeScript: web-ng-m3/src/generated.commonjs/
• Google APIs: env/dependencies/googleapis/

Key Java Classes

• JSON Serialization: com.google.protobuf.util.JsonFormat
• Project Utilities:
  • org.codetricks.construction.code.assistant.io.Proto - JSON serialization helpers
  • org.codetricks.construction.code.assistant.io.TextProtoLoader - Text proto parsing
• Enum Utilities: com.google.protobuf.Descriptors.EnumValueDescriptor
• Custom Options: descriptor.getOptions().getExtension()