Skip to main content

API Integration Testing with gRPC

This document provides comprehensive guidance on end-to-end (E2E) integration testing of the PermitProof backend using gRPC calls from Java test code.

Overview

The project uses Java-based gRPC integration tests to validate complete user journeys by making actual gRPC network calls to running servers. These tests verify the entire stack including network serialization, message size limits, authentication, and business logic.

Key Achievement

🎯 Real Network Testing: We've implemented comprehensive integration tests that make actual gRPC calls to deployed environments, testing the complete request-response cycle including authentication, authorization, and data persistence.

Available Integration Tests

The gRPC integration test suite is located in src/test/java/ and includes the following test categories:

Core Integration Tests

Authentication & Authorization Tests

Full Pipeline E2E Test

Quick Start: Running E2E Tests

The fastest way to run the full E2E integration test:

# Using the helper script (recommended)
./cli/utils/run-e2e-test.sh crsr

# Or with custom options
./cli/utils/run-e2e-test.sh crsr --prefix=SmokeTest

Stage Skip Options

For faster iteration during development, you can skip individual stages:

OptionDescription
--skip-uploadSkip PDF upload stage
--skip-ingestionSkip page ingestion (Cloud Run Job)
--skip-applicabilitySkip applicability analysis (Cloud Run Job)
--skip-complianceSkip compliance report generation
# Example: Skip compliance for faster iteration
./cli/utils/run-e2e-test.sh crsr --skip-compliance

# Example: Skip multiple stages
./cli/utils/run-e2e-test.sh crsr --skip-ingestion --skip-applicability

Resume from Existing Project

To rerun specific stages on an existing project:

# Resume from an existing project, skip early stages
./cli/utils/run-e2e-test.sh crsr \
--baseline-project=IntegrationTest-commit.abc123-time.10.30.00-PST \
--skip-ingestion \
--skip-applicability

Configuration Reference

OptionDescription
--prefix=<name>Prefix for new project names (default: IntegrationTest)
--owners=<emails>Comma-separated emails to share project with
--books=<ids>ICC book IDs for applicability (default: 2217)
--baseline-project=<id>Use existing project instead of creating new
--baseline-file-id=<id>Use existing file ID (implies --skip-upload)

Advanced Maven pass-through (for edge cases):

./cli/utils/run-e2e-test.sh crsr -Dsome.custom.property=value

Prerequisites

  1. ICC Book Data: Sync to target environment:

    gsutil -m cp -r gs://construction-code-expert-repo/api/icc/content/ \
    gs://construction-code-expert-<env>/api/icc/content/
  2. Test User: ai-swe-agent-test@codetricks.org must exist in Firebase

  3. Environment Deployed: Latest backend changes must be deployed

End-to-End Test Scenarios

Implemented Scenarios

  1. Complete Project Workflow (CostAnalysisIntegrationTest)

    • Copy baseline project with sharing settings
    • Get project input files
    • Start page ingestion task
    • Monitor task completion via Firestore
    • Validate cost analysis data
    • Verify task status and progress tracking
  2. Compliance Report Generation (ComplianceReportIntegrationTest)

    • Project copying with GCS file structure verification
    • Compliance report generation via gRPC
    • GCS output file validation
    • Firestore task tracking with cost analysis
    • Multi-page compliance report validation
  3. File Upload Operations (ArchitecturalPlanWriteServiceGrpcIntegrationTest)

    • Small file upload (< 1MB)
    • Large file upload (> 50MB)
    • Message size limit testing (100MB)
    • Network serialization validation
    • Server configuration verification
  4. Project Management

    • Project creation with metadata
    • Project copying with file structure
    • Project deletion (soft and hard delete)
    • Sharing settings management
  5. Access Control Validation

    • Firebase authentication token generation
    • Role-based permission checking
    • Admin role preservation
    • User invitation workflow

Potential Future Scenarios

The following scenarios can be implemented using the existing gRPC infrastructure:

  1. Multi-User Collaboration Workflows

    • Share project with multiple users at different permission levels
    • Verify concurrent access and modifications
    • Test permission inheritance and revocation
    • Validate real-time collaboration features
  2. Code Analysis Pipeline

    • Upload architectural plans
    • Ingest multiple pages in parallel
    • Run code applicability analysis on all pages
    • Generate comprehensive compliance reports
    • Download and validate report artifacts
  3. Background Task Orchestration

    • Start multiple async tasks concurrently
    • Monitor task progress via Firestore
    • Handle task failures and retries
    • Verify task cancellation
    • Test task timeout scenarios
  4. Billing and Cost Tracking

    • Validate cost accumulation across operations
    • Test billing profile updates
    • Verify transaction history
    • Check balance deductions
    • Test payment processing integration
  5. Performance and Load Testing

    • Concurrent project operations
    • Large batch file uploads
    • Stress testing with multiple users
    • Memory leak detection
    • Response time validation
  6. Error Handling and Resilience

    • Network failure scenarios
    • Invalid input validation
    • Authorization failures
    • Resource exhaustion handling
    • Graceful degradation testing

Running gRPC Integration Tests

Prerequisites

  1. Environment Setup: Test environment must be deployed with latest backend changes
  2. Authentication: Firebase Web API key and service account credentials configured
  3. Test Data: Baseline projects (e.g., SanJose-baseline) must exist in test environment
  4. Network Access: Connectivity to Cloud Run services (test or local)

Local Development Testing

# Set Java environment (dev container)
export JAVA_HOME=/usr/lib/jvm/temurin-23-jdk-arm64

# Start local gRPC server (in one terminal)
./cli/start-local-server.sh

# Set authentication token (in another terminal)
export CODEPROOF_DEMO_USER_FIREBASE_TOKEN="your_firebase_token"

# Run specific integration test
mvn test -Dtest=ArchitecturalPlanWriteServiceGrpcIntegrationTest#testServerConnectivity

# Run small file upload test
mvn test -Dtest=ArchitecturalPlanWriteServiceGrpcIntegrationTest#testUploadSmallFileViaGrpc

# Run large file upload test with custom parameters
mvn test -Dtest=ArchitecturalPlanWriteServiceGrpcIntegrationTest#testUploadLargeFileViaGrpc \
-Dpdf.path="/path/to/your/large-file.pdf" \
-Dproject.id="test-large-upload-$(date +%s)" \
-Dupload.filename="test-plan.pdf" \
-Dauth.token="$CODEPROOF_DEMO_USER_FIREBASE_TOKEN"

# Run project creation test
mvn test -Dtest=ArchitecturalPlanWriteServiceGrpcIntegrationTest#testCreateProjectViaGrpc

Testing Against Deployed Environment

# Full E2E test on any environment
./cli/utils/run-e2e-test.sh crsr # Cursor environment
./cli/utils/run-e2e-test.sh agy # Antigravity environment
./cli/utils/run-e2e-test.sh gcli # Gemini CLI environment

# With custom options
./cli/utils/run-e2e-test.sh crsr --prefix=Nightly

# Skip stages for faster iteration
./cli/utils/run-e2e-test.sh crsr --skip-compliance

# Show all available options
./cli/utils/run-e2e-test.sh --help

Manual: Using Maven Directly

# Set environment variables
export FIREBASE_WEB_API_KEY="your_firebase_web_api_key"
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"

# Run compliance report integration test against test environment
mvn test \
-Dtest=ComplianceReportIntegrationTest \
-Dserver.host=construction-code-expert-test-fhj3jauezq-uc.a.run.app \
-Dserver.port=443 \
-Dproject.id=construction-code-expert-test \
-DFIREBASE_WEB_API_KEY="$FIREBASE_WEB_API_KEY"

# Run cost analysis integration test
mvn test \
-Dtest=CostAnalysisIntegrationTest \
-Dserver.host=construction-code-expert-test-fhj3jauezq-uc.a.run.app \
-Dserver.port=443 \
-Dproject.id=construction-code-expert-test

# Run all integration tests
mvn test -Dtest="*IntegrationTest"

Using Helper Scripts

# Run compliance report integration test with all prerequisites
./run-compliance-integration-test.sh

# This script automatically:
# - Verifies environment variables are set
# - Checks Google Application Default Credentials
# - Compiles the project
# - Runs the integration test with proper configuration

Running Specific Test Suites

# Run only authentication/RBAC tests
mvn test -Dtest="*RbacTest,*AccessControlTest"

# Run only file upload tests
mvn test -Dtest="ArchitecturalPlanWriteServiceGrpcIntegrationTest#testUpload*"

# Run only project management tests
mvn test -Dtest="ProjectCopyFunctionalityTest,ArchitecturalPlanWriteServiceGrpcIntegrationTest#testCreateProjectViaGrpc"

# Run tests with specific tag or pattern
mvn test -Dtest="*IntegrationTest" -Dmaven.test.failure.ignore=false

Authentication in gRPC Integration Tests

How Authentication Works

The gRPC integration tests use Firebase authentication with bearer tokens:

  1. Token Generation: Tests generate Firebase custom tokens using the firebase-token-generator/generate-token.sh script or Firebase Web API
  2. gRPC Interceptor: Tests inject bearer tokens into gRPC call headers using client interceptors
  3. Server Validation: Backend validates tokens and enforces RBAC policies
  4. Test User: Tests use ai-swe-agent-test@codetricks.org with appropriate permissions

Creating Authenticated gRPC Stubs

Example from ArchitecturalPlanWriteServiceGrpcIntegrationTest.java:

/**
* Creates a blocking stub with authorization header
*/
private ArchitecturalPlanWriteServiceGrpc.ArchitecturalPlanWriteServiceBlockingStub
createAuthenticatedStub(String authToken) {

ClientInterceptor authInterceptor = new ClientInterceptor() {
@Override
public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(
MethodDescriptor<ReqT, RespT> method,
CallOptions callOptions,
Channel next) {
return new ForwardingClientCall.SimpleForwardingClientCall<ReqT, RespT>(
next.newCall(method, callOptions)) {
@Override
public void start(ClientCall.Listener<RespT> responseListener, Metadata headers) {
// Add Bearer token to authorization header
headers.put(
Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER),
"Bearer " + authToken
);
super.start(responseListener, headers);
}
};
}
};

return blockingStub.withInterceptors(authInterceptor)
.withDeadlineAfter(60, TimeUnit.SECONDS);
}

Token Generation Methods

Example from CostAnalysisIntegrationTest.java:

private void setupAuthentication() throws Exception {
String firebaseWebApiKey = System.getProperty("FIREBASE_WEB_API_KEY");
if (firebaseWebApiKey == null || firebaseWebApiKey.isEmpty()) {
firebaseWebApiKey = System.getenv("FIREBASE_WEB_API_KEY");
}

if (firebaseWebApiKey == null || firebaseWebApiKey.isEmpty()) {
throw new IllegalStateException("FIREBASE_WEB_API_KEY must be set");
}

// Generate custom token using firebase-token-generator script
ProcessBuilder pb = new ProcessBuilder(
"./firebase-token-generator/generate-token.sh",
TEST_USER_EMAIL
);
pb.redirectErrorStream(true);
Process process = pb.start();

BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getInputStream())
);
String customToken = reader.readLine();

if (customToken == null || customToken.isEmpty()) {
throw new IllegalStateException("Failed to generate custom token");
}

// Exchange custom token for ID token via Firebase Web API
authToken = exchangeCustomTokenForIdToken(customToken, firebaseWebApiKey);
}

private String exchangeCustomTokenForIdToken(String customToken, String apiKey)
throws Exception {
// Make HTTP POST to Firebase Auth REST API
String url = "https://identitytoolkit.googleapis.com/v1/accounts:signInWithCustomToken?key=" + apiKey;

// ... HTTP request implementation ...

return idToken;
}

Method 2: Using Environment Variable

String authToken = System.getProperty("auth.token");
if (authToken == null || authToken.isEmpty()) {
authToken = System.getenv("CODEPROOF_DEMO_USER_FIREBASE_TOKEN");
}
if (authToken == null || authToken.isEmpty()) {
System.out.println("Skipping test - no auth token provided");
return;
}

Setting Up gRPC Channel

@Before
public void setUp() throws Exception {
// Generate authentication token
setupAuthentication();

// Initialize gRPC channel for test environment
String serverHost = System.getProperty("server.host", "localhost");
int serverPort = Integer.parseInt(System.getProperty("server.port", "8080"));

ManagedChannelBuilder<?> channelBuilder = ManagedChannelBuilder
.forAddress(serverHost, serverPort);

// Use TLS for production/test environments
if (serverPort == 443) {
channelBuilder.useTransportSecurity();
} else {
channelBuilder.usePlaintext();
}

channel = channelBuilder
.maxInboundMessageSize(100 * 1024 * 1024) // 100MB
.build();

// Create authenticated stubs
ClientInterceptor authInterceptor = createAuthInterceptor();
planService = ArchitecturalPlanServiceGrpc
.newBlockingStub(channel)
.withInterceptors(authInterceptor);
}

@After
public void tearDown() throws Exception {
if (channel != null) {
channel.shutdown().awaitTermination(5, TimeUnit.SECONDS);
}
}

Example Test Pattern with Authentication

@Test
public void testCompleteWorkflow() throws Exception {
// Step 1: Copy baseline project
CopyArchitecturalPlanRequest copyRequest = CopyArchitecturalPlanRequest.newBuilder()
.setSourceProjectId("SanJose-baseline")
.setTargetProjectId("test-project-" + UUID.randomUUID())
.setCopySharingSettings(true)
.build();

CopyArchitecturalPlanResponse copyResponse = writeService.copyArchitecturalPlan(copyRequest);
assertTrue("Project copy should succeed", copyResponse.getSuccess());
String projectId = copyResponse.getTargetProjectId();

// Step 2: Start page ingestion task
StartAsyncIngestFileRequest ingestRequest = StartAsyncIngestFileRequest.newBuilder()
.setProjectId(projectId)
.setFilename("building-plan.pdf")
.addPageNumbers(1)
.addPageNumbers(2)
.build();

StartAsyncIngestFileResponse ingestResponse = asyncWriteService
.startAsyncIngestFile(ingestRequest);
assertTrue("Ingestion should start successfully", ingestResponse.getSuccess());
String taskId = ingestResponse.getTaskId();

// Step 3: Monitor task completion via Firestore
Map<String, Object> taskData = waitForTaskCompletion(taskId);
assertEquals("Task should complete", "COMPLETE", taskData.get("status"));

// Step 4: Validate results
GetArchitecturalPlanRequest getRequest = GetArchitecturalPlanRequest.newBuilder()
.setArchitecturalPlanId(projectId)
.build();

ArchitecturalPlan plan = planService.getArchitecturalPlan(getRequest);
assertNotNull("Project should exist", plan);
assertTrue("Project should have ingested pages", plan.getPagesCount() > 0);
}

CI/CD Integration

Maven Test Execution

Integration tests can be run as part of the Maven build lifecycle:

# Run all tests (unit + integration)
mvn test

# Run only integration tests
mvn test -Dtest="*IntegrationTest"

# Skip tests during build
mvn clean package -DskipTests

# Run tests with specific profile
mvn test -P integration-tests

GitHub Actions Workflow

Example workflow for running gRPC integration tests:

name: gRPC Integration Tests

on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
schedule:
- cron: '0 2 * * *' # Run daily at 2 AM

jobs:
grpc-integration-tests:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Setup Java
uses: actions/setup-java@v3
with:
java-version: '23'
distribution: 'temurin'

- name: Setup Google Cloud credentials
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
echo "$GCP_SA_KEY" > $HOME/gcp-key.json
export GOOGLE_APPLICATION_CREDENTIALS=$HOME/gcp-key.json

- name: Setup Firebase credentials
env:
FIREBASE_WEB_API_KEY: ${{ secrets.FIREBASE_WEB_API_KEY }}
run: |
echo "FIREBASE_WEB_API_KEY=$FIREBASE_WEB_API_KEY" >> $GITHUB_ENV

- name: Run gRPC integration tests
run: |
mvn test \
-Dtest="*IntegrationTest" \
-Dserver.host=construction-code-expert-test-fhj3jauezq-uc.a.run.app \
-Dserver.port=443 \
-Dproject.id=construction-code-expert-test \
-DFIREBASE_WEB_API_KEY=$FIREBASE_WEB_API_KEY

- name: Upload test results
if: always()
uses: actions/upload-artifact@v3
with:
name: test-results
path: target/surefire-reports/

Cloud Build Integration

For Google Cloud Build, add a test step to your cloudbuild.yaml:

steps:
# ... other build steps ...

- name: 'maven:3.9-eclipse-temurin-23'
id: 'integration-tests'
entrypoint: 'bash'
args:
- '-c'
- |
mvn test \
-Dtest="*IntegrationTest" \
-Dserver.host=construction-code-expert-test-fhj3jauezq-uc.a.run.app \
-Dserver.port=443 \
-Dproject.id=construction-code-expert-test \
-DFIREBASE_WEB_API_KEY=$$FIREBASE_WEB_API_KEY
env:
- 'GOOGLE_APPLICATION_CREDENTIALS=/workspace/.secrets/service-account.json'
secretEnv:
- 'FIREBASE_WEB_API_KEY'

availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/firebase-web-api-key/versions/latest
env: 'FIREBASE_WEB_API_KEY'

Common Gotchas and Troubleshooting

1. Authentication Token Expiration

Problem: Tests fail with UNAUTHENTICATED status after running for a while

Solution:

  • Firebase ID tokens expire after 1 hour
  • Regenerate tokens for long-running test suites
  • Implement token refresh logic in test setup
  • Use custom tokens with longer expiration for test environments
// Check token expiration before each test
@Before
public void refreshTokenIfNeeded() throws Exception {
if (isTokenExpired(authToken)) {
setupAuthentication(); // Regenerate token
}
}

2. Message Size Limits

Problem: RESOURCE_EXHAUSTED error when uploading large files

Solution:

  • Configure client-side message size: .maxInboundMessageSize(100 * 1024 * 1024)
  • Verify server-side limits match client configuration
  • Check ESPv2 gateway limits if using REST API transcoding
  • Consider chunking for files > 100MB
channel = ManagedChannelBuilder.forAddress(serverHost, serverPort)
.usePlaintext()
.maxInboundMessageSize(100 * 1024 * 1024) // 100MB
.maxOutboundMessageSize(100 * 1024 * 1024) // 100MB
.build();

3. Firestore Access Issues

Problem: Tests fail to read/write Firestore documents

Solution:

  • Verify GOOGLE_APPLICATION_CREDENTIALS is set correctly
  • Ensure service account has Firestore permissions
  • Check Firestore project ID matches test environment
  • Use Application Default Credentials in dev container
// Initialize Firestore with explicit project ID
firestore = FirestoreOptions.newBuilder()
.setProjectId(TEST_PROJECT_ID)
.build()
.getService();

4. Test Data Dependencies

Problem: Tests fail because baseline projects don't exist

Solution:

  • Document required baseline projects in test javadoc
  • Create baseline projects as part of test setup
  • Use project copying to create test fixtures
  • Implement cleanup to remove test projects after execution
@Before
public void ensureBaselineProjectExists() throws Exception {
try {
GetArchitecturalPlanRequest request = GetArchitecturalPlanRequest.newBuilder()
.setArchitecturalPlanId("SanJose-baseline")
.build();
planService.getArchitecturalPlan(request);
} catch (StatusRuntimeException e) {
if (e.getStatus().getCode() == Status.Code.NOT_FOUND) {
fail("Baseline project 'SanJose-baseline' does not exist. Please create it first.");
}
}
}

5. Network Connectivity Issues

Problem: Tests timeout or fail to connect to gRPC server

Solution:

  • Verify server host and port are correct
  • Check firewall rules and network policies
  • Test basic connectivity with testServerConnectivity()
  • Use IPv4 explicitly if IPv6 causes issues: -Djava.net.preferIPv4Stack=true
# Test basic connectivity first
mvn test -Dtest=ArchitecturalPlanWriteServiceGrpcIntegrationTest#testServerConnectivity \
-Dserver.host=localhost \
-Dserver.port=8080

6. Async Task Timeouts

Problem: Tests timeout waiting for background tasks to complete

Solution:

  • Increase task timeout for LLM-intensive operations
  • Poll Firestore more frequently for task status
  • Implement exponential backoff for polling
  • Add detailed logging to understand task progress
private Map<String, Object> waitForTaskCompletion(String taskId) throws Exception {
int maxAttempts = 60; // 10 minutes with 10-second intervals
int attempt = 0;

while (attempt < maxAttempts) {
DocumentSnapshot doc = firestore.collection("tasks")
.document(taskId)
.get()
.get();

if (doc.exists()) {
Map<String, Object> data = doc.getData();
String status = (String) data.get("status");

System.out.println("Task " + taskId + " status: " + status +
" (attempt " + (attempt + 1) + "/" + maxAttempts + ")");

if ("COMPLETE".equals(status) || "FAILED".equals(status)) {
return data;
}
}

Thread.sleep(10000); // Wait 10 seconds
attempt++;
}

throw new TimeoutException("Task did not complete within timeout period");
}

7. GCS File Access Issues

Problem: Tests fail to read/write files from Google Cloud Storage

Solution:

  • Verify service account has Storage Object Admin role
  • Check bucket names and paths are correct
  • Ensure GCS bucket exists in test environment
  • Use proper file naming conventions
// Validate GCS file exists after upload
Storage storage = StorageOptions.newBuilder()
.setProjectId(TEST_PROJECT_ID)
.build()
.getService();

String bucketName = "construction-code-expert-test-data";
String blobName = "projects/" + projectId + "/input/" + filename;

Blob blob = storage.get(bucketName, blobName);
assertNotNull("File should exist in GCS", blob);
assertTrue("File should have content", blob.getSize() > 0);

Test Configuration Parameters

The following system properties can be used to configure integration tests:

ParameterDescriptionDefaultExample
server.hostgRPC server hostnamelocalhostconstruction-code-expert-test-fhj3jauezq-uc.a.run.app
server.portgRPC server port8080443
project.idGCP project ID for test environmentN/Aconstruction-code-expert-test
auth.tokenFirebase ID token for authentication$CODEPROOF_DEMO_USER_FIREBASE_TOKENeyJhbGciOiJSUzI1...
pdf.pathPath to PDF file for upload testsHardcoded path/path/to/test.pdf
upload.filenameFilename to use when uploadingOriginal filenametest-plan.pdf
FIREBASE_WEB_API_KEYFirebase Web API key for token exchangeN/AAIzaSyC...
GOOGLE_APPLICATION_CREDENTIALSPath to service account JSONN/A/path/to/sa.json

Best Practices

  1. Use Unique Project IDs: Generate unique test project IDs using UUID to avoid conflicts
  2. Clean Up Test Data: Always delete test projects in @After methods
  3. Document Prerequisites: Clearly document required baseline projects and environment setup
  4. Test Isolation: Each test should be independent and not rely on other tests
  5. Meaningful Assertions: Use descriptive assertion messages to aid debugging
  6. Comprehensive Logging: Add detailed logging to understand test flow and failures
  7. Handle Async Operations: Implement proper waiting and polling for background tasks
  8. Validate End-to-End: Don't just check gRPC responses, validate actual data persistence
  9. Test Error Scenarios: Include negative test cases for error handling
  10. Use Helper Methods: Extract common patterns into reusable helper methods

External Resources