feat: Add functional health checks for test prerequisites (#65)

Replaces basic "service is up" checks with functional verification that tests can actually use the services. ## Changes ### New Health Checks - **RPC Proxy**: Verifies eth_call works and deployed contracts are accessible - **GraphQL**: Confirms Ponder has indexed data with non-zero stats - **Web App**: Validates endpoint accessibility ### Improvements - Clear error messages explain what failed and how to fix it - Checks verify actual functionality, not just HTTP 200 responses - Fails fast before tests run with cryptic errors ### Files - `tests/setup/health-checks.ts` - Core health check functions - `tests/setup/stack.ts` - Integration with waitForStackReady() - `tests/HEALTH_CHECKS.md` - Documentation and troubleshooting guide ## Error Message Example Before: ``` RPC health check failed with status 404 ``` After: ``` ❌ Stack health check failed Failed services: • RPC Proxy: RPC proxy returned HTTP 404 Expected 200, got 404. Check if Anvil is running and RPC_URL is correct. • GraphQL Indexer: GraphQL has no indexed data yet Ponder is running but has not indexed contract events. Troubleshooting: 1. Check stack logs: tail tests/.stack.log 2. Verify services are running: ./scripts/dev.sh status 3. Restart stack: ./scripts/dev.sh restart --full ``` ## Benefits - ✅ Tests fail fast with clear error messages - ✅ Catches configuration issues before tests run - ✅ Verifies services are actually usable, not just running resolves #61 Co-authored-by: johba <johba@harb.eth> Reviewed-on: https://codeberg.org/johba/harb/pulls/65
2025-10-05 20:03:07 +02:00 · 2025-10-05 20:03:07 +02:00 · b1f40374cd
commit b1f40374cd
parent 1645865c5a
3 changed files with 472 additions and 42 deletions
--- a/tests/HEALTH_CHECKS.md
+++ b/tests/HEALTH_CHECKS.md
@ -0,0 +1,133 @@
+# Functional Health Checks
+
+## Overview
+
+The test suite includes functional health checks that verify services are actually usable, not just running. This prevents cryptic test failures and provides clear error messages when prerequisites aren't met.
+
+## What Gets Checked
+
+### 1. RPC Proxy Functionality
+- ✅ Verifies RPC endpoint responds to requests
+- ✅ Confirms `eth_chainId` returns valid chain ID
+- ✅ Tests contract accessibility via `eth_call` to deployed Kraiken contract
+- ✅ Validates deployed contracts from `onchain/deployments-local.json` are accessible
+
+**Why:** Catches RPC proxy misconfigurations (404s), deployment issues, and contract accessibility problems before tests run.
+
+### 2. GraphQL Indexed Data
+- ✅ Verifies GraphQL endpoint is responsive
+- ✅ Confirms Ponder has indexed contract events
+- ✅ Validates stats data contains non-zero values (actual indexed data)
+
+**Why:** Ensures Ponder has completed indexing before tests attempt to query data. Prevents "no data found" errors in tests.
+
+### 3. Web App Accessibility
+- ✅ Confirms webapp endpoint returns 200 or 308 status
+- ✅ Validates routing is functional
+
+**Why:** Basic check that frontend is accessible for UI tests.
+
+## How It Works
+
+Health checks run automatically in `beforeAll` hooks via `waitForStackReady()`:
+
+```typescript
+test.beforeAll(async () => {
+  await startStack();
+  await waitForStackReady({
+    rpcUrl: STACK_RPC_URL,
+    webAppUrl: STACK_WEBAPP_URL,
+    graphqlUrl: STACK_GRAPHQL_URL,
+  });
+});
+```
+
+The checks poll every 2 seconds with a 3-minute timeout, providing clear feedback if any service fails.
+
+## Error Messages
+
+### Before (Generic)
+```
+RPC health check failed with status 404
+```
+
+### After (Specific)
+```
+❌ Stack health check failed
+
+Failed services:
+  • RPC Proxy: RPC proxy returned HTTP 404
+    Expected 200, got 404. Check if Anvil is running and RPC_URL is correct.
+  • GraphQL Indexer: GraphQL has no indexed data yet
+    Ponder is running but has not indexed contract events. Wait longer or check Ponder logs.
+
+Troubleshooting:
+  1. Check stack logs: tail tests/.stack.log
+  2. Verify services are running: ./scripts/dev.sh status
+  3. Restart stack: ./scripts/dev.sh restart --full
+```
+
+## Implementation
+
+### Files
+- `tests/setup/health-checks.ts` - Core health check functions
+- `tests/setup/stack.ts` - Integration with test lifecycle
+
+### API
+
+```typescript
+// Individual checks
+const result = await checkRpcFunctional('http://127.0.0.1:8545');
+if (!result.success) {
+  console.error(`${result.service}: ${result.message}`);
+  console.error(result.details);
+}
+
+// Run all checks
+const results = await runAllHealthChecks({
+  rpcUrl: 'http://127.0.0.1:8545',
+  webAppUrl: 'http://localhost:5173',
+  graphqlUrl: 'http://localhost:42069/graphql',
+});
+
+// Format errors
+const errorMessage = formatHealthCheckError(results);
+```
+
+## Benefits
+
+✅ **Fail Fast** - Tests stop immediately with clear errors instead of cryptic failures
+✅ **Clear Diagnostics** - Error messages explain what failed and how to fix it
+✅ **Catch Config Issues** - Finds env var problems, proxy misconfigurations, and missing data
+✅ **Better DX** - Developers know exactly what's wrong without debugging test code
+
+## Troubleshooting
+
+If health checks fail:
+
+1. **Check stack logs**
+   ```bash
+   tail -f tests/.stack.log
+   ```
+
+2. **Verify services are running**
+   ```bash
+   ./scripts/dev.sh status
+   curl http://127.0.0.1:8545 -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1,"method":"eth_chainId","params":[]}'
+   curl http://localhost:42069/graphql -X POST -H "Content-Type: application/json" -d '{"query":"{__typename}"}'
+   ```
+
+3. **Restart the stack**
+   ```bash
+   ./scripts/dev.sh restart --full
+   ```
+
+4. **Check deployments**
+   ```bash
+   cat onchain/deployments-local.json
+   ```
+
+5. **Verify Ponder indexing**
+   ```bash
+   curl -X POST http://localhost:42069/graphql -H "Content-Type: application/json" -d '{"query":"{ stats(id:\"0x01\"){kraikenTotalSupply}}"}'
+   ```
--- a/tests/setup/health-checks.ts
+++ b/tests/setup/health-checks.ts
@ -0,0 +1,320 @@
+import { readFile } from 'node:fs/promises';
+import { dirname, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+const repoRoot = resolve(__dirname, '..', '..');
+
+export interface HealthCheckResult {
+  success: boolean;
+  service: string;
+  message: string;
+  details?: string;
+}
+
+/**
+ * Verify RPC proxy is functional by:
+ * 1. Checking eth_chainId returns the expected chain
+ * 2. Making an eth_call to verify contract accessibility
+ * 3. Confirming deployed contracts are accessible
+ */
+export async function checkRpcFunctional(rpcUrl: string): Promise<HealthCheckResult> {
+  const service = 'RPC Proxy';
+
+  try {
+    // Step 1: Verify basic RPC connectivity
+    const chainIdResponse = await fetch(rpcUrl, {
+      method: 'POST',
+      headers: { 'content-type': 'application/json' },
+      body: JSON.stringify({
+        jsonrpc: '2.0',
+        id: 1,
+        method: 'eth_chainId',
+        params: []
+      }),
+    });
+
+    if (!chainIdResponse.ok) {
+      return {
+        success: false,
+        service,
+        message: `RPC proxy returned HTTP ${chainIdResponse.status}`,
+        details: `Expected 200, got ${chainIdResponse.status}. Check if Anvil is running and RPC_URL is correct.`
+      };
+    }
+
+    const chainIdPayload = await chainIdResponse.json();
+    if (!chainIdPayload?.result) {
+      return {
+        success: false,
+        service,
+        message: 'RPC returned no chain ID',
+        details: 'RPC responded but returned no result. Check if Anvil is properly initialized.'
+      };
+    }
+
+    // Step 2: Load deployed contracts and verify accessibility
+    const deploymentsPath = resolve(repoRoot, 'onchain', 'deployments-local.json');
+    let deployments: { contracts: { Kraiken: string } };
+
+    try {
+      const deploymentsContent = await readFile(deploymentsPath, 'utf-8');
+      deployments = JSON.parse(deploymentsContent);
+    } catch (error) {
+      return {
+        success: false,
+        service,
+        message: 'Failed to load contract deployments',
+        details: `Could not read ${deploymentsPath}. Run stack setup first.`
+      };
+    }
+
+    // Step 3: Verify we can make eth_call to deployed Kraiken contract
+    const kraikenAddress = deployments.contracts.Kraiken;
+    const totalSupplyCalldata = '0x18160ddd'; // totalSupply() selector
+
+    const callResponse = await fetch(rpcUrl, {
+      method: 'POST',
+      headers: { 'content-type': 'application/json' },
+      body: JSON.stringify({
+        jsonrpc: '2.0',
+        id: 2,
+        method: 'eth_call',
+        params: [{
+          to: kraikenAddress,
+          data: totalSupplyCalldata
+        }, 'latest']
+      }),
+    });
+
+    if (!callResponse.ok) {
+      return {
+        success: false,
+        service,
+        message: 'eth_call to Kraiken contract failed',
+        details: `HTTP ${callResponse.status}. Contract may not be deployed or RPC proxy is misconfigured.`
+      };
+    }
+
+    const callPayload = await callResponse.json();
+
+    if (callPayload.error) {
+      return {
+        success: false,
+        service,
+        message: 'Contract call returned RPC error',
+        details: `${callPayload.error.message}. Kraiken contract at ${kraikenAddress} may not be deployed.`
+      };
+    }
+
+    if (!callPayload?.result) {
+      return {
+        success: false,
+        service,
+        message: 'Contract call returned no result',
+        details: 'eth_call responded but returned no data. Contract may not be deployed.'
+      };
+    }
+
+    // Verify we got a valid response (should be a hex-encoded uint256)
+    const result = callPayload.result as string;
+    if (!result.startsWith('0x')) {
+      return {
+        success: false,
+        service,
+        message: 'Contract call returned invalid data',
+        details: `Expected hex-encoded value, got: ${result}`
+      };
+    }
+
+    return {
+      success: true,
+      service,
+      message: 'RPC proxy is functional and contracts are accessible'
+    };
+
+  } catch (error) {
+    const errorMessage = error instanceof Error ? error.message : String(error);
+    return {
+      success: false,
+      service,
+      message: 'RPC health check failed with exception',
+      details: `${errorMessage}. Check if RPC endpoint ${rpcUrl} is accessible.`
+    };
+  }
+}
+
+/**
+ * Verify GraphQL has indexed contract data by:
+ * 1. Checking GraphQL endpoint responds
+ * 2. Querying for stats to verify Ponder has indexed events
+ * 3. Confirming indexed data is non-zero (indicates successful indexing)
+ */
+export async function checkGraphqlIndexed(graphqlUrl: string): Promise<HealthCheckResult> {
+  const service = 'GraphQL Indexer';
+
+  try {
+    // Query for stats which should be populated after indexing
+    const response = await fetch(graphqlUrl, {
+      method: 'POST',
+      headers: { 'content-type': 'application/json' },
+      body: JSON.stringify({
+        query: `
+          query {
+            stats(id: "0x01") {
+              kraikenTotalSupply
+              stakeTotalSupply
+            }
+          }
+        `
+      }),
+    });
+
+    if (!response.ok) {
+      return {
+        success: false,
+        service,
+        message: `GraphQL endpoint returned HTTP ${response.status}`,
+        details: `Expected 200, got ${response.status}. Check if Ponder is running.`
+      };
+    }
+
+    const payload = await response.json();
+
+    if (payload.errors) {
+      const errorMsg = payload.errors.map((e: { message: string }) => e.message).join(', ');
+      return {
+        success: false,
+        service,
+        message: 'GraphQL query returned errors',
+        details: `${errorMsg}. Check Ponder logs for indexing issues.`
+      };
+    }
+
+    // Verify stats data exists
+    const stats = payload?.data?.stats;
+    if (!stats) {
+      return {
+        success: false,
+        service,
+        message: 'GraphQL has no indexed data yet',
+        details: 'Ponder is running but has not indexed contract events. Wait longer or check Ponder logs.'
+      };
+    }
+
+    // Verify we have actual indexed data (non-zero supply indicates indexing happened)
+    const kraikenSupply = BigInt(stats.kraikenTotalSupply || '0');
+    if (kraikenSupply === 0n) {
+      return {
+        success: false,
+        service,
+        message: 'GraphQL indexed data is empty',
+        details: 'Stats exist but Kraiken supply is 0. Indexing may not have completed or contracts not initialized.'
+      };
+    }
+
+    return {
+      success: true,
+      service,
+      message: 'GraphQL has successfully indexed contract data'
+    };
+
+  } catch (error) {
+    const errorMessage = error instanceof Error ? error.message : String(error);
+    return {
+      success: false,
+      service,
+      message: 'GraphQL health check failed with exception',
+      details: `${errorMessage}. Check if GraphQL endpoint ${graphqlUrl} is accessible.`
+    };
+  }
+}
+
+/**
+ * Verify webapp is accessible
+ */
+export async function checkWebAppAccessible(webAppUrl: string): Promise<HealthCheckResult> {
+  const service = 'Web App';
+
+  try {
+    const url = webAppUrl.endsWith('/') ? `${webAppUrl}app/` : `${webAppUrl}/app/`;
+    const response = await fetch(url, {
+      method: 'GET',
+      redirect: 'manual',
+    });
+
+    if (!response.ok && response.status !== 308) {
+      return {
+        success: false,
+        service,
+        message: `Web app returned HTTP ${response.status}`,
+        details: `Expected 200 or 308, got ${response.status}. Check if webapp is running.`
+      };
+    }
+
+    return {
+      success: true,
+      service,
+      message: 'Web app is accessible'
+    };
+
+  } catch (error) {
+    const errorMessage = error instanceof Error ? error.message : String(error);
+    return {
+      success: false,
+      service,
+      message: 'Web app health check failed',
+      details: `${errorMessage}. Check if webapp endpoint ${webAppUrl} is accessible.`
+    };
+  }
+}
+
+/**
+ * Run all functional health checks and return detailed results
+ */
+export async function runAllHealthChecks(options: {
+  rpcUrl: string;
+  webAppUrl: string;
+  graphqlUrl: string;
+}): Promise<HealthCheckResult[]> {
+  const results = await Promise.all([
+    checkRpcFunctional(options.rpcUrl),
+    checkWebAppAccessible(options.webAppUrl),
+    checkGraphqlIndexed(options.graphqlUrl),
+  ]);
+
+  return results;
+}
+
+/**
+ * Format health check results into a readable error message
+ */
+export function formatHealthCheckError(results: HealthCheckResult[]): string {
+  const failures = results.filter(r => !r.success);
+
+  if (failures.length === 0) {
+    return 'All health checks passed';
+  }
+
+  const lines: string[] = [
+    '❌ Stack health check failed',
+    '',
+    'Failed services:',
+  ];
+
+  for (const failure of failures) {
+    lines.push(`  • ${failure.service}: ${failure.message}`);
+    if (failure.details) {
+      lines.push(`    ${failure.details}`);
+    }
+  }
+
+  lines.push('');
+  lines.push('Troubleshooting:');
+  lines.push('  1. Check stack logs: tail tests/.stack.log');
+  lines.push('  2. Verify services are running: ./scripts/dev.sh status');
+  lines.push('  3. Restart stack: ./scripts/dev.sh restart --full');
+
+  return lines.join('\n');
+}
--- a/tests/setup/stack.ts
+++ b/tests/setup/stack.ts
@ -3,6 +3,11 @@ import { readFile } from 'node:fs/promises';
 import { dirname, resolve } from 'node:path';
 import { fileURLToPath } from 'node:url';
 import { promisify } from 'node:util';
+import {
+  runAllHealthChecks,
+  formatHealthCheckError,
+  type HealthCheckResult,
+} from './health-checks.js';

 const exec = promisify(execCallback);

@ -33,44 +38,19 @@ async function run(command: string): Promise<void> {
  await exec(command, { cwd: repoRoot, shell: true });
 }

-async function checkRpcReady(rpcUrl: string): Promise<void> {
-  const response = await fetch(rpcUrl, {
-    method: 'POST',
-    headers: { 'content-type': 'application/json' },
-    body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'eth_chainId', params: [] }),
-  });
+/**
+ * Run functional health checks that verify services are actually usable, not just running
+ */
+async function checkStackFunctional(
+  rpcUrl: string,
+  webAppUrl: string,
+  graphqlUrl: string
+): Promise<void> {
+  const results = await runAllHealthChecks({ rpcUrl, webAppUrl, graphqlUrl });

-  if (!response.ok) {
-    throw new Error(`RPC health check failed with status ${response.status}`);
-  }
-
-  const payload = await response.json();
-  if (!payload?.result) {
-    throw new Error('RPC health check returned no result');
-  }
-}
-
-async function checkWebAppReady(webAppUrl: string): Promise<void> {
-  const url = webAppUrl.endsWith('/') ? `${webAppUrl}app/` : `${webAppUrl}/app/`;
-  const response = await fetch(url, {
-    method: 'GET',
-    redirect: 'manual',
-  });
-
-  if (!response.ok && response.status !== 308) {
-    throw new Error(`Web app health check failed with status ${response.status}`);
-  }
-}
-
-async function checkGraphqlReady(graphqlUrl: string): Promise<void> {
-  const response = await fetch(graphqlUrl, {
-    method: 'POST',
-    headers: { 'content-type': 'application/json' },
-    body: JSON.stringify({ query: '{ __typename }' }),
-  });
-
-  if (!response.ok) {
-    throw new Error(`GraphQL health check failed with status ${response.status}`);
+  const failures = results.filter(r => !r.success);
+  if (failures.length > 0) {
+    throw new Error(formatHealthCheckError(results));
  }
 }

@ -103,11 +83,8 @@ export async function waitForStackReady(options: {

  while (Date.now() - start < timeoutMs) {
    try {
-      await Promise.all([
-        checkRpcReady(rpcUrl),
-        checkWebAppReady(webAppUrl),
-        checkGraphqlReady(graphqlUrl),
-      ]);
+      await checkStackFunctional(rpcUrl, webAppUrl, graphqlUrl);
+      console.log('✅ All stack health checks passed');
      return;
    } catch (error) {
      const message = error instanceof Error ? error.message : String(error);