4277f19b68
Co-authored-by: openhands <openhands@all-hands.dev> Reviewed-on: https://codeberg.org/johba/harb/pulls/84
240 lines
6.7 KiB
Markdown
240 lines
6.7 KiB
Markdown
# Migration Status Report
|
|
|
|
**Date**: 2025-11-20
|
|
**Branch**: feature/ci
|
|
**Commit**: 8c6b6c4
|
|
|
|
## ✅ Completed Steps
|
|
|
|
### 1. Podman → Docker Migration ✅
|
|
- Updated `/etc/woodpecker/agent.env` to use Docker socket
|
|
- Added `ci` user to `docker` group
|
|
- Restarted Woodpecker agent
|
|
- **Verified**: Agent running successfully with Docker backend
|
|
|
|
### 2. Composite Integration Service Created ✅
|
|
- Created `docker/Dockerfile.integration` (self-contained stack image)
|
|
- Created `docker/integration-entrypoint.sh` (orchestration script)
|
|
- Created `docker-compose.ci.yml` (local testing interface)
|
|
- Created `scripts/build-integration-image.sh` (build automation)
|
|
- Created refactored `.woodpecker/e2e.yml` pipeline
|
|
|
|
### 3. Integration Image Built ✅
|
|
- **Image**: `registry.sovraigns.network/harb/integration:latest`
|
|
- **Size**: 515MB (491MB compressed)
|
|
- **Status**: Built locally, ready for push
|
|
- **Build time**: ~45 seconds
|
|
|
|
### 4. Pipeline Activated ✅
|
|
- Backed up old E2E pipeline to `.woodpecker/e2e-old.yml`
|
|
- Activated new pipeline in `.woodpecker/e2e.yml`
|
|
- All changes committed to git
|
|
|
|
### 5. Documentation Created ✅
|
|
- `CI_MIGRATION.md` - Complete technical documentation
|
|
- `MIGRATION_SUMMARY.md` - Executive summary
|
|
- `QUICKSTART_MIGRATION.md` - Step-by-step testing guide
|
|
- `MIGRATION_STATUS.md` - This file
|
|
|
|
---
|
|
|
|
## ⚠️ Remaining Actions
|
|
|
|
### Action 1: Push Integration Image to Registry
|
|
|
|
**Status**: Blocked - requires registry authentication
|
|
|
|
**What to do**:
|
|
```bash
|
|
# Option A: Login with credentials (requires password)
|
|
docker login registry.sovraigns.network -u ciuser
|
|
# Password: <ask admin>
|
|
|
|
# Option B: Build image in CI (recommended)
|
|
# The E2E pipeline can build the image on first run
|
|
# Add a build step before the service in e2e.yml
|
|
```
|
|
|
|
**Recommendation**: For now, let the CI build the image on first run. This tests the full build process in CI and doesn't require manual registry access.
|
|
|
|
### Action 2: Test New E2E Pipeline
|
|
|
|
**Options**:
|
|
|
|
**A. Let CI build image (recommended)**
|
|
1. Add build step to `.woodpecker/e2e.yml`:
|
|
```yaml
|
|
steps:
|
|
- name: build-integration-image
|
|
image: docker:27-dind
|
|
privileged: true
|
|
environment:
|
|
DOCKER_HOST: tcp://docker:2375
|
|
commands:
|
|
- ./scripts/build-integration-image.sh
|
|
- docker tag registry.sovraigns.network/harb/integration:latest harb-integration:local
|
|
|
|
services:
|
|
- name: stack
|
|
image: harb-integration:local # Use locally built image
|
|
...
|
|
```
|
|
|
|
**B. Push image manually (requires sudo/password)**
|
|
```bash
|
|
# Get registry password from admin or check htpasswd
|
|
docker login registry.sovraigns.network -u ciuser
|
|
docker push registry.sovraigns.network/harb/integration:latest
|
|
```
|
|
|
|
**C. Test locally first**
|
|
```bash
|
|
# Start the integration container
|
|
docker run --rm --privileged -p 8081:8081 \
|
|
registry.sovraigns.network/harb/integration:latest
|
|
|
|
# In another terminal, wait for healthy
|
|
timeout 300 sh -c 'until curl -sf http://localhost:8081/api/graphql; do sleep 5; done'
|
|
|
|
# Run E2E tests
|
|
npm run test:e2e
|
|
```
|
|
|
|
---
|
|
|
|
## Current State
|
|
|
|
### Files Changed (10 files, +1067/-97 lines)
|
|
```
|
|
M .dockerignore (updated to exclude more build artifacts)
|
|
A .woodpecker/e2e-old.yml (backup of old DinD pipeline)
|
|
M .woodpecker/e2e.yml (new composite service pipeline)
|
|
A CI_MIGRATION.md (technical documentation)
|
|
A MIGRATION_SUMMARY.md (executive summary)
|
|
A QUICKSTART_MIGRATION.md (testing guide)
|
|
A docker-compose.ci.yml (local testing interface)
|
|
A docker/Dockerfile.integration (integration image)
|
|
A docker/integration-entrypoint.sh (entrypoint script)
|
|
A scripts/build-integration-image.sh (build automation)
|
|
```
|
|
|
|
### Commit Hash
|
|
```
|
|
8c6b6c4 - ci: migrate to composite integration service + Docker backend
|
|
```
|
|
|
|
### Branch
|
|
```
|
|
feature/ci
|
|
```
|
|
|
|
---
|
|
|
|
## Next Steps (Choose One)
|
|
|
|
### Option A: Build in CI (Recommended)
|
|
1. Modify `.woodpecker/e2e.yml` to add build step (see above)
|
|
2. Commit change
|
|
3. Push to remote
|
|
4. Watch CI build and test
|
|
|
|
**Pros**: Tests full CI build process, no registry credentials needed
|
|
**Cons**: First run will be slower (~5-10 min extra)
|
|
|
|
### Option B: Push Image Manually
|
|
1. Get registry password from admin
|
|
2. `docker login registry.sovraigns.network -u ciuser`
|
|
3. `docker push registry.sovraigns.network/harb/integration:latest`
|
|
4. Push branch to remote
|
|
5. Watch CI test
|
|
|
|
**Pros**: Faster first CI run
|
|
**Cons**: Requires registry credentials
|
|
|
|
### Option C: Local Test First
|
|
1. Run integration container locally (see commands above)
|
|
2. Run E2E tests against it
|
|
3. Verify everything works
|
|
4. Then proceed with Option A or B
|
|
|
|
**Pros**: Catch issues before CI
|
|
**Cons**: Takes more time upfront
|
|
|
|
---
|
|
|
|
## Performance Expectations
|
|
|
|
### Old Pipeline (DinD)
|
|
- Stack startup: ~180-240s
|
|
- Total E2E: ~8-10 minutes
|
|
- Complexity: High (nested containers)
|
|
|
|
### New Pipeline (Composite)
|
|
- Stack startup: ~60-90s (if image pre-built) OR ~5-10 min (first build)
|
|
- Total E2E: ~5-6 minutes (after first build)
|
|
- Complexity: Low (single service)
|
|
|
|
### After First CI Run
|
|
- **Image cached**: Subsequent runs will be fast (~5-6 min total)
|
|
- **Improvement**: ~3-5 minutes faster per run
|
|
- **Simplification**: 1 service instead of DinD + 8 nested containers
|
|
|
|
---
|
|
|
|
## Rollback Instructions
|
|
|
|
If something goes wrong:
|
|
|
|
```bash
|
|
# Restore old E2E pipeline
|
|
git checkout HEAD~1 .woodpecker/e2e.yml
|
|
|
|
# Or manually
|
|
mv .woodpecker/e2e-old.yml .woodpecker/e2e.yml
|
|
|
|
# Commit and push
|
|
git add .woodpecker/e2e.yml
|
|
git commit -m "ci: rollback to DinD E2E pipeline"
|
|
git push
|
|
```
|
|
|
|
To rollback Podman migration (requires sudo):
|
|
```bash
|
|
# Edit agent config
|
|
sudo nano /etc/woodpecker/agent.env
|
|
# Change: WOODPECKER_BACKEND_DOCKER_HOST=unix:///run/user/1001/podman/podman.sock
|
|
|
|
# Restart agent
|
|
sudo systemctl restart woodpecker-agent
|
|
```
|
|
|
|
---
|
|
|
|
## Success Criteria
|
|
|
|
- [x] Podman → Docker migration complete
|
|
- [x] Integration Dockerfile created
|
|
- [x] docker-compose.ci.yml created
|
|
- [x] Build script created
|
|
- [x] New E2E pipeline created
|
|
- [x] Documentation complete
|
|
- [x] Integration image builds successfully
|
|
- [ ] Image pushed to registry OR build-in-CI implemented
|
|
- [ ] CI E2E pipeline tested and passing
|
|
- [ ] Performance improvement verified (~3-5 min faster)
|
|
|
|
**Current Status**: 8/10 complete - Ready for final testing
|
|
|
|
---
|
|
|
|
## Recommendation
|
|
|
|
I recommend **Option A (Build in CI)** because:
|
|
1. No registry credentials needed
|
|
2. Tests the full build process in CI environment
|
|
3. Image will be cached for subsequent runs
|
|
4. First run will validate everything works end-to-end
|
|
|
|
The only downside is the first run will take longer (~5-10 min extra for image build), but all subsequent runs will be much faster.
|
|
|
|
Would you like me to modify the E2E pipeline to build the image in CI?
|