4coder/triage/build-system-simplification-plan.md

# 4coder Build System Simplification Plan

## Overview

This document outlines a comprehensive plan to replace the current complex meta-build system with simple, direct bash scripts that call clang with appropriate flags. The plan maintains all essential functionality while eliminating unnecessary complexity.

## Goals

1. **Simplicity**: Replace meta-build C++ program with direct bash scripts
2. **Maintainability**: Clear, readable build scripts with inline documentation
3. **Standardization**: Use clang across all platforms for consistency
4. **Functionality**: Preserve all essential build features and outputs

## Current vs Target Architecture

### Current System
```
Platform Script → Compile 4ed_build.cpp → Execute 4ed_build → Generate Commands → Build
```

### Target System
```
Platform Script → Direct Clang Calls → Build
```

## Phase 1: Foundation and Analysis (2-3 days) ✅ COMPLETED

### TODO 1.1: Set Up New Build Directory Structure ✅
- [x] Create `build_new/` directory for new build scripts
- [x] Create subdirectories:
  - `build_new/scripts/` - Main build scripts
  - `build_new/helpers/` - Helper scripts and utilities
  - `build_new/config/` - Build configuration files
  - `build_new/temp/` - Temporary files during build

### TODO 1.2: Extract Essential Build Parameters ✅
- [x] Analyze `4ed_build.cpp` to extract all compiler flags per platform
- [x] Document all linker flags and library dependencies
- [x] Create configuration file with all essential build parameters:
  ```bash
  # build_new/config/build-config.sh
  # Platform-specific compiler flags, library paths, etc.
  ```

### TODO 1.3: Create Platform Detection Utility ✅
- [x] Write `build_new/helpers/detect-platform.sh`:
  ```bash
  #!/bin/bash
  # Returns: win32, linux, or macos
  ```

### TODO 1.4: Verify Clang Installation Requirements ✅
- [x] Document clang version requirements for each platform
- [x] Create `build_new/helpers/check-toolchain.sh` to verify clang availability
- [x] Document any additional platform-specific setup needed

**Phase 1 Results:**
- ✅ Build directory structure created
- ✅ Essential build parameters extracted and documented in `build_new/config/build-config.sh`
- ✅ Platform detection utility working (`build_new/helpers/detect-platform.sh`)
- ✅ Toolchain verification script working (`build_new/helpers/check-toolchain.sh`)
- ✅ Current environment verified: macOS with clang 16.0, all dependencies available

## Phase 2: Core Build Script Development (4-5 days)

### TODO 2.1: Create Master Build Script
- [ ] Create `build_new/scripts/build.sh` as main entry point:
  ```bash
  #!/bin/bash
  # Usage: ./build.sh [platform] [config] [arch]
  # Examples: 
  #   ./build.sh macos debug x64
  #   ./build.sh linux release x64
  #   ./build.sh win32 debug x64
  ```

### TODO 2.2: Implement Platform-Specific Build Functions
- [ ] Create `build_new/scripts/build-macos.sh`:
  ```bash
  #!/bin/bash
  # macOS-specific build logic using clang
  # Handles Objective-C++ files, frameworks, etc.
  ```
- [ ] Create `build_new/scripts/build-linux.sh`:
  ```bash
  #!/bin/bash
  # Linux-specific build logic using clang
  # Handles X11, pthread, fontconfig, etc.
  ```
- [ ] Create `build_new/scripts/build-win32.sh`:
  ```bash
  #!/bin/bash
  # Windows-specific build logic using clang
  # Handles Windows APIs, resource compilation, etc.
  ```

### TODO 2.3: Implement Core Engine Build
- [ ] Create `build_new/scripts/build-core.sh`:
  ```bash
  #!/bin/bash
  # Builds 4ed_app.so (core application logic)
  # Builds 4ed executable (platform layer)
  ```
- [ ] Extract and implement all essential compiler flags
- [ ] Extract and implement all essential linker flags
- [ ] Handle FreeType and other external libraries

### TODO 2.4: Implement Custom Layer Build
- [ ] Create `build_new/scripts/build-custom.sh`:
  ```bash
  #!/bin/bash
  # Multi-stage custom layer build process
  # 1. Preprocess with -DMETA_PASS
  # 2. Build metadata generator
  # 3. Generate metadata files
  # 4. Compile custom layer shared library
  ```
- [ ] Preserve metadata generation functionality
- [ ] Preserve API binding generation

## Phase 3: Advanced Features (3-4 days)

### TODO 3.1: Implement Build Configuration System
- [ ] Create `build_new/config/flags-common.sh` - Common compiler flags
- [ ] Create `build_new/config/flags-debug.sh` - Debug-specific flags
- [ ] Create `build_new/config/flags-release.sh` - Release-specific flags
- [ ] Create `build_new/config/libs-[platform].sh` - Platform-specific libraries

### TODO 3.2: Implement Resource Management
- [ ] Create `build_new/scripts/copy-resources.sh`:
  ```bash
  #!/bin/bash
  # Copies themes, fonts, configs to build directory
  # Handles platform-specific resource requirements
  ```

### TODO 3.3: Add Build Validation
- [ ] Create `build_new/scripts/validate-build.sh`:
  ```bash
  #!/bin/bash
  # Verifies all expected build outputs exist
  # Checks library dependencies are resolved
  # Validates executable functionality
  ```

### TODO 3.4: Implement Clean Build Support
- [ ] Create `build_new/scripts/clean.sh`:
  ```bash
  #!/bin/bash
  # Removes all build artifacts
  # Supports selective cleaning (debug/release, platform-specific)
  ```

## Phase 4: Packaging and Distribution (2-3 days)

### TODO 4.1: Create Packaging System
- [ ] Create `build_new/scripts/package.sh`:
  ```bash
  #!/bin/bash
  # Usage: ./package.sh [platform] [tier] [arch]
  # Creates distribution archives
  ```
- [ ] Implement directory structure creation
- [ ] Implement file copying for distribution
- [ ] Implement archive creation (ZIP format)

### TODO 4.2: Implement Version Management
- [ ] Create `build_new/scripts/version.sh`:
  ```bash
  #!/bin/bash
  # Extracts version from 4coder_version.h
  # Generates version-specific naming
  ```

### TODO 4.3: Create Distribution Validation
- [ ] Create `build_new/scripts/validate-package.sh`:
  ```bash
  #!/bin/bash
  # Validates package contents
  # Checks all required files are present
  ```

## Phase 5: Testing and Validation (3-4 days)

### TODO 5.1: Create Test Suite
- [ ] Create `build_new/tests/test-build.sh`:
  ```bash
  #!/bin/bash
  # Comprehensive build testing
  # Tests all platform/config combinations
  ```
- [ ] Test debug builds on all platforms
- [ ] Test release builds on all platforms
- [ ] Test packaging on all platforms

### TODO 5.2: Performance Comparison
- [ ] Create `build_new/tests/benchmark-build.sh`:
  ```bash
  #!/bin/bash
  # Compares build times: old vs new system
  # Measures clean build times
  # Measures incremental build times
  ```

### TODO 5.3: Regression Testing
- [ ] Verify all build outputs are identical to current system
- [ ] Test custom layer loading functionality
- [ ] Test metadata generation correctness
- [ ] Test API binding generation correctness

## Phase 6: Integration and Migration (2-3 days)

### TODO 6.1: Update Project Files
- [ ] Update `code/project.4coder` to use new build scripts
- [ ] Update `code/custom/project.4coder` for custom layer builds
- [ ] Update F-key bindings to use new build system

### TODO 6.2: Create Migration Documentation
- [ ] Document differences between old and new systems
- [ ] Create migration guide for developers
- [ ] Update CLAUDE.md with new build commands

### TODO 6.3: Backup and Transition
- [ ] Create backup of current build system
- [ ] Implement gradual transition plan
- [ ] Create rollback procedure if needed

## Implementation Details

### Script Structure Template

Each build script should follow this structure:

```bash
#!/bin/bash
set -e  # Exit on error

# Script: build-[platform].sh
# Purpose: Build 4coder for [platform] using clang
# Usage: ./build-[platform].sh [debug|release] [x64|x86]

# =============================================================================
# Configuration
# =============================================================================

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
BUILD_ROOT="$SCRIPT_DIR/../../build"
CONFIG_DIR="$SCRIPT_DIR/../config"

# Source configuration files
source "$CONFIG_DIR/build-config.sh"
source "$CONFIG_DIR/flags-common.sh"

# =============================================================================
# Platform-specific settings
# =============================================================================

setup_platform_vars() {
    # Set platform-specific variables
    # - Compiler flags
    # - Library paths
    # - Include paths
    # - Linker flags
}

# =============================================================================
# Build functions
# =============================================================================

build_core_engine() {
    echo "Building core engine..."
    
    # Document each clang call with comments
    # Example:
    # Build 4ed_app.so (core application logic)
    clang++ \
        -shared \
        -std=c++11 \
        -I"$CODE_DIR" \
        -I"$FOREIGN_DIR/freetype2" \
        -DFTECH_64_BIT \
        -DFRED_SUPER \
        -O2 \
        -o "$BUILD_DIR/4ed_app.so" \
        "$CODE_DIR/4ed_app_target.cpp" \
        "$FOREIGN_DIR/x64/libfreetype-mac.a"
}

build_custom_layer() {
    echo "Building custom layer..."
    
    # Stage 1: Preprocess with meta macros
    clang++ \
        -I"$CODE_DIR" \
        -I"$CUSTOM_DIR" \
        -DMETA_PASS \
        -E \
        "$CUSTOM_DIR/4coder_default_bindings.cpp" \
        -o "$BUILD_DIR/4coder_default_bindings.i"
    
    # Stage 2: Build metadata generator
    clang++ \
        -std=c++11 \
        -I"$CODE_DIR" \
        -I"$CUSTOM_DIR" \
        -o "$BUILD_DIR/metadata_generator" \
        "$CUSTOM_DIR/4coder_metadata_generator.cpp"
    
    # Stage 3: Generate metadata
    "$BUILD_DIR/metadata_generator" \
        -R "$CUSTOM_DIR/generated/" \
        "$BUILD_DIR/4coder_default_bindings.i"
    
    # Stage 4: Compile custom layer
    clang++ \
        -shared \
        -std=c++11 \
        -I"$CODE_DIR" \
        -I"$CUSTOM_DIR" \
        -I"$CUSTOM_DIR/generated" \
        -DFTECH_64_BIT \
        -DFRED_SUPER \
        -O2 \
        -o "$BUILD_DIR/custom_4coder.so" \
        "$CUSTOM_DIR/4coder_default_bindings.cpp"
}

# =============================================================================
# Main execution
# =============================================================================

main() {
    setup_platform_vars
    create_build_dirs
    build_core_engine
    build_custom_layer
    copy_resources
    validate_build
    
    echo "Build completed successfully!"
}

main "$@"
```

### Error Handling Strategy

Each script should include:
- `set -e` for exit on error
- Validation of input parameters
- Checking for required tools and dependencies
- Clear error messages with suggested fixes
- Cleanup of temporary files on failure

### Documentation Standards

Each script should include:
- Purpose and usage in header comments
- Inline documentation for each major step
- Clear variable names and consistent formatting
- Examples of usage in comments

## Success Criteria

The new build system will be considered successful when:

1. **Functionality**: All current build outputs are preserved
2. **Simplicity**: No C++ meta-build program required
3. **Maintainability**: Clear, readable bash scripts
4. **Performance**: Build times equal or better than current system
5. **Reliability**: No regression in build success rates
6. **Documentation**: Clear usage instructions and examples

## Risks and Mitigation

### Risk 1: Platform-specific Compatibility Issues
- **Mitigation**: Extensive testing on all target platforms
- **Fallback**: Maintain current system until new system is fully validated

### Risk 2: Missing Build Features
- **Mitigation**: Comprehensive analysis of current system before implementation
- **Fallback**: Gradual migration with feature parity validation

### Risk 3: Performance Regression
- **Mitigation**: Benchmark testing and optimization
- **Fallback**: Performance profiling and targeted improvements

### Risk 4: Developer Disruption
- **Mitigation**: Clear migration documentation and parallel system support
- **Fallback**: Quick rollback procedure if issues arise

## Timeline Summary

- **Phase 1**: 2-3 days - Foundation and analysis
- **Phase 2**: 4-5 days - Core build script development
- **Phase 3**: 3-4 days - Advanced features
- **Phase 4**: 2-3 days - Packaging and distribution
- **Phase 5**: 3-4 days - Testing and validation
- **Phase 6**: 2-3 days - Integration and migration

**Total Estimated Time**: 16-22 days

This plan provides a comprehensive roadmap for replacing the complex meta-build system with simple, maintainable bash scripts while preserving all essential functionality and ensuring a smooth transition for developers.