diff --git a/triage/build-system-description.md b/triage/build-system-description.md new file mode 100644 index 00000000..d4963978 --- /dev/null +++ b/triage/build-system-description.md @@ -0,0 +1,203 @@ +# 4coder Build System Analysis + +## Overview + +The 4coder build system is a sophisticated multi-tier architecture designed to handle cross-platform compilation for Windows, Linux, and macOS. The system consists of three main components: + +1. **Meta-Build System**: A C++ program that generates platform-specific build commands +2. **Platform-Specific Entry Scripts**: Shell/batch scripts that compile and execute the meta-build program +3. **Custom Layer Build System**: Separate compilation process for the customizable editor layer + +## Architecture Deep Dive + +### 1. Meta-Build System (`code/bin/4ed_build.cpp`) + +The core of the build system is a 720-line C++ program that serves as a build configuration generator. Key features: + +- **Platform Detection**: Automatically detects Windows, Linux, or macOS +- **Compiler Support**: Handles CL (MSVC), GCC, and Clang compilers +- **Architecture Support**: Supports both x64 and x86 architectures +- **Build Tiers**: Manages Demo vs Super build configurations +- **Flag Management**: Centralized handling of compilation flags and options + +#### Key Data Structures: +```cpp +typedef u32 Platform_Code; +enum { Platform_Windows, Platform_Linux, Platform_Mac }; + +typedef u32 Compiler_Code; +enum { Compiler_CL, Compiler_GCC, Compiler_Clang }; + +typedef u32 Arch_Code; +enum { Arch_X64, Arch_X86 }; + +typedef u32 Tier_Code; +enum { Tier_Demo, Tier_Super }; +``` + +#### Build Flags System: +```cpp +enum { + OPTS = 0x1, // Compiler warnings/options + LIBS = 0x2, // Link libraries + ICON = 0x4, // Windows icon resource + SHARED_CODE = 0x8, // Build as shared library + DEBUG_INFO = 0x10, // Debug symbols + OPTIMIZATION = 0x20, // Optimization level + SUPER = 0x40, // Super tier features + INTERNAL = 0x80, // Internal build flags + SHIP = 0x100, // Shipping build +}; +``` + +### 2. Platform-Specific Entry Scripts + +Each platform has dedicated build scripts that: +1. Compile the meta-build system (`4ed_build.cpp`) +2. Execute the compiled meta-build program with appropriate flags +3. Handle platform-specific compiler setup + +#### macOS (`code/bin/build-mac-dev.sh`): +```bash +#!/bin/bash +g++ -std=c++11 -DDEV_BUILD -Wno-write-strings -Wno-comment -Wno-null-dereference -Wno-logical-op-parentheses -Wno-switch -o ../build/4ed_build 4ed_build.cpp +../build/4ed_build $1 +``` + +#### Linux (`code/bin/build-linux.sh`): +```bash +#!/bin/bash +g++ -g -std=c++11 -DDEV_BUILD -Wno-write-strings -Wno-comment -Wno-null-dereference -Wno-logical-op-parentheses -Wno-switch -o ../build/4ed_build 4ed_build.cpp +../build/4ed_build $1 +``` + +#### Windows (`code/bin/build.bat`): +```batch +@echo off +IF NOT EXIST ..\build mkdir ..\build +pushd ..\build +cl /I..\code /DDEV_BUILD /std:c++11 /Zi /Fe:4ed_build.exe ..\code\bin\4ed_build.cpp /link /INCREMENTAL:NO /RELEASE > NUL +4ed_build.exe %1 +popd +``` + +### 3. Custom Layer Build System + +The custom layer (editor behavior/commands) uses a sophisticated multi-stage build process: + +#### Stage 1: Preprocessing with Meta Macros +```bash +clang++ -I../code -I../code/custom -I../code/custom/generated -DMETA_PASS -E 4coder_default_bindings.cpp -o 4coder_default_bindings.i +``` + +#### Stage 2: Metadata Generator Compilation +```bash +clang++ -I../code -I../code/custom -I../code/custom/generated -o metadata_generator 4coder_metadata_generator.cpp +``` + +#### Stage 3: Metadata Generation +```bash +./metadata_generator -R ../code/custom/generated/ 4coder_default_bindings.i +``` + +#### Stage 4: Final Custom Layer Compilation +```bash +clang++ -I../code -I../code/custom -I../code/custom/generated -shared -fPIC -o custom_4coder.so 4coder_default_bindings.cpp +``` + +### 4. Compiler-Specific Configurations + +#### Visual Studio (CL): +```cpp +#define CL_OPTS \ +"-W4 -wd4310 -wd4100 -wd4201 -wd4505 -wd4996 " \ +"-wd4127 -wd4510 -wd4512 -wd4610 -wd4390 " \ +"-wd4611 -wd4189 -WX -GR- -EHa- -nologo -FC" + +#define CL_LIBS_COMMON \ +"user32.lib winmm.lib gdi32.lib opengl32.lib comdlg32.lib userenv.lib " +``` + +#### GCC (Linux): +```cpp +#define GCC_OPTS \ +"-Wno-write-strings " \ +"-D_GNU_SOURCE -fPIC " \ +"-fno-threadsafe-statics -pthread " \ +"-Wno-unused-result " \ +"-std=c++11" + +#define GCC_LIBS_COMMON \ +"-lX11 -lpthread -lm -lrt " \ +"-lGL -ldl -lXfixes -lfreetype -lfontconfig" +``` + +#### Clang (macOS): +```cpp +#define CLANG_OPTS \ +"-Wno-write-strings -Wno-deprecated-declarations " \ +"-Wno-comment -Wno-switch -Wno-null-dereference " \ +"-Wno-tautological-compare -Wno-unused-result " \ +"-Wno-missing-declarations -Wno-nullability-completeness " \ +"-std=c++11 " + +#define CLANG_LIBS_COMMON \ +"-framework Cocoa -framework QuartzCore " \ +"-framework CoreServices " \ +"-framework OpenGL -framework IOKit -framework Metal -framework MetalKit " +``` + +### 5. Build Process Flow + +#### Development Build Flow: +1. **Entry Script** (platform-specific) compiles `4ed_build.cpp` +2. **Meta-Build Program** generates compiler commands based on platform/arch/flags +3. **Custom Layer Build** (parallel process): + - Preprocesses source with meta macros + - Builds metadata generator + - Generates API bindings + - Compiles custom layer as shared library +4. **Core Engine Build**: + - Compiles `4ed_app_target.cpp` as shared library (`4ed_app.so`) + - Compiles platform layer as main executable (`4ed`) +5. **Resource Management**: Copies themes, fonts, configs to build directory + +#### Packaging Build Flow: +1. **Standard Build** (as above) +2. **Distribution Preparation**: + - Creates versioned directory structure + - Copies executables and shared libraries + - Includes source code (Super tier only) + - Copies documentation and resources +3. **Archive Creation**: Creates ZIP files for distribution + +### 6. Key Build Outputs + +The build system produces: +- **`4ed`**: Main editor executable (platform layer) +- **`4ed_app.so`**: Core application logic (shared library) +- **`custom_4coder.so`**: Custom layer with user commands (shared library) +- **Resources**: Themes, fonts, configuration files +- **Debug Symbols**: `.dSYM` directories (macOS), `.pdb` files (Windows) + +### 7. Build Complexity Analysis + +The current system is complex due to: + +1. **Multi-tier Architecture**: Three separate compilation units with interdependencies +2. **Cross-platform Abstraction**: Different compilers, flags, and linking requirements +3. **Dynamic API Generation**: Metadata generation for extensible command system +4. **Packaging Requirements**: Multiple build tiers and distribution formats +5. **Debug/Release Variants**: Different optimization and debugging configurations + +### 8. Simplification Opportunities + +The build system could be simplified by: + +1. **Unified Build Scripts**: Single bash script per platform instead of meta-build system +2. **Direct Compiler Calls**: Eliminate C++ meta-build program in favor of direct clang/gcc invocations +3. **Simplified Metadata**: Reduce or eliminate runtime metadata generation +4. **Merged Compilation Units**: Combine core and custom layers into single executable +5. **Standardized Toolchain**: Use consistent compiler (clang) across all platforms + +The current complexity stems from the editor's plugin architecture and cross-platform requirements, but many of these features could be maintained with a simpler build system that uses direct bash scripts calling clang with appropriate flags. \ No newline at end of file diff --git a/triage/build-system-simplification-plan.md b/triage/build-system-simplification-plan.md new file mode 100644 index 00000000..81c70416 --- /dev/null +++ b/triage/build-system-simplification-plan.md @@ -0,0 +1,396 @@ +# 4coder Build System Simplification Plan + +## Overview + +This document outlines a comprehensive plan to replace the current complex meta-build system with simple, direct bash scripts that call clang with appropriate flags. The plan maintains all essential functionality while eliminating unnecessary complexity. + +## Goals + +1. **Simplicity**: Replace meta-build C++ program with direct bash scripts +2. **Maintainability**: Clear, readable build scripts with inline documentation +3. **Standardization**: Use clang across all platforms for consistency +4. **Functionality**: Preserve all essential build features and outputs + +## Current vs Target Architecture + +### Current System +``` +Platform Script → Compile 4ed_build.cpp → Execute 4ed_build → Generate Commands → Build +``` + +### Target System +``` +Platform Script → Direct Clang Calls → Build +``` + +## Phase 1: Foundation and Analysis (2-3 days) + +### TODO 1.1: Set Up New Build Directory Structure +- [ ] Create `build_new/` directory for new build scripts +- [ ] Create subdirectories: + - `build_new/scripts/` - Main build scripts + - `build_new/helpers/` - Helper scripts and utilities + - `build_new/config/` - Build configuration files + - `build_new/temp/` - Temporary files during build + +### TODO 1.2: Extract Essential Build Parameters +- [ ] Analyze `4ed_build.cpp` to extract all compiler flags per platform +- [ ] Document all linker flags and library dependencies +- [ ] Create configuration file with all essential build parameters: + ```bash + # build_new/config/build-config.sh + # Platform-specific compiler flags, library paths, etc. + ``` + +### TODO 1.3: Create Platform Detection Utility +- [ ] Write `build_new/helpers/detect-platform.sh`: + ```bash + #!/bin/bash + # Returns: win32, linux, or macos + ``` + +### TODO 1.4: Verify Clang Installation Requirements +- [ ] Document clang version requirements for each platform +- [ ] Create `build_new/helpers/check-toolchain.sh` to verify clang availability +- [ ] Document any additional platform-specific setup needed + +## Phase 2: Core Build Script Development (4-5 days) + +### TODO 2.1: Create Master Build Script +- [ ] Create `build_new/scripts/build.sh` as main entry point: + ```bash + #!/bin/bash + # Usage: ./build.sh [platform] [config] [arch] + # Examples: + # ./build.sh macos debug x64 + # ./build.sh linux release x64 + # ./build.sh win32 debug x64 + ``` + +### TODO 2.2: Implement Platform-Specific Build Functions +- [ ] Create `build_new/scripts/build-macos.sh`: + ```bash + #!/bin/bash + # macOS-specific build logic using clang + # Handles Objective-C++ files, frameworks, etc. + ``` +- [ ] Create `build_new/scripts/build-linux.sh`: + ```bash + #!/bin/bash + # Linux-specific build logic using clang + # Handles X11, pthread, fontconfig, etc. + ``` +- [ ] Create `build_new/scripts/build-win32.sh`: + ```bash + #!/bin/bash + # Windows-specific build logic using clang + # Handles Windows APIs, resource compilation, etc. + ``` + +### TODO 2.3: Implement Core Engine Build +- [ ] Create `build_new/scripts/build-core.sh`: + ```bash + #!/bin/bash + # Builds 4ed_app.so (core application logic) + # Builds 4ed executable (platform layer) + ``` +- [ ] Extract and implement all essential compiler flags +- [ ] Extract and implement all essential linker flags +- [ ] Handle FreeType and other external libraries + +### TODO 2.4: Implement Custom Layer Build +- [ ] Create `build_new/scripts/build-custom.sh`: + ```bash + #!/bin/bash + # Multi-stage custom layer build process + # 1. Preprocess with -DMETA_PASS + # 2. Build metadata generator + # 3. Generate metadata files + # 4. Compile custom layer shared library + ``` +- [ ] Preserve metadata generation functionality +- [ ] Preserve API binding generation + +## Phase 3: Advanced Features (3-4 days) + +### TODO 3.1: Implement Build Configuration System +- [ ] Create `build_new/config/flags-common.sh` - Common compiler flags +- [ ] Create `build_new/config/flags-debug.sh` - Debug-specific flags +- [ ] Create `build_new/config/flags-release.sh` - Release-specific flags +- [ ] Create `build_new/config/libs-[platform].sh` - Platform-specific libraries + +### TODO 3.2: Implement Resource Management +- [ ] Create `build_new/scripts/copy-resources.sh`: + ```bash + #!/bin/bash + # Copies themes, fonts, configs to build directory + # Handles platform-specific resource requirements + ``` + +### TODO 3.3: Add Build Validation +- [ ] Create `build_new/scripts/validate-build.sh`: + ```bash + #!/bin/bash + # Verifies all expected build outputs exist + # Checks library dependencies are resolved + # Validates executable functionality + ``` + +### TODO 3.4: Implement Clean Build Support +- [ ] Create `build_new/scripts/clean.sh`: + ```bash + #!/bin/bash + # Removes all build artifacts + # Supports selective cleaning (debug/release, platform-specific) + ``` + +## Phase 4: Packaging and Distribution (2-3 days) + +### TODO 4.1: Create Packaging System +- [ ] Create `build_new/scripts/package.sh`: + ```bash + #!/bin/bash + # Usage: ./package.sh [platform] [tier] [arch] + # Creates distribution archives + ``` +- [ ] Implement directory structure creation +- [ ] Implement file copying for distribution +- [ ] Implement archive creation (ZIP format) + +### TODO 4.2: Implement Version Management +- [ ] Create `build_new/scripts/version.sh`: + ```bash + #!/bin/bash + # Extracts version from 4coder_version.h + # Generates version-specific naming + ``` + +### TODO 4.3: Create Distribution Validation +- [ ] Create `build_new/scripts/validate-package.sh`: + ```bash + #!/bin/bash + # Validates package contents + # Checks all required files are present + ``` + +## Phase 5: Testing and Validation (3-4 days) + +### TODO 5.1: Create Test Suite +- [ ] Create `build_new/tests/test-build.sh`: + ```bash + #!/bin/bash + # Comprehensive build testing + # Tests all platform/config combinations + ``` +- [ ] Test debug builds on all platforms +- [ ] Test release builds on all platforms +- [ ] Test packaging on all platforms + +### TODO 5.2: Performance Comparison +- [ ] Create `build_new/tests/benchmark-build.sh`: + ```bash + #!/bin/bash + # Compares build times: old vs new system + # Measures clean build times + # Measures incremental build times + ``` + +### TODO 5.3: Regression Testing +- [ ] Verify all build outputs are identical to current system +- [ ] Test custom layer loading functionality +- [ ] Test metadata generation correctness +- [ ] Test API binding generation correctness + +## Phase 6: Integration and Migration (2-3 days) + +### TODO 6.1: Update Project Files +- [ ] Update `code/project.4coder` to use new build scripts +- [ ] Update `code/custom/project.4coder` for custom layer builds +- [ ] Update F-key bindings to use new build system + +### TODO 6.2: Create Migration Documentation +- [ ] Document differences between old and new systems +- [ ] Create migration guide for developers +- [ ] Update CLAUDE.md with new build commands + +### TODO 6.3: Backup and Transition +- [ ] Create backup of current build system +- [ ] Implement gradual transition plan +- [ ] Create rollback procedure if needed + +## Implementation Details + +### Script Structure Template + +Each build script should follow this structure: + +```bash +#!/bin/bash +set -e # Exit on error + +# Script: build-[platform].sh +# Purpose: Build 4coder for [platform] using clang +# Usage: ./build-[platform].sh [debug|release] [x64|x86] + +# ============================================================================= +# Configuration +# ============================================================================= + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +BUILD_ROOT="$SCRIPT_DIR/../../build" +CONFIG_DIR="$SCRIPT_DIR/../config" + +# Source configuration files +source "$CONFIG_DIR/build-config.sh" +source "$CONFIG_DIR/flags-common.sh" + +# ============================================================================= +# Platform-specific settings +# ============================================================================= + +setup_platform_vars() { + # Set platform-specific variables + # - Compiler flags + # - Library paths + # - Include paths + # - Linker flags +} + +# ============================================================================= +# Build functions +# ============================================================================= + +build_core_engine() { + echo "Building core engine..." + + # Document each clang call with comments + # Example: + # Build 4ed_app.so (core application logic) + clang++ \ + -shared \ + -std=c++11 \ + -I"$CODE_DIR" \ + -I"$FOREIGN_DIR/freetype2" \ + -DFTECH_64_BIT \ + -DFRED_SUPER \ + -O2 \ + -o "$BUILD_DIR/4ed_app.so" \ + "$CODE_DIR/4ed_app_target.cpp" \ + "$FOREIGN_DIR/x64/libfreetype-mac.a" +} + +build_custom_layer() { + echo "Building custom layer..." + + # Stage 1: Preprocess with meta macros + clang++ \ + -I"$CODE_DIR" \ + -I"$CUSTOM_DIR" \ + -DMETA_PASS \ + -E \ + "$CUSTOM_DIR/4coder_default_bindings.cpp" \ + -o "$BUILD_DIR/4coder_default_bindings.i" + + # Stage 2: Build metadata generator + clang++ \ + -std=c++11 \ + -I"$CODE_DIR" \ + -I"$CUSTOM_DIR" \ + -o "$BUILD_DIR/metadata_generator" \ + "$CUSTOM_DIR/4coder_metadata_generator.cpp" + + # Stage 3: Generate metadata + "$BUILD_DIR/metadata_generator" \ + -R "$CUSTOM_DIR/generated/" \ + "$BUILD_DIR/4coder_default_bindings.i" + + # Stage 4: Compile custom layer + clang++ \ + -shared \ + -std=c++11 \ + -I"$CODE_DIR" \ + -I"$CUSTOM_DIR" \ + -I"$CUSTOM_DIR/generated" \ + -DFTECH_64_BIT \ + -DFRED_SUPER \ + -O2 \ + -o "$BUILD_DIR/custom_4coder.so" \ + "$CUSTOM_DIR/4coder_default_bindings.cpp" +} + +# ============================================================================= +# Main execution +# ============================================================================= + +main() { + setup_platform_vars + create_build_dirs + build_core_engine + build_custom_layer + copy_resources + validate_build + + echo "Build completed successfully!" +} + +main "$@" +``` + +### Error Handling Strategy + +Each script should include: +- `set -e` for exit on error +- Validation of input parameters +- Checking for required tools and dependencies +- Clear error messages with suggested fixes +- Cleanup of temporary files on failure + +### Documentation Standards + +Each script should include: +- Purpose and usage in header comments +- Inline documentation for each major step +- Clear variable names and consistent formatting +- Examples of usage in comments + +## Success Criteria + +The new build system will be considered successful when: + +1. **Functionality**: All current build outputs are preserved +2. **Simplicity**: No C++ meta-build program required +3. **Maintainability**: Clear, readable bash scripts +4. **Performance**: Build times equal or better than current system +5. **Reliability**: No regression in build success rates +6. **Documentation**: Clear usage instructions and examples + +## Risks and Mitigation + +### Risk 1: Platform-specific Compatibility Issues +- **Mitigation**: Extensive testing on all target platforms +- **Fallback**: Maintain current system until new system is fully validated + +### Risk 2: Missing Build Features +- **Mitigation**: Comprehensive analysis of current system before implementation +- **Fallback**: Gradual migration with feature parity validation + +### Risk 3: Performance Regression +- **Mitigation**: Benchmark testing and optimization +- **Fallback**: Performance profiling and targeted improvements + +### Risk 4: Developer Disruption +- **Mitigation**: Clear migration documentation and parallel system support +- **Fallback**: Quick rollback procedure if issues arise + +## Timeline Summary + +- **Phase 1**: 2-3 days - Foundation and analysis +- **Phase 2**: 4-5 days - Core build script development +- **Phase 3**: 3-4 days - Advanced features +- **Phase 4**: 2-3 days - Packaging and distribution +- **Phase 5**: 3-4 days - Testing and validation +- **Phase 6**: 2-3 days - Integration and migration + +**Total Estimated Time**: 16-22 days + +This plan provides a comprehensive roadmap for replacing the complex meta-build system with simple, maintainable bash scripts while preserving all essential functionality and ensuring a smooth transition for developers. \ No newline at end of file