4coder/triage/build-system-description.md

7.1 KiB

4coder Build System Analysis

Overview

The 4coder build system is a sophisticated multi-tier architecture designed to handle cross-platform compilation for Windows, Linux, and macOS. The system consists of three main components:

  1. Meta-Build System: A C++ program that generates platform-specific build commands
  2. Platform-Specific Entry Scripts: Shell/batch scripts that compile and execute the meta-build program
  3. Custom Layer Build System: Separate compilation process for the customizable editor layer

Architecture Deep Dive

1. Meta-Build System (code/bin/4ed_build.cpp)

The core of the build system is a 720-line C++ program that serves as a build configuration generator. Key features:

  • Platform Detection: Automatically detects Windows, Linux, or macOS
  • Compiler Support: Handles CL (MSVC), GCC, and Clang compilers
  • Architecture Support: Supports both x64 and x86 architectures
  • Build Tiers: Manages Demo vs Super build configurations
  • Flag Management: Centralized handling of compilation flags and options

Key Data Structures:

typedef u32 Platform_Code;
enum { Platform_Windows, Platform_Linux, Platform_Mac };

typedef u32 Compiler_Code;
enum { Compiler_CL, Compiler_GCC, Compiler_Clang };

typedef u32 Arch_Code;
enum { Arch_X64, Arch_X86 };

typedef u32 Tier_Code;
enum { Tier_Demo, Tier_Super };

Build Flags System:

enum {
    OPTS = 0x1,          // Compiler warnings/options
    LIBS = 0x2,          // Link libraries
    ICON = 0x4,          // Windows icon resource
    SHARED_CODE = 0x8,   // Build as shared library
    DEBUG_INFO = 0x10,   // Debug symbols
    OPTIMIZATION = 0x20, // Optimization level
    SUPER = 0x40,        // Super tier features
    INTERNAL = 0x80,     // Internal build flags
    SHIP = 0x100,        // Shipping build
};

2. Platform-Specific Entry Scripts

Each platform has dedicated build scripts that:

  1. Compile the meta-build system (4ed_build.cpp)
  2. Execute the compiled meta-build program with appropriate flags
  3. Handle platform-specific compiler setup

macOS (code/bin/build-mac-dev.sh):

#!/bin/bash
g++ -std=c++11 -DDEV_BUILD -Wno-write-strings -Wno-comment -Wno-null-dereference -Wno-logical-op-parentheses -Wno-switch -o ../build/4ed_build 4ed_build.cpp
../build/4ed_build $1

Linux (code/bin/build-linux.sh):

#!/bin/bash
g++ -g -std=c++11 -DDEV_BUILD -Wno-write-strings -Wno-comment -Wno-null-dereference -Wno-logical-op-parentheses -Wno-switch -o ../build/4ed_build 4ed_build.cpp
../build/4ed_build $1

Windows (code/bin/build.bat):

@echo off
IF NOT EXIST ..\build mkdir ..\build
pushd ..\build
cl /I..\code /DDEV_BUILD /std:c++11 /Zi /Fe:4ed_build.exe ..\code\bin\4ed_build.cpp /link /INCREMENTAL:NO /RELEASE > NUL
4ed_build.exe %1
popd

3. Custom Layer Build System

The custom layer (editor behavior/commands) uses a sophisticated multi-stage build process:

Stage 1: Preprocessing with Meta Macros

clang++ -I../code -I../code/custom -I../code/custom/generated -DMETA_PASS -E 4coder_default_bindings.cpp -o 4coder_default_bindings.i

Stage 2: Metadata Generator Compilation

clang++ -I../code -I../code/custom -I../code/custom/generated -o metadata_generator 4coder_metadata_generator.cpp

Stage 3: Metadata Generation

./metadata_generator -R ../code/custom/generated/ 4coder_default_bindings.i

Stage 4: Final Custom Layer Compilation

clang++ -I../code -I../code/custom -I../code/custom/generated -shared -fPIC -o custom_4coder.so 4coder_default_bindings.cpp

4. Compiler-Specific Configurations

Visual Studio (CL):

#define CL_OPTS \
"-W4 -wd4310 -wd4100 -wd4201 -wd4505 -wd4996 " \
"-wd4127 -wd4510 -wd4512 -wd4610 -wd4390 " \
"-wd4611 -wd4189 -WX -GR- -EHa- -nologo -FC"

#define CL_LIBS_COMMON \
"user32.lib winmm.lib gdi32.lib opengl32.lib comdlg32.lib userenv.lib "

GCC (Linux):

#define GCC_OPTS \
"-Wno-write-strings " \
"-D_GNU_SOURCE -fPIC " \
"-fno-threadsafe-statics -pthread " \
"-Wno-unused-result " \
"-std=c++11"

#define GCC_LIBS_COMMON \
"-lX11 -lpthread -lm -lrt " \
"-lGL -ldl -lXfixes -lfreetype -lfontconfig"

Clang (macOS):

#define CLANG_OPTS \
"-Wno-write-strings -Wno-deprecated-declarations " \
"-Wno-comment -Wno-switch -Wno-null-dereference " \
"-Wno-tautological-compare -Wno-unused-result " \
"-Wno-missing-declarations -Wno-nullability-completeness " \
"-std=c++11 "

#define CLANG_LIBS_COMMON \
"-framework Cocoa -framework QuartzCore " \
"-framework CoreServices " \
"-framework OpenGL -framework IOKit -framework Metal -framework MetalKit "

5. Build Process Flow

Development Build Flow:

  1. Entry Script (platform-specific) compiles 4ed_build.cpp
  2. Meta-Build Program generates compiler commands based on platform/arch/flags
  3. Custom Layer Build (parallel process):
    • Preprocesses source with meta macros
    • Builds metadata generator
    • Generates API bindings
    • Compiles custom layer as shared library
  4. Core Engine Build:
    • Compiles 4ed_app_target.cpp as shared library (4ed_app.so)
    • Compiles platform layer as main executable (4ed)
  5. Resource Management: Copies themes, fonts, configs to build directory

Packaging Build Flow:

  1. Standard Build (as above)
  2. Distribution Preparation:
    • Creates versioned directory structure
    • Copies executables and shared libraries
    • Includes source code (Super tier only)
    • Copies documentation and resources
  3. Archive Creation: Creates ZIP files for distribution

6. Key Build Outputs

The build system produces:

  • 4ed: Main editor executable (platform layer)
  • 4ed_app.so: Core application logic (shared library)
  • custom_4coder.so: Custom layer with user commands (shared library)
  • Resources: Themes, fonts, configuration files
  • Debug Symbols: .dSYM directories (macOS), .pdb files (Windows)

7. Build Complexity Analysis

The current system is complex due to:

  1. Multi-tier Architecture: Three separate compilation units with interdependencies
  2. Cross-platform Abstraction: Different compilers, flags, and linking requirements
  3. Dynamic API Generation: Metadata generation for extensible command system
  4. Packaging Requirements: Multiple build tiers and distribution formats
  5. Debug/Release Variants: Different optimization and debugging configurations

8. Simplification Opportunities

The build system could be simplified by:

  1. Unified Build Scripts: Single bash script per platform instead of meta-build system
  2. Direct Compiler Calls: Eliminate C++ meta-build program in favor of direct clang/gcc invocations
  3. Simplified Metadata: Reduce or eliminate runtime metadata generation
  4. Merged Compilation Units: Combine core and custom layers into single executable
  5. Standardized Toolchain: Use consistent compiler (clang) across all platforms

The current complexity stems from the editor's plugin architecture and cross-platform requirements, but many of these features could be maintained with a simpler build system that uses direct bash scripts calling clang with appropriate flags.