## Summary ccgo-generated Go code is overwhelmingly data initialization (99%+) due to verbose union initialization in `initializerUnion`. For tree-sitter grammars this produces Go files that are 1.5-2x the C source, mostly from repetitive inline struct literals. ## Real-world example: tree-sitter-python | Metric | Value | |--------|-------| | C source lines | 130,440 | | Generated Go lines | 243,736 | | Data portion (before `init()`) | 242,872 (99.6%) | | Code portion (after `init()`) | 864 (0.4%) | Four static arrays account for **91%** of the file: | Array | Elements | Type | Lines | |-------|----------|------|-------| | `_ts_small_parse_table` | 118,009 | `uint16` | 117,908 | | `_ts_parse_actions` | 5,042 | `TTSParseActionEntry` (union) | 73,934 | | `_ts_parse_table` | — | — | 18,313 | | `_ts_lex_modes` | — | — | 12,202 | The LR parse table is the dominant consumer regardless of language — every tree-sitter grammar has one. ## Root cause `initializerUnion` (`v4/lib/init.go:378`) generates a full inline struct definition + field assignments + `unsafe.Pointer` wrapper for **every single union element**: ```go // Each union element: 15+ lines 3: *(*TTSParseActionEntry)(unsafe.Pointer(&struct { _ [0]byte f struct { Fcount Tuint8_t Freusable uint8 } _ [6]byte }{f: struct { Fcount Tuint8_t Freusable uint8 }{ Fcount: uint8(1), Freusable: libc.BoolUint8(true1 != 0), }})), ``` For `TTSParseActionEntry` (4 bytes actual), this expands to 15+ lines per element — a **15:1 bloat ratio**. ## Proposed solutions ### Phase 1: Compact union byte literals (minimal ccgo change) Modify `initializerUnion` to emit a compact byte array instead of inline struct: ```go // Before (15 lines): *(*TTSParseActionEntry)(unsafe.Pointer(&struct{_ [0]byte; f struct{...}; _ [6]byte}{...})) // After (1 line): *(*TTSParseActionEntry)(unsafe.Pointer(&[4]byte{0x01, 0x00, 0x00, 0x00})) ``` Since ccgo already knows all type sizes and field offsets, computing the exact byte representation is straightforward. This alone would reduce tree-sitter-python.go from 243K → ~58K lines (76% reduction). ### Phase 2: `-compact-data` flag with `//go:embed` (larger feature) For arrays above a size threshold, write data as binary companion files and embed at compile time: ```go // Instead of 242K lines of data: //go:embed _ts_parse_actions.bin var _ts_parse_actions_data []byte var _ts_parse_actions = (*[5042]TTSParseActionEntry)(unsafe.Pointer(&_ts_parse_actions_data[0])) ``` This would reduce the Go file to ~15K lines (94% reduction) and also speed up compilation by removing massive literal initialization from the Go compiler's workload. ### Phase 1 implementation sketch The key insight is that `initializerUnion` already produces correct values — it just produces them in a verbose form. Converting to byte literals needs: 1. A helper that walks the union's active member and writes each field's value as bytes at the correct offset 2. Use `binary.LittleEndian` / `binary.BigEndian` per target ABI for multi-byte fields 3. Handle nested structs/arrays recursively ## Questions for maintainer 1. Would you accept a Phase 1 PR (compact union byte literals)? 2. Is `-compact-data` with `//go:embed` something you'd want in ccgo itself, or would it be better as a post-processing tool? 3. Are there ABI considerations (endianness, alignment) that the byte-literal approach must handle that the current struct-literal approach implicitly gets right? --- *Discovered while converting tree-sitter grammars to cgo-free Go with ccgo: tree-sitter, tree-sitter-c, tree-sitter-python. MR !24 fixed the blocking `exprUintptr` bug; this issue addresses the next scalability concern.*