generated code size: union initialization produces 15x bloat for data-heavy projects

Summary

ccgo-generated Go code is overwhelmingly data initialization (99%+) due to verbose union initialization in initializerUnion. For tree-sitter grammars this produces Go files that are 1.5-2x the C source, mostly from repetitive inline struct literals.

Real-world example: tree-sitter-python

Metric	Value
C source lines	130,440
Generated Go lines	243,736
Data portion (before init())	242,872 (99.6%)
Code portion (after init())	864 (0.4%)

Four static arrays account for 91% of the file:

Array	Elements	Type	Lines
_ts_small_parse_table	118,009	uint16	117,908
_ts_parse_actions	5,042	TTSParseActionEntry (union)	73,934
_ts_parse_table	—	—	18,313
_ts_lex_modes	—	—	12,202

The LR parse table is the dominant consumer regardless of language — every tree-sitter grammar has one.

Root cause

initializerUnion (v4/lib/init.go:378) generates a full inline struct definition + field assignments + unsafe.Pointer wrapper for every single union element:

// Each union element: 15+ lines
3: *(*TTSParseActionEntry)(unsafe.Pointer(&struct {
    _ [0]byte
    f struct {
        Fcount    Tuint8_t
        Freusable uint8
    }
    _ [6]byte
}{f: struct {
    Fcount    Tuint8_t
    Freusable uint8
}{
    Fcount:    uint8(1),
    Freusable: libc.BoolUint8(true1 != 0),
}})),

For TTSParseActionEntry (4 bytes actual), this expands to 15+ lines per element — a 15:1 bloat ratio.

Proposed solutions

Phase 1: Compact union byte literals (minimal ccgo change)

Modify initializerUnion to emit a compact byte array instead of inline struct:

// Before (15 lines):
*(*TTSParseActionEntry)(unsafe.Pointer(&struct{_ [0]byte; f struct{...}; _ [6]byte}{...}))

// After (1 line):
*(*TTSParseActionEntry)(unsafe.Pointer(&[4]byte{0x01, 0x00, 0x00, 0x00}))

Since ccgo already knows all type sizes and field offsets, computing the exact byte representation is straightforward. This alone would reduce tree-sitter-python.go from 243K → ~58K lines (76% reduction).

Phase 2: -compact-data flag with //go:embed (larger feature)

For arrays above a size threshold, write data as binary companion files and embed at compile time:

// Instead of 242K lines of data:
//go:embed _ts_parse_actions.bin
var _ts_parse_actions_data []byte
var _ts_parse_actions = (*[5042]TTSParseActionEntry)(unsafe.Pointer(&_ts_parse_actions_data[0]))

This would reduce the Go file to ~15K lines (94% reduction) and also speed up compilation by removing massive literal initialization from the Go compiler's workload.

Phase 1 implementation sketch

The key insight is that initializerUnion already produces correct values — it just produces them in a verbose form. Converting to byte literals needs:

1. A helper that walks the union's active member and writes each field's value as bytes at the correct offset
2. Use binary.LittleEndian / binary.BigEndian per target ABI for multi-byte fields
3. Handle nested structs/arrays recursively

Questions for maintainer

1. Would you accept a Phase 1 PR (compact union byte literals)?
2. Is -compact-data with //go:embed something you'd want in ccgo itself, or would it be better as a post-processing tool?
3. Are there ABI considerations (endianness, alignment) that the byte-literal approach must handle that the current struct-literal approach implicitly gets right?

---

Discovered while converting tree-sitter grammars to cgo-free Go with ccgo: tree-sitter, tree-sitter-c, tree-sitter-python. MR !24 (merged) fixed the blocking exprUintptr bug; this issue addresses the next scalability concern.