rows: cache the column decltype lookup once per result set

Background

The Next() hot path calls (*rows).ColumnTypeDatabaseTypeName(i) for every TEXT column on every row (unconditionally, via the TEXT branch in rows.go), and for every INTEGER column on every row when intToTime is set. Each call ran:

return strings.ToUpper(r.c.columnDeclType(r.pstmt, index))

That is one libc.GoString to materialise the C-side decltype into Go memory, plus a (cheap, allocation-free for already-uppercase inputs) strings.ToUpper. The declared type of a result column is fixed for the lifetime of a prepared statement, so the libc.GoString cost is paid N_text_cols * N_rows times for nothing.

What this MR does

Move the lookup to newRows() and cache the uppercased decltype into a new rows.decltypes []string. ColumnTypeDatabaseTypeName, the Next() DATETIME branch (which goes through it), and ColumnTypeScanType now read from the cache instead of redoing the C round-trip per row. The case-sensitive switch in ColumnTypeScanType is rewritten against the cached uppercase values to drop a per-column strings.ToLower as well.

The TEXT branch in Next() calls ColumnTypeDatabaseTypeName on every row regardless of any DSN flag, so the cache benefits any SELECT that returns TEXT columns, not only _texttotime users. The only queries that pay any new cost are result sets with no TEXT or INTEGER columns and no caller-side ColumnTypes() use: they now do n decltype lookups once at newRows() time instead of zero. The lookups are per-result-set (not per-row), so the eager-population tradeoff stays net-positive.

No API change. ColumnTypeDatabaseTypeName / ScanType return identical values for every input.

Benchmark

darwin/arm64 (Apple M3), _texttotime=1, 1000-row SELECT of all DATETIME columns, -benchtime=2s, before -> after:

Columns allocs/op B/op ns/op
1 11010 -> 10012 400354 -> 392393 646068 -> 601121 (-7%)
5 55014 -> 50020 2000499 -> 1960654 2992839 -> 2908393 (-3%)

Saving is exactly 1 alloc + 8 bytes per TEXT column per row (the libc.GoString for the "DATETIME" body). It scales 1:1 with N_text_cols * N_rows. The ns/op delta is modest in percentage because the per-row scan path is dominated by SQLite step and time parsing, but the alloc reduction is real on every SELECT with TEXT columns.

Tests

  • TestColumnTypeDatabaseTypeNameCache builds a mixed-case CREATE TABLE across all SQLite storage classes (INTEGER / TEXT / BLOB / DATETIME / DATE / BOOLEAN) and asserts that every column returns the expected uppercase DatabaseTypeName. Runs under -race.
  • TestColumnTypeScanTypeDecltypeCache (added in 8a6f33ce) is a table-driven test that locks down every arm of the cached ColumnTypeScanType switch: INTEGER + BOOLEAN -> bool, INTEGER + DATE/DATETIME/TIME/TIMESTAMP -> time.Time, INTEGER fallback -> int64, TEXT default -> string, TEXT under _texttotime=1 -> time.Time for the four recognised decltypes and string otherwise. Mixed-case decltypes exercise the case-folding path. 15 sub-cases, all pass under -race.

Full go test -count=1 ./... stays green (467s on darwin/arm64).

Backward compatibility

Pure perf fix; no API surface change, no behavior change visible to callers. The returned ColumnTypeDatabaseTypeName strings have the same values as before, and ColumnTypeScanType returns the same reflect.Type for every decltype it used to.

Edited by Ian Chechin

Merge request reports

Loading