Loading
Commits on Source 42
-
Ian Chechin authoredAdds the public PageCacheModule API, the SQLITE_CONFIG_PCACHE2 wiring, the lifecycle gate between RegisterPageCacheModule and the first Open, and tests for the API surface. The production pool-backed reference implementation and the memory-utilization benchmark that demonstrate the reduction in #204 are deferred to a follow-up MR so the API shape can be reviewed independently of the implementation. New files: - pagecache.go: public PageCacheModule struct, Page / PageEq interfaces, RegisterPageCacheModule / MustRegisterPageCacheModule entry points, ErrPageCacheTooLate / ErrPageCacheConflict sentinels. - pagecache_alias_new.go, pagecache_alias_old.go: per-arch type alias shims so pagecache.go can refer to a single pcacheMethods2 type. Most arches emit the SQLite struct as Tsqlite3_pcache_methods2; the three old-generator arches (freebsd_386, freebsd_arm, netbsd_amd64) emit it as Sqlite3_pcache_methods2. The two build-tagged shims keep the body of pagecache.go arch-agnostic. - pagecache_test.go: TestPCacheMethods2Layout pins the struct shape against regeneration drift, TestRegisterPageCacheModuleValidation covers nil / missing-required-field rejection, TestRegisterPageCacheModuleLifecycle exercises the gate (conflict, too-late, same-pointer idempotency), and TestOpenGateConcurrentReaders runs the openGate RWMutex under load. Modified files: - conn.go: newConn now wraps c.openV2 in withOpenGate so an in-flight Open holds pcacheState.openGate.RLock for the duration of sqlite3_open_v2. A concurrent RegisterPageCacheModule call takes the write lock, drains all readers, and then either installs the methods table or returns ErrPageCacheTooLate. Design notes: - SQLITE_CONFIG_PCACHE2 is global and one-shot. The first successful install commits pcacheState.registered = m and pcacheState.cMethods; any subsequent call with a different pointer returns ErrPageCacheConflict, with the same pointer returns nil. Reload is not supported in this MR. - The configOnce.Do body uses a defer / recover guard so a panic during Xsqlite3_config or populateCMethods leaves configErr set to a panic message and rolls back the half-set state. - populateCMethods uses named-field writes (FiVersion, FpArg, FxInit, ...) rather than hardcoded byte offsets, so the wiring is portable across all supported GOOS / GOARCH pairs. - Page / PageEq interfaces are exported as a stable surface for follow-up impl authors; the binding itself does not use them in this MR. Tested locally on darwin/arm64 with go test -race -short. Cross-build clean for linux/amd64, linux/386, linux/arm64, darwin/arm64, darwin/amd64, windows/amd64, openbsd/amd64; existing pre-MR upstream build failures on freebsd/386, freebsd/arm, netbsd/amd64 are unrelated to this change. Updates #204 Signed-off-by:
Ian Chechin <ian00chechin@gmail.com>
-
Ian Chechin authored
sqlite: pcache2 rework per !126 review (idiomatic Go API, internal trampolines, binding-owned stubs) Replaces the raw-ccgo-ABI surface from the prior commit with an idiomatic Go interface set, per the maintainer's review at gitlab.com/cznic/sqlite/-/merge_requests/126#note. The wiring, lifecycle gate, and tests are kept and extended; the user-facing surface is rebuilt. Public API changes: - PageCache (factory) replaces PageCacheModule. Single method: Create(pageSize, extraSize int, purgeable bool) (Cache, error). - Cache (per-database instance) replaces the prior PageCacheModule struct of function fields. All methods are required, which eliminates the SIGSEGV class the maintainer reproduced: a module with nil Rekey or Shrink. Methods: SetSize, PageCount, Fetch, Unpin, Rekey, Truncate, Destroy, Shrink. - Page (unchanged) is the raw-memory boundary the binding cannot hide: SQLite reads pBuf/pExtra directly and the addresses must stay put. - FetchMode enum (FetchLookup / FetchCreateEasy / FetchCreateForce) replaces a bare int32 createFlag. - RegisterPageCache / MustRegisterPageCache replace RegisterPageCacheModule / MustRegisterPageCacheModule. - PageEq removed: not consumed by anything in this MR; can be added later if the production impl shows it pays. - Both per-arch type alias shims (pagecache_alias_new.go, pagecache_alias_old.go) are removed. The three old-generator arches they covered (freebsd_386, freebsd_arm, netbsd_amd64) are not in build_all_targets, and they fail to build for unrelated upstream reasons today, so the shim never bought a working binary. pagecache.go now declares the single `type pcacheMethods2 = sqlite3.Tsqlite3_pcache_methods2` inline. Internal trampolines (pagecache_trampolines.go, new file): - 11 package-level trampolines wired into the C methods table via cFuncPointer. Closures and method values are never passed to cFuncPointer (only top-level decls, per its godoc). - Per-database pcacheBinding holds the user-supplied Cache plus two maps: byKey (uint32 SQLite key -> sqlite3_pcache_page stub uintptr) and byStub (stub -> {Page, key}). The binding owns the stub end-to-end so SQLite always sees the same opaque sqlite3_pcache_page pointer for the same (cache, key); fixes the stale-PgHdr->pPage hazard that the prior submission's MR description called out under §2. - pcacheIDGen is a bitset-backed handle allocator (uintptr starting at 1; 0 reserved as sentinel). Identical in shape to vtab's idGen, reproduced locally to keep pcache self-contained. - Unpin and Rekey use a check-call-recheck pattern: the user callback runs outside b.mu, then b.mu is re-acquired and the trampoline verifies byStub[pPage] still exists before mutating the maps or calling libc.Xfree. Protects against a concurrent Truncate sweep racing in and freeing the same stub. Strict Buf/Extra contract (Cache.Unpin docstring): - Cache implementations MUST keep Buf and Extra valid at the same addresses on Unpin(discard=false). This is stricter than SQLite's xUnpin contract, which permits unilateral eviction by the cache. The binding caches the stub per key and re-returns it on every subsequent Fetch without re-invoking the user callback, so a silent eviction would hand SQLite a stub pointing at freed memory. The strict contract makes that failure mode unreachable. Tests: - TestPCacheMethods2Layout pins field names and the lower bound on struct size. The prior NumField()==13 assertion is dropped because netbsd_amd64 emits a 14-field layout with explicit F__ccgo_pad1 padding, and counting fields would force an arch-conditional check that the named-field assertions already cover. - TestRegisterPageCacheNil, TestRegisterPageCacheLifecycle (DifferentValuesConflict / TooLate / TooLateIdempotentForSameValue), TestOpenGateConcurrentReaders, TestPCacheIDGen. - An in-tree end-to-end test (real sqlite open + queries through the wrapper) is not feasible in this package: vec_test.go imports modernc.org/sqlite/vec which calls Xsqlite3_auto_extension at package init, which itself calls Xsqlite3_initialize. By the time any test here runs, sqlite3_config(PCACHE2) returns SQLITE_MISUSE. The maintainer exercised the C path end-to-end with a minimal page cache (see MR thread); we defer adding an isolated e2e test to a follow-up MR. Tested locally on darwin/arm64 with go test -race -short. Cross-build clean for linux/amd64, linux/386, linux/arm64, darwin/arm64, darwin/amd64, windows/amd64, openbsd/amd64, freebsd/amd64. The pre-existing upstream build failures on freebsd/386, freebsd/arm, netbsd/amd64 are unchanged by this commit and unrelated to it. Refs !126 review Updates #204 Signed-off-by:
-
Ian Chechin authored
sqlite: pcache2 round-2 fixes per !126 review (always-call-Fetch, doc tightening, CHANGELOG + doc.go) Addresses cznic's second-round review at gitlab.com/cznic/sqlite/-/merge_requests/126#note_3434353434. Three contract changes plus housekeeping. 1. Always-call-Fetch design (pagecache_trampolines.go) pcacheTrampolineFetch now invokes Cache.Fetch on every SQLite request and reuses the cached sqlite3_pcache_page stub only when the returned Page value equals the previously-stored Page for that key. When the implementation evicted the entry and returned either nil or a different Page, the binding retires the stale stub via libc.Xfree and either returns NullStub (impl reports miss) or mints a fresh stub (impl returned a new Page). This unblocks the canonical use case for plugging in a custom page cache: a bounded purgeable cache that evicts on Unpin(discard=false) to honour cache_size. The previous design's stub-caching shortcut silently leaked a stale stub to SQLite the nex...
-
-
Ian Chechin authored
Implements the production page cache deferred to MR-B from !126. pool.go: PageCache factory minting per-database caches backed by libc.Xmalloc / libc.Xcalloc pages with a strict cache_size cap and LRU-tail eviction. Page identity is *page (comparable). Pool aggregates hit/miss/alloc/eviction counters across every cache it creates. pool_test.go: 16 unit tests covering empty state, retain/replace across Fetch cycles, FetchCreateEasy refusing at cap, FetchCreateForce evicting LRU tail, SetSize shrink + overcommit-on-pinned, Rekey with colliding eviction, Truncate pinned eviction, Shrink, Destroy no-panic. e2e_test.go: real-DB harness mirroring the !126 validation workload: cache_size=16, 4000 BLOB rows + DELETE + incremental_vacuum, integrity_check=ok under -race. Multi-DB test asserts one Cache per opened database. bench_test.go: BenchmarkPoolBoundedCache reports per-insert allocs + evictions + go-heap-inuse delta for #204 memory-utilization measurement. BenchmarkPoolEvictionChurn drives steady-state 1:1 alloc/eviction churn at cache_size=16.
-
-
An opt-in `_dqs` DSN query parameter disables SQLite's double-quoted string literal compatibility quirk on a per-connection basis. When set to a false value (`_dqs=0` or any `strconv.ParseBool` false), the driver calls `sqlite3_db_config` with both `SQLITE_DBCONFIG_DQS_DDL` and `SQLITE_DBCONFIG_DQS_DML` set to off in `newConn`, after `sqlite3_open_v2` and before any statement is prepared. Default (absent or `_dqs=1`) leaves SQLite's built-in behavior unchanged so existing DSNs continue to work byte-for-byte. The config call goes through a new `(*conn).dbConfigBool` helper that hand-lays the mixed `(int onoff, int *pRes)` vararg form for the cgo-free transpilation: two pointer-sized slots, the second a NULL pointer because callers in this driver only need the side effect. Tests cover the FFI shape directly (`TestDQSConfigCallVaList`), end-to-end behavior through `database/sql` for the default, explicit-on, and off cases (`TestDQSOptIn`), and the unparseable-val...
-
cznic authored
Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
cznic authored
# Conflicts: # CHANGELOG.md
-
cznic authored
dbConfigBool sized its va_list buffer as 2*unsafe.Sizeof(uintptr(0)), i.e. 8 bytes on 32-bit targets. libc.VaList packs every argument into a fixed 8-byte slot regardless of pointer width (an int is widened to 8 bytes), so the (int onoff, int *pRes) pair writes 12 bytes on 32-bit (int64 at [0,8), pointer at [8,12)) and overruns the 8-byte allocation by 4 bytes on linux/386 and linux/arm. The overflow is currently masked by modernc.org/memory allocator over-allocation and is caught by neither go vet nor cross-builds; -race is unavailable on 386. Size the buffer for two 8-byte VaList slots. Verified: a fill-and-probe shows VaList writes [0,12) within a 16-byte buffer on 386; TestDQS* pass on amd64 and 386; full -short suite green. Co-Authored-By: -
cznic authored
The _dqs changelog entry added by the original !128 commit was lost when master was merged into the dqs-opt-in branch and the CHANGELOG.md conflict was resolved in master's favor. Re-add it under the v1.53.0 section. Co-Authored-By:
-
-
Ian Chechin authored
An opt-in `_error_rc` DSN query parameter switches the connection into a stricter error-string reporting mode: when set to a true value (`_error_rc=1` or any `strconv.ParseBool` true), the synthesised `*Error.Error()` only appends `sqlite3_errmsg(db)` when `sqlite3_extended_errcode(db)` is consistent with the operation rc. The match check tries the full extended code first and falls back to the primary code (`rc & 0xff`); on mismatch the canonical `sqlite3_errstr(rc)` is used alone. This stops the legacy `errstr: errmsg` form from carrying a stale errmsg from the temporary db handle on open-time failures: a SQLITE_CANTOPEN no longer reads as "unable to open database file: out of memory" when the temporary handle was last touched by an unrelated initialisation path. Absence of the parameter or `_error_rc=0` preserves the legacy form byte-for-byte so callers parsing error strings remain unaffected. `*Error.Code()` returns the same SQLite result code in both...
-
Three non-blocking follow-ups raised by cznic on the !127 merge, collected into a single polish pass: (1) Stats.Evictions documentation tightened to match actual behavior. The field counts LRU eviction, Unpin(discard=true), and Shrink releases; bulk frees performed by Truncate, Rekey collisions, and Destroy are not counted. The old "LRU-driven page releases" docstring read narrower than the implementation. (2) New Stats.EasyRefusals counter. FetchCreateEasy refuses at cap even when there are recyclable unpinned pages, while pcache1 would recycle one without spilling; the counter records each refusal so the I/O pressure of the strict Easy contract is observable. SQLite reacts to a refusal by spilling dirty pages and retrying with FetchCreateForce, so EasyRefusals/op is a direct proxy for that spill rate. The two existing benchmarks now report easy-refusals/op alongside the page-allocs and page-evictions metrics, and TestEasyHonoursCacheSize asserts the counter increments on each Easy refusal. (3) TestPoolRoundTripIntegrity comment fix. The previous wording claimed the DELETE + incremental_vacuum workload exercised xRekey ~15 times, which the actual run does not confirm (Rekeys reports 0 on every platform). The corrected comment notes that the SQL surface does not reliably emit xRekey here and that the code path is covered by the unit tests (TestRekey, TestRekeyEvictsCollider) instead. Open question for !127 follow-up I/O comparison: a direct side-by-side vs the in-engine pcache1 (e.g. SQLITE_DBSTATUS_CACHE_SPILL) would require either exposing sqlite3_db_status through the parent driver or running two separate test binaries with and without sqlite.MustRegisterPageCache. Asking which approach you prefer before extending the benchmark in that direction; for now EasyRefusals/op is the in-package proxy. Builds clean across the 8 supported GOOS/GOARCH pairs, gofmt clean, go vet only the two pre-existing unsafe.Pointer notices on Buf/Extra. go test -race -short ./pcache/ ok (16 unit + 2 e2e + new TestEasyHonoursCacheSize assertions).
-
Adds an empty sharing.go that captures the three open design questions for MR-C. The file compiles but introduces no behavior; it exists so cznic can react to the directional choices in a focused diff before any concurrency primitive, wrapper type, or PageCache contract change is committed. Q1. Concurrency primitive — sync.Mutex (current lean), sync.RWMutex, or finer-grained per-bucket. Q2. Locking surface — lock on the existing cache struct, a separate sharedCache wrapper, or an opt-in ConcurrentSafe hook on the sqlite.Cache contract. Q3. Discovery — extend PageCache.Create, add PageCache.CreateShared, or detect at the binding level from the parent conn URI. A canonical TestSharedCacheTwoConns_Integrity (two connections with cache=shared, concurrent writes, PRAGMA integrity_check) is reserved in the file but not added until the locking shape is settled. Builds clean, gofmt clean, go vet introduces no new warnings beyond the pre-existing pool.go unsafe.Pointer notices on Buf/Extra. Existing pcache tests unchanged and green. References !127 review note "the assumption MR-C will need to revisit" and the original !126 description's deferred-to-MR-C scope. -
Ian Chechin authored
(1) Rework BenchmarkPoolEvictionChurn to use a rotating-residue DELETE (k % 3 = i % 3) with a matching-batch re-insert per cycle. Cycle i removes batchPerCycle rows from residue (i % 3) and re-inserts batchPerCycle rows back into the same residue in a fresh disjoint key range, so the next visit (three cycles later) finds rows to scan. Per-cycle work is constant from cycle 0 onward: 200 deletes + 200 inserts + 1 incremental_vacuum, with the seed pre-populated as three 200-row partitions so cycle 0 is already in steady state. easy-refusals/op and page-evictions/op are now rates that hold flat across benchtime values rather than a fixed first-cycle cost divided by b.N (was: 5.36, 2.68, 1.34, 0.67 at 25x/50x/100x/200x; now: ~60.32 throughout). (2) Tighten Stats.Evictions docstring to mention Unpin(discard =false) trimming back to target after a FetchCreateForce overcommit, per cznic's optional nit on round 1. (3) CHANGELOG: replace "merge request #N" placeholder with #130.
-
Ian Chechin authored
Round-1 review follow-ups by cznic on !129 (#230): (1) Fix inverted boolean in the getErrorRcMode docstring. The "Absent parameter or true value preserves..." sentence said the opposite of what the code does and contradicted both the conn.errorRcMode field comment and the driver.go paragraph. Swap the true/false words so the doc agrees with the code. (2) Add TestErrstrForDBSuppressOnMismatch, a deterministic unit test that calls errstrForDB directly with a healthy db handle (sqlite3_extended_errcode = SQLITE_OK, sqlite3_errmsg = "not an error") and a deliberately mismatched rc = SQLITE_CANTOPEN. In legacy mode the formatter appends the stale "not an error" as the helpful detail; under errorRcMode=true the conditional suppress branch fires and the canonical errstr(rc) is used alone. Code() returns SQLITE_CANTOPEN in both modes. This pins the new behavior across SQLite versions and platforms independently of the host-specific open-time failure path that TestErrorRcOpenTimeUnopenable was relying on (cznic noted he can no longer reproduce the original #230 symptom on the current tree, so that test's "no out of memory" assertion now passes without the suppress branch ever firing).
-
-
Ian Chechin authored
Resolves the question raised by cznic on !127 about what MR-C would need to revisit: the pool is already correct under SQLite's shared-cache mode, because every callback into a given Cache is serialised internally by sqlite3BtreeEnter on the BtShared mutex (verified empirically with a lock-free in-flight probe: max-in-flight = 1 on the canonical two-connection workload, 4 on a positive control with goroutines hitting the cache directly). The Go race detector, however, does not recognise SQLite's libc mutex as a happens-before edge and reports false-positive races on Fetch vs Unpin reads/writes of the per-cache state, which surfaces as DATA RACE failures for any user who registers the pool and runs their suite under -race. Take a sync.Mutex on the cache type on every public method (SetSize, PageCount, Fetch, Unpin, Rekey, Truncate, Destroy, Shrink), always. On the common non-shared-cache path the lock is uncontended (one atomic CAS per Lock/Unlock pair, negligible next to the SQLite work it bookends); on the shared-cache path it just rubber-stamps the order SQLite's BtShared mutex already established. Drop sharedCacheStub from sharing.go (cznic noted it triggers staticcheck U1000 and breaks make all). Rewrite sharing.go as a design record describing why the lock is always taken and the alternatives considered (always-taken vs conditional, document-as-unsupported vs reject-at-Create). The design-questions RFC scaffold is gone. Add TestSharedCacheTwoConns_Integrity in e2e_test.go: two sql.Conn against the same cache=shared URI with concurrent writers + PRAGMA integrity_check, runs cleanly under -race. CHANGELOG entry under TBC vNEXT.
-
Ian Chechin authored
The round-2 rotating-residue rework reliably triggers the b-tree rebalance paths that emit xRekey through the SQL surface (~13 Rekeys per cycle, 325 over 25 cycles, scaling linearly with b.N), so the benchmark now complements the dedicated xRekey unit tests (TestRekey, TestRekeyEvictsCollider) rather than deferring to them, contrary to what the existing comment claimed. Comment-only change per cznic's round-3 review on !130; TestPoolRoundTripIntegrity exercises a different (unchanged) workload and its comment stays accurate as-is.
-
cznic authored
The committed lib/sqlite_netbsd_amd64.go was a stale old-generator transpile that no longer built (the mu.enter/mu.leave undefined break in issue #246) and was out of sync with the new-generator code every other platform uses. Re-transpile SQLite 3.53.2 for netbsd/amd64 (generated on NetBSD 10.1 / Go 1.26.3) and re-vendor: - vendor_libs/main.go: add netbsd/amd64 to the libsqlite3 and libsqlite_vec target lists. Also skip emitting an un-prefixed type alias when it would collide with a const of the same name — netbsd's transpile emits spurious `const <typename> = 0` macro-eval artifacts (off_t, gid_t, ...) that otherwise redeclare the generated alias. No effect on the other platforms' output. - lib/sqlite_netbsd_amd64.go: regenerated (replaces the stale file). - vec/vec_netbsd_amd64.go: vendor sqlite-vec for netbsd; vec/patches.go and vec_test.go: include netbsd so the extension registers and is tested. - Makefile, builder.json: add netbsd/amd64 to build_all_targets and the builder test matrix. pagecache.go: drop netbsd from the stale old-generator comment. The full suite (incl. pcache, vec, and the -race concurrency test) passes on NetBSD 10.1 / Go 1.26.3. Requires the matching modernc.org/libc Xmmap PAD-ABI fix. Co-Authored-By:
-
cznic authored
Picks up the netbsd/amd64 Xmmap PAD-ABI fix. modernc.org/sqlite#246. Co-Authored-By: -
cznic authored
Document the issue #246 NetBSD/amd64 support status: Tier-2 done, the libc mmap PAD SIGBUS fix (v1.73.1), the cc/ccgo Tier-1 toolchain gate, the cascade landed on master, and the remaining maintainer tag cascade. Co-Authored-By:
-
cznic authored
Completes the netbsd/amd64 cascade — picks up the race-free netbsd Xabort (and the earlier mmap PAD fix). ABI-preserving for the vendored lib/vec and a no-op for non-netbsd targets (v1.73.1..v1.73.3 touch only libc_netbsd.go). Co-Authored-By: -
cznic authored
Co-Authored-By: -
cznic authored
Regenerate the vendored transpiles from the now-released siblings (was generated from in-development trees during the netbsd/amd64 work). vec/* update to the v0.3.0 transpile; lib/sqlite_netbsd_amd64.go picks up the v1.14.0 netbsd diff (other targets unchanged — same SQLite version). build_all_targets passes; TestVec passes. Co-Authored-By: -
cznic authored
The revived port has green CI across the chain but zero production mileage, so it stays out of the supported-platforms list in doc.go pending broader real-world testing (~a month). Expand the note with the libc mmap-PAD and abort(3) fixes and the call for users to evaluate it and report via #246. Co-Authored-By:
-
cznic authored
The per-target transpiles in lib/ (SQLite) and vec/ (sqlite-vec) ship one generated Go file per GOOS/GOARCH. Declarations byte-identical across targets are now folded into build-tagged shared files -- lib/sqlite.go + lib/sqlite_g_<hex>.go and vec/vec.go + vec/vec_g_<hex>.go -- by modernc.org/undup, wired into `make vendor`. This is a packaging change only. Go's build constraints make every target compile exactly the same declarations as before; the public API and behavior are unchanged, `make build_all_targets` is green on all ~20 platforms, and the suite (incl. TestVec) passes. Hand-written platform files (libsqlite3_*.go, hooks_*.go, ...) carry no generated-code marker and are untouched. 421 files changed, 1,523,713 insertions(+), 4,437,082 deletions(-) lib 164.8 -> 62.2 MB (2.65x) vec 8.6 -> 2.1 MB (4.19x) Net ~2.9M fewer lines of vendored generated code -- good news for clone, fetch, and build times. The motivation... -
cznic authored
The latest real tag is v1.52.0; v1.53.0 and v1.54.0 existed only as CHANGELOG sections, splitting one pending release across two version numbers. Renumber the premature v1.54.0 header to v1.53.0 and fuse the two sections so the next tag is honestly v1.53.0. No released (tagged) section is touched. Co-Authored-By: -
-
cznic authored
Vendor fresh SQLite 3.53.2 + sqlite-vec transpiles for freebsd/386 and freebsd/arm (replacing the stale 3.41 freebsd_386 file) via make vendor, and enable them in vendor_libs, build_all_targets, and builder.json test. Requires modernc.org/libc v1.73.4: with the previous libc, freebsd/arm's WAL shared-memory mmap faulted (SIGBUS) because the 64-bit off_t was mis-encoded for 32-bit; v1.73.4 fixes the per-arch mmap off_t encoding. Runtime-tested on both freebsd/386 and freebsd/arm (core + WAL/concurrency + vec). doc.go's supported-targets list is left for a follow-up after broader testing, mirroring how netbsd/amd64 was introduced. Co-Authored-By: -
cznic authored
The freebsd/arm sqlite-vec transpile was generated against the older libsqlite3 (SQLite 3.53.1) while the rest of the tree is 3.53.2; re-vendored from libsqlite_vec regenerated at 3.53.2 so all targets are consistent. Functionally unchanged (3.53.1->3.53.2 is a patch, no ABI change; freebsd/386+arm runtime tests already passed). Co-Authored-By: -
-
cznic authored
Co-Authored-By: -
Ian Chechin authored
Adds a Go binding for sqlite3_db_status, the per-connection runtime counters (cache hit/miss/write/spill, schema/statement memory, lookaside usage, deferred FKs), as discussed on the !130 review. dbstatus.go: DBStatus interface implemented by *conn and reached via (*sql.Conn).Raw(), mirroring the FileControl surface cznic pointed at. DBStatusOp is a distinct typed enum of the SQLITE_DBSTATUS_* verbs so a constant from another op family will not compile in its place; all 14 ops the transpiled lib defines are exposed. Status(op, reset) returns the (current, high) pair via the tls.Alloc(8) two-int32 pattern from cznic's skeleton and surfaces an out-of-range op as an error. dbstatus_test.go: exercises the three counter families through Raw() - SchemaUsed (memory high-water) grows after DDL; CacheHit (running counter) resets; LookasideHit reports its value in high not current; an out-of-range op errors. pcache/bench_test.go: BenchmarkPoolSpillIO reads the pager-level CACHE_SPILL/CACHE_WRITE/CACHE_HIT/CACHE_MISS counters through the new API, replacing the EasyRefusals proxy with the real I/O numbers cznic asked for on the !127 review. The pager maintains these identically for pcache1 and the pool, so the comparison is apples-to-apples. On the rotating-residue churn at cache_size=16 the pool spills ~3.5x more than pcache1 (cache-spill/op 31.96 vs 8.96) for ~3% more writes (450 vs 436) at identical hit/miss, quantifying the strict Easy contract's I/O cost.
-
cznic authored
- CHANGELOG.md: re-add the "See merge request #131" line that the MR diff dropped, so the pcache -race-clean entry keeps its attribution instead of reading as part of !132. - dbstatus.go: make the op-family doc match the transpiled sqlite3_db_status64 behavior — only DBStatusLookasideUsed maintains a high-water mark; CacheUsed/SchemaUsed/StmtUsed/CacheUsedShared report high==0 with the reset flag ignored; DBStatusDeferredFKs is a 0/1 flag (reset ignored); DBStatusTempbufSpill is a running byte counter (was undocumented). No code/behavior change. Co-Authored-By:
-
-
cznic authored
Pre-v1.53.0 release prep; documentation and module hygiene only, no change to any compiled source: - CHANGELOG.md: add an experimental-status entry for freebsd/386 and freebsd/arm (MR #119, Olivier Cochard-Labbé / @ocochard), mirroring the netbsd/amd64 framing -- shipped and CI-tested but intentionally not yet in doc.go's supported-platforms list, pending broader real-world testing. Also bump the v1.53.0 entry date to the tag date. - CLAUDE.md: correct the stale "transpiled SQLite 3.53.1" to 3.53.2. - go.sum: go mod tidy, pruning orphan toolchain hashes; go.mod and all selected module versions are unchanged. - Remove issue246-tracker.md, an internal status tracker not meant to ship in the released module. Co-Authored-By:
-
Ian Chechin authored
MAX/MIN/COALESCE over a DATETIME TEXT column drop the declared type (sqlite3_column_decltype returns "" for aggregates), so Next delivered a raw string and a Scan into *time.Time failed with "unsupported Scan, storing driver.Value type string into type *time.Time". The column-metadata route the reporter suggested does not help: on master sqlite3_column_origin_name already resolves direct columns (ColumnInfo, !113) but returns "" through MAX/MIN/COALESCE, so it cannot recover the source type. This does the runtime best-effort parse instead: an empty-decltype TEXT column under _texttotime is parsed via parseTime; success yields time.Time, failure falls back to the original string, so no Scan that worked before can newly fail. Gated on _texttotime so non-opt-in callers are unaffected. ColumnTypeScanType is left reporting string for empty decltype, since it is called before any row and cannot know an aggregate is date-shaped; Next ...
-
Ian Chechin authored
Addresses !133 review: reword the rows.go comment to cover every empty-decltype TEXT case (aggregates/expressions, subqueries, typeless real columns) rather than only MAX/MIN/COALESCE, document the *string RFC3339Nano reformat consequence on the _texttotime DSN doc, and add the v1.54.0 CHANGELOG entry.
-
-
cznic authored
Re-vendor lib/ and vec/ from libsqlite3 v1.14.2 and libsqlite_vec v0.3.1 (SQLite 3.53.3) and bump the pinned modernc.org/libc to v1.74.1, the version the transpile was generated against. Updates doc.go, CLAUDE.md, and the pending v1.54.0 CHANGELOG section. make vendor build_all_targets passes; local test suite (root incl. TestVec, pcache, vfs) is green. Co-Authored-By: