Loading
Commits on Source 80
-
Two thin wrappers around the existing sqlite3_backup_remaining and sqlite3_backup_pagecount C symbols. They expose the underlying backup progress counters that the database/sql layer already keeps but that Go callers cannot currently read without dropping to lib/* directly. The motivation is the standard progress-UI use case for online backups: for { more, err := bck.Step(pagesPerTick) if err != nil { return err } ui.Update(bck.PageCount()-bck.Remaining(), bck.PageCount()) if !more { break } } Without these wrappers a caller has to either skip the progress display or fall back to unsafe per-call SQL queries against pragma_page_count. API shape mirrors !115 (FileControlDataVersion): named after the SQLite C function with the s/sqlite3_// prefix stripped and CamelCase applied, documented inline with a link to the official C API page, and added on the existing public *Backup receiver so no new interface or escape hatch is required. The C functions are zero-arg lookups against the sqlite3_backup object and cannot fail, so the Go wrappers return int with no error. Per the SQLite docs, both return 0 before the first Step and Remaining returns 0 after SQLITE_DONE; the new TestBackupProgress test exercises all three phases (before any Step, after a partial Step, after DONE) and asserts the documented relationships hold (Remaining = PageCount - copied, PageCount stable across the final Step). Test suite (go test -count=1 ./...) stays green. -
-
cznic authored
Co-Authored-By:Claude Opus 4.7 (1M context) <noreply@anthropic.com>
-
Ian Chechin authored
(*conn).columnText currently allocates twice per TEXT column per row: once for the make([]byte, len) buffer that receives the SQLite-owned UTF-8 bytes, and once again inside the string(b) conversion that runtime.slicebytetostring performs because the compiler must assume the caller could mutate b. Here b is local to columnText and is never touched again after the copy from the C buffer, so the second copy is redundant. Replacing string(b) with unsafe.String(unsafe.SliceData(b), len) builds the returned string directly on top of b. The string is immutable from Go's perspective, the GC keeps b alive for as long as the string is reachable, and no aliasing is possible because b becomes unreachable as []byte the moment the function returns. The same pattern is already used in sqlite.go (!120) for the volatile-args path. Benchmark on darwin/arm64 (Apple M3), 1000-row SELECT of a single TEXT column, -benchtime=2s, before -> after: Short (16-byte TEXT): 4009 -> 4009 allocs/op (Go runtime already short-circuits string(b) for slices below the inline threshold; no regression either) 52348 -> 52348 B/op 157342 -> 155746 ns/op Medium (256-byte TEXT): 5009 -> 4009 allocs/op (-1000 allocs/op = -1 per row) 548351 -> 292350 B/op (-256 KB/op = the second 256-byte copy) 226863 -> 204730 ns/op (-10%) Long (4096-byte TEXT): 5009 -> 4009 allocs/op (-1000 allocs/op = -1 per row) 8228510 -> 4132413 B/op (-4 MB/op = the second 4 KB copy) 1605640 -> 1135113 ns/op (-29%) The saving scales linearly with TEXT column length, since the eliminated work is exactly one memcpy of the column bytes. No change to (*conn). columnBlob, which already returns its make([]byte, len) buffer directly and pays only one alloc + memcpy per row. TestColumnTextScan exercises the path under -race over the three branches of columnText: empty (short-circuit), short (Go-fast-path) and long (allocating) TEXT, including a multi-byte / emoji payload to confirm UTF-8 is preserved bit-for-bit. Full go test -count=1 ./... stays green.
-
-
cznic authored
Co-Authored-By: -
Ian Chechin authored
The Next() hot path calls (*rows).ColumnTypeDatabaseTypeName(i) for every TEXT column on every row when _texttotime=1, and for every INTEGER column on every row when intToTime is set. Each call ran: return strings.ToUpper(r.c.columnDeclType(r.pstmt, index)) which is one libc.GoString to materialise the C decltype string into Go memory, plus a (cheap, allocation-free for already-uppercase inputs) strings.ToUpper. The declared type of a result column is fixed for the lifetime of a prepared statement, so the libc.GoString cost is paid N_text_cols * N_rows times for nothing. Move the lookup to newRows() and cache the uppercased decltype into a new rows.decltypes []string. ColumnTypeDatabaseTypeName, the Next() DATETIME branch (which goes through it), and ColumnTypeScanType now read from the cache instead of redoing the C round-trip per row. The case-sensitive switch in ColumnTypeScanType is rewritten against the cached uppercase values to drop a per-column strings.ToLower as well. Benchmark (darwin/arm64 Apple M3, _texttotime=1, 1000-row SELECT of all DATETIME columns, -benchtime=2s, before -> after): 1 column: 11010 -> 10012 allocs/op (-1000 = -1 per row, the libc.GoString) 400354 -> 392393 B/op (-8 KB = -8 bytes per row, "DATETIME" string body) 646068 -> 601121 ns/op (-7%) 5 columns: 55014 -> 50020 allocs/op (-5000 = -5 per row, -1 per col per row) 2000499 -> 1960654 B/op (-40 KB, scales linearly with columns) 2992839 -> 2908393 ns/op (-3%) The saving scales 1:1 with N_text_cols * N_rows for queries that hit the time-conversion path. Workloads using _texttotime, _time_format, or _intToTime DSN flags benefit; queries without those flags do not touch ColumnTypeDatabaseTypeName per row and see no behavior change. TestColumnTypeDatabaseTypeNameCache covers a mixed-case CREATE TABLE across all SQLite storage classes (INTEGER / TEXT / BLOB / DATETIME / DATE / BOOLEAN), reads the cache once at result-set start and again inside the Next loop for every row, and asserts the values never drift. The full go test -count=1 ./... suite stays green. -
Ian Chechin authored
Per @cznic on !124: the decltype cache rewrites the lowercase decltype switch in ColumnTypeScanType to a cached-uppercase switch, but the existing TestColumnTypeDatabaseTypeNameCache only exercises the DatabaseTypeName side. Add a table-driven TestColumnTypeScanTypeDecltypeCache that covers every arm of the cached switch: - INTEGER + BOOLEAN (any case) -> bool - INTEGER + DATE/DATETIME/TIME/TIMESTAMP -> time.Time - INTEGER + plain / unrecognised decltype -> int64 - TEXT (default) -> string - TEXT + DATETIME-shaped decltype (no flag) -> string - TEXT + DATE/DATETIME/TIME/TIMESTAMP under _texttotime=1 -> time.Time - TEXT + unrecognised decltype under _texttotime=1 -> string Each case uses a mixed-case declared type to keep the case-folding path covered, and inserts one row before SELECT so sqlite3_column_type sees the actual storage class instead of SQLITE_NULL (which would short- circuit ColumnTypeScanType to reflect.TypeOf(nil)). All 15 sub-cases pass under -race.
-
-
cznic authored
Co-Authored-By: -
Ian Chechin authored
(*conn).parseTime ran on every TEXT-stored DATETIME / DATE / TIMESTAMP column read in Next(). The function tried (*conn).parseTimeString first and then walked parseTimeFormats[0..6] sequentially until time.Parse matched the row's value. For the canonical SQLite TEXT datetime format ("2006-01-02 15:04:05.999999999", index 2) every row paid two failed time.Parse attempts in the warmup, plus the one successful match. Each failed Parse allocates a ParseError, so the per-row cost on a steady 1000-row scan was ~5 allocs per row from the format-search alone. Add a sticky per-column hint cache: - rows.parseFmtIdx []int8, sized once at newRows() to the column count, initialised to -1 (no match recorded). - (*conn).parseTime now takes hintIdx int and returns the index that actually matched (or -1 when parseTimeString matched / all formats failed). It tries hintIdx first if in range, then walks the list skipping the index it just tried. ... -
-
cznic authored
Tighten the parseFmtIdx doc comment: a mixed-format column pays at most one extra format probe (on rows whose matching format precedes the cached index), not just the original fall-through cost. Add the !125 CHANGELOG entry. No code/behavior change. Co-Authored-By:
-
cznic authored
-
Ian Chechin authored
Adds the public PageCacheModule API, the SQLITE_CONFIG_PCACHE2 wiring, the lifecycle gate between RegisterPageCacheModule and the first Open, and tests for the API surface. The production pool-backed reference implementation and the memory-utilization benchmark that demonstrate the reduction in #204 are deferred to a follow-up MR so the API shape can be reviewed independently of the implementation. New files: - pagecache.go: public PageCacheModule struct, Page / PageEq interfaces, RegisterPageCacheModule / MustRegisterPageCacheModule entry points, ErrPageCacheTooLate / ErrPageCacheConflict sentinels. - pagecache_alias_new.go, pagecache_alias_old.go: per-arch type alias shims so pagecache.go can refer to a single pcacheMethods2 type. Most arches emit the SQLite struct as Tsqlite3_pcache_methods2; the three old-generator arches (freebsd_386, freebsd_arm, netbsd_amd64) emit it as Sqlite3_pcache_methods2. The two build-tagged shims keep the body of pagecache.go arch-agnostic. - pagecache_test.go: TestPCacheMethods2Layout pins the struct shape against regeneration drift, TestRegisterPageCacheModuleValidation covers nil / missing-required-field rejection, TestRegisterPageCacheModuleLifecycle exercises the gate (conflict, too-late, same-pointer idempotency), and TestOpenGateConcurrentReaders runs the openGate RWMutex under load. Modified files: - conn.go: newConn now wraps c.openV2 in withOpenGate so an in-flight Open holds pcacheState.openGate.RLock for the duration of sqlite3_open_v2. A concurrent RegisterPageCacheModule call takes the write lock, drains all readers, and then either installs the methods table or returns ErrPageCacheTooLate. Design notes: - SQLITE_CONFIG_PCACHE2 is global and one-shot. The first successful install commits pcacheState.registered = m and pcacheState.cMethods; any subsequent call with a different pointer returns ErrPageCacheConflict, with the same pointer returns nil. Reload is not supported in this MR. - The configOnce.Do body uses a defer / recover guard so a panic during Xsqlite3_config or populateCMethods leaves configErr set to a panic message and rolls back the half-set state. - populateCMethods uses named-field writes (FiVersion, FpArg, FxInit, ...) rather than hardcoded byte offsets, so the wiring is portable across all supported GOOS / GOARCH pairs. - Page / PageEq interfaces are exported as a stable surface for follow-up impl authors; the binding itself does not use them in this MR. Tested locally on darwin/arm64 with go test -race -short. Cross-build clean for linux/amd64, linux/386, linux/arm64, darwin/arm64, darwin/amd64, windows/amd64, openbsd/amd64; existing pre-MR upstream build failures on freebsd/386, freebsd/arm, netbsd/amd64 are unrelated to this change. Updates #204 Signed-off-by:
Ian Chechin <ian00chechin@gmail.com>
-
Ian Chechin authored
sqlite: pcache2 rework per !126 review (idiomatic Go API, internal trampolines, binding-owned stubs) Replaces the raw-ccgo-ABI surface from the prior commit with an idiomatic Go interface set, per the maintainer's review at gitlab.com/cznic/sqlite/-/merge_requests/126#note. The wiring, lifecycle gate, and tests are kept and extended; the user-facing surface is rebuilt. Public API changes: - PageCache (factory) replaces PageCacheModule. Single method: Create(pageSize, extraSize int, purgeable bool) (Cache, error). - Cache (per-database instance) replaces the prior PageCacheModule struct of function fields. All methods are required, which eliminates the SIGSEGV class the maintainer reproduced: a module with nil Rekey or Shrink. Methods: SetSize, PageCount, Fetch, Unpin, Rekey, Truncate, Destroy, Shrink. - Page (unchanged) is the raw-memory boundary the binding cannot hide: SQLite reads pBuf/pExtra directly and the addresses must stay put. - FetchMode enum (FetchLookup / FetchCreateEasy / Fet...
-
Ian Chechin authored
sqlite: pcache2 round-2 fixes per !126 review (always-call-Fetch, doc tightening, CHANGELOG + doc.go) Addresses cznic's second-round review at gitlab.com/cznic/sqlite/-/merge_requests/126#note_3434353434. Three contract changes plus housekeeping. 1. Always-call-Fetch design (pagecache_trampolines.go) pcacheTrampolineFetch now invokes Cache.Fetch on every SQLite request and reuses the cached sqlite3_pcache_page stub only when the returned Page value equals the previously-stored Page for that key. When the implementation evicted the entry and returned either nil or a different Page, the binding retires the stale stub via libc.Xfree and either returns NullStub (impl reports miss) or mints a fresh stub (impl returned a new Page). This unblocks the canonical use case for plugging in a custom page cache: a bounded purgeable cache that evicts on Unpin(discard=false) to honour cache_size. The previous design's stub-caching shortcut silently leaked a stale stub to SQLite the nex...
-
-
Ian Chechin authored
Implements the production page cache deferred to MR-B from !126. pool.go: PageCache factory minting per-database caches backed by libc.Xmalloc / libc.Xcalloc pages with a strict cache_size cap and LRU-tail eviction. Page identity is *page (comparable). Pool aggregates hit/miss/alloc/eviction counters across every cache it creates. pool_test.go: 16 unit tests covering empty state, retain/replace across Fetch cycles, FetchCreateEasy refusing at cap, FetchCreateForce evicting LRU tail, SetSize shrink + overcommit-on-pinned, Rekey with colliding eviction, Truncate pinned eviction, Shrink, Destroy no-panic. e2e_test.go: real-DB harness mirroring the !126 validation workload: cache_size=16, 4000 BLOB rows + DELETE + incremental_vacuum, integrity_check=ok under -race. Multi-DB test asserts one Cache per opened database. bench_test.go: BenchmarkPoolBoundedCache reports per-insert allocs + evictions + go-heap-inuse delta for #204 memory-utilization measurement. BenchmarkPoolEvictionChurn drives steady-state 1:1 alloc/eviction churn at cache_size=16.
-
-
An opt-in `_dqs` DSN query parameter disables SQLite's double-quoted string literal compatibility quirk on a per-connection basis. When set to a false value (`_dqs=0` or any `strconv.ParseBool` false), the driver calls `sqlite3_db_config` with both `SQLITE_DBCONFIG_DQS_DDL` and `SQLITE_DBCONFIG_DQS_DML` set to off in `newConn`, after `sqlite3_open_v2` and before any statement is prepared. Default (absent or `_dqs=1`) leaves SQLite's built-in behavior unchanged so existing DSNs continue to work byte-for-byte. The config call goes through a new `(*conn).dbConfigBool` helper that hand-lays the mixed `(int onoff, int *pRes)` vararg form for the cgo-free transpilation: two pointer-sized slots, the second a NULL pointer because callers in this driver only need the side effect. Tests cover the FFI shape directly (`TestDQSConfigCallVaList`), end-to-end behavior through `database/sql` for the default, explicit-on, and off cases (`TestDQSOptIn`), and the unparseable-value error path (`TestDQSInvalid`). Documented next to `_pragma` and `_txlock` in driver.go. CHANGELOG entry under TBC vNEXT. Resolves #61.
-
cznic authored
Co-Authored-By: -
cznic authored
# Conflicts: # CHANGELOG.md
-
cznic authored
dbConfigBool sized its va_list buffer as 2*unsafe.Sizeof(uintptr(0)), i.e. 8 bytes on 32-bit targets. libc.VaList packs every argument into a fixed 8-byte slot regardless of pointer width (an int is widened to 8 bytes), so the (int onoff, int *pRes) pair writes 12 bytes on 32-bit (int64 at [0,8), pointer at [8,12)) and overruns the 8-byte allocation by 4 bytes on linux/386 and linux/arm. The overflow is currently masked by modernc.org/memory allocator over-allocation and is caught by neither go vet nor cross-builds; -race is unavailable on 386. Size the buffer for two 8-byte VaList slots. Verified: a fill-and-probe shows VaList writes [0,12) within a 16-byte buffer on 386; TestDQS* pass on amd64 and 386; full -short suite green. Co-Authored-By: -
cznic authored
The _dqs changelog entry added by the original !128 commit was lost when master was merged into the dqs-opt-in branch and the CHANGELOG.md conflict was resolved in master's favor. Re-add it under the v1.53.0 section. Co-Authored-By:
-
-
Ian Chechin authored
An opt-in `_error_rc` DSN query parameter switches the connection into a stricter error-string reporting mode: when set to a true value (`_error_rc=1` or any `strconv.ParseBool` true), the synthesised `*Error.Error()` only appends `sqlite3_errmsg(db)` when `sqlite3_extended_errcode(db)` is consistent with the operation rc. The match check tries the full extended code first and falls back to the primary code (`rc & 0xff`); on mismatch the canonical `sqlite3_errstr(rc)` is used alone. This stops the legacy `errstr: errmsg` form from carrying a stale errmsg from the temporary db handle on open-time failures: a SQLITE_CANTOPEN no longer reads as "unable to open database file: out of memory" when the temporary handle was last touched by an unrelated initialisation path. Absence of the parameter or `_error_rc=0` preserves the legacy form byte-for-byte so callers parsing error strings remain unaffected. `*Error.Code()` returns the same SQLite result code in both modes; the cznic review refinement was to change the message only, never the code. The parameter is parsed before `sqlite3_open_v2` because the failure path that motivated the issue runs during open itself, on a temporary db handle. A new `getErrorRcMode` parser sits next to `getVFSName` in sqlite.go and is consulted before the `conn` struct is created. The flag is then stored on `conn.errorRcMode` and threaded through `errstrForDB`'s new `errorRcMode bool` parameter to the three call sites (`(*conn).openV2`, `(*conn).errstr`, and the backup-init error path which uses the destination connection's mode). Tests cover the open-time SQLITE_CANTOPEN reproducer in all three modes (default, explicit-off, on) with structural assertions that hold across platforms; the non-regression case where syntax errors must preserve the helpful "no such table" detail in every mode; and the invalid-value error path. Documented next to `_pragma`, `_txlock`, and `_dqs` in driver.go. CHANGELOG entry under TBC vNEXT. Resolves #230.
-
Three non-blocking follow-ups raised by cznic on the !127 merge, collected into a single polish pass: (1) Stats.Evictions documentation tightened to match actual behavior. The field counts LRU eviction, Unpin(discard=true), and Shrink releases; bulk frees performed by Truncate, Rekey collisions, and Destroy are not counted. The old "LRU-driven page releases" docstring read narrower than the implementation. (2) New Stats.EasyRefusals counter. FetchCreateEasy refuses at cap even when there are recyclable unpinned pages, while pcache1 would recycle one without spilling; the counter records each refusal so the I/O pressure of the strict Easy contract is observable. SQLite reacts to a refusal by spilling dirty pages and retrying with FetchCreateForce, so EasyRefusals/op is a direct proxy for that spill rate. The two existing benchmarks now report easy-refusals/op alongside the page-allocs and page-evictions metrics, and TestEasyHonoursCacheSize asserts the counter increments on each Easy refusal. (3) TestPoolRoundTripIntegrity comment fix. The previous wording claimed the DELETE + incremental_vacuum workload exercised xRekey ~15 times, which the actual run does not confirm (Rekeys reports 0 on every platform). The corrected comment notes that the SQL surface does not reliably emit xRekey here and that the code path is covered by the unit tests (TestRekey, TestRekeyEvictsCollider) instead. Open question for !127 follow-up I/O comparison: a direct side-by-side vs the in-engine pcache1 (e.g. SQLITE_DBSTATUS_CACHE_SPILL) would require either exposing sqlite3_db_status through the parent driver or running two separate test binaries with and without sqlite.MustRegisterPageCache. Asking which approach you prefer before extending the benchmark in that direction; for now EasyRefusals/op is the in-package proxy. Builds clean across the 8 supported GOOS/GOARCH pairs, gofmt clean, go vet only the two pre-existing unsafe.Pointer notices on Buf/Extra. go test -race -short ./pcache/ ok (16 unit + 2 e2e + new TestEasyHonoursCacheSize assertions).
-
Adds an empty sharing.go that captures the three open design questions for MR-C. The file compiles but introduces no behavior; it exists so cznic can react to the directional choices in a focused diff before any concurrency primitive, wrapper type, or PageCache contract change is committed. Q1. Concurrency primitive — sync.Mutex (current lean), sync.RWMutex, or finer-grained per-bucket. Q2. Locking surface — lock on the existing cache struct, a separate sharedCache wrapper, or an opt-in ConcurrentSafe hook on the sqlite.Cache contract. Q3. Discovery — extend PageCache.Create, add PageCache.CreateShared, or detect at the binding level from the parent conn URI. A canonical TestSharedCacheTwoConns_Integrity (two connections with cache=shared, concurrent writes, PRAGMA integrity_check) is reserved in the file but not added until the locking shape is settled. Builds clean, gofmt clean, go vet introduces no new warnings beyond the pre-existing pool.go unsafe.Pointer notices on Buf/Extra. Existing pcache tests unchanged and green. References !127 review note "the assumption MR-C will need to revisit" and the original !126 description's deferred-to-MR-C scope. -
Ian Chechin authored
(1) Rework BenchmarkPoolEvictionChurn to use a rotating-residue DELETE (k % 3 = i % 3) with a matching-batch re-insert per cycle. Cycle i removes batchPerCycle rows from residue (i % 3) and re-inserts batchPerCycle rows back into the same residue in a fresh disjoint key range, so the next visit (three cycles later) finds rows to scan. Per-cycle work is constant from cycle 0 onward: 200 deletes + 200 inserts + 1 incremental_vacuum, with the seed pre-populated as three 200-row partitions so cycle 0 is already in steady state. easy-refusals/op and page-evictions/op are now rates that hold flat across benchtime values rather than a fixed first-cycle cost divided by b.N (was: 5.36, 2.68, 1.34, 0.67 at 25x/50x/100x/200x; now: ~60.32 throughout). (2) Tighten Stats.Evictions docstring to mention Unpin(discard =false) trimming back to target after a FetchCreateForce overcommit, per cznic's optional nit on round 1. (3) CHANGELOG: replace "merge request #N" placeholder with #130.
-
Ian Chechin authored
Round-1 review follow-ups by cznic on !129 (#230): (1) Fix inverted boolean in the getErrorRcMode docstring. The "Absent parameter or true value preserves..." sentence said the opposite of what the code does and contradicted both the conn.errorRcMode field comment and the driver.go paragraph. Swap the true/false words so the doc agrees with the code. (2) Add TestErrstrForDBSuppressOnMismatch, a deterministic unit test that calls errstrForDB directly with a healthy db handle (sqlite3_extended_errcode = SQLITE_OK, sqlite3_errmsg = "not an error") and a deliberately mismatched rc = SQLITE_CANTOPEN. In legacy mode the formatter appends the stale "not an error" as the helpful detail; under errorRcMode=true the conditional suppress branch fires and the canonical errstr(rc) is used alone. Code() returns SQLITE_CANTOPEN in both modes. This pins the new behavior across SQLite versions and platforms independently of the host-specific open-time failure path that TestErrorRcOpenTimeUnopenable was relying on (cznic noted he can no longer reproduce the original #230 symptom on the current tree, so that test's "no out of memory" assertion now passes without the suppress branch ever firing).
-
-
Ian Chechin authored
Resolves the question raised by cznic on !127 about what MR-C would need to revisit: the pool is already correct under SQLite's shared-cache mode, because every callback into a given Cache is serialised internally by sqlite3BtreeEnter on the BtShared mutex (verified empirically with a lock-free in-flight probe: max-in-flight = 1 on the canonical two-connection workload, 4 on a positive control with goroutines hitting the cache directly). The Go race detector, however, does not recognise SQLite's libc mutex as a happens-before edge and reports false-positive races on Fetch vs Unpin reads/writes of the per-cache state, which surfaces as DATA RACE failures for any user who registers the pool and runs their suite under -race. Take a sync.Mutex on the cache type on every public method (SetSize, PageCount, Fetch, Unpin, Rekey, Truncate, Destroy, Shrink), always. On the common non-shared-cache path the lock is uncontended (one atomic CAS per Lock/Unlock pair, negligible next to the SQLite work it bookends); on the shared-cache path it just rubber-stamps the order SQLite's BtShared mutex already established. Drop sharedCacheStub from sharing.go (cznic noted it triggers staticcheck U1000 and breaks make all). Rewrite sharing.go as a design record describing why the lock is always taken and the alternatives considered (always-taken vs conditional, document-as-unsupported vs reject-at-Create). The design-questions RFC scaffold is gone. Add TestSharedCacheTwoConns_Integrity in e2e_test.go: two sql.Conn against the same cache=shared URI with concurrent writers + PRAGMA integrity_check, runs cleanly under -race. CHANGELOG entry under TBC vNEXT.
-
Ian Chechin authored
The round-2 rotating-residue rework reliably triggers the b-tree rebalance paths that emit xRekey through the SQL surface (~13 Rekeys per cycle, 325 over 25 cycles, scaling linearly with b.N), so the benchmark now complements the dedicated xRekey unit tests (TestRekey, TestRekeyEvictsCollider) rather than deferring to them, contrary to what the existing comment claimed. Comment-only change per cznic's round-3 review on !130; TestPoolRoundTripIntegrity exercises a different (unchanged) workload and its comment stays accurate as-is.
-
cznic authored
The committed lib/sqlite_netbsd_amd64.go was a stale old-generator transpile that no longer built (the mu.enter/mu.leave undefined break in issue #246) and was out of sync with the new-generator code every other platform uses. Re-transpile SQLite 3.53.2 for netbsd/amd64 (generated on NetBSD 10.1 / Go 1.26.3) and re-vendor: - vendor_libs/main.go: add netbsd/amd64 to the libsqlite3 and libsqlite_vec target lists. Also skip emitting an un-prefixed type alias when it would collide with a const of the same name — netbsd's transpile emits spurious `const <typename> = 0` macro-eval artifacts (off_t, gid_t, ...) that otherwise redeclare the generated alias. No effect on the other platforms' output. - lib/sqlite_netbsd_amd64.go: regenerated (replaces the stale file). - vec/vec_netbsd_amd64.go: vendor sqlite-vec for netbsd; vec/patches.go and vec_test.go: include netbsd so the extension registers and is tested. - Makefile, builder.json: add netbsd/amd64...
-
cznic authored
Picks up the netbsd/amd64 Xmmap PAD-ABI fix. modernc.org/sqlite#246. Co-Authored-By: -
cznic authored
Document the issue #246 NetBSD/amd64 support status: Tier-2 done, the libc mmap PAD SIGBUS fix (v1.73.1), the cc/ccgo Tier-1 toolchain gate, the cascade landed on master, and the remaining maintainer tag cascade. Co-Authored-By:
-
cznic authored
Completes the netbsd/amd64 cascade — picks up the race-free netbsd Xabort (and the earlier mmap PAD fix). ABI-preserving for the vendored lib/vec and a no-op for non-netbsd targets (v1.73.1..v1.73.3 touch only libc_netbsd.go). Co-Authored-By: -
cznic authored
Co-Authored-By: -
cznic authored
Regenerate the vendored transpiles from the now-released siblings (was generated from in-development trees during the netbsd/amd64 work). vec/* update to the v0.3.0 transpile; lib/sqlite_netbsd_amd64.go picks up the v1.14.0 netbsd diff (other targets unchanged — same SQLite version). build_all_targets passes; TestVec passes. Co-Authored-By: -
cznic authored
The revived port has green CI across the chain but zero production mileage, so it stays out of the supported-platforms list in doc.go pending broader real-world testing (~a month). Expand the note with the libc mmap-PAD and abort(3) fixes and the call for users to evaluate it and report via #246. Co-Authored-By:
-
cznic authored
The per-target transpiles in lib/ (SQLite) and vec/ (sqlite-vec) ship one generated Go file per GOOS/GOARCH. Declarations byte-identical across targets are now folded into build-tagged shared files -- lib/sqlite.go + lib/sqlite_g_<hex>.go and vec/vec.go + vec/vec_g_<hex>.go -- by modernc.org/undup, wired into `make vendor`. This is a packaging change only. Go's build constraints make every target compile exactly the same declarations as before; the public API and behavior are unchanged, `make build_all_targets` is green on all ~20 platforms, and the suite (incl. TestVec) passes. Hand-written platform files (libsqlite3_*.go, hooks_*.go, ...) carry no generated-code marker and are untouched. 421 files changed, 1,523,713 insertions(+), 4,437,082 deletions(-) lib 164.8 -> 62.2 MB (2.65x) vec 8.6 -> 2.1 MB (4.19x) Net ~2.9M fewer lines of vendored generated code -- good news for clone, fetch, and build times. The motivation is Go's 500 MB per-tag module download cap (golang.org/x/mod/zip MaxZipFile), which the un-deduplicated tree was approaching as targets were added. README documents how to expand the sources back to one self-contained file per target for debugging. Co-Authored-By: -
cznic authored
The latest real tag is v1.52.0; v1.53.0 and v1.54.0 existed only as CHANGELOG sections, splitting one pending release across two version numbers. Renumber the premature v1.54.0 header to v1.53.0 and fuse the two sections so the next tag is honestly v1.53.0. No released (tagged) section is touched. Co-Authored-By: -
-
cznic authored
Vendor fresh SQLite 3.53.2 + sqlite-vec transpiles for freebsd/386 and freebsd/arm (replacing the stale 3.41 freebsd_386 file) via make vendor, and enable them in vendor_libs, build_all_targets, and builder.json test. Requires modernc.org/libc v1.73.4: with the previous libc, freebsd/arm's WAL shared-memory mmap faulted (SIGBUS) because the 64-bit off_t was mis-encoded for 32-bit; v1.73.4 fixes the per-arch mmap off_t encoding. Runtime-tested on both freebsd/386 and freebsd/arm (core + WAL/concurrency + vec). doc.go's supported-targets list is left for a follow-up after broader testing, mirroring how netbsd/amd64 was introduced. Co-Authored-By: -
cznic authored
The freebsd/arm sqlite-vec transpile was generated against the older libsqlite3 (SQLite 3.53.1) while the rest of the tree is 3.53.2; re-vendored from libsqlite_vec regenerated at 3.53.2 so all targets are consistent. Functionally unchanged (3.53.1->3.53.2 is a patch, no ABI change; freebsd/386+arm runtime tests already passed). Co-Authored-By: -
-
cznic authored
Co-Authored-By: -
Ian Chechin authored
Adds a Go binding for sqlite3_db_status, the per-connection runtime counters (cache hit/miss/write/spill, schema/statement memory, lookaside usage, deferred FKs), as discussed on the !130 review. dbstatus.go: DBStatus interface implemented by *conn and reached via (*sql.Conn).Raw(), mirroring the FileControl surface cznic pointed at. DBStatusOp is a distinct typed enum of the SQLITE_DBSTATUS_* verbs so a constant from another op family will not compile in its place; all 14 ops the transpiled lib defines are exposed. Status(op, reset) returns the (current, high) pair via the tls.Alloc(8) two-int32 pattern from cznic's skeleton and surfaces an out-of-range op as an error. dbstatus_test.go: exercises the three counter families through Raw() - SchemaUsed (memory high-water) grows after DDL; CacheHit (running counter) resets; LookasideHit reports its value in high not current; an out-of-range op errors. pcache/bench_test.go: BenchmarkPoolSpillIO reads the pager-level CACHE_SPILL/CACHE_WRITE/CACHE_HIT/CACHE_MISS counters through the new API, replacing the EasyRefusals proxy with the real I/O numbers cznic asked for on the !127 review. The pager maintains these identically for pcache1 and the pool, so the comparison is apples-to-apples. On the rotating-residue churn at cache_size=16 the pool spills ~3.5x more than pcache1 (cache-spill/op 31.96 vs 8.96) for ~3% more writes (450 vs 436) at identical hit/miss, quantifying the strict Easy contract's I/O cost.
-
cznic authored
- CHANGELOG.md: re-add the "See merge request #131" line that the MR diff dropped, so the pcache -race-clean entry keeps its attribution instead of reading as part of !132. - dbstatus.go: make the op-family doc match the transpiled sqlite3_db_status64 behavior — only DBStatusLookasideUsed maintains a high-water mark; CacheUsed/SchemaUsed/StmtUsed/CacheUsedShared report high==0 with the reset flag ignored; DBStatusDeferredFKs is a 0/1 flag (reset ignored); DBStatusTempbufSpill is a running byte counter (was undocumented). No code/behavior change. Co-Authored-By:
-
-
cznic authored
Pre-v1.53.0 release prep; documentation and module hygiene only, no change to any compiled source: - CHANGELOG.md: add an experimental-status entry for freebsd/386 and freebsd/arm (MR #119, Olivier Cochard-Labbé / @ocochard), mirroring the netbsd/amd64 framing -- shipped and CI-tested but intentionally not yet in doc.go's supported-platforms list, pending broader real-world testing. Also bump the v1.53.0 entry date to the tag date. - CLAUDE.md: correct the stale "transpiled SQLite 3.53.1" to 3.53.2. - go.sum: go mod tidy, pruning orphan toolchain hashes; go.mod and all selected module versions are unchanged. - Remove issue246-tracker.md, an internal status tracker not meant to ship in the released module. Co-Authored-By:
-
Ian Chechin authored
MAX/MIN/COALESCE over a DATETIME TEXT column drop the declared type (sqlite3_column_decltype returns "" for aggregates), so Next delivered a raw string and a Scan into *time.Time failed with "unsupported Scan, storing driver.Value type string into type *time.Time". The column-metadata route the reporter suggested does not help: on master sqlite3_column_origin_name already resolves direct columns (ColumnInfo, !113) but returns "" through MAX/MIN/COALESCE, so it cannot recover the source type. This does the runtime best-effort parse instead: an empty-decltype TEXT column under _texttotime is parsed via parseTime; success yields time.Time, failure falls back to the original string, so no Scan that worked before can newly fail. Gated on _texttotime so non-opt-in callers are unaffected. ColumnTypeScanType is left reporting string for empty decltype, since it is called before any row and cannot know an aggregate is date-shaped; Next is the only place that upgrades.
-
Ian Chechin authored
Addresses !133 review: reword the rows.go comment to cover every empty-decltype TEXT case (aggregates/expressions, subqueries, typeless real columns) rather than only MAX/MIN/COALESCE, document the *string RFC3339Nano reformat consequence on the _texttotime DSN doc, and add the v1.54.0 CHANGELOG entry.
-
-
cznic authored
Re-vendor lib/ and vec/ from libsqlite3 v1.14.2 and libsqlite_vec v0.3.1 (SQLite 3.53.3) and bump the pinned modernc.org/libc to v1.74.1, the version the transpile was generated against. Updates doc.go, CLAUDE.md, and the pending v1.54.0 CHANGELOG section. make vendor build_all_targets passes; local test suite (root incl. TestVec, pcache, vfs) is green. Co-Authored-By: -
For improved DSN compatibility, the following keys have been added: _foreign_keys | _fk _busy_timeout | _timeout _journal_mode | _journal _synchronous | _sync _auto_vacuum | _vacuum _query_only Their values are passed as-is to exec for their respective PRAGMAs and not validated in any way. The compatibility here is intended when switching between modernc.org/sqlite and mattn/go-sqlite3, where it is handy to have higher DSN compatibility and to avoid dangerous mistakes like not having foreign keys enabled. -
Ian Chechin authored
Follow-up to the DSN shorthand keys carried over from !73: document _busy_timeout/_fk/_journal/_sync/_vacuum/_query_only in the driver DSN reference, note the fixed apply order (busy_timeout first, query_only last) at the query_only call site, and add a combined-DSN test asserting the keys coexist in a single DSN regardless of the order they appear in it.
-
Ian Chechin authored
Addresses the !134 review: - Apply _auto_vacuum before the _pragma list and the other shorthand keys. auto_vacuum only takes effect while the database is new; a _journal_mode change materialises page 1 and locks it in, so the previous order made _journal_mode=wal&_auto_vacuum=1 silently resolve to auto_vacuum=0. The "Test combined DSN" case now includes _auto_vacuum and asserts it reads back as 1, proving the fixed order. - Validate every shorthand value against the set github.com/mattn/go-sqlite3 accepts and return an error on anything else, instead of silently ignoring it, so a _synchronous or _foreign_keys typo no longer downgrades durability or drops enforcement. This also removes the DSN-injection surface these keys had, since a value is now either a known token or an error. - When a key and its alias are both present, the alias wins, matching mattn (_foreign_keys=off&_fk=on -> on). - Document the apply order, precedence, accepted values, and that only _pragma values are executed verbatim and must be trusted. - CHANGELOG entry for #134.
-
cznic authored
Support more underscore keys in DSN (continues !73) See merge request !134 Co-Authored-By:
-
cznic authored
The v1.55.0 entry said the new mattn-compatible keys were "parsed but had no effect in prior releases". They were not parsed at all; applyQueryParams never looked at them. It also described an unrecognized value as failing "instead of being silently ignored", a contrast against an unreleased iteration of !134 rather than against v1.54.0, which would read to a user upgrading as though their DSN had previously been tolerated and ignored. Restate both breaks against the last released version: keys that were ignored now take effect (with the consequence spelled out rather than just the key named), and a value outside the accepted set now fails the connection where the same DSN previously opened successfully - e.g. a duration-style _busy_timeout=5s, which is not the integer that key requires. The latter was missing entirely and is the sharper of the two. Co-Authored-By:
-
cznic authored
dsnPick treated an empty value as absent, so "_foreign_keys=on&_fk=" fell back to the primary key and enabled foreign keys. mattn/go-sqlite3 selects between a key and its alias by presence alone and then reads that key's value, so the empty alias wins and suppresses the PRAGMA entirely. Match that: pick on presence, return the selected key's value as-is. An empty value still skips the PRAGMA at the call sites, and skips validation with it, so an empty shorthand is not an error. Co-Authored-By: -
cznic authored
applyQueryParams validated each parameter as it reached it, so a DSN whose last parameter was bad still executed every PRAGMA ahead of it before failing. PRAGMA journal_mode and auto_vacuum are persistent changes to the database file, so file:x.db?_journal_mode=wal&_synchronous=bogus failed the connection and left x.db converted to WAL regardless. A failed Open must not change the database. Split the function into a validation phase and an apply phase: everything checkable is rejected before the first c.exec. Assignments to c stay in the validation phase, since newConn closes and discards the connection when this returns an error and they cannot outlive the failure. The documented apply order is unchanged and still covered by the combined-DSN test. This predates the !134 shorthand keys - master already behaved this way for _pragma combined with a late-rejected _txlock, which is why the new test covers that case too. _pragma remains the o...
-
cznic authored
Adds a v1.55.0 entry for validating every DSN parameter before applying any of them. It gets its own bullet rather than folding into the !134 paragraph because it changes behavior for parameters that have shipped for years - _txlock, _timezone, _time_format, _time_integer_format, _inttotime and _texttotime - so it concerns readers who never touch the new mattn-compatible keys. The alias-selection fix is folded into the !134 paragraph instead. v1.55.0 is not tagged, so no release ever shipped the fallback behavior and describing it as a fix would document a bug nobody could have hit. Also corrects that paragraph's closing claim that "all other existing parameters are unchanged", which the validation-order change made false: those parameters are affected in when they are validated, though not in what they accept or what they mean. Co-Authored-By:
-
cznic authored
database/sql offers no way to reach a registered driver: sql.Drivers returns names only and there is no sql.Driver(name). The single route from the "sqlite" name back to the driver value is (*sql.DB).Driver(), which is why callers who need it resort to db, _ := sql.Open("sqlite", "") drv := db.Driver() db.Close() That opens nothing - it works only because sql.Open does not connect and *Driver does not implement driver.DriverContext. Both are properties this package could change without noticing it had broken anyone. Callers want the driver in order to interpose on the physical connections database/sql opens: tracing, metrics, connection-scoped setup. Handing out the driver alone would not be enough, because the only way to get a wrapper into a *sql.DB through sql.Open is sql.Register, which is process-global, panics on a name it has already seen and cannot be undone - so a library has to invent a unique name per configuration. TestConnectionHook in all_test.go is an in-tree instance of that workaround. sql.OpenDB takes a connector directly and registers nothing, so a Connector answers both halves. Connect goes through d.Open rather than newConn, so the connections carry every function, collation, connection hook and vtab module registered on the package-level driver. A caller-constructed &sqlite.Driver{} carries none of them: the fields are unexported, so it silently yields connections missing all of it. The 2022 Connector prototype on danp-embed called newConn and had exactly that bug. Deliberately not done: - No accessor for the package-level driver. driver.Connector requires a Driver() method, so the singleton stays reachable, but as the driver.Driver interface - one method - and only through a type assertion godoc gives no hint of. Nothing returns *sqlite.Driver. - No OpenConnector on *Driver. Implementing driver.DriverContext would make sql.Open eager and move where DSN errors surface for every existing user. The sql.Open path is untouched. - No eager validation of parameter values. That validation is interleaved with assignments to *conn in applyQueryParams, and separating it is a larger change than this feature warrants. NewConnector checks the query string's syntax alone and documents that it does. NewConnector returns the driver.Connector interface rather than a concrete type, keeping the added surface to one symbol. That does not foreclose the fs.FS/VFS-lifecycle idea from the danp-embed prototype: database/sql discovers an optional Close on the connector by type assertion. Resolves #253. Co-Authored-By: -
cznic authored
Driver has been exported since 65f6d7e4 (2017), where "Make sqlite public" renamed the unexported sqlite type as part of publishing the package. The struct then held a connection counter and a mutex, so constructing one cost nothing and lost nothing. a9227519 (2022) moved the function and collation registries onto it and introduced the package-level instance, which is when a constructed Driver started silently lacking things; 14082cad (2023) added (*Driver).RegisterConnectionHook, which is only meaningful on an instance the caller builds, and so made the export load-bearing. The result is a type that is legitimately constructible for the private-hook pattern yet carries none of what the package-level Register* functions install. This documents that on the type and on the method, including the consequence easiest to miss: a registered function replaces a SQLite built-in of the same name, so a constructed Driver can evaluate upper(x) or date(x) differently from a connection opened through sql.Open. The half-global virtual table module behavior is documented as it stands rather than changed. registerModules reads the package-level driver, so modules do reach a constructed Driver while functions and collations do not. Making that consistent breaks somebody in either direction - inheriting silently changes query results for anyone whose registration shadows a built-in, isolating turns a working CREATE VIRTUAL TABLE into "no such module" - so it wants an issue of its own rather than a doc commit. No behavior changes. Co-Authored-By:
-
cznic authored
NewConnector parsed the query string itself to reject a malformed dsn early. getVFSName is what newConn calls for the same purpose, and it does strictly more: the same url.ParseQuery, plus a check for conflicting vfs parameters. Calling it instead drops an import, removes the repeated parse, and keeps what NewConnector rejects aligned with what opening a connection rejects by construction rather than by matching two call sites by hand. The eager contract widens accordingly: "file:x?vfs=a&vfs=b" is now reported by NewConnector rather than by the first Connect. It was already an error either way, so no dsn changes from accepted to rejected. Everything a connection must exist to check - unknown parameters, out-of-range values - is still reported by Connect, as documented. v1.56.0 is not tagged, so no release shipped the narrower behavior. Co-Authored-By: -
cznic authored
Re-vendor the transpiled sources from ../libsqlite3 and ../libsqlite_vec. SQLite stays at 3.53.3; what changes is that the amalgamation now carries libsqlite3's sqlite_superjournal.patch, fixing an upstream 3.53.3 data-corruption bug in journal rollback. After a crash during the commit of a multi-database (ATTACH) transaction the super-journal name and its checksum can be left zeroed while the name length and the trailing magic survive; the checksum is a plain byte sum, so an all-zero name still validates, readSuperJournal() hands back a non-NULL pointer to an empty string, and pager_playback() deletes the hot journal without replaying it. All 19 targets carry the patch; master had it on none. Verified by expanding both the old and the new trees and diffing per target: 17 of 19 targets differ by exactly that one line. linux/s390x additionally picks up modernc.org/cc/v4 v4.29.1's MSB-first big-endian bit-field allocation, it being this module's only big-endian target. linux/riscv64 was regenerated on GCC 11.4.0 rather than 13.3.0, which only moves the COMPILER= entry PRAGMA compile_options reports and drops some unexported predefined macro constants; no SQLite code generation differs. go.mod goes to the current releases throughout. modernc.org/libc lands on v1.74.4 rather than the v1.74.3 ../libsqlite3 pins, that version being retracted upstream for a freeaddrinfo lock leak that deadlocks name resolution. Documentation: - openbsd/amd64 and openbsd/arm64 join the supported platforms table. Both have been in builder.json's test matrix since January and are cross-built by make build_all_targets, but were never listed. - Document the vfs DSN query parameter on Driver.Open. - Rewrite "Debug and development versions". It described a GO_GENERATE environment variable that is no longer read anywhere and ccgo/v3; drop with it the //go:generate naming generator.go, deleted from this repo back at SQLite 3.45.1, which made go generate ./... fail. - Add the package doc comments vec and vfs were missing. - Correct two stale claims in CLAUDE.md. make build_all_targets and make test are green (249 pass, 0 fail, 2 skip), as are ./vfs/..., ./pcache/... and ./vtab/.... Co-Authored-By: -
-
cznic authored
Documentation, one new validation rule, and test coverage on top of wsman's GitHub PR #6. No change to what _defensive itself does. conn.go: the comment above the db_config calls had been generalized to "connection-level sqlite3_db_config options are applied before applyQueryParams because the SQLite contract requires sqlite3_db_config(SQLITE_DBCONFIG_DQS_*) to be set before any statement is prepared". That contract is DQS-specific and does not extend to DEFENSIVE, which may be toggled at any point in a connection's life; defensive goes first because the PRAGMAs that follow must run under the restriction, which is a choice, not an API requirement. Restore the DQS comment verbatim on the DQS call and give _defensive its own rationale. sqlite.go: reject _defensive=1 together with _journal_mode=OFF (or _journal=OFF). SQLite turns PRAGMA journal_mode=OFF into a no-op that still reports success under defensive mode, so the combination previously opened a connection in which neither parameter had been honoured and nothing was reported. The check sits in the validation phase from v1.55.0, before any statement executes, so a rejected DSN still cannot leave the database half-configured. _pragma stays the documented exception: _pragma =journal_mode(OFF) alongside _defensive=1 runs and is silently ignored by SQLite. Only DSNs using _defensive can be affected and that parameter is new, so no DSN that opened before changes behavior. driver.go: expand the _defensive documentation to the depth of its _dqs and _error_rc neighbours -- what the mode observably does (writable_schema=ON, journal_mode=OFF and schema_version=N become silent no-ops; shadow-table and sqlite_dbpage writes error; reads, ordinary virtual table use and VACUUM are unaffected), the new _journal_mode conflict, and the two limits the name invites callers to overlook: it is a hardening measure rather than a sandbox for hostile files (this build has neither SQLITE_TRUSTED_SCHEMA=0 nor SQLITE_DQS=0 and there is no authorizer), and it is a property of the connection, not of the database file. defensive_test.go: rename TestUnmodifiedBehaviorIsNotAnADEDefensiveControl, whose "ADE" is an acronym from the contributor's downstream and means nothing here, to TestDefensiveAbsentOrFalseIsBaseline, and TestUnprotectedPoolIsA NegativeControl to TestDefensiveOffPoolIsANegativeControl so every test in the file groups under the feature. Add TestDefensiveRejectsJournalModeOff for the new rule, TestDefensiveDSNForms for the file: URI shape -- which keeps its query string all the way into sqlite3_open_v2, so SQLite parses _defensive too and must ignore it -- plus :memory: and shared-cache memory URIs, TestDefensiveLeavesOrdinaryUseIntact for the other half of the contract (fts5 create/insert/MATCH, foreign keys, VACUUM, integrity_check and every other DSN parameter that could collide, with direct fts5 shadow-table and sqlite_dbpage writes refused), and TestDefensiveIsPerConnection to pin the documented scope. Also adopt the file's `package sqlite // import ...` form. CHANGELOG.md: document both the parameter and the new conflict rule.
-
cznic authored
TestConcurrentGoroutines re-invokes itself under -race and treats anything the recursive run prints other than two known "cannot run here" messages as a failure. With CGO_ENABLED=0 the go tool refuses with "-race requires cgo; enable cgo by setting CGO_ENABLED=1", which matched neither, so the whole suite failed on an environment where the check simply cannot run -- and a CGo-free driver is a natural thing to build and test with cgo disabled. Accept that message alongside the existing two and skip, as the test already does for a toolchain without race support and for an unsupported VMA range. Nothing changes when cgo is available: the recursive -race run still executes and still has to pass. Found while reviewing GitHub PR #6, whose author hit it in a CGO_ENABLED=0 lane; unrelated to that change.
-
-
cznic authored
vec/ has carried the transpiled sqlite-vec sources since v1.47.0, but the module shipped only its own BSD-3-Clause LICENSE and the public-domain SQLite notice. sqlite-vec is Copyright (c) 2024 Alex Garcia and dual-licensed Apache-2.0 OR MIT; modernc.org/libsqlite_vec's generator elects MIT, whose terms require the copyright and permission notice to accompany substantial portions of the software. 2.8 MB of transpiled vec/ is a substantial portion. Attribution was never absent -- vec's package documentation names the extension, pins v0.1.9 and links upstream -- but the license text was. LICENSE-SQLITE_VEC: the notice, byte-identical to LICENSE-MIT in the upstream v0.1.9 archive and to the file libsqlite_vec extracts it into. vendor_libs/main.go: the omission was mechanical -- the tool copied the per-target transpiles and nothing else, so a plain cp of the notice would have survived only until the next `make vendor`. Copy it alongside the sources it belongs to, and treat a missing source as fatal: shipping the code without the notice is worse than not vendoring at all. SQLITE-LICENSE -> LICENSE-SQLITE, contents unchanged. This matches the new file beside it and the LICENSE-<upstream> convention the rest of the modernc.org repositories follow, but it is not only cosmetic. `go mod vendor` picks the metadata files it copies into a downstream vendor/ tree by matching each name against a fixed prefix list (cmd/go/internal/modcmd/vendor.go, metaPrefixes) that includes LICENSE, so a name merely ending in LICENSE was never propagated. Both notices now reach vendored builds, which is where the MIT terms on vec/ keep applying. Direct links to the old path will break. vec/patches.go: a License section on the package documentation, so an importer of vec sees on pkg.go.dev that this package is under a different license from the rest of the module. Found by an SBOM audit of the published module. Co-Authored-By: -
cznic authored
The vendoring reads one full per-target file at a time from ../libsqlite3 and ../libsqlite_vec, which assumes those checkouts ship expanded. They do today, but either may adopt the deduplicated layout that modernc.org/wa2c already ships (modernc.org/builder's NW autogen; see NW_GENERALIZATION_HANDOFF.md there) — and then those per-target files hold only each target's residue, so vendoring them would silently drop most of the package. Detect it instead of remembering it later. srcDir applies undup's own rule (base.go or base_g_*.go present means folded) and: - returns an expanded checkout as is: no copy, no subprocess, exactly the path this tool has always taken; - copies a deduplicated one to a temp directory and expands it THERE, never in place — expanding the sibling would leave that checkout dirty and tempt a "restore" that discards whatever else is uncommitted in it. go.mod and go.sum travel with the copy because undup resolves each unaliased import's package name by running "go list" inside the directory it expands. The undup pin is passed in from the Makefile, so the repo keeps one version of record wherever undup is invoked, and expansion that leaves shared files behind is a hard error naming the pin — that is what a version too old for the layout looks like. Also adds -libsqlite3 / -libsqlite_vec flags so this can be exercised against scratch copies without touching the real checkouts. Verified both ways: output byte-identical to the previous binary on today's expanded checkouts (38/38 files); and from deduplicated copies of both siblings, a complete vendoring whose lib/ and vec/ build for linux/{amd64,s390x}, darwin/arm64, windows/{amd64,386}, freebsd/386 and netbsd/amd64. Expect one-time textual churn whenever a sibling does flip: undup reconstructs declarations in its own order and recomputes imports, dropping ccgo's `var _ = math.Pi` and friends along with the imports they exist to keep. Same package, different bytes. Co-Authored-By: -
cznic authored
TestDBPageVtab's comment about -DSQLITE_ENABLE_DBPAGE_VTAB carried a trailing space, which made all_test.go the one hand-written file in the tree that gofmt -l reports. Comment text only; nothing else changes. Noticed by Ian Chechin while preparing GitLab merge request #135, which does not touch this file. Co-Authored-By:
-
cznic authored
The test registered its driver under the fixed name "sqlite_conn_hook_test". sql.Register panics on a name it has already seen and offers no way to undo a registration, so the second iteration of go test -count=2 took the whole test binary down with "sql: Register called twice for driver sqlite_conn_hook_test" -- not a failure of the code under test, and it hid whatever the remaining iterations would have found. Derive the name from an atomic counter instead, so each invocation registers its own driver, and close the sql.DB the test opens while here: it was leaked once per iteration. Nothing changes for a single run. Ian Chechin's driver_register_test.go in GitLab merge request #135 solves the same problem with a uniqueDriverName helper; the two can be folded together once that lands. Co-Authored-By:
-
Ian Chechin authored
Driver holds four categories of registration state but only connection hooks could be put on a constructed one. Functions and collations were reachable through the package-level API alone, and modules through the package-level driver only, so a constructed Driver was half-built: its modules field was written and read through the package-level instance, making it process-global state wearing a per-instance field. This is the additive half of #254, with one narrow, loud exception spelled out below: - registerFunction and registerCollation become methods on *Driver. The package-level RegisterFunction, RegisterScalarFunction, RegisterDeterministicScalarFunction and RegisterCollationUtf8 keep targeting the package-level driver, spelled explicitly now rather than by relying on the receiver named d shadowing the package variable of the same name. newDriver is renamed defaultDriver, since it read...
-
Ian Chechin authored
-
-
cznic authored
All three shipped as experimental in v1.53.0 and were deliberately left out of the supported platforms table until they had accumulated some real-world exposure. That period has elapsed: they have been in the builder matrix and in build_all_targets since, they pass on this release's commit alongside the seventeen platforms already listed, and no open issue reports a defect in any of them. The table therefore listed seventeen entries while the module shipped, cross-built and tested twenty; it now lists all twenty. Also set the v1.57.0 CHANGELOG date to the release date, and drop a stale claim in lib/hooks_linux_arm64.go that this module is "stuck on libc@v1.55.3" -- go.mod has pinned v1.74.4 since v1.56.0. The comment now says what the run-time patch is actually for.