Loading
Commits on Source 24
-
For improved DSN compatibility, the following keys have been added: _foreign_keys | _fk _busy_timeout | _timeout _journal_mode | _journal _synchronous | _sync _auto_vacuum | _vacuum _query_only Their values are passed as-is to exec for their respective PRAGMAs and not validated in any way. The compatibility here is intended when switching between modernc.org/sqlite and mattn/go-sqlite3, where it is handy to have higher DSN compatibility and to avoid dangerous mistakes like not having foreign keys enabled. -
Ian Chechin authoredFollow-up to the DSN shorthand keys carried over from !73: document _busy_timeout/_fk/_journal/_sync/_vacuum/_query_only in the driver DSN reference, note the fixed apply order (busy_timeout first, query_only last) at the query_only call site, and add a combined-DSN test asserting the keys coexist in a single DSN regardless of the order they appear in it.
-
Ian Chechin authored
Addresses the !134 review: - Apply _auto_vacuum before the _pragma list and the other shorthand keys. auto_vacuum only takes effect while the database is new; a _journal_mode change materialises page 1 and locks it in, so the previous order made _journal_mode=wal&_auto_vacuum=1 silently resolve to auto_vacuum=0. The "Test combined DSN" case now includes _auto_vacuum and asserts it reads back as 1, proving the fixed order. - Validate every shorthand value against the set github.com/mattn/go-sqlite3 accepts and return an error on anything else, instead of silently ignoring it, so a _synchronous or _foreign_keys typo no longer downgrades durability or drops enforcement. This also removes the DSN-injection surface these keys had, since a value is now either a known token or an error. - When a key and its alias are both present, the alias wins, matching mattn (_foreign_keys=off&_fk=on -> on). - Document the apply order, precedence, accepted values, and that only _pragma values are executed verbatim and must be trusted. - CHANGELOG entry for #134.
-
cznic authoredSupport more underscore keys in DSN (continues !73) See merge request !134 Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com>
-
cznic authored
The v1.55.0 entry said the new mattn-compatible keys were "parsed but had no effect in prior releases". They were not parsed at all; applyQueryParams never looked at them. It also described an unrecognized value as failing "instead of being silently ignored", a contrast against an unreleased iteration of !134 rather than against v1.54.0, which would read to a user upgrading as though their DSN had previously been tolerated and ignored. Restate both breaks against the last released version: keys that were ignored now take effect (with the consequence spelled out rather than just the key named), and a value outside the accepted set now fails the connection where the same DSN previously opened successfully - e.g. a duration-style _busy_timeout=5s, which is not the integer that key requires. The latter was missing entirely and is the sharper of the two. Co-Authored-By:
-
cznic authored
dsnPick treated an empty value as absent, so "_foreign_keys=on&_fk=" fell back to the primary key and enabled foreign keys. mattn/go-sqlite3 selects between a key and its alias by presence alone and then reads that key's value, so the empty alias wins and suppresses the PRAGMA entirely. Match that: pick on presence, return the selected key's value as-is. An empty value still skips the PRAGMA at the call sites, and skips validation with it, so an empty shorthand is not an error. Co-Authored-By: -
cznic authored
applyQueryParams validated each parameter as it reached it, so a DSN whose last parameter was bad still executed every PRAGMA ahead of it before failing. PRAGMA journal_mode and auto_vacuum are persistent changes to the database file, so file:x.db?_journal_mode=wal&_synchronous=bogus failed the connection and left x.db converted to WAL regardless. A failed Open must not change the database. Split the function into a validation phase and an apply phase: everything checkable is rejected before the first c.exec. Assignments to c stay in the validation phase, since newConn closes and discards the connection when this returns an error and they cannot outlive the failure. The documented apply order is unchanged and still covered by the combined-DSN test. This predates the !134 shorthand keys - master already behaved this way for _pragma combined with a late-rejected _txlock, which is why the new test covers that case too. _pragma remains the o...
-
cznic authored
Adds a v1.55.0 entry for validating every DSN parameter before applying any of them. It gets its own bullet rather than folding into the !134 paragraph because it changes behavior for parameters that have shipped for years - _txlock, _timezone, _time_format, _time_integer_format, _inttotime and _texttotime - so it concerns readers who never touch the new mattn-compatible keys. The alias-selection fix is folded into the !134 paragraph instead. v1.55.0 is not tagged, so no release ever shipped the fallback behavior and describing it as a fix would document a bug nobody could have hit. Also corrects that paragraph's closing claim that "all other existing parameters are unchanged", which the validation-order change made false: those parameters are affected in when they are validated, though not in what they accept or what they mean. Co-Authored-By:
-
cznic authored
database/sql offers no way to reach a registered driver: sql.Drivers returns names only and there is no sql.Driver(name). The single route from the "sqlite" name back to the driver value is (*sql.DB).Driver(), which is why callers who need it resort to db, _ := sql.Open("sqlite", "") drv := db.Driver() db.Close() That opens nothing - it works only because sql.Open does not connect and *Driver does not implement driver.DriverContext. Both are properties this package could change without noticing it had broken anyone. Callers want the driver in order to interpose on the physical connections database/sql opens: tracing, metrics, connection-scoped setup. Handing out the driver alone would not be enough, because the only way to get a wrapper into a *sql.DB through sql.Open is sql.Register, which is process-global, panics on a name it has already seen and cannot be undone - so a library has to invent a unique name per configuration. TestConnectionHook in all_test.go is an in-tree instance of that workaround. sql.OpenDB takes a connector directly and registers nothing, so a Connector answers both halves. Connect goes through d.Open rather than newConn, so the connections carry every function, collation, connection hook and vtab module registered on the package-level driver. A caller-constructed &sqlite.Driver{} carries none of them: the fields are unexported, so it silently yields connections missing all of it. The 2022 Connector prototype on danp-embed called newConn and had exactly that bug. Deliberately not done: - No accessor for the package-level driver. driver.Connector requires a Driver() method, so the singleton stays reachable, but as the driver.Driver interface - one method - and only through a type assertion godoc gives no hint of. Nothing returns *sqlite.Driver. - No OpenConnector on *Driver. Implementing driver.DriverContext would make sql.Open eager and move where DSN errors surface for every existing user. The sql.Open path is untouched. - No eager validation of parameter values. That validation is interleaved with assignments to *conn in applyQueryParams, and separating it is a larger change than this feature warrants. NewConnector checks the query string's syntax alone and documents that it does. NewConnector returns the driver.Connector interface rather than a concrete type, keeping the added surface to one symbol. That does not foreclose the fs.FS/VFS-lifecycle idea from the danp-embed prototype: database/sql discovers an optional Close on the connector by type assertion. Resolves #253. Co-Authored-By: -
cznic authored
Driver has been exported since 65f6d7e4 (2017), where "Make sqlite public" renamed the unexported sqlite type as part of publishing the package. The struct then held a connection counter and a mutex, so constructing one cost nothing and lost nothing. a9227519 (2022) moved the function and collation registries onto it and introduced the package-level instance, which is when a constructed Driver started silently lacking things; 14082cad (2023) added (*Driver).RegisterConnectionHook, which is only meaningful on an instance the caller builds, and so made the export load-bearing. The result is a type that is legitimately constructible for the private-hook pattern yet carries none of what the package-level Register* functions install. This documents that on the type and on the method, including the consequence easiest to miss: a registered function replaces a SQLite built-in of the same name, so a constructed Driver can evaluate upper(x) or date(x) differently from a connection opened through sql.Open. The half-global virtual table module behavior is documented as it stands rather than changed. registerModules reads the package-level driver, so modules do reach a constructed Driver while functions and collations do not. Making that consistent breaks somebody in either direction - inheriting silently changes query results for anyone whose registration shadows a built-in, isolating turns a working CREATE VIRTUAL TABLE into "no such module" - so it wants an issue of its own rather than a doc commit. No behavior changes. Co-Authored-By:
-
cznic authored
NewConnector parsed the query string itself to reject a malformed dsn early. getVFSName is what newConn calls for the same purpose, and it does strictly more: the same url.ParseQuery, plus a check for conflicting vfs parameters. Calling it instead drops an import, removes the repeated parse, and keeps what NewConnector rejects aligned with what opening a connection rejects by construction rather than by matching two call sites by hand. The eager contract widens accordingly: "file:x?vfs=a&vfs=b" is now reported by NewConnector rather than by the first Connect. It was already an error either way, so no dsn changes from accepted to rejected. Everything a connection must exist to check - unknown parameters, out-of-range values - is still reported by Connect, as documented. v1.56.0 is not tagged, so no release shipped the narrower behavior. Co-Authored-By: -
cznic authored
Re-vendor the transpiled sources from ../libsqlite3 and ../libsqlite_vec. SQLite stays at 3.53.3; what changes is that the amalgamation now carries libsqlite3's sqlite_superjournal.patch, fixing an upstream 3.53.3 data-corruption bug in journal rollback. After a crash during the commit of a multi-database (ATTACH) transaction the super-journal name and its checksum can be left zeroed while the name length and the trailing magic survive; the checksum is a plain byte sum, so an all-zero name still validates, readSuperJournal() hands back a non-NULL pointer to an empty string, and pager_playback() deletes the hot journal without replaying it. All 19 targets carry the patch; master had it on none. Verified by expanding both the old and the new trees and diffing per target: 17 of 19 targets differ by exactly that one line. linux/s390x additionally picks up modernc.org/cc/v4 v4.29.1's MSB-first big-endian bit-field allocation, it being this module's onl...
-
-
cznic authored
Documentation, one new validation rule, and test coverage on top of wsman's GitHub PR #6. No change to what _defensive itself does. conn.go: the comment above the db_config calls had been generalized to "connection-level sqlite3_db_config options are applied before applyQueryParams because the SQLite contract requires sqlite3_db_config(SQLITE_DBCONFIG_DQS_*) to be set before any statement is prepared". That contract is DQS-specific and does not extend to DEFENSIVE, which may be toggled at any point in a connection's life; defensive goes first because the PRAGMAs that follow must run under the restriction, which is a choice, not an API requirement. Restore the DQS comment verbatim on the DQS call and give _defensive its own rationale. sqlite.go: reject _defensive=1 together with _journal_mode=OFF (or _journal=OFF). SQLite turns PRAGMA journal_mode=OFF into a no-op that still reports success under defensive mode, so the combination previously opened ...
-
cznic authored
TestConcurrentGoroutines re-invokes itself under -race and treats anything the recursive run prints other than two known "cannot run here" messages as a failure. With CGO_ENABLED=0 the go tool refuses with "-race requires cgo; enable cgo by setting CGO_ENABLED=1", which matched neither, so the whole suite failed on an environment where the check simply cannot run -- and a CGo-free driver is a natural thing to build and test with cgo disabled. Accept that message alongside the existing two and skip, as the test already does for a toolchain without race support and for an unsupported VMA range. Nothing changes when cgo is available: the recursive -race run still executes and still has to pass. Found while reviewing GitHub PR #6, whose author hit it in a CGO_ENABLED=0 lane; unrelated to that change.
-
-
cznic authored
vec/ has carried the transpiled sqlite-vec sources since v1.47.0, but the module shipped only its own BSD-3-Clause LICENSE and the public-domain SQLite notice. sqlite-vec is Copyright (c) 2024 Alex Garcia and dual-licensed Apache-2.0 OR MIT; modernc.org/libsqlite_vec's generator elects MIT, whose terms require the copyright and permission notice to accompany substantial portions of the software. 2.8 MB of transpiled vec/ is a substantial portion. Attribution was never absent -- vec's package documentation names the extension, pins v0.1.9 and links upstream -- but the license text was. LICENSE-SQLITE_VEC: the notice, byte-identical to LICENSE-MIT in the upstream v0.1.9 archive and to the file libsqlite_vec extracts it into. vendor_libs/main.go: the omission was mechanical -- the tool copied the per-target transpiles and nothing else, so a plain cp of the notice would have survived only until the next `make vendor`. Copy it alongside the sources it belongs to, and treat a missing source as fatal: shipping the code without the notice is worse than not vendoring at all. SQLITE-LICENSE -> LICENSE-SQLITE, contents unchanged. This matches the new file beside it and the LICENSE-<upstream> convention the rest of the modernc.org repositories follow, but it is not only cosmetic. `go mod vendor` picks the metadata files it copies into a downstream vendor/ tree by matching each name against a fixed prefix list (cmd/go/internal/modcmd/vendor.go, metaPrefixes) that includes LICENSE, so a name merely ending in LICENSE was never propagated. Both notices now reach vendored builds, which is where the MIT terms on vec/ keep applying. Direct links to the old path will break. vec/patches.go: a License section on the package documentation, so an importer of vec sees on pkg.go.dev that this package is under a different license from the rest of the module. Found by an SBOM audit of the published module. Co-Authored-By: -
cznic authored
The vendoring reads one full per-target file at a time from ../libsqlite3 and ../libsqlite_vec, which assumes those checkouts ship expanded. They do today, but either may adopt the deduplicated layout that modernc.org/wa2c already ships (modernc.org/builder's NW autogen; see NW_GENERALIZATION_HANDOFF.md there) — and then those per-target files hold only each target's residue, so vendoring them would silently drop most of the package. Detect it instead of remembering it later. srcDir applies undup's own rule (base.go or base_g_*.go present means folded) and: - returns an expanded checkout as is: no copy, no subprocess, exactly the path this tool has always taken; - copies a deduplicated one to a temp directory and expands it THERE, never in place — expanding the sibling would leave that checkout dirty and tempt a "restore" that discards whatever else is uncommitted in it. go.mod and go.sum travel with the copy be... -
cznic authored
TestDBPageVtab's comment about -DSQLITE_ENABLE_DBPAGE_VTAB carried a trailing space, which made all_test.go the one hand-written file in the tree that gofmt -l reports. Comment text only; nothing else changes. Noticed by Ian Chechin while preparing GitLab merge request #135, which does not touch this file. Co-Authored-By:
-
cznic authored
The test registered its driver under the fixed name "sqlite_conn_hook_test". sql.Register panics on a name it has already seen and offers no way to undo a registration, so the second iteration of go test -count=2 took the whole test binary down with "sql: Register called twice for driver sqlite_conn_hook_test" -- not a failure of the code under test, and it hid whatever the remaining iterations would have found. Derive the name from an atomic counter instead, so each invocation registers its own driver, and close the sql.DB the test opens while here: it was leaked once per iteration. Nothing changes for a single run. Ian Chechin's driver_register_test.go in GitLab merge request #135 solves the same problem with a uniqueDriverName helper; the two can be folded together once that lands. Co-Authored-By:
-
Ian Chechin authored
Driver holds four categories of registration state but only connection hooks could be put on a constructed one. Functions and collations were reachable through the package-level API alone, and modules through the package-level driver only, so a constructed Driver was half-built: its modules field was written and read through the package-level instance, making it process-global state wearing a per-instance field. This is the additive half of #254, with one narrow, loud exception spelled out below: - registerFunction and registerCollation become methods on *Driver. The package-level RegisterFunction, RegisterScalarFunction, RegisterDeterministicScalarFunction and RegisterCollationUtf8 keep targeting the package-level driver, spelled explicitly now rather than by relying on the receiver named d shadowing the package variable of the same name. newDriver is renamed defaultDriver, since it read...
-
Ian Chechin authored
-
-
cznic authored
All three shipped as experimental in v1.53.0 and were deliberately left out of the supported platforms table until they had accumulated some real-world exposure. That period has elapsed: they have been in the builder matrix and in build_all_targets since, they pass on this release's commit alongside the seventeen platforms already listed, and no open issue reports a defect in any of them. The table therefore listed seventeen entries while the module shipped, cross-built and tested twenty; it now lists all twenty. Also set the v1.57.0 CHANGELOG date to the release date, and drop a stale claim in lib/hooks_linux_arm64.go that this module is "stuck on libc@v1.55.3" -- go.mod has pinned v1.74.4 since v1.56.0. The comment now says what the run-time patch is actually for.