Skip to content
  • Jamie Tanna's avatar
    def9666e
    Wrap all `INSERT`s with transactions · def9666e
    Jamie Tanna authored
    The performance of DMD hasn't been particularly great - hence why
    there's been the need to introduce a progress bar for most of the
    operations - and it turns out it's to do with SQLite's transactions[0].
    
    As we're not wrapping any of our updates in a transaction, SQLite is
    producing one for us, which is slower than us managing it ourselves, as
    we know how many `INSERT`s we'll be doing at a given time.
    
    This speeds up considerably, for instance the `integration_test` phase
    currently takes ~90 seconds instead of ~240 seconds, and results in no
    longer even seeing the progress bar on `import`s.
    
    Closes #89.
    
    [0]: https://stackoverflow.com/questions/3852068/sqlite-insert-very-slow
    def9666e
    Wrap all `INSERT`s with transactions
    Jamie Tanna authored
    The performance of DMD hasn't been particularly great - hence why
    there's been the need to introduce a progress bar for most of the
    operations - and it turns out it's to do with SQLite's transactions[0].
    
    As we're not wrapping any of our updates in a transaction, SQLite is
    producing one for us, which is slower than us managing it ourselves, as
    we know how many `INSERT`s we'll be doing at a given time.
    
    This speeds up considerably, for instance the `integration_test` phase
    currently takes ~90 seconds instead of ~240 seconds, and results in no
    longer even seeing the progress bar on `import`s.
    
    Closes #89.
    
    [0]: https://stackoverflow.com/questions/3852068/sqlite-insert-very-slow
Loading