Skip to content
  1. Aug 17, 2024
    • Antoine Beaupré's avatar
      tentative fixes for syncthing clusterfuckery · 899a35c8
      Antoine Beaupré authored
      Here are the notes i painfully wrote down to get there, then realizing
      those conflicts *kept* coming and I had to fix the underlying
      issues. this is kept here in case they come back.
      
      # Sample sync problems
      ## Book 1
      
      ### Conflict 20240802-232230-P2R7L33.lua
      
      ```
      3,4c3,19
      <     ["annotations"] = {},
      <     ["bookmarks"] = {},
      ---
      >     ["annotations"] = {
      >         [1] = {
      >             ["chapter"] = "Chapter 7",
      >             ["datetime"] = "2024-07-26 02:43:05",
      >             ["page"] = "/body/DocFragment[10]/body/div/section/p[33]/text()[1].295",
      >             ["pageno"] = 110,
      >             ["text"] = "in Chapter 7",
      >         },
      >     },
      >     ["bookmarks"] = {
      >         [1] = {
      >             ["chapter"] = "Chapter 7",
      >             ["datetime"] = "2024-07-26 02:43:05",
      >             ["notes"] = "in Chapter 7",
      >             ["page"] = "/body/DocFragment[10]/body/div/section/p[33]/text()[1].295",
      >         },
      >     },
      66c81
      <     ["last_xpointer"] = "/body/DocFragment[4]/body/div/section/h1/a[2]/text().0",
      ---
      >     ["last_xpointer"] = "/body/DocFragment[27]/body/div/section/h1.0",
      70c85
      <     ["percent_finished"] = 0.011834319526627,
      ---
      >     ["percent_finished"] = 0.90532544378698,
      ```
      
      A is the conflict file, B is the good version. good version has more
      percent and extra annotations.
      
      ### Conflict 20240803-022141-P2R7L33.lua and 20240803-022217-P2R7L33.lua
      
      Identical diff. 
      
      ### .old conflict files
      
      We assume those are useless (and, by the way, find the
      `metadata.lua.old` to be equally garbage).
      
      ## Book 2
      
      ### Conflict 20240802-232233-P2R7L33.opf
      
      ```
      anarcat@angela:Mastering Emacs (1668)[1]$ diff  metadata.sync-conflict-20240802-232233-P2R7L33.opf metadata.opf
      8c8
      <         <dc:contributor opf:file-as="calibre" opf:role="bkp">calibre (6.26.0) [https://calibre-ebook.com]</dc:contributor>
      ---
      >         <dc:contributor opf:file-as="calibre" opf:role="bkp">calibre (6.13.0) [https://calibre-ebook.com]</dc:contributor>
      11a12,14
      >         <meta name="calibre:author_link_map" content="{&quot;Mickey Petersen&quot;: &quot;&quot;}"/>
      >         <meta name="calibre:series" content="Mastering Emacs"/>
      >         <meta name="calibre:series_index" content="28"/>
      19c22
      <         <reference type="cover" title="Couverture" href="cover.jpg"/>
      ---
      >         <reference type="cover" title="Cover" href="cover.jpg"/>
      ```
      
      spurious calibre metadata changes, latter has more fields, deleted
      conflict.
      
      `20240803-022300-P2R7L33.opf ` and `20240803-022211-P2R7L33`
      identical.
      
      ## Book 3
      
      ### -conflict-20240724-090036-CVJZX7H.lua 
      
      ```
      1c1
      < -- /mnt/onboard/Ben Collier/Tor (1724)/Tor - Ben Collier.sdr/metadata.epub.lua
      ---
      > -- /storage/emulated/0/books/Ben Collier/Tor (1724)/Tor - Ben Collier.sdr/metadata.epub.lua
      2a3
      >     ["annotations"] = {},
      6c7
      <     ["cache_file_path"] = "./cache/cr3cache/Tor_-_Ben_Collier.epub.a8bfab68.1.cr3",
      ---
      >     ["cache_file_path"] = "/storage/emulated/0/koreader/cache/cr3cache/Tor_-_Ben_Collier.epub.a8bfab68.1.cr3",
      17c18
      <     ["copt_font_size"] = 24,
      ---
      >     ["copt_font_size"] = 34,
      39,40c40,41
      <     ["doc_pages"] = 373,
      <     ["doc_path"] = "/mnt/onboard/Ben Collier/Tor (1724)/Tor - Ben Collier.epub",
      ---
      >     ["doc_pages"] = 470,
      >     ["doc_path"] = "/storage/emulated/0/books/Ben Collier/Tor (1724)/Tor - Ben Collier.epub",
      52d52
      <     ["fulltext_search_last_search_text"] = "redacted",
      66c66
      <     ["last_xpointer"] = "/body/DocFragment[18]/body/section/section[19]/ul/li[75]/text().0",
      ---
      >     ["last_xpointer"] = "/body/DocFragment[12]/body/section/p[21]/text().0",
      72c72
      <     ["percent_finished"] = 1,
      ---
      >     ["percent_finished"] = 0.4936170212766,
      74c74
      <     ["readermenu_tab_index"] = 1,
      ---
      >     ["readermenu_tab_index"] = 7,
      81c81
      <         ["pages"] = 373,
      ---
      >         ["pages"] = 470,
      87c87
      <         ["modified"] = "2024-07-22",
      ---
      >         ["modified"] = "2024-07-23",
      ```
      
      harder. conflicted version is marked as finished while current one
      isn't, which is wrong. Note that the latter comes from the Android
      device (`/storage/emulated`) and has a later timestamp: it's likely
      the file was open there *after* the book was finished and that
      erroneous state was saved.
      
      Conflict file moved in place instead of actual file.
      
      ## Book 3
      
      ### conflict-20240810-010430-P2R7L33
      
      ```
      3c3,6
      <     ["annotations"] = {},
      ---
      >     ["book_style_tweak"] = "\
      > p.chapopeningtextno-indentcotx::before {content: '●'; display: block; text-align: center; font-size: 50%; margin: 1em}\
      > h1 + p.chapopeningtextno-indentcotx::before {content: ''; display: block; text-align: center; font-size: 50%; margin: 1em}",
      >     ["book_style_tweak_enabled"] = true,
      6c9,20
      <         [2] = 1,
      ---
      >         [2] = 176,
      >     },
      >     ["bookmarks"] = {
      >         [1] = {
      >             ["chapter"] = "Chapter Five: Breakout",
      >             ["datetime"] = "2024-08-04 23:51:03",
      >             ["highlighted"] = true,
      >             ["notes"] = "Now, at risk of being accused of sexist stereotyping, I’d like to note that a lot of retail psychology (and sales) depends on the fact that men and women shop (or are trained to shop) in different ways. Broadly: women forage while men hunt. This is especially true of clothing, where I’ve noticed Mo can spend all afternoon searching for exactly the right pair of shoes and end up with a jacket, two bras, a skirt, and an umbrella—while I begin to sweat bullets and edge close to a panic attack if I can’t find exactly the correct size of plain black tee-shirt in Marks and Spencer within thirty seconds of entering the front door.",
      >             ["page"] = "/body/DocFragment[13]/body/section/div/p[37]/text()[1].0",
      >             ["pos0"] = "/body/DocFragment[13]/body/section/div/p[37]/text()[1].0",
      >             ["pos1"] = "/body/DocFragment[13]/body/section/div/p[37]/text()[2].372",
      >         },
      8d21
      <     ["bookmarks"] = {},
      11,12c24,25
      <     ["cache_file_path"] = "/storage/emulated/0/koreader/cache/cr3cache/The_Delirium-_Stross.epub.b1bcea22.1.cr3",
      <     ["config_panel_index"] = 1,
      ---
      >     ["cache_file_path"] = "./cache/cr3cache/The_Delirium-_Stross.epub.b1bcea22.1.cr3",
      >     ["config_panel_index"] = 6,
      14c27
      <     ["copt_block_rendering_mode"] = 3,
      ---
      >     ["copt_block_rendering_mode"] = 2,
      43,45c56,58
      <     ["css"] = "./data/epub.css",
      <     ["doc_pages"] = 413,
      <     ["doc_path"] = "/storage/emulated/0/books/Charles Stross/The Delirium Brief (1705)/The Delirium Brief - Charles Stross.epub",
      ---
      >     ["css"] = "./data/html5.css",
      >     ["doc_pages"] = 634,
      >     ["doc_path"] = "/mnt/onboard/Charles Stross/The Delirium Brief (1705)/The Delirium Brief - Charles Stross.epub",
      56c69
      <     ["floating_punctuation"] = 1,
      ---
      >     ["floating_punctuation"] = 0,
      64c77,88
      <     ["highlight"] = {},
      ---
      >     ["highlight"] = {
      >         [255] = {
      >             [1] = {
      >                 ["chapter"] = "Chapter Five: Breakout",
      >                 ["datetime"] = "2024-08-04 23:51:03",
      >                 ["drawer"] = "lighten",
      >                 ["pos0"] = "/body/DocFragment[13]/body/section/div/p[37]/text()[1].0",
      >                 ["pos1"] = "/body/DocFragment[13]/body/section/div/p[37]/text()[2].372",
      >                 ["text"] = "Now, at risk of being accused of sexist stereotyping, I’d like to note that a lot of retail psychology (and sales) depends on the fact that men and women shop (or are trained to shop) in different ways. Broadly: women forage while men hunt. This is especially true of clothing, where I’ve noticed Mo can spend all afternoon searching for exactly the right pair of shoes and end up with a jacket, two bras, a skirt, and an umbrella—while I begin to sweat bullets and edge close to a panic attack if I can’t find exactly the correct size of plain black tee-shirt in Marks and Spencer within thirty seconds of entering the front door.",
      >             },
      >         },
      >     },
      69c93
      <     ["hyph_trust_soft_hyphens"] = true,
      ---
      >     ["hyph_trust_soft_hyphens"] = false,
      72c96
      <     ["last_xpointer"] = "/body/DocFragment[10]/body/section/div/h1/a/text().0",
      ---
      >     ["last_xpointer"] = "/body/DocFragment[23]/body/section/div/p[1]/text().0",
      76c100
      <     ["percent_finished"] = 0.19128329297821,
      ---
      >     ["percent_finished"] = 1,
      78c102
      <     ["readermenu_tab_index"] = 2,
      ---
      >     ["readermenu_tab_index"] = 1,
      82c106
      <         ["highlights"] = 0,
      ---
      >         ["highlights"] = 1,
      85c109
      <         ["pages"] = 413,
      ---
      >         ["pages"] = 634,
      91c115,118
      <         ["font_family_all_inherit"] = true,
      ---
      >         ["footnote-inpage_classic_classnames_smaller"] = true,
      >         ["footnote-inpage_epub_smaller"] = false,
      >         ["footnote-inpage_fb2"] = false,
      >         ["text_align_most_justify"] = true,
      94,95c121,122
      <         ["modified"] = "2024-07-29",
      <         ["status"] = "reading",
      ---
      >         ["modified"] = "2024-08-06",
      >         ["status"] = "complete",
      ```
      
      compatibility problem? one uses highlight, the other bookmarks. latter
      is finished, other isn't, and latter has a book style tweak. kept
      latter (normal version).
      
      `sync-conflict-20240810-133125-P2R7L33.lua` and `conflict-20240810-115213-P2R7L33.lua` identical.
      
      ## Book 4
      
      ### conflicts on covers
      
      opened a file browser, looked at thumbnails, deleted garbage.
      
      Note that there were similar conflicts on similar files here:
      ```
      cover.jpg
      cover.sync-conflict-20240802-232234-P2R7L33.jpg
      cover.sync-conflict-20240803-022210-P2R7L33.jpg
      cover.sync-conflict-20240803-022300-P2R7L33.jpg
      metadata.opf
      metadata.sync-conflict-20240802-232231-P2R7L33.opf
      metadata.sync-conflict-20240803-022210-P2R7L33.opf
      metadata.sync-conflict-20240803-022300-P2R7L33.opf
      ```
      
      Assumed the OPF conflicts were like the cover.
      
      ## Book 5
      
      ### conflict-20240802-232235-P2R7L33.opf
      
      ```
      16c16
      <         <meta name="calibre:timestamp" content="2024-02-21T02:50:30.862801+00:00"/>
      ---
      >         <meta name="calibre:timestamp" content="2024-02-21T02:50:30+00:00"/>
      ```
      
      spurious calibre metadata change (timestamp *resolution*!),
      deleted. two more sync conflicts
      (`metadata.sync-conflict-20240803-022210-P2R7L33.opf`
      `metadata.sync-conflict-20240803-022300-P2R7L33.opf`) were of
      identical checksum
      
      ## Book 6
      
      
      ### conflict-20240802-232231-P2R7L33.opf
      
      ```
      2c2
      < <package xmlns="http://www.idpf.org/2007/opf" unique-identifier="uuid_id">
      ---
      > <package xmlns="http://www.idpf.org/2007/opf" unique-identifier="uuid_id" version="2.0">
      8c8
      <         <dc:contributor opf:file-as="calibre" opf:role="bkp">calibre (0.8.51) [http://calibre-ebook.com]</dc:contributor>
      ---
      >         <dc:contributor opf:file-as="calibre" opf:role="bkp">calibre (6.13.0) [https://calibre-ebook.com]</dc:contributor>
      13,17c13,18
      <         <meta content="{&quot;Émile Nelligan&quot;: &quot;&quot;}" name="calibre:author_link_map"/>
      <         <meta content="2012-09-22T03:01:11+00:00" name="calibre:timestamp"/>
      <         <meta content="Poésies complètes" name="calibre:title_sort"/>
      <         <meta name="calibre:user_metadata:#emprunteur" content="{&quot;is_category&quot;: true, &quot;#extra#&quot;: null, &quot;kind&quot;: &quot;field&quot;, &quot;is_custom&quot;: true, &quot;is_csp&quot;: false, &quot;colnum&quot;: 1, &quot;column&quot;: &quot;value&quot;, &quot;rec_index&quot;: 22, &quot;search_terms&quot;: [&quot;#emprunteur&quot;], &quot;link_column&quot;: &quot;value&quot;, &quot;is_multiple2&quot;: {&quot;cache_to_list&quot;: &quot;|&quot;, &quot;ui_to_list&quot;: &quot;&amp;&quot;, &quot;list_to_ui&quot;: &quot; &amp; &quot;}, &quot;is_multiple&quot;: &quot;|&quot;, &quot;datatype&quot;: &quot;text&quot;, &quot;#value#&quot;: [], &quot;category_sort&quot;: &quot;value&quot;, &quot;table&quot;: &quot;custom_column_1&quot;, &quot;is_editable&quot;: true, &quot;label&quot;: &quot;emprunteur&quot;, &quot;display&quot;: {&quot;is_names&quot;: true}, &quot;name&quot;: &quot;Emprunté par&quot;}"/>
      <         <meta name="calibre:user_metadata:#read" content="{&quot;is_category&quot;: false, &quot;#extra#&quot;: null, &quot;kind&quot;: &quot;field&quot;, &quot;is_custom&quot;: true, &quot;is_csp&quot;: false, &quot;colnum&quot;: 2, &quot;column&quot;: &quot;value&quot;, &quot;rec_index&quot;: 23, &quot;search_terms&quot;: [&quot;#read&quot;], &quot;link_column&quot;: &quot;value&quot;, &quot;is_multiple2&quot;: {}, &quot;is_multiple&quot;: null, &quot;datatype&quot;: &quot;bool&quot;, &quot;#value#&quot;: false, &quot;category_sort&quot;: &quot;value&quot;, &quot;table&quot;: &quot;custom_column_2&quot;, &quot;is_editable&quot;: true, &quot;label&quot;: &quot;read&quot;, &quot;display&quot;: {}, &quot;name&quot;: &quot;Lu&quot;}"/>
      ---
      >         <meta name="calibre:author_link_map" content="{&quot;Émile Nelligan&quot;: &quot;&quot;}"/>
      >         <meta name="calibre:timestamp" content="2012-09-22T03:01:11+00:00"/>
      >         <meta name="calibre:title_sort" content="Poésies complètes"/>
      >         <meta name="calibre:user_metadata:#emprunteur" content="{&quot;table&quot;: &quot;custom_column_1&quot;, &quot;column&quot;: &quot;value&quot;, &quot;datatype&quot;: &quot;text&quot;, &quot;is_multiple&quot;: &quot;|&quot;, &quot;kind&quot;: &quot;field&quot;, &quot;name&quot;: &quot;Emprunté par&quot;, &quot;search_terms&quot;: [&quot;#emprunteur&quot;], &quot;label&quot;: &quot;emprunteur&quot;, &quot;colnum&quot;: 1, &quot;display&quot;: {&quot;is_names&quot;: true}, &quot;is_custom&quot;: true, &quot;is_category&quot;: true, &quot;link_column&quot;: &quot;value&quot;, &quot;category_sort&quot;: &quot;value&quot;, &quot;is_csp&quot;: false, &quot;is_editable&quot;: true, &quot;rec_index&quot;: 22, &quot;#value#&quot;: [], &quot;#extra#&quot;: null, &quot;is_multiple2&quot;: {&quot;cache_to_list&quot;: &quot;|&quot;, &quot;ui_to_list&quot;: &quot;&amp;&quot;, &quot;list_to_ui&quot;: &quot; &amp; &quot;}}"/>
      >         <meta name="calibre:user_metadata:#mm_annotations" content="{&quot;table&quot;: &quot;custom_column_3&quot;, &quot;column&quot;: &quot;value&quot;, &quot;datatype&quot;: &quot;comments&quot;, &quot;is_multiple&quot;: null, &quot;kind&quot;: &quot;field&quot;, &quot;name&quot;: &quot;Annotations&quot;, &quot;search_terms&quot;: [&quot;#mm_annotations&quot;], &quot;label&quot;: &quot;mm_annotations&quot;, &quot;colnum&quot;: 3, &quot;display&quot;: {}, &quot;is_custom&quot;: true, &quot;is_category&quot;: false, &quot;link_column&quot;: &quot;value&quot;, &quot;category_sort&quot;: &quot;value&quot;, &quot;is_csp&quot;: false, &quot;is_editable&quot;: true, &quot;rec_index&quot;: 23, &quot;#value#&quot;: null, &quot;#extra#&quot;: null, &quot;is_multiple2&quot;: {}}"/>
      >         <meta name="calibre:user_metadata:#read" content="{&quot;table&quot;: &quot;custom_column_2&quot;, &quot;column&quot;: &quot;value&quot;, &quot;datatype&quot;: &quot;bool&quot;, &quot;is_multiple&quot;: null, &quot;kind&quot;: &quot;field&quot;, &quot;name&quot;: &quot;Lu&quot;, &quot;search_terms&quot;: [&quot;#read&quot;], &quot;label&quot;: &quot;read&quot;, &quot;colnum&quot;: 2, &quot;display&quot;: {}, &quot;is_custom&quot;: true, &quot;is_category&quot;: false, &quot;link_column&quot;: &quot;value&quot;, &quot;category_sort&quot;: &quot;value&quot;, &quot;is_csp&quot;: false, &quot;is_editable&quot;: true, &quot;rec_index&quot;: 24, &quot;#value#&quot;: false, &quot;#extra#&quot;: null, &quot;is_multiple2&quot;: {}}"/>
      20c21
      <         <reference href="cover.jpg" type="cover" title="Couverture"/>
      ---
      >         <reference type="cover" title="Cover" href="cover.jpg"/>
      ```
      
      seems like metadata garbage, unreadable. other sync conflicts
      identical
      
      
      `metadata.sync-conflict-20240802-232231-P2R7L33.opf`
      `metadata.sync-conflict-20240803-022210-P2R7L33.opf`
      `metadata.sync-conflict-20240803-022300-P2R7L33.opf`
      
      ## Book 7
      
      Another book was similar.
      
      # Other notes
      
      After dealing with *all* those conflicts, for some reason conflict
      files *returned*. Again with the device ID `P2R7L3` which is, no
      surprise, the kobo reader. That device has this weird issue that it
      seems to rescan all files at each restart, and treats *all* of them as
      new. This takes a ridiculously long time, of course, but more
      crucially, is likely the cause of those conflicts.
      
      In fact, while being on both devices, I cannot convince the kobo to
      accept certain changes, namely deleting files. 
      899a35c8
  2. Apr 29, 2024
  3. Sep 06, 2022
  4. Jul 27, 2021
  5. Jul 22, 2021
  6. Jul 21, 2021
Loading