Skip to content
  • Matthew Fernandez's avatar
    fix 'merge_chain' assertion failure · 0a3d2434
    Matthew Fernandez authored
    When calling `merge_chain` from `interclexp`, this code apparently did not
    anticipate that both the condition `ND_rank(agtail(e)) == ND_rank(aghead(e))`
    and `ED_to_virt(prev) != NULL` could be true at once. In this case, the merge
    can happen but the `.to_virt` member of `e` needs to be overwritten; it is not
    `NULL` on entry to `merge_chain`.
    
    This issue appears to have existed since the first revision of Graphviz.
    
    These is a lot of uncertainty around what the expected behavior of this code is
    and whether this fix is correct. So here is some of my background thoughts:¹
    
      `merge_chain` has a leading `assert(ED_to_virt(e) == NULL)`, presumably trying
      to express the assumption that it is not overwriting an existing back pointer.
      But the code here in `interclexp` seems to be deliberately trying to overwrite
      this pointer. There seemed to me two possible local fixes:
    
        1. Remove the assertion
        2. Write `NULL` to `ED_to_virt(e)` in advance
    
      The first possibility seemed more risky. This would allow other functions to
      accidentally call into merge_chain with a non-null `ED_to_virt(e)` and for
      this to go undetected.
    
      On the other hand, it's possible my change here is mistaken and what the
      original author intended was for the true case of the branch on line 181 to
      continue after updating `ED_to_virt(e) = prev`, making the call to
      `merge_chain` not reached in this scenario. It certainly looks pretty weird
      for the code after my changes to set `ED_to_virt(e)` on lines 181-184 and then
      subsequently overwrite this value on line 187. But I was guessing the initial
      write is relevant for the case where we continue.
    
      As you can tell, all of this is (barely) educated guess work. I find the
      intent of this code very hard to determine.
    
    Gitlab: fixes #121
    
    ¹ Quoted from discussion on
      graphviz/graphviz!2724 (comment 1005393861).
    0a3d2434
To find the state of this project's repository at the time of any of these versions, check out the tags.