Skip to content

WIP: partial fix for #56

Matthew Fernandez requested to merge smattr/graphviz:4737a3f0-6a1b-462a-895e-8bf86ad36bf9 into master

This is some work towards fixing #56 (closed). I've marked it as WIP because it seems this fix actually doesn't help the example in #56 (closed). After bypassing the trapezoid exhaustion issue, this example later causes Graphviz to allocate a huge array that exhausts memory. ASan back trace follows:

$ dot -Tsvg 56.dot -o /dev/null
Warning: Orthogonal edges do not currently handle edge labels. Try using xlabels.
==30743==AddressSanitizer's allocator is terminating the process instead of returning 0
==30743==If you don't like this behavior set allocator_may_return_null=1
==30743==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_allocator.cc:216 "((0)) != (0)" (0x0, 0x0)
    #0 0x7f8362ee4fa5  (/lib/x86_64-linux-gnu/libasan.so.5+0xf3fa5)
    #1 0x7f8362efff39 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/lib/x86_64-linux-gnu/libasan.so.5+0x10ef39)
    #2 0x7f8362eea006  (/lib/x86_64-linux-gnu/libasan.so.5+0xf9006)
    #3 0x7f8362eea043  (/lib/x86_64-linux-gnu/libasan.so.5+0xf9043)
    #4 0x7f8362e1832d  (/lib/x86_64-linux-gnu/libasan.so.5+0x2732d)
    #5 0x7f8362eda4f1 in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0xe94f1)
    #6 0x7f8362c36e61 in gcalloc /home/matthew/graphviz-56/lib/common/memory.c:42
    #7 0x7f8362d0eeb5 in convertSPtoRoute /home/matthew/graphviz-56/lib/ortho/ortho.c:158
    #8 0x7f8362d198b2 in orthoEdges /home/matthew/graphviz-56/lib/ortho/ortho.c:1346
    #9 0x7f835e92cc4e in _dot_splines /home/matthew/graphviz-56/lib/dotgen/dotsplines.c:280
    #10 0x7f835e931642 in dot_splines /home/matthew/graphviz-56/lib/dotgen/dotsplines.c:522
    #11 0x7f835e8f5120 in dotLayout /home/matthew/graphviz-56/lib/dotgen/dotinit.c:323
    #12 0x7f835e8f65ee in doDot /home/matthew/graphviz-56/lib/dotgen/dotinit.c:450
    #13 0x7f835e8f6bc9 in dot_layout /home/matthew/graphviz-56/lib/dotgen/dotinit.c:496
    #14 0x7f8362be376d in gvLayoutJobs /home/matthew/graphviz-56/lib/gvc/gvlayout.c:85
    #15 0x558075192960 in main /home/matthew/graphviz-56/cmd/dot/dot.c:132
    #16 0x7f836298f09a in __libc_start_main ../csu/libc-start.c:308
    #17 0x558075192269 in _start (/tmp/tmp.cfvBUBCvnb/bin/dot+0x2269)

I need some further direction on what to do here. Should I continue to debug this next, revealed error? Is the input graph actually invalid somehow? Is this out-of-memory scenario a duplicate or some other open issue?

Note the suggested fix on #56 (closed) to expand TRSIZE is invalid. ASan detects that this results in an out-of-bounds heap access.

Merge request reports