coding.c: fix undefined behavior with pointer arithmetics

Alexander Usrequested to merge
alexander-us/libtasn1:clang_10_ptr_arith_ub into master

asn1_der_coding contained unchecked pointer arithmetics.

As source code shows, ider can be NULL pointer. C standard states that arithmetic using NULL pointers gives undefined behavior (C99, 6.5.6, clause 8). LLVM (beginning from version 10) started to optimize pointer arithmetic in comparisons with NULL with assumptions that addition of NULL and non-zero value will produce undefined behavior (https://reviews.llvm.org/D66608). This means that code like ptr + x == NULL will be turned to ptr == NULL because if ptr is NULL then NULL + x will be UB. Short summary of what happened in asn1_der_coding is in this code: https://godbolt.org/z/DAo0Bt.

Without my patch clang UBSan reports problems in copynode and Test_tree tests and these tests fail with -O2 flag (CC=clang-10 CFLAGS="-O2 -fsanitize=undefined"):

../../libtasn1/lib/coding.c:1207:56: runtime error: applying non-zero offset 7 to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../libtasn1/lib/coding.c:1207:56 in
../../libtasn1/lib/coding.c:1225:54: runtime error: applying non-zero offset 287 to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../../libtasn1/lib/coding.c:1225:54 in
LIBTASN1 ERROR: VALUE_NOT_VALID
Cannot copy node
FAIL copynode (exit status: 1)

Checklist

  • Code modified for feature
  • Test suite updated with functionality tests
  • Test suite updated with negative tests
  • Documentation updated

Reviewer's checklist:

  • There is a test suite reasonably covering new functionality or modifications
  • Function naming, parameters, return values, types, etc., are consistent with other code
  • This feature/change has adequate documentation added
  • No obvious mistakes in the code

Merge request reports