mkvmerge does not support reading HD DVD subtitles files which often use the file extension .sup. This choice was made because HD DVD was the clear loser to Blu-rays in the fight for the HD content storage medium.
However, mkvmerge does support Blu-ray subtitles, also called PGS subtitles, which often use the file extension .sub. If you want to mux existing HD DVD subtitles, you have to convert them to the Blu-ray format first. Fortunately there's a freeware application called BDSup2Sub which does just that.