bgpscanner fails, but bgpdump does not? ---> bad packet detected (Oversized or inconsistent BGP update NLRI field)
Hi,
I downloaded the file rib.20200709.0000.bz2 from https://www.isolario.it/Isolario_MRT_data/Korriban/2020_07/ and when I run bgpscanner, at a given point it starts printing thousands and thousands of lines saying what you find below. Clearly some entries fail to be read.
- From which feeder-AS?
- Is this being printed in the middle or at the end?
I ask this since all these prints in stdout seem to seriously compromise the performance of bgpscanner. I had to kill the process and, comparing results with bgpdump, I see differences. Taking AS199036 as an example:
$ bgpdump -m -O bgpdump.rib rib.20200709.0000.bz2 --> finishes OK.
$ grep "|199036|" bgpdump.rib | wc -l
78308686
$ bgpscanner -a 199036 -o bgpscanner.rib rib.20200709.0000.bz2 --> prints lots of error messages, had to stop it.
$ wc -l bgpscanner.rib
70638882 bgpscanner.rib
As you see, I am actually missing some entries. It seems the prints to stdout where done while reading... but this gets the process "stuck", or at least it taking too long.
I need to parse probably multiple RIBs of the Korriban collector. This problem makes me doubt about the feasibility relying on bgpscanner...it seems not possible to read the data in a reasonable time.
Regards, Julian
bgpscanner: rib.20200709.0000.bz2: bad packet detected (Oversized or inconsistent BGP update NLRI field) binary packet dump follows: ASN32BIT: yes ADDPATH: yes 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0x00 | 0x62 | 0x02 | 0x00 | 0x00 | 0x00 | 0x4b | 0x40 | 0x01 | 0x01 | 0x00 | 0x40 | 0x02 | 0x12 | 0x02 | 0x04 | 0x00 | 0x03 | 0x09 | 0x7c | 0x00 | 0x00 | 0x0d | 0xa3 | 0x00 | 0x00 | 0x0d | 0x1c | 0x00 | 0x00 | 0x09 | 0xb5 | 0xc0 | 0x08 | 0x0c | 0x0d | 0xa3 | 0x07 | 0xd0 | 0x0d | 0xa3 | 0x07 | 0xd4 | 0x0d | 0xa3 | 0x23 | 0x2a | 0x80 | 0x0e | 0x20 | 0x10 | 0x20 | 0x01 | 0x10 | 0x20 | 0x01 | 0x08 | 0x88 | 0x20 | 0x01 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x01 | 0x30 | 0x00 | 0x00 | 0x00 | 0x00 | 0x5a | 0x30 | 0x20 | 0x01 | 0x06 | 0x7c | 0x13 | 0x48
bgpscanner: rib.20200709.0000.bz2: bad packet detected (Oversized or inconsistent BGP update NLRI field) binary packet dump follows: ASN32BIT: yes ADDPATH: yes 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0x00 | 0x7d | 0x02 | 0x00 | 0x00 | 0x00 | 0x66 | 0x40 | 0x01 | 0x01 | 0x00 | 0x40 | 0x02 | 0x16 | 0x02 | 0x05 | 0x00 | 0x03 | 0x09 | 0x7c | 0x00 | 0x03 | 0x1d | 0x3c | 0x00 | 0x00 | 0x84 | 0xe3 | 0x00 | 0x00 | 0x08 | 0x98 | 0x00 | 0x00 | 0x09 | 0xb5 | 0x80 | 0x04 | 0x04 | 0x00 | 0x00 | 0x00 | 0x00 | 0xc0 | 0x08 | 0x1c | 0x84 | 0xe3 | 0xfd | 0xe8 | 0x84 | 0xe3 | 0xfe | 0x10 | 0x84 | 0xe3 | 0xfe | 0x11 | 0xfb | 0xf0 | 0x00 | 0x64 | 0xfb | 0xf0 | 0x08 | 0x66 | 0xfb | 0xf0 | 0x84 | 0xe3 | 0xff | 0xe8 | 0x4e | 0x21 | 0x80 | 0x0e | 0x20 | 0x10 | 0x20 | 0x01 | 0x10 | 0x20 | 0x01 | 0x08 | 0x88 | 0x20 | 0x01 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x00 | 0x01 | 0x30 | 0x00 | 0x00 | 0x00 | 0x00 | 0x1d | 0x30 | 0x20 | 0x01 | 0x06 | 0x7c | 0x13 | 0x48