Issue Description
TCP BBR does not increase its rate after ProbeRTT (or loss). Here is a brief demo (Github Gist).
Fix
After some time of debugging, I found that TcpRateSample::m_deliveryRate is smaller than the actual rate. The root cause is that m_ackElapsed is not updated correctly.
As reference, Linux calculates ack interval in function tcp_rate_gen(), which is TcpRateLinux::GenerateSample() in ns-3.
But ns-3 calculates m_ackElapsed in TcpRateSample::SkbDelivered(). The initial implementation is as below:
if (m_rateSample.m_priorDelivered == 0 || skbInfo.m_delivered > m_rateSample.m_priorDelivered)
{
m_rateSample.m_priorDelivered = skbInfo.m_delivered;
m_rateSample.m_priorTime = skbInfo.m_deliveredTime;
m_rateSample.m_isAppLimited = skbInfo.m_isAppLimited;
m_rateSample.m_sendElapsed = skb->GetLastSent() - skbInfo.m_firstSent;
m_rateSample.m_ackElapsed = Simulator::Now() - m_rateSample.m_priorTime;
m_rateSampleTrace(m_rateSample);
m_rate.m_firstSentTime = skb->GetLastSent();
}The case that above code goes wrong is: two skbs can be sent in the interval between receiving ACKs during x1.25 cycle of ProbeBW. These two skbs have same m_deliveredTime, so that only the first one can update m_ackElapsed, while m_delivered is increased for each of them.
As a result, m_deliveryRate is wrong because m_delivered and m_ackElapsed is inconsistent.
The initial implementation of tcp-rate-ops would over-estimate the rate when receiving acks of packets sent in x1.25 cycle of ProbeBW.
Commit e16ff5ee made things weirder: it still calculates m_ackElapsed in the above if-clause, but before m_priorTime is updated, which over-estimates the ack interval. m_deliveryRate is correct only when the bottleneck link is full (queue formed) and m_ackElapsed is not updated for one ack as explained above.