Draft: Force inline in debug builds (!718) · Merge requests · nsnam / ns-3-dev

Gabriel Ferreira requested to merge Gabrielcarvfer/ns-3-dev:forceInline into master Sep 06, 2021
The debug builds are pretty slow and there are many reasons for that.
One of them is that -O0 doesn't not force inlining, resulting in a bunch of additional jumps, that leads to increased instruction cache miss rates.
How much more? A lot. Look at the LogComponent.IsEnabled line in the profiling results.
Notice the two L2_CACHE_MISS columns. The first is for instructions and the second is for data.
Both of them are L2 misses caused by L1 misses, which will lead to L3/memory seeks and slow things down.
Profiling
Forced inlines should speed things up by 1.17x. Currently, it gets enabled by defining NS3_FORCE_INLINE and I've included that define by default in debug builds. This is an alternative to using a higher optimization option (e.g. -Og).
P.S.: I've included a little script in the utils folder: speedup.py. It compares the output of test.py -d runs and prints the absolute and geometric speedups to keep track of that. ./utils/speedup.py previous_run.txt current_run.txt will output the results to the terminal, -o output_file.txt will save it to the file and -s will sort the tests/examples by speedup instead of by name.
Update: updating results
Upstream vs this MR
./utils/speedup.py notforceinlined2.txt forceinlined_branch.txt -f -td
     Before (s)       After (s)  Difference (s) Test name
        899.229         730.217        -169.012 Example examples/wireless/wifi-he-network --simulationTime=0.3 --frequency=5 --useRts=0 --useExtendedBlockAck=1 --minExpectedThroughput=6 --maxExpectedThroughput=1033
        855.049         691.041        -164.008 Example examples/wireless/wifi-he-network --simulationTime=0.3 --frequency=5 --useRts=1 --minExpectedThroughput=6 --maxExpectedThroughput=745
        741.144         602.799        -138.345 Example examples/wireless/wifi-he-network --simulationTime=0.25 --frequency=5 --useRts=0 --minExpectedThroughput=6 --maxExpectedThroughput=844
        489.053         404.385         -84.668 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=4 --clientNss=4 --stepTime=0.1
        320.324         241.907         -78.417 Example examples/wireless/wifi-80211n-mimo --simulationTime=0.1 --step=10
        475.143         398.730         -76.413 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=4 --clientNss=4 --stepTime=0.1
        491.821         419.308         -72.513 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=4 --clientNss=4 --stepTime=0.1
        325.124         254.817         -70.307 Example examples/wireless/wifi-he-network --simulationTime=0.3 --frequency=2.4 --useRts=1 --minExpectedThroughput=6 --maxExpectedThroughput=223
        393.815         326.513         -67.302 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=4 --clientNss=4 --stepTime=0.1
        492.694         428.784         -63.910 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=4 --clientNss=4 --stepTime=0.1
        399.176         335.555         -63.621 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=4 --clientNss=4 --stepTime=0.1
        276.209         214.036         -62.173 Example examples/wireless/wifi-he-network --simulationTime=0.25 --frequency=2.4 --useRts=0 --minExpectedThroughput=6 --maxExpectedThroughput=238
        338.545         278.575         -59.970 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=4 --clientNss=4 --stepTime=0.1
        338.581         282.298         -56.283 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=4 --clientNss=4 --stepTime=0.1
        343.969         298.607         -45.362 TestSuite lte-frequency-reuse
        168.363         133.229         -35.134 Example examples/wireless/wifi-vht-network --simulationTime=0.2 --useRts=0  --minExpectedThroughput=5 --maxExpectedThroughput=583
        159.107         127.300         -31.807 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
        163.988         136.472         -27.516 Example examples/wireless/wifi-vht-network --simulationTime=0.2 --useRts=1  --minExpectedThroughput=5 --maxExpectedThroughput=543
        147.047         121.432         -25.615 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
        152.111         130.478         -21.633 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
        112.573          91.317         -21.256 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ac --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=400 --clientShortGuardInterval=400 --serverNss=1 --clientNss=1 --stepTime=0.1
        144.220         123.369         -20.851 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
        145.825         125.242         -20.583 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
        145.415         125.185         -20.230 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
        144.109         123.999         -20.110 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
        145.851         126.817         -19.034 Example src/wifi/examples/wifi-bianchi --validate --phyMode=OfdmRate6Mbps --nMinStas=5 --nMaxStas=10 --duration=15
         74.443          57.635         -16.808 Example examples/wireless/wifi-ht-network --simulationTime=0.2 --frequency=2.4 --useRts=0 --minExpectedThroughput=5 --maxExpectedThroughput=132
        119.560         103.142         -16.418 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         75.225          59.489         -15.736 Example examples/wireless/wifi-ht-network --simulationTime=0.2 --frequency=5 --useRts=1 --minExpectedThroughput=5 --maxExpectedThroughput=131
        107.989          92.278         -15.711 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ac --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
        108.056          92.838         -15.218 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ac --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
        140.799         125.671         -15.128 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
         75.766          60.689         -15.077 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
         80.139          65.077         -15.062 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
         72.662          57.793         -14.869 Example examples/wireless/wifi-ht-network --simulationTime=0.2 --frequency=5 --useRts=0 --minExpectedThroughput=5 --maxExpectedThroughput=135
         73.947          59.309         -14.638 Example examples/wireless/wifi-ht-network --simulationTime=0.2 --frequency=2.4 --useRts=1 --minExpectedThroughput=5 --maxExpectedThroughput=128
         64.305          49.845         -14.460 Example examples/tcp/tcp-validation --firstTcpType=cubic --linkRate=50Mbps --baseRtt=50ms --queueUseEcn=0 --stopTime=20s --validate=cubic-50ms-no-ecn
        141.458         127.932         -13.526 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
        118.617         105.111         -13.506 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         58.085          44.669         -13.416 TestSuite three-gpp-v2v-channel-condition-model
        107.970          94.874         -13.096 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ac --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=400 --clientShortGuardInterval=400 --serverNss=1 --clientNss=1 --stepTime=0.1
         79.014          66.684         -12.330 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
         89.683          77.672         -12.011 TestSuite lte-x2-handover-measures
        145.984         134.087         -11.897 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=160 --clientChannelWidth=160 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
         80.750          68.927         -11.823 Example src/lte/examples/lena-radio-link-failure --numberOfEnbs=2 --interSiteDistance=700 --simTime=17
         60.313          48.528         -11.785 Example examples/tcp/tcp-validation --firstTcpType=cubic --linkRate=50Mbps --baseRtt=50ms --queueUseEcn=1 --stopTime=20s --validate=cubic-50ms-ecn
         80.649          69.274         -11.375 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         69.227          57.988         -11.239 Example src/lte/examples/lena-radio-link-failure --numberOfEnbs=1 --simTime=17
         72.116          60.901         -11.215 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
         62.277          51.508         -10.769 TestSuite propagation-channel-condition-model
         79.230          68.471         -10.759 Example src/lte/examples/lena-radio-link-failure --numberOfEnbs=2 --useIdealRrc=0 --interSiteDistance=700 --simTime=17
         95.245          84.582         -10.663 Example examples/tcp/tcp-validation --firstTcpType=dctcp --linkRate=50Mbps --baseRtt=80ms --queueUseEcn=1 --stopTime=40s --validate=dctcp-80ms
         67.859          57.529         -10.330 Example src/lte/examples/lena-radio-link-failure --numberOfEnbs=1 --useIdealRrc=0 --simTime=17
         72.005          62.310          -9.695 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-6GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
         71.782          62.547          -9.235 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
         93.596          84.374          -9.222 TestSuite lte-carrier-aggregation
         76.164          66.964          -9.200 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=1600 --clientShortGuardInterval=1600 --serverNss=1 --clientNss=1 --stepTime=0.1
         52.687          43.782          -8.905 TestSuite lte-aggregation-throughput-scale
         57.003          48.105          -8.898 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ac --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         56.186          47.563          -8.623 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ac --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=400 --clientShortGuardInterval=400 --serverNss=1 --clientNss=1 --stepTime=0.1
         81.968          73.545          -8.423 Example src/uan/examples/uan-cw-example
         67.437          59.209          -8.228 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11ax-5GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         39.881          32.145          -7.736 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-2.4GHz --serverChannelWidth=40 --clientChannelWidth=40 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1
         43.445          36.060          -7.385 TestSuite probabilistic-v2v-channel-condition-model
         78.442          71.229          -7.213 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-6GHz --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         40.163          33.376          -6.787 Example examples/tcp/tcp-variants-comparison
         50.980          44.198          -6.782 Example examples/wireless/wifi-mixed-network --simulationTime=1
         37.666          30.944          -6.722 Example src/lr-wpan/examples/lr-wpan-error-distance-plot
         30.783          24.091          -6.692 Example src/wifi/examples/wifi-manager-example --wifiManager=MinstrelHt --standard=802.11n-5GHz --serverChannelWidth=40 --clientChannelWidth=40 --serverShortGuardInterval=400 --clientShortGuardInterval=400 --serverNss=4 --clientNss=4 --stepTime=0.1
         64.096          57.459          -6.637 Example examples/wireless/wifi-multi-tos --simulationTime=1 --nWifi=16 --useRts=1 --useShortGuardInterval=1
         54.859          48.303          -6.556 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ac --serverChannelWidth=80 --clientChannelWidth=80 --serverShortGuardInterval=800 --clientShortGuardInterval=800 --serverNss=1 --clientNss=1 --stepTime=0.1
         37.243          30.912          -6.331 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11n-2.4GHz --serverChannelWidth=40 --clientChannelWidth=40 --serverShortGuardInterval=400 --clientShortGuardInterval=400 --serverNss=4 --clientNss=4 --stepTime=0.1
         40.773          34.454          -6.319 Example src/wifi/examples/wifi-manager-example --wifiManager=Ideal --standard=802.11ax-5GHz --serverChannelWidth=40 --clientChannelWidth=40 --serverShortGuardInterval=3200 --clientShortGuardInterval=3200 --serverNss=1 --clientNss=1 --stepTime=0.1


556 improved, 172 worsened, 9 stable
Absolute Speedup: 1.180x (Before 04h 35m - After 03h 53m)
forceinlined_branch.txt notforceinlined2.txt forceinlined_branch_noptr.txt
Edited Feb 10, 2022 by Gabriel Ferreira
Draft: Force inline in debug builds

Merge request reports
