TCP: Enhance Unseen Ack detection
The latest changes for ACKed unseen segments only partially improved their detection while introducing side-effects.
Reconciliation with maxseqtobeacked after loss events is too optmistically pushing forward when the receiver might not have acknowledged yet all the packets seen. We now rely on the unacked structure and better manage the maxseqtobeacked information in order to be more accurate.
See #8404 (closed) #11506 (closed) #18558 (closed) #18633
Merge request reports
Activity
mentioned in merge request !8862 (merged)
-
before-after comparison p31 p36 p99 shouldn't be marked, while p85 should.
p17 p28 p32 shouldn't be marked :
p6 - p26 should be marked, but not p28 which covers the ACK of p27 :
Edited by Eugène Adell added 12 commits
-
14de7f39...f96885ab - 11 commits from branch
wireshark:master - fa292032 - Draft: TCP: Enhance Unseen Ack detection
-
14de7f39...f96885ab - 11 commits from branch
added 1056 commits
-
fa292032...dfc2f43a - 1055 commits from branch
wireshark:master - 64aa0822 - TCP: Enhance Unseen Ack detection
-
fa292032...dfc2f43a - 1055 commits from branch
mentioned in issue #18558 (closed)
added 434 commits
-
64aa0822...2be5b24d - 433 commits from branch
wireshark:master - 6baa052b - TCP: Enhance Unseen Ack detection
-
64aa0822...2be5b24d - 433 commits from branch
Should not **Reassemble out-of-order segments ** allow to remove additional TCP ACKed unseen segment here: https://ask.wireshark.org/question/28210/loads-of-tcp-retransmission-tcp-out-of-order-tcp-dups/
-
Sorry for late reply, Valerii. No, you are talking about 2 unrelated different things. OoO reassembly allows the subdissectors (FTP, MySQL,..) to work on ordered data otherwise they might not be able to do their own job, while the other is a visual indication for the user based on the Sequence Analysis. While this indication might help him to go further in his investigation, it has no impact on subdissectors.
But in my case the millisecond resolution on windows just does not allow to get two TCP packets in the correct order. I get two packets with the same timestamp perfectly. And they are ordered wrong.
Edited by Valerii Zapod.Sure. Reproduced the very first moment (see frames 5, 6, 7 that are all the same timestamp). strangetimestamp.pcapng
Also here are the keys needed to decode TLS 1.3 from inside windows OS Schannel using https://b.poc.fun/decrypting-schannel-tls-part-1/
(the command that produces this is ffplay.exe https://www.modartt.com/data/audio/kivir/Guy-Sigsworth_Italian-Concerto-Slow-Movement.mp3 )
Got client random from SslHashHandshake: 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 Got client random from SslHashHandshake: 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 CLIENT_HANDSHAKE_TRAFFIC_SECRET 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 0faf2aceae863303bcbf7796c4e3e16dfcc23cfbd9841a835074ca5b1e84def7cbcfa817bbc670df76ccdd71da12cd8c SERVER_HANDSHAKE_TRAFFIC_SECRET 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 a4f87790f5a6a493e2708ea7c2f1ddc8ce0e73b2d62c168a108cc5ab66367cf2f335db24852951900609419d3a2eea33 EXPORTER_SECRET 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 bd79dfa5957d00560592f20074363aeb49d6b0a879f42dac0680e886a385c43524f661b794c735634c207a47581ed58f CLIENT_TRAFFIC_SECRET_0 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 eb5692256f1ae1577d9ba21db2c18a53fa993cc9f6a7e197fae242bcd7e4d2537b03c1a952f73afc1716973cb98a555b SERVER_TRAFFIC_SECRET_0 0d596240a28c3cf387647f6b04cadb9a7d3ad4d4c1683b6bdf9144d6d776fac5 e26aebca65caaf52804787d0a014fc9ebf7e01daf4bb1ecddcfc80b8ba17a622c983714e5de49f219eb2e8f07e335707 Got client random from SslHashHandshake: 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 Got client random from SslHashHandshake: 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 Got client random from SslHashHandshake: 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 Got client random from SslHashHandshake: 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 got key from SslGenerateMasterKey CLIENT_RANDOM 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 3578ad936bae7f40f036e0ff0144c74428a4938994096ee417a73cfb3256af9b1b35c974d5c33848baab81a4f422ea69 Got client random from SslGenerateSessionKeys's pParameterList: 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 got key from SslGenerateSessionKeys CLIENT_RANDOM 64721edfff7edb81a6a38759a38224386f4acd9b6e79e74eb4ff2992e7cfe412 3578ad936bae7f40f036e0ff0144c74428a4938994096ee417a73cfb3256af9b1b35c974d5c33848baab81a4f422ea69 Got client random from SslHashHandshake: 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af Got client random from SslHashHandshake: 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af Got client random from SslHashHandshake: 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af Got client random from SslHashHandshake: 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af got key from SslGenerateMasterKey CLIENT_RANDOM 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af 96f4ed788e03ae8112f6a38c8b54fce65b476ba0c8ac95c5b20c4c3be73fb1a9d3604cae4b3f78b1fd1adc1e0e81bf0c Got client random from SslGenerateSessionKeys's pParameterList: 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af got key from SslGenerateSessionKeys CLIENT_RANDOM 4079adbd0853859126a4f09bbdbd0f0e699393fd0439f6f40605bbde92ae41af 96f4ed788e03ae8112f6a38c8b54fce65b476ba0c8ac95c5b20c4c3be73fb1a9d3604cae4b3f78b1fd1adc1e0e81bf0c Got client random from SslHashHandshake: 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff Got client random from SslHashHandshake: 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff Got client random from SslHashHandshake: 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff Got client random from SslHashHandshake: 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff got key from SslGenerateMasterKey CLIENT_RANDOM 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff ad06b6d4eb425dc72027af4b88dc81f30937543ea972b76254d4498a31ae9dc5adb5be442e7c5df5140a27aeff8c60e0 Got client random from SslGenerateSessionKeys's pParameterList: 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff got key from SslGenerateSessionKeys CLIENT_RANDOM 64721ef7533c578f1855f566a858dae8c5b949d1e89caa7ad7b66aedddee5dff ad06b6d4eb425dc72027af4b88dc81f30937543ea972b76254d4498a31ae9dc5adb5be442e7c5df5140a27aeff8c60e0 Got client random from SslHashHandshake: 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 Got client random from SslHashHandshake: 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 Got client random from SslHashHandshake: 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 Got client random from SslHashHandshake: 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 CLIENT_HANDSHAKE_TRAFFIC_SECRET 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 47d7585e3ab5d5de68344ded71f3c18af10e43667590a3063d5047033c9f144ef89cc559dfa37e31fd33a8b491b71d4b SERVER_HANDSHAKE_TRAFFIC_SECRET 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 2cbca9deb6eb2f554fc7ba4672bd39b8c811adb3c58d31c065cd35c65517fe6ac2d34fc24eacb7238ccb70618757f4b0 EXPORTER_SECRET 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 177d90989c3a69d8eaf0f45d41374fc18ecaebc230202d7c1db0be30cf4ea62aeef1f55b83a5d3078fb10ec827725cd8 CLIENT_TRAFFIC_SECRET_0 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 67881f75851df6dd4949cec4703fe4c4f3592b674d914ec9e776bd3a2acfb91c7af86e1401b5f9c6cadbf8ca6dfc5c90 SERVER_TRAFFIC_SECRET_0 7ddd343d28a9172d38b058e97340bd26c26e11b45d8d7f685fb694510236ede2 b4a43d1f102a0d457b2864892a6c3fe8229c4229c9b30c1eb7064deedd00a4c23773c40a64e3d8fd46d87c6ee79a876c Got client random from SslHashHandshake: e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c Got client random from SslHashHandshake: e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c Got client random from SslHashHandshake: e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c Got client random from SslHashHandshake: e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c CLIENT_HANDSHAKE_TRAFFIC_SECRET e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c e0581fcb59eca7034b09afa13d309c073492f03617708cdd0747b6a116d4767bef4b98e4bbdc625d5315a4a76f01e17a SERVER_HANDSHAKE_TRAFFIC_SECRET e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c ce4deda28982589dcf2d90d866894ecebf59fdf4e68e96e36f85e918e05489ea862af5e554531fe731f342c52a93bb50 EXPORTER_SECRET e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c 007f227b2526c84e9f982c1705237f8c9d5c4424e11ecd39795383b436e7c89d39cce75af5e4282ceda4116962360faf CLIENT_TRAFFIC_SECRET_0 e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c edbddd2d0287d05860db7e75fc14e1a8973742e24393f16d6facb057bb70ca41803118c09694fb55dd7ba6c533a2e9ed SERVER_TRAFFIC_SECRET_0 e97e809c1e736caa6e068dc1a3b535bf77801c071c68e805f0cd7ba3a67d977c f0b89f2f60805f656ee8fffd6e0c4dc396068466d24357fa8385f5f559ca7788d3f5fdb4a2398b2f8d9b3010bd89ee05 Got client random from SslHashHandshake: b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 Got client random from SslHashHandshake: b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 Got client random from SslHashHandshake: b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 Got client random from SslHashHandshake: b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 CLIENT_HANDSHAKE_TRAFFIC_SECRET b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 ac23214ce9db3e5018a8f796f78b4582dd19c4764d8ae4f1bb0fc02497970d4b7d552a9a83f082d12a652a958ff295ca SERVER_HANDSHAKE_TRAFFIC_SECRET b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 da08a818dd29a292824454fe891a390e8456f642a1950e8f737a14a6742f2956ea9049e5f48c0a3ea1f29e73a08ebc50 EXPORTER_SECRET b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 b3ddcabe5f6ca7b513f39033f5ea62f6592d749bc596ec92c59e170135fb697ee623ac419d36371333390776f32ff4f5 CLIENT_TRAFFIC_SECRET_0 b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 8a6d688ecd2808971412f896d4470c4889258716ff785add8286085ec25aafa0081ff8836b510c12059cd6bc4cb978da SERVER_TRAFFIC_SECRET_0 b8ee755630a4a4a30d977ff48a07b2817881a6b163fadd5308995a63efc86489 212d3a9d03573fc8607bb9998e0295829f8ea03bc33ee60a0ef5a2774b8866e568dd20f581c6991e619d31765d7fafe7 Got client random from SslHashHandshake: 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad Got client random from SslHashHandshake: 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad Got client random from SslHashHandshake: 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad Got client random from SslHashHandshake: 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad got key from SslGenerateMasterKey CLIENT_RANDOM 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad 79fee00e931603215e3c4fca665ac3db30e0b0031d9be86f8ad5dd4f2edb7a0c92d5825932e98f82e107800ac6a74551 Got client random from SslGenerateSessionKeys's pParameterList: 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad got key from SslGenerateSessionKeys CLIENT_RANDOM 64721efb31fbe1cbc46d84d69795ac6483b93124eaffc28825cf5794205834ad 79fee00e931603215e3c4fca665ac3db30e0b0031d9be86f8ad5dd4f2edb7a0c92d5825932e98f82e107800ac6a74551 Got client random from SslHashHandshake: 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 Got client random from SslHashHandshake: 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 Got client random from SslHashHandshake: 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 Got client random from SslHashHandshake: 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 got key from SslGenerateMasterKey CLIENT_RANDOM 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 e57b1e8c6577a767e13aa5c0400a8b4480fe8ef809785009b3a5c74f98d5577093f2d86351240831605b090db495374f Got client random from SslGenerateSessionKeys's pParameterList: 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 got key from SslGenerateSessionKeys CLIENT_RANDOM 64721efde4082822c4ca1ef225ae047fc04d1a7ba74317bd41243f554c0e8f22 e57b1e8c6577a767e13aa5c0400a8b4480fe8ef809785009b3a5c74f98d5577093f2d86351240831605b090db495374f Got client random from SslHashHandshake: 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 Got client random from SslHashHandshake: 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 Got client random from SslHashHandshake: 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 Got client random from SslHashHandshake: 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 got key from SslGenerateMasterKey CLIENT_RANDOM 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 337ccb94026bcdc81e6c22cec2a7b997ba4550d89e1dd8e1fee2d4ee4471735c0aedcc7b387b92dbc4726b14f6920537 Got client random from SslGenerateSessionKeys's pParameterList: 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 got key from SslGenerateSessionKeys CLIENT_RANDOM 64721efd8bdb9462c58c4abe89eb48d5dda2c1a49d7bb4aa7e794e806684fdf3 337ccb94026bcdc81e6c22cec2a7b997ba4550d89e1dd8e1fee2d4ee4471735c0aedcc7b387b92dbc4726b14f6920537Edited by Valerii Zapod.-
The interpretation of packets 5,6,7 is good as the packet 5 should be after 6,7. Their timestamps being the same have nothing to do with that, they have been recorded in that order by the software used for that purpose (Wireshark too according to the file properties), and read in that same order according to First In First Out principle.
Why would packets arrive in a bad order, or in the right order but still be managed in a wrong way by the receiving computer (OS, network card or else) giving a capture file not reflecting what really happened on the wire, we can't say. That's why some people use specific expensive hardware for recording captures when the stakes are high. Meanwhile if your recording material always systematically produces captures with unordered packets, that would indicate a problem with the recording process itself. In that case I can only recommend to upgrade your OS or network driver or proceed with another material. I hope that helps.
Or you (or npcap) are using Microsoft APIs wrong. I mean of course you do because OS has no problems. If the packets have the same timestamp you need to read some other property to understand which packet came first. I use Windows 11 22H2 latest update and DCH driver for my wifi adapter is also latest. So...
Edited by Valerii Zapod.-
Or you (or npcap) are using Microsoft APIs wrong. I mean of course you do because OS has no problems.
Please gather technical elements that someone can work on, such as 2 captures by 2 different recording software (wireshark and another) of the very same traffic on the same computer, showing this wrong behavior. Ideally a 3rd capture from a network tap close to the computer would help. Then open a ticket for this specific problem.
NPCAP does not support 100 nanoseconds, even if it does new KeQuerySystemTimePrecise()
https://github.com/nmap/npcap/issues/583
And there is no other way like HW timestamps to get nanoseconds...
Edited by Valerii Zapod.-
I'm just an ordinary contributor and my opinion doesn't matter that much. Anyway, I believe the packets are in that order on the Wireshark GUI because they were recorded in that order on the file, and even with more precise timestamps you would get the very same result (packets in that order in the GUI, and same interpretation in the Info column). You might think that your recording system on Windows 11 is just perfect, but reliable recording hardware is expensive and probably for a good reason.
Two things tend to confirm that the problem is not in the recording software, but something else, ending with packets arriving in the wrong order.
First, only packets from 213.186.33.18 (not the connection initiator which is running the recording) are in the wrong order or spuriously retransmitted. Second, and more significantly, there are DUP ACKs from the connection initiator which is running the recording. That is the proof that the packets were received by the application in a wrong order, as the application is informing the sender that at least a packet is missing (once we have all the capture, we know it's not missing, it just arrived in the wrong order).
Again, I still don't see any wrong behavior in Wireshark here although there 1,4k issues opened. On the contrary, it is exactly pointing to the real issue you are facing to (packets arrived at the application level in the wrong order). You might want to investigate this further, but your suggestion to reorder TCP packets with the same timestamps would bring more confusing than helping.
I hope this helps.
mentioned in issue #19109
