Segmentation fault for large files with at least 54 million packets
Summary
After opening a large capture file with at least 54 million packets, Wireshark Qt crashes (segmentation fault) at the moment it tries to render the GUI packet table layout.
Steps to reproduce
Generate a file with 54 million packets (1.6 GB big): randpkt -c 54000000 -t arp -b 15 randpkt.pcap
Open in wireshark (WARNING: this consumes about 9 GB of RAM during loading!): wireshark randpkt.pcap
What is the current bug behavior?
Crash (segmentation fault).
As can be seen in the following logs, QVector<QTreeViewItem>::realloc
causes the segfault and this is because of a known limitation in Qt, see https://doc.qt.io/qt-5/qvector.html#maximum-size-and-out-of-memory-conditions and https://forum.qt.io/topic/114950/qvector-realloc-throwing-sigsegv-when-very-large-surface3d-is-rendered. According to my calculations (for exact details, see below (#16908 (comment 430876048))), this translates to the 54 million packet boundary where things start to go wrong.
It seems that Qt 6 will resolve the issue (see e.g. https://github.com/qt/qtbase/commit/215ca735341b9487826023a7983382851ce8bf26), but I just wanted to mention this issue here, as we might want to look for a workaround in the meantime (or not...)?
What is the expected correct behavior?
No crash :)
Sample capture file
Generate a file with 54 million packets (1.6 GB big): randpkt -c 54000000 -t arp -b 15 randpkt.pcap
Relevant logs and/or screenshots
Debugging with gdb
and debug symbols installed for Qt and Wireshark reveals the following stack trace:
Thread 1 "wireshark" received signal SIGSEGV, Segmentation fault.
0x00007ffff1e88bba in QVector<QTreeViewItem>::realloc(int, QFlags<QArrayData::AllocationOption>) () from /lib64/libQt5Widgets.so.5
(gdb) bt
#0 0x00007ffff1e88bba in QVector<QTreeViewItem>::realloc(int, QFlags<QArrayData::AllocationOption>) () from /lib64/libQt5Widgets.so.5
#1 0x00007ffff1e88d0b in QVector<QTreeViewItem>::resize(int) () from /lib64/libQt5Widgets.so.5
#2 0x00007ffff1e80040 in QTreeViewPrivate::layout(int, bool, bool) () from /lib64/libQt5Widgets.so.5
#3 0x00007ffff1e86411 in QTreeView::doItemsLayout() () from /lib64/libQt5Widgets.so.5
#4 0x00007ffff1e7d35e in QTreeView::visualRect(QModelIndex const&) const () from /lib64/libQt5Widgets.so.5
#5 0x00007ffff1e78520 in QTreeView::visualRegionForSelection(QItemSelection const&) const () from /lib64/libQt5Widgets.so.5
#6 0x00007ffff1e0ab63 in QAbstractItemView::selectionChanged(QItemSelection const&, QItemSelection const&) () from /lib64/libQt5Widgets.so.5
#7 0x00007ffff1e8820e in QTreeView::selectionChanged(QItemSelection const&, QItemSelection const&) () from /lib64/libQt5Widgets.so.5
#8 0x00005555557c9c38 in PacketList::selectionChanged (this=0x5555565b6ea0, selected=..., deselected=...) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/packet_list.cpp:459
#9 0x00007ffff0dbbfb0 in void doActivate<false>(QObject*, int, void**) () from /lib64/libQt5Core.so.5
#10 0x00007ffff0d51dcb in QItemSelectionModel::selectionChanged(QItemSelection const&, QItemSelection const&) () from /lib64/libQt5Core.so.5
#11 0x00007ffff0d5a096 in QItemSelectionModel::select(QItemSelection const&, QFlags<QItemSelectionModel::SelectionFlag>) () from /lib64/libQt5Core.so.5
#12 0x00007ffff0d557a1 in QItemSelectionModel::select(QModelIndex const&, QFlags<QItemSelectionModel::SelectionFlag>) () from /lib64/libQt5Core.so.5
#13 0x00007ffff0d53564 in QItemSelectionModel::setCurrentIndex(QModelIndex const&, QFlags<QItemSelectionModel::SelectionFlag>) () from /lib64/libQt5Core.so.5
#14 0x00005555557c5a70 in PacketList::goFirstPacket (user_selected=false, this=0x5555565b6ea0) at /usr/include/qt5/QtCore/qabstractitemmodel.h:60
#15 PacketList::goFirstPacket (this=0x5555565b6ea0, user_selected=<optimized out>) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/packet_list.cpp:1493
#16 0x000055555565417f in cf_read (cf=<optimized out>, reloading=<optimized out>) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/file.c:699
#17 0x00005555557b9265 in MainWindow::openCaptureFile (this=0x555555d4fc00, cf_path=..., read_filter=..., type=<optimized out>, is_tempfile=0) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/main_window_slots.cpp:254
#18 0x00005555557b9970 in MainWindow::openCaptureFile (display_filter=..., cf_path=..., this=0x555555d4fc00) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/main_window.h:316
#19 MainWindow::on_actionFileOpen_triggered (this=0x555555d4fc00) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/main_window_slots.cpp:1684
#20 0x00005555556886f3 in MainWindow::qt_metacall (this=0x555555d4fc00, _c=QMetaObject::InvokeMetaMethod, _id=91, _a=0x7fffffffc9b0) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/qtui_autogen/EWIEGA46WW/moc_main_window.cpp:1868
#21 0x00007ffff0dbbc98 in void doActivate<false>(QObject*, int, void**) () from /lib64/libQt5Core.so.5
#22 0x00007ffff1ba9876 in QAction::triggered(bool) () from /lib64/libQt5Widgets.so.5
#23 0x00007ffff1bac161 in QAction::activate(QAction::ActionEvent) () from /lib64/libQt5Widgets.so.5
#24 0x00007ffff1d29d5a in QMenuPrivate::activateCausedStack(QVector<QPointer<QWidget> > const&, QAction*, QAction::ActionEvent, bool) () from /lib64/libQt5Widgets.so.5
#25 0x00007ffff1d313c2 in QMenuPrivate::activateAction(QAction*, QAction::ActionEvent, bool) () from /lib64/libQt5Widgets.so.5
#26 0x00007ffff1bf136e in QWidget::event(QEvent*) () from /lib64/libQt5Widgets.so.5
#27 0x00007ffff1bb0063 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#28 0x00007ffff1bb5891 in QApplication::notify(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#29 0x00007ffff0d8cfc0 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /lib64/libQt5Core.so.5
#30 0x00007ffff1d3215f in QMenuPrivate::mouseEventTaken(QMouseEvent*) () from /lib64/libQt5Widgets.so.5
#31 0x00007ffff1d322f6 in QMenu::mouseReleaseEvent(QMouseEvent*) () from /lib64/libQt5Widgets.so.5
#32 0x00007ffff1bf136e in QWidget::event(QEvent*) () from /lib64/libQt5Widgets.so.5
#33 0x00007ffff1bb0063 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#34 0x00007ffff1bb5891 in QApplication::notify(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#35 0x00007ffff0d8cfc0 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /lib64/libQt5Core.so.5
#36 0x00007ffff1bb478a in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) () from /lib64/libQt5Widgets.so.5
#37 0x00007ffff1c0a98f in QWidgetWindow::handleMouseEvent(QMouseEvent*) () from /lib64/libQt5Widgets.so.5
#38 0x00007ffff1c0cdde in QWidgetWindow::event(QEvent*) () from /lib64/libQt5Widgets.so.5
#39 0x00007ffff1bb0063 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#40 0x00007ffff0d8cfc0 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /lib64/libQt5Core.so.5
#41 0x00007ffff117116b in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) () from /lib64/libQt5Gui.so.5
#42 0x00007ffff115315b in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Gui.so.5
#43 0x00007fffdfc51c9e in xcbSourceDispatch(_GSource*, int (*)(void*), void*) () from /lib64/libQt5XcbQpa.so.5
#44 0x00007ffff160878f in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#45 0x00007ffff1608b18 in g_main_context_iterate.constprop () from /lib64/libglib-2.0.so.0
#46 0x00007ffff1608be3 in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#47 0x00007ffff0dd8b73 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Core.so.5
#48 0x00007ffff0d8b91b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Core.so.5
#49 0x00007ffff0d935a6 in QCoreApplication::exec() () from /lib64/libQt5Core.so.5
#50 0x000055555564d1f1 in main (argc=<optimized out>, qt_argv=<optimized out>) at /usr/src/debug/wireshark-3.2.6-1.fc32.x86_64/ui/qt/main.cpp:964
Build information
Wireshark 3.2.6 (Git commit 4f9257fb8ccc)
Copyright 1998-2020 Gerald Combs <gerald@wireshark.org> and contributors.
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/gpl-2.0.html>
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiled (64-bit) with Qt 5.14.2, with libpcap, with POSIX capabilities (Linux),
with libnl 3, with GLib 2.64.4, with zlib 1.2.11, with SMI 0.4.8, with c-ares
1.15.0, with Lua 5.1.5, with GnuTLS 3.6.14 and PKCS #11 support, with Gcrypt
1.8.5, with MIT Kerberos, with MaxMind DB resolver, with nghttp2 1.41.0, without
brotli, without LZ4, without Zstandard, without Snappy, without libxml2, with
QtMultimedia, without automatic updates, with SpeexDSP (using bundled
resampler), without SBC, without SpanDSP, without bcg729.
Running on Linux 5.8.13-200.fc32.x86_64, with Intel(R) Core(TM) i7-8665U CPU @
1.90GHz (with SSE4.2), with 15815 MB of physical memory, with locale
en_US.UTF-8, with libpcap version 1.9.1 (with TPACKET_V3), with GnuTLS 3.6.15,
with Gcrypt 1.8.5, with zlib 1.2.11, binary plugins supported (0 loaded).
Built using gcc 10.2.1 20200723 (Red Hat 10.2.1-1).