- What is a bug?
- Reporting bugs
- What to do if you think you've found a bug
- Writing a good bug report
What is a bug?
A bug is a defect in the software or supporting system: something that doesn't work as it should. FullMonte's master branch must always build automatically on Gitlab CI, and should (to the extent this is possible for a complex system) be easy to install, run, and develop on other systems. To the extent that the experience of new users, experienced users, and developers are part of how the software should work, unnecessary frustration is also a bug. Bugs can be present in the program code, build system, test cases, examples, or documentation. Sometimes a bug involves the interaction of more than one factor.
Examples of defects:
- Incorrect results
- Crashes or other failures to produce results
- Failures due to user error that do not give clear error messages
- Incorrect documentation
- Missing documentation
If the process is ill-specified, confusing, or does not work conditions where it is expected to, that is a defect in code, documentation, or both.
Bug reports are how we know what's not working as intended and produce better software. The Gitlab issue tracker is the one and only place to report bugs and gather relevant information. Emails, group discussions, in-person conversations, etc are OK of course (necessary to use the software, solve problems, and implement features) but should be considered not to exist from the point of view of project management. Those discussions are are very much at risk of being lost; open issues should be tracked, assigned, and worked on until closed. Even after close, all necessary information is preserved (in case we didn't quite solve the problem) and can be linked from merge requests to indicate when and how it was solved. If you have a problem with the software (including docs & user experience) and want it solved, the tracker is the place to post it.
We use the issue tracker because:
- It provides a single location to see everything that we know needs addressing with the software
- It allows people to see if something they're dealing with is a known issue
- It allows conversations and supporting data to be stored at a central place
- Issues can be matched to source code branches, commits, and merge requests
- It is easily searchable
- It helps share tasks and manage workload
- It helps prioritize tasks
- It permits assigning tasks and notifies people who are mentioned in discussions
- It clearly indicates which issues are open and which are deemed closed
If there is a problem with documentation, code, or any other aspects of the system listed above then that is the place to put it.
What to do if you think you've found a bug
Double-check the documentation (including examples you may be working from) and ensure it's not a user mistake or misinterpretation
Ensure you are using the most current version (those using the Docker image should always be on the latest master branch)
Go to the issue tracker page to view open issues and see if anyone else has reported a similar problem. If so, please comment on that issue to add any additional information.
If not, create a new issue.
Writing a good bug report
A bug report should state as specifically as possible how the actual state diverges from the expected or desired state. It should contain the information needed to illustrate what's wrong and reproduce the problem. Ask yourself: could I, using only the text and attachments of the bug report, reproduce the problem?
For FullMonte, that means including the data, script, and any other context required to run it, as well as the error message or other indication of what the problem is.
For documentation, it may be a description of a step that was missing/unclear.
Handy tips for formatting
The text in issues is interpreted as Gitlab-flavoured Markdown which has handy text-formatting facilities.
Single backticks (the key below
ESC) allow inline formatting of verbatim text (eg. shell commands & output).
Triple backticks on their own line delineate blocks of verbatim text (code or shell output). This block is formatted that way.
You can also mention someone by name eg.
@bob and they will receive notification.
You can (and should, when dealing with meshes or 3D output) include images
You can also refer to:
!123 merge requests
9deadbeef commits by SHA identifier
Please do not re-type or paraphrase text such as commands run or error messages. The best thing to do is copy-paste the exact text into a code block like this:
> command_I_ran bla bla bla ERROR!
Please also avoid screen shots where possible. They don't format as well, are not searchable, and cannot be copy-pasted when trying to replicate a problem. For really really big transcripts, you can attach a text file.
The best way to replicate what you're trying to do in FullMonte is to provide the Tcl script that you're running. It specifies exactly what should be done. If the data is provided, then it
Large data files can be attached. Please compress them (7zip is preferred) to economize on space and download time. Gitlab has attachment-size limits which may make compression necessary.
Visualizations of input meshes and output data
You can save a visualization setup in Paraview as a state file . If discussing defects in meshes or output data, the ideal would be to include a screenshot,
.vtk file (compressed), and a Paraview
.pvsm state file to replicate the screenshot.
In some cases when there is a defect in the build process, eg. it won't build on some other machine, detailed information about the system in use will be needed. That may mean OS, packages installed, library versions, tool versions, type of processor, CMake options, etc. as needed. When dealing with build issues be sure to include all information that might even possibly be relevant, including the Git commit being built.