Improve memory used in the Web IDE state by moving properties away from the file object
Description
There's opportunity for improving the memory footprint of the Vuex state in the Web IDE... This includes both the actual data being stored but also the number of callbacks and functions registered.
Original description
Over time in the Web IDE we have added more data into the Vuex store, most of the data doesn't get used. We should do an audit on the data & remove the data that is no longer required.
We also need to figure out the hanging when receiving the file list. For this we pass the data into a web worker to put into the format that we require, however when the data comes back Vue makes it reactive which then makes the browser hang (the more files, the larger the hang). I'm not really sure what we can do here, but its worth investigating as its noticeable especially with GitLab CE.
Actionable items
Remove dead code
We'll clearly gain performance / reduce memory usage if we remove dead code. Even if the benefit is like 1ms, we should do it. I have identified that the following properties on the file object are not used anymore:
-
parentTreeUrl
!33654 (merged) -
permalink
!33654 (merged) -
commitsPath
!33654 (merged) -
blamePath
!33654 (merged) -
replaces
!33654 (merged) -
renderError
!33654 (merged) -
lastCommitPath
!33654 (merged) -
html
!33654 (merged) -
renderError
!33654 (merged) -
lastCommit
Move some properties to view / utils
Some properties on the file object are used in certain views. It is quite wasteful to store that property on every file, instead of computing the property in the particular view:
-
previewMode
-> should be moved torepo_editor.vue
, compute based on file extension. !33870 (merged) -
eol
-> should be moved toide_status_list.vue
, compute based on file contents. !33870 (merged) -
fileLanguage
-> we can write a util function that asks Monaco what language a file is using based on its extension. This would probably break in the future when we add support for custom languages via.gitattributes
, but we can handle that later when we begin implementing that feature. -
base64
-> When we upload a new file in the IDE,base64
is set to true, and is false in all other cases. Finally when the file is about to be committed, we use this property to send a keyencoding
to the backend. The key has two values:text
orbase64
. We can get rid of thebase64
property and when we commit, we can use a utility function calledisBase64
that computes on the file'srawPath
(which will always be a data URL for a base64 file). !33870 (merged) -
projectId
-> This properties are unnecessary on the file object since at a time only one project and branch can be open in the Web IDE. The usage can be replaced withcurrentProjectId
in state. -
branchId
-> Same as above. Can be replaced withcurrentBranchId
in state. -
url
-> Can be computed based on thecurrentBranchId
,currentProjectId
andfile.path
. We can define a getter calledgetFileURL
, that does this job. -
parentPath
-> There is a util calledgetPathParent
that does this. So we don't need this property on every file. -
name
-> The name of the file can be computed again by getting thelastPathComponent
. -
binary
-> We have a util function calledisTextFile
. We can check if a file is binary or not by doing!isTextFile(...)
. -
changed
-> We already have an array ofchangedFiles
. We can create a getter calledisFileChanged
, that looks up the file in that array. This makes our lookup inchangedFiles
O(n), where n is the size of changed files, but it also reduces memory usage by O(n), where n is the number of files in a repository. So I think this optimises the average case, but worsens the worst case (where a very large folder is renamed and a lot of files are changed).Also, since the
changedFiles
array just contains references to thestate.entries
object, this can be turned into aSet
. -
staged
-> Same as above, but forstagedFiles
array. Except the items in this array are not references, so this array cannot be turned into aSet
. -
opened
-> Similarly, we have anopenFiles
array. But this one is tricky because we use this property for folders too to determine which folders were open in the file tree. So to solve this one for both files and folders, we might need to introduce a new property in state calledopenFolders
Also, if we get rid of the three properties above, we wouldn't need to clone objects to use in
staged
andopened
arrays. So those could be converted toSet
s or array of references too. -
active
-> At a time, there can only be one file active. So it makes sense to turn the existing getteractiveFile
into a state property instead of storingactive
on each file.
When we fetch data for a file, do not store all properties the backend returns in Vue state
-
We should only store the properties we really need. See c85b8763.