On GitLab's Frontend teams we use JavaScript for writing all of our FE code with EcmaScript 6 as our generation of JavaScript. We use VueJS to write our reactive features and we use webpack to compile our code with Babel and we are beginning to use code splitting. We have a lot of code written in JQuery and use many other little libraries included with Bootstrap.
We should be type checking our code. A statically types language is a language that does type checking (the process of verifying and enforcing the constraints of types) at compile-time as opposed to run-time. We do have a compile step, which is babel transforming our ES6 back into ES5, but that step does not include type checking. There are many many options available today, and I believe we should start to think about which is the best. I would like to list them all here and discuss the pros and cons of each. I will present my research thus far others can feel free to correct me where I am wrong and add in their opinions.
Flow
Flow - This is a Facebook project with an MIT license. It is not so much a full fledged language itself, but more of a superset of JavaScript with type annotations. It can also do type inferences. So you can write the follow to infer the type.
Or you can write the follow to add a strict type annotation.
// @flowfunction square(n: number): number { return n * n;}square("2"); // Error!
A pro would be that it would potentially be the simplest to implement because it is the most minimal.
The cons are that it would be a potential challenge to integrate it with VueJS. Although if it is just compiled into JavaScript I do not think that our Vue code would be impacted at all.
Another con is that we are at the mercy of Facebook to support it as they come out with many other new languages. It does not have a huge community for support. However the implementation seems straight forward.
TypeScript
TypeScript - is a Microsoft project with an Apache-2.0 license. It is a full fledged language with tons of features. It has static type checking built in and looks very similar to the ES6 generation of JavaScript. It also has other advanced features like generics. It can infer types like Flow. It can also has namespaces which would benefit us greatly.
One major pro is that Vue supports it right out of the box.
Another pro is that is has a much larger community and support base. A lot of other people seem to be using it. It also has a low barrier to entry, as it is really simple.
I really like all the advanced features of TypeScript. I do not know how well it compiles down JS or how optimized that code will be. For example: does it leave out unused code like some of the other langs I am going to mention do. Like OCaml with Bucklescript is extremely beautifully optimized. Much more than a 1 to 1 conversion.
Reason which just put out a version 3, is a Facebook project which is licensed under BSD-3-Clause. It is based on OCaml and attempts to be friendlier to the writer that is familiar with JavaScript. It uses Bucklescript (mentioned above, and not a FB project, AFAIK) to compile to JavaScript. BuckleScript compiles to highly optimized JS. I really appreciate the JS that BuckleScript outputs. The goal of Reason, as I understand it, is to use the "industrial strength" of OCaml while making it easier and more friendly to learn.
To be clear: Reason is not a new language, it is a new syntax and toolchain based on OCaml.
Pros: You can write much more concise code with Reason. It is based off of OCaml which is super skookum. Because of the way that Reason is written you should see many less regression.
Cons: The license is not ideal and it's a FB project in case anyone has any problems there. It's also relatively new. With the initial release in May 2016.
Pros/Cons: Reason 3 which was release yesterday has some major syntax changes. The development seems to be mainly happening between a few developers. They have refactored 50% of messenger.com (the online facebook messenger) to Reason, and they reported a major reduction in their bugs.
Seems like a strong candidate but the learning curve is steep as it is based on OCaml and that's a completely different but awesome language.
Elm
Elm is a language which can compile to HTML, CSS, and JS licensed under BSD-3-Clause. It has it's own coding styles and way of doing things already defined. Straight from the website
Unlike hand-written JavaScript, Elm code does not produce runtime exceptions in practice. Instead, Elm uses type inference to detect problems during compilation and give friendly hints. This way problems never make it to your users. NoRedInk has 80k+ lines of Elm, and after more than a year in production, it still has not produced a single runtime exception.
A similar situation with OCaml and Reason. Because the strict type checking happens while writing code the users shouldn't see those type of bugs.
Great community and great support available.
Along with Reason and OCaml it is also a functional language and make some very strict rules, similar to OCaml. Like dividing two numbers. There is int and float and you can't mix them.
The big difference here being that it is a self contained unit. When you create webapps with Elm you aren't using Vue and I don't think it is meant to be used with Vue. Using it with Vue would be a challenge. It has it's own web framework which is a completely package and VERY well written. It is based on the ideas of Vue and React in that it creates 'reactive' webapps and it has it's own implementation of a virtual DOM. I wrote a few small apps with it and I have to say that it solves a lot of the problems we have, in that it 1. has fantastic performance, and 2. has it's own architecture which when followed everything is flawless. Model — the state of your application, Update — a way to update your state, View — a way to view your state as HTML. It really is a very good way of doing things.
The language is not hard to learn but is a completely new language. It is easier than Reason IMO, and if we wanted to we could do a POC. You can inject it into a page so it wouldn't have to be a refactor. It can take over a #element. Once I started writing it I was kind of mind blown at how awesome it is. I have no idea how you would implement it with RoR or our existing arch.
OCaml
OCaml was released in 1996 making it 21 years ago. It is used all over the place in big companies like FB, Citrix, Docker and tons of others.
It is an awesome functional language with a solid learning curve. Through Bucklescript (mentioned above) written by Bloomberg you can output highly optimized JavaScript. I am sure we could use it to integrate with Vue, but I wasn't able to figure out how in the short time that I used it.
If we did not want to use Reason then we could just go straight to OCaml which Reason is built on. If we did that I believe we might be trying to solve a lot of the problems that Reason solves in the first place. But I am not 100% sure. It seems like we might be able to just use Bucketscript to compile our JS, which I know we can but how would it integrate with Vue, I am not sure. If we could pull it off I believe this would be the coolest solution. I might not be reasonable to ask everyone to learn OCaml but I believe we would could potentially get the most solid code out of this solution.
Go try a few OCaml tutorials and I think you'll love it.
Haxe
Haxe is a toolkit and language which has been around since 2005 (12 years)! It is a statically typed simple language which writes a lot like a mixture of C#/ActionScript 3/JavaScript if it had static type checking. It outputs to optimized JavaScript (it can also output Java or C#), optimized in that it won't output any unused code. It is solid as a rock and has been around forever. It has a huge community. Writing it is a joy.
Haxe and TypeScript are very similar except that Haxe has been around for much longer. It's not the cool kid anymore. It also has it's own unit testing framework if we wanted to use it. The learning curve is small and anyone from our team should be able to pick it up in about 30 minutes. It has many other fantastic features like TypeScript has like Generics and Abstracts.
It's always fun to drive the latest and greatest car, but the US Postal Service uses a Grumman LLV, because it has a lifespan of 30 years, it's economical, and it's built to last.
Conclusion
There may be other static type checkers for JS. If there are let me know and I can add them to the list.
I know that there is another issue just about Flow but I wanted to compile all of them into one list. So we can close that issue.
Also we could start to be like FB and really separate out GitLab. I am not sure if it's reasonable to say this but it might be a good idea to write our $1 million chat idea in a different FE language then the rest of GL. Maybe it's a good idea to write that in OCaml from the start. Or something else. It's the macro idea not the specific suggestion. Maybe the CI would be better suited for another tool. I don't think it's unreasonable to choose the right tool for the goal we want to achieve.
@jschatz1 nice research and thanks for putting all these together. Here is my thoughts
Flow
Flow has several advantages like it's easy to implement, learn and start with. However I am not sure how hard it would be to integrate with dot vue files. I made a quick search and it looks like it's doable. See this blog post and this and that issue from main Vue repo.
TypeScript
TypeScript is awesome but I don't think it is not a good fit for us right now. It's a completely new language and it would significantly suffer our development time since it's a steep learning curve[1]. However it has some advantages like the growing community, being actively in development, great support, easy and full integration with Vue. I really want to us moving to TypeScript but I personally think it is a huge decision. It may be the best decision in long term but may not be the best decision in short to medium term.
[1] What I mean with steep learning curve is, it's not just about the language itself. Language may be simple and easy to understand and start with but it's something completely new and it surely will take some time for all of us to get some speed with it.
Reason
I have very limited idea of what is Reason and how it could be helpful for us. @jschatz1looks like you are in love with Reason so maybe you can show us what it is in a very quick call. WDYT?
Elm
Elm is also super cool and hipsterish. I know that it requires a paradigm shift to functional programming so I think if we will switch to a completely new language it should be TypeScript instead of Elm.
Conclusion
I think, if we
want something simple and easy to start with => Flow
are ready to make some investment for the future and know that it would not be the simple one in the short term => TypeScript
Thanks @fatihacet... When we look at these frameworks and languages we must look at them with an open mind and forget if they are hipsterish or not. I will give everyone a quick tour of some of them. We do have to pay attention to the licensing though.
Also @fatihacet where do you see that Typescript has a steep learning curve? I don't see that anywhere. I have been studying it for a while and took a deep dive this weekend and to me it's very straightforward. Would love to hear your thoughts though.
where do you see that Typescript has a steep learning curve?
[1] What I mean with steep learning curve is, it's not just about the language itself. Language may be simple and easy to understand and start with but it's something completely new and it surely will take some time for all of us to get some speed with it.
Steep learning curve may be an incorrect term, the above quote was actually expressing what I want to say. It's a completely something new and may be hard enough to get used to it. I don't have strong feeling about it but just sayin.
My vote is for either flow or Typescript - both are straightforward to add to our existing code, and can be picked up by JS developers - using something close to regular javascript allows greater number of people to contribute without needing to learn a new language/framework/etc.
Reason/Elm/etc have some great features, but limited adoption. We could try some separate applications using them, but our current codebase is fragmented enough without introducing more frameworks/languages. They also need separate build toolchain (flow and TS can both use babel/webpack).
What do we see as goals of adding Types? Here are some of mine:
eliminate cannot read property x of undefined errors
increase our confidence in code correctness
people without experience with Type system can still contribute
reduce number/string coercion when using + operator
act like docstrings - you know what arguments functions want from looking at signature
integrate easily with our existing build tools
well documented and used/tested. We don't want to be the first to discover issues
Lots of good arguments on both sides; Flow vs. TypeScript.
Having worked in moderately sized Angular 2 project in the past, I really like TypeScript as a language and find it more organized and intuitive for large projects, and while it has ES7+ features, I found that TypeScript in Angular 2 feels more pragmatic as Angular 2 framework itself uses lots of niche language features like dependency injection, class decorators, et al. which is something that simpler framework like Vue doesn't have (yet), and that's where I think adopting this language just for static type support is an overkill.
If only problem we're trying to solve is to ensure type safety, we're better off using something simpler that can be removed (or replaced) at will if we feel like in the future. Not to mention that TypeScript to JS transpilation could be a serious bottleneck in development workflow as the code size grows as it doesn't have native transpiler written in C/C++, while on contrary, as the time goes on, we can slowly phase out Babel transpilation of our JS code as browsers get mature in ES6 support.
Flow is heavily adopted in the React community, but just about everywhere else (including in the Vue ecosystem), TypeScript reigns supreme. There are a few blog posts on integrating Flow with Vue, and there are some community-made Flow type definitions for Vue, but TypeScript has official support and plenty of documentation. I think we'll run into the least number of issues by using a well-trodden path that others have laid out. This is the line of reasoning we took when adopting .vue files instead of JSX. Sure, we could do either one, but one clearly had better support.
I also think the common line of reasoning that TypeScript is a "new language" and Flow is a light-weight layer on top of JS is misleading. Despite the fact that Flow still uses a *.js file extension, it is in fact not JavaScript. If a browser encounters function square(n: number): number { return n * n; } it will not know how to run it. Both languages need to be transpiled, and both are effectively a superset of ES6+, so let's not be misled by an illusion of relative simplicity.
I would just like to point out that it appears that Flow is used in the VueJS source code, but I agree that we should take on TypeScript. I will put something together and take a stab at it.
We still have relicts from CoffeeScript in the code base which we didn't clean up a year later.
If we move our code base to TypeScript now, will we have relicts from two other languages in our code base?
Also if ECMAScript 2018 introduces decorators (currently stage 2: https://github.com/tc39/proposal-decorators), we can have type annotations in JavaScript, I think. Would that mean we switch back from TypeScript to JavaScript in a year?
Also if ECMAScript 2018 introduces decorators (currently stage 2: https://github.com/tc39/proposal-decorators), we can have type annotations in JavaScript, I think. Would that mean we switch back from TypeScript to JavaScript in a year?
Exactly my concern , like we adopted Yarn for its easier dependency locking and more importantly, faster dependency installations, but NPM5 is as fast as Yarn is (and in some cases even faster), and in future, we'll still have Yarn as dependency even if Node (and NPM) have solved the problem we aimed for while adopting Yarn, this is just one small example of why we shouldn't hurry on this IMO.
I don't think there is a reason to covert everything to TypeScript. Do you?
Also @kushalpandya we adopt things as we need them. We can technically change later, but yarn did and still does solve a problem for us. Even though other stuff has come out since then, we don't have to necessarily use it because our problem is still solved.
Even though other stuff has come out since then, we don't have to necessarily use it because our problem is still solved.
I think it's generally a good idea to pick a solution that is standardized because more people may know it, tool support may be better and it has a higher chance to exist in the future. This may not be important enough for switching back from yarn but it may be an argument to stick with ECMAScript.
Rewriting everything. I am not sure if we can do that. It would have to be only for new stuff. I would love to see if we can figure how much of bugs come from not having a type checking system. What do you think?
If we go for TypeScript, I propose we wait until we can upgrade to babel 7 (which we should do soon) because it has built-in support for TypeScript. Without this we would essentially be running two separate language transpilers - lots of unnecessary overhead when compiling assets, non-DRY config, and potential feature parity issues.
I think we should proceed with TypeScript. Better to get those sweet sweet Static Benefits sooner, though not sure what the babel 7 release timeline is like. I see main differences as syntax (fairly minor) and tool/docs support (such as Vue, Babel 7, and VSCode).
It is also reasonably reversible if we later decide it's terrible and we miss .js
we could configure the typescript on specific folders/files, maybe have a specific namespace for our sentry logs?
we can opt in with the .ts file extension (or possibly other ways). This means new code can be typed, or files can be converted in a similar way to how we currently add tests for existing code.
The codebase is also now written in TypeScript. Although this will make proficiency in TypeScript a pre-requisite for contributing to the new codebase, we believe the type information and IDE support will actually make it easier for a new contributor to make meaningful contributions.
In the past, my biggest struggle with TypeScript was wrestling with out-of-date third party typing definitions. I think the community and infrastructure has come a long way since then.
IMO, if TypeScript compiling can be nicely included in our webpack and babel stack (and if we have a revert plan just in case), we could benefit from exploring this, especially with our initiative for GraphQL.
Hey from the future You're definitely right, here's a repository maintained by the community for javascript libraries without type definitions: https://github.com/DefinitelyTyped/DefinitelyTyped (just leaving here for transparency).
I am trying to add Typescript outside of the webpack workflow, so that type-checking is separate from our bundling/transpiling efforts.
@leipert yes, that would be the way to do it. We won't use typescript at compile time, we'd use it more or less as a linter. We would set up babel to strip the type definitions when compiling (which babel 7 finally supports).
The nice thing about typescript is that we can get a lot of the benefits of using it by just setting it up and running it against our existing .js codebase. It will infer types from JSDoc comment headers and still provide some useful checks for us out of the gate.
In fact, webpack uses TypeScript entirely this way.
We won't use typescript at compile time, we'd use it more or less as a linter. We would set up babel to strip the type definitions when compiling (which babel 7 finally supports).
IMO, there's some TypeScript features that we actually want to compile. Decorators seem to be needed for most of the GraphQL/TypeScript libraries I've seen (among other things). Does babel transpile this as well or does it simply strip the types @mikegreiling?
Also, why not let TypeScript check and compile at the same time? It sounds like needless processing power to only use TypeScript as a linter and then do the same thing again when babel needs to compile it.
Also, why not let TypeScript check and compile at the same time? It sounds like needless processing power to only use TypeScript as a linter and then do the same thing again when babel needs to compile it.
For the initial implementation (assuming we decide to do this), this would be the least invasive change.
TypeScript currently duplicates a lot of the functionality that babel provides. It does type checking and transpiling to legacy javascript, however it does not have the same feature set as babel. For one, we're soon going to be using babel-preset-env aggressively to target both legacy and modern browsers, and we will also be using the "useBuiltIns": "used" feature to automatically add browser polyfills based on target browsers. TypeScript does not support this, so if we transpile some files with babel and others with TypeScript we'll end up with divergent feature sets and I think it will cause a lot of unnecessary confusion.
I have not checked out how this interacts with decorators yet. It's possible it converts them into es6 stage-2 proposal decorators which babel can handle. We normally only support stage-3 and up, but maybe we would make an exception.
TypeScript also strips out the type annotations. The big pro of setting it up separately, we can iterate on it and if on some future point we want to jump from babel to tsc, we have a zero barrier.
I've been thinking about this off and on and I'm pretty torn about it now. I recently got to see some TypeScript developers at work and I can't get over how cumbersome it all seemed. This also was my first impression some years ago ...
Google must have known that I was having these thoughts and recommended this article "The TypeScript Tax". The author is a little opinionated. He's a fan of TypeScript, but concludes that it has a negative ROI when compared to modern JavaScript development with modern tooling.
IMO, any quality issues that we hope TypeScript will solve, can also be solved with appropriate levels of automated testing (which I prioritize over type "safety"). I'm scared that if we start adopting TypeScript, it will slow development and build time, while only preventing trivial quality issues which should be caught in tests anyways.
Open question: What's the actual value of type checking which automated tests doesn't bring?
At a previous company I worked for we were in a similar situation. Our entire codebase was written in ES6 and we started outweighing some of the benefits that TypeScript might bring to the table.
Some of the benefits that I personally liked:
Typescript integrates with IDEs extremely well.
We got real-time compilation feedback from the IDE.
Acurate auto-imports and auto-completion as you’re coding.
Got clear feedback of API definitions.
More structured clean code base.
We ended up running ES6 and TypeScript in parallel which meant we could refactor the codebase to TypeScript over time.
We started seeing an increase in throughput and saw more backend developers feeling a lot more comfortable to contribute to the frontend as well.
We won't use typescript at compile time, we'd use it more or less as a linter.
+1 to that.
I was basically forced to use Typescript at my old company when working on a big Angular 4/5/6 (I think the version increased a couple times during that time) project. I was sceptical in the beginning but I now feel that TS can add a lot of value to a codebase. Besides the benefits @jerasmus mentioned in his comment (I just love the integration in VSCode) I think it's great for code documentation.
@pslaughter - Thanks for all your thoughtful comments! Regarding your question:
What's the actual value of type checking which automated tests doesn't bring?
Not having to write those tests in the first place!
I'm of the opinion that less code is better, even when it comes to tests. If we can replace a whole class of automated tests with the TypeScript compiler, I think that's a big win.
...I can't get over how cumbersome it all seemed
Interesting - any specific examples of features/issues you found cumbersome?
I've always thought TypeScript did a good job of sprinkling in type safety to JavaScript without requiring too much change to how the code is actually written.
...slow build time
Definitely a valid concern. In my experience, the TypeScript compiler is pretty quick at incremental changes and doesn't add too much time to the feedback loop. That being said, I haven't worked with TypeScript on a codebase as large as GitLab's, so it would be worth doing a proof-of-concept.
I'd love to see a proof-of-concept that adds TypeScript support to our webpack build. I think a concrete implementation would answer a lot of the questions being asked in this thread. @leipert, it sounds like you might have been working on this? (https://gitlab.com/gitlab-org/gitlab-ce/issues/39574#note_131825207) Did you make any progress? If not, I might take a look after the current milestone is closed. Would be a good to chance to familiarize myself with our current webpack setup .
@nfriend This is the WIP MR: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24792 It has a lot of errors still, but module resolution works (apart from EE obviously). We still have ~1700 errors which are probably due to missing types, TS not knowing about the stuff we add to window, jQuery et al. But it is working and we could go from there and improve on it step by step.
I realize this may be too late and I am very new, but I wanted to add another voice as a not being convinced by Typescript at all — it's been a big burden in my experience (and has made implementing a lot of nice dynamic work challenging) and rarely catches issues that tests to do not (as @pslaughter mentions above). It is also not a sound type system so does not come with a lot of the guarantees some folks think it does.
I'm glad to chat with people more about the vague things I've mentioned if you are interested but did not want to write an essay if that ship has sailed.
Just piping in with no real facts and just some opinion as it looks like most issues have been discussed already. I would love to see static typing in the codebase especially as we continue to move to Graphql and are continually refactoring code.
Personally I love Reason because of the speed of the BS compiler and soundness of the type system, but i'll definately concede its problematic as mentioned above and probably not the safest option yet (license, , learning curve)
We won't use typescript at compile time, we'd use it more or less as a linter.
+1 This sounds like a good move, id be happy with this compromise.
I've heard the argument before that TypeScript's type system is unsound. I can appreciate the argument, but I feel like this is "letting perfect be the enemy of good", to use a cliché . Even if TypeScript only catches, say, 80% of type errors, isn't this a lot better than 0%?
Oh yea, @nfriend, I know it is intentional; I just think it undermines the safety argument, especially when balanced against tests and effort. In that case, Typescript is itself the pursuit of the perfect over the good enough.
It is unlikely that all type errors are slipping through without Typescript (most are not*) and at some point the velocity vs. unlikely error tradeoff is too much. To me, with GraphQL (typing where it is useful) and things like tests, including property testing, you can have most of the safety and none of the encumbrances. You don't get your coworkers training your IDE autocomplete, but that seems like the wrong place for that work anyways.
(I actually have a whole Medium post about this, but it is from 2017, so there are probably some thoughts I have evolved on.)
Anyways, it is nice to chat, for sure, and I am sure I will have more thoughts to share, especially on types.
[*] Most in most codebases are not. I have only worked here for 2.2 days, so I can't be specific to the Gitlab setup.
Just a user here, looking for a way to programmatically control the Web IDE through e.g. a userscript/Chrome extension. To help with my understanding, I cloned the source code, looked at the VueX store, and immediately have some questions, e.g.
Trying to find an answer to this leads to quite an adventure...
See the adventure
Finally I found the function in utils.js that defines the data structure, but…
If the types of an entry is documented via types, finding the answer to this would have been much easier for newcomers to the project like me (just a Cmd+Click away). From my experience using TypeScript a lot in my projects, I found that the most beneficial value gained is documentation, much more than type-safety/soundness. So, would love to see this. :D
I do genuinely like TypeScript! But let me tell you a story...
One of my previous workplaces started migrating their ES2015 codebase to TypeScript about 6 months before I joined. New work was done entirely in TypeScript, and the idea was that devs would migrate legacy JS files to TypeScript if they had to touch them for any reason. By the time I joined, that rarely happened, as all the easy-to-migrate files had already been done. All that remained was the awkward legacy stuff very deep in the guts of the system, which no one wanted to touch for fear of breaking everything horribly.
The only way the entire codebase would get ported would be if actual development time was scheduled and dedicated to it. Unfortunately, the business did not allow it. I suspect it'll remain a JS/TypeScript chimaera forever.
Because the codebase was part JS, part TypeScript, none of the strict type checking options were enabled. Most insidious was not having --noImplicitAny set, which meant that if TypeScript couldn't infer the type of the expression, it'd fallback to any, which is equivalent to not having any type checking at all. While TypeScript's type inference is impressive, it's definitely not perfect, and there are lots of times it needs explicit typing. Not having --strictNullChecks enabled was almost as bad, since without it, TypeScript does not warn about possible cannot read property x of (undefined|null) errors!
While I was there, I explored enabling some of the strict type checking options (see what --strict enables in the docs), but each one showed literally tens of thousands of violations in the codebase. Even with allotted development time to tackle them (which I didn't have), merge conflicts would be inevitable if new features and bug fixes were worked on at the same time by other devs, which of course would be the case.
Yes, TypeScript's type system is unsound. But it's actually worse than "unsound" without its strict checks enabled, since it gives a false sense of security about the type safety it's giving you. Sure, you still get the benefits of being able to define/document types in your system, and smart IDE autocompletion/linking goodness (assuming the expression you're looking at hasn't been implicitly cast to any), but the cost-to-benefit ratio is extremely high in this case, i.e., bad.
You might be wondering why I said I like TypeScript, given what I've said so far! Well, I think there is a happy path with TypeScript, where the cost-to-benefit ratio is low, i.e., good. The happy path is quite simple:
All strict type checks enabled, right from the start.
Without these, you're opting out of one of the main reasons for investing in TypeScript in the first place: type safety! Again, yes, TypeScript's type system is at best unsound, but there's already a good rebuttal as to why that's not a problem.
TypeScript-only codebase - or, a realistic and dedicated effort to port everything to TypeScript
This is largely required for the first point to be possible, unless someone invests the time into writing copious .d.ts typings for JS imports. The problem with these, like all non-executable code, is that they can easily get out of sync with the real implementation.
On the happy path, TypeScript is great, for lots reasons that others have listed in this issue. I don't see how we can be on the happy path if we start writing TypeScript directly in GitLab. I think it'd forever be a JS/TS chimaera, with all of the costs of TypeScript and few of the benefits.
But I'd love to be proven wrong!
What I propose
If we want to adopt TypeScript, let's do so in isolated libraries/dependencies/projects; at least initially. For instance, gitlab-ui might be a reasonable candidate. Right now it's mostly thin wrappers around bootstrap-vue, so perhaps it's of limited utility so far, but at least porting over all of it to TypeScript quickly wouldn't be too onerous. It'd give us a much better feel for how to use TypeScript effectively with Vue 2.x. As we start moving vue_shared components from GitLab to gitlab-ui, we'd start to see the benefits, as we'd have actual code rather than just wrappers.
But maybe there are better candidate libraries/dependencies/projects than gitlab-ui to do this with. Any ideas, anyone?
Just a passing thought: this discussion I think highlights the value of a strict type system.
The question in the discussion is, should a function which usually returns an array of objects return null in the case that there are no objects, or an empty array? Written with null-aware types, the answer (I think) is clear. The return type would have to be Thing[] | null in the first case, and simply Thing[] in the second. Much better!
Hey @timzallmann, given the discourse and the length of time this issue has been opened, should you make an executive decision on whether or not we will implement static type checking as a frontend organization?
Personal note: I apologize for the delay, I appreciate everyone's patience on this!
This discussion has been two years in the making, so I'd first like to thank everyone for giving their thoughtful input on this. Introducing something like this can certainly be contentious. However, the discourse here has been very healthy and productive.
After reviewing the discussions with the frontend engineering managers, we've decided not to move forward with implementing TypeScript at this point.
Several factors led us to this decision:
Overhead
There will be additional overhead with onboarding at GitLab, which will continue indefinitely, with mentorship/training new team members, especially given the ambitious FY2021 hiring plan. We also have to factor in how this would affect our overall throughput.
There is also an initial and continuing training investment that we have to consider that we've not yet quantified.
External contribution barrier
There's a lot to orient yourself around as an external contributor, and this adds another level of complexity that we think will decrease the number of external frontend contributions to our product.
Cognitive load
We have excellent quality control processes already in place with regards to our linters, testing culture, and review process.
We need to be sure that the investment in TypeScript adds to, and doesn't take away from our existing structures.
Getting closer to the code
Our near-term focus is on performance and getting closer to how our code functions in production through the use of Kibana, Prometheus, and Sentry
Being closer to the code with a tighter feedback loop could highlight the need and value for TypeScript in the future.
SentryJS will help us identify and be more proactive with the bugs that do get out to production.
Fragmentation
Current Vue/TS support, huge codebase, HAML -> JS/Vue is not a black/white decision, we will fragment further, and that will make progress a bit more difficult at this time.
There is no denying that TypeScript will solve some problems of ours, but we don't feel it's the right time to introduce something like this.
That said, this discussion isn't final. The current thinking is that we can revisit this in around nine months, roughly around the time of the next Contribute, after the Prague event.
If anyone would like to re-open this discussion, we would greatly appreciate a cost-benefit analysis addressing some of the factors mentioned above, highlighting the pros and cons concerning the inputs and outputs required to make TypeScript successful in our organization, to help inform the decision.
Thanks to everyone for their invaluable input in this discussion. If you have any concerns or feedback, we're all ears!
Disclaimer: I'm actually a big fan of TypeScript, so I'd be keen to work with people to re-open this topic when we've done a bit more digging into how we can make TypeScript successful at GitLab!
@dennis I'm going to reopen this discussion. Typescript has become increasingly ubiquitous over the years since this RFC was created. I wouldn't have agreed with the points you made 2 years ago and in hindsight I think the argument for TS is even more sound.
Overhead
There will be additional overhead with onboarding at GitLab, which will continue indefinitely, with mentorship/training new team members, especially given the ambitious FY2021 hiring plan. We also have to factor in how this would affect our overall throughput.
There is also an initial and continuing training investment that we have to consider that we've not yet quantified.
In 2022 I would consider lack of familiarity or at least exposure to Typescript a red flag when hiring frontend/fullstack engineers. These days understanding TS is almost a prerequisite to contributing to the JS open source ecosystem as virtually every popular npm package is either written in TS or at least provides TS definitions (including Vue). If you are using editors like VSCode that support the language server protocol, you are benefiting from the TS compiler even when you are writing vanilla JS.
External contribution barrier
There's a lot to orient yourself around as an external contributor, and this adds another level of complexity that we think will decrease the number of external frontend contributions to our product.
I think the complete opposite is true. Navigating an unfamiliar codebase with strict types is way easier. Static type checking makes it easy to introduce new changes because it won't allow you to call APIs incorrectly. In my experience contributing to loosely typed codebases you spend as much time digging into the implementation details of the functions you are trying to call to understand what they expect as you do writing new code. In a well-typed TS codebase, the intellisense autocomplete practically writes everything for you.
Getting closer to the code
Our near-term focus is on performance and getting closer to how our code functions in production through the use of Kibana, Prometheus, and Sentry
Being closer to the code with a tighter feedback loop could highlight the need and value for TypeScript in the future.
SentryJS will help us identify and be more proactive with the bugs that do get out to production.
Ideally you want to catch bugs before they hit production. SentryJS is not a reason to not have compile-time checking in place as well. According to this article:
Airbnb decided to make TypeScript its official language for frontend web development after looking at six months of postmortems and discovering that TypeScript could have prevented 38% of the bugs that resulted in incidents in production.
Cognitive load
We have excellent quality control processes already in place with regards to our linters, testing culture, and review process.
We need to be sure that the investment in TypeScript adds to, and doesn't take away from our existing structures.
I'm not really sure what you are getting at here. TS is a strict superset of JS. There is no cognitive overhead. If you know JS, you know TS. It is compatible with essentially all of the same tooling and, if anything, provides more robust linting and QC capabilities.
When we launched the beta version of our TypeScript platform support, more than 200 projects opted into TypeScript in the first year alone. Zero projects went back.
Fragmentation
Current Vue/TS support, huge codebase, HAML -> JS/Vue is not a black/white decision, we will fragment further, and that will make progress a bit more difficult at this time.
This is not fragmentation in the same way that HAML/Vue is today. JS code is callable from TS code and vice-versa. As others have mentioned in this thread there are numerous iterative paths we can take towards adopting TS. The best example is by starting with gitlab-ui. Consumers of code/libraries written in TS benefit from type definitions even when writing JS code.
@marshall007 one important point: Vue 2 is not written in TS, it's written in Flow which makes typing Vue components in the current version cumbersome. This also applies to Vuex and its string-based mutation types. I wouldn't recommend starting with introducing TS to Vue codebase until we fully migrate to Vue 3 where using TS in components is much more pleasant and doesn't require that much manual work compared to Vue 2.
I wouldn't recommend starting with introducing TS to Vue codebase until we fully migrate to Vue 3 where using TS in components is much more pleasant and doesn't require that much manual work compared to Vue 2.
For context, we are in a similar position on the backend.
However, we are not yet on Ruby 3, although we are getting close (it's still not, AFAIK, an official high-level priority with dedicated allocation, but that's a separate discussion).
So, just like frontend, the backend is waiting on a major dependency upgrade before we can (or should) start making any significant towards taking advantage of strong typing.
Thanks for starting the conversation @marshall007!
suggestion: This issue is very old and the scope has naturally changed since it was first opened... What do you think of opening two separate RFC's:
Prefer TypeScript for new projects and packages
Migrate GitLab monolith to TypeScript
That way we can have a more focused discussion.
food for thought:
TypeScript is really nice, but it has some costs and risks which we need to evaluate in different scenarios. Also, its pro's are not guaranteed. Type safety is not a silver bullet that makes code bug-free and easier to maintain. No abstraction is safe from the leering noodles of spaghetti code.
These days understanding TS is almost a prerequisite to contributing to the JS open source ecosystem as virtually every popular npm package is either written in TS or at least provides TS definitions (including Vue)
I think understanding TS is a prerequisite to contributing to a TS project, but clearly there are many popular packages which aren't in TS
Personally, I'd be more interested that someone understands the gotcha's of Vue reactivity than the syntax of TypeScript (which the compiler will lovingly reprimand you for anyway).
TS is a strict superset of JS. There is no cognitive overhead. If you know JS, you know TS.
JS is a subset of TS, so knowing JS means you only know part of TS. TypeScript has a lot of features like generics, interfaces, type aliases, which would be unfamiliar with someone not coming from a typed language.
That's not a bad thing, but it is a level of mental "overhead" (especially if you end up in generic hell)
opinion summary: I think TypeScript is great, but it can give a false sense of security (regarding both bugs and maintainability). If a project adopts TypeScript, it should be strict, with linting (avoiding unsafe casts and any), and uphold the same commitment to scrutinizing test coverage and maintainability. Otherwise, the costs of TypeScript will likely outweigh the benefits.
IMHO, we should strongly consider using TypeScript for new packages and projects, but it's a whole other discussion when considering this for the main GitLab project. I think there's a way to get there, but we'll need to be really intentional about it.
Personally, I'd be more interested that someone understands the gotcha's of Vue reactivity than the syntax of TypeScript (which the compiler will lovingly reprimand you for anyway).
I would also second your point that usage of typescript without quite good understanding of its caveats and limitations leads to false sense of security even with strict config and proper typescript-eslint config in place
Ideally you want to catch bugs before they hit production
after looking at six months of postmortems and discovering that TypeScript could have prevented 38% of the bugs that resulted in incidents in production.
As a person who gave multiple talks on reliability and "price" of the typescript I should note that unfortunately Airbnb never disclosed (although requested multiple times, including my request) how these 38% were calculated and what other error-preventing techniques were in place for the project where this number was calculated.
I'm not questioning the statement that typescript prevents certain bugs, however I'm questioning effort-to-results balance for introducing typescript
I'm not questioning the usage of typescript and % believe that this was a right choice I'm very confused that making such a significant change to quite a big part of our frontend ecosystem was done bypassing RFC process
I'd be more interested that someone understands the gotcha's of Vue reactivity than the syntax of TypeScript
I am not sure I understand what you mean here. What makes it the case that if someone learns TypeScript, they cannot learn the gotchas of Vue's reactivity?
(I'm assuming one answer could be related to time constraints (e.g. one has time either to learn TypeScript or to learn Vue reactivity gotcha), how would you prioritise what one would learn?)
Thanks, @xanf for sharing the paper The results and conclusions of the authors are quite promising! Being able to catch 15% of real-world bugs is not a trivial achievement. I especially liked this part of the article:
At first glance, 15% may not appear to be a large number. In practice, however, even small changes in the number of checked-in bugs can be quite valuable. When we presented the results to an engineering manager at Microsoft, he responded “That’s shocking. If you could make a change to the way we do development that would reduce the number of bugs being checked in by 10% or more overnight, that’s a no-brainer. Unless it doubles development time or something, we’d do it.”.
I'd be more interested that someone understands the gotcha's of Vue reactivity than the syntax of TypeScript
I am not sure I understand what you mean here. What makes it the case that if someone learns TypeScript, they cannot learn the gotchas of Vue's reactivity?
@agulina If I may take the mic from Paul here, I read this as a response to the comment above that a FE engineer applying to be hired at GitLab that doesn't know TS shouldn't be hired.
In 2022 I would consider lack of familiarity or at least exposure to Typescript a red flag when hiring frontend/fullstack engineers.
Given our codebase is in Vue with Vanilla JS, I think Paul's comment just meant that having experience in Vue is more valuable than TS experience, which we don't even use ourselves
There's a lot to orient yourself around as an external contributor, and this adds another level of complexity that we think will decrease the number of external frontend contributions to our product.
I think the complete opposite is true. Navigating an unfamiliar codebase with strict types is way easier. Static type checking makes it easy to introduce new changes because it won't allow you to call APIs incorrectly. In my experience contributing to loosely typed codebases you spend as much time digging into the implementation details of the functions you are trying to call to understand what they expect as you do writing new code. In a well-typed TS codebase, the intellisense autocomplete practically writes everything for you.
Huge +1 here. Typing is documentation, navigating typing makes it SO MUCH EASIER to drop in in an unknown codebase, and navigate what each class has available, what are the constraints, where to find further definitions. My experience with being onboarded into typed codebases (go, java, kotlin, typed python) was that it was considerably smoother than non-typed ones (JS, RoR, non-typed python)
Still, I think the rationale behind the question stays. The point on considering TypeScript skills did not seem about hiring per se, but about how important it is in contributing to OSS projects (one can argue for or against it, of course).
But even if it was a point about the skills we would seek in engineers, nothing there implies we should limit to one additional requirement and therefore have to choose TypeScript over Vue knowledge. It could be Vue AND TypeScript, or Vue OR TypeScript.
As a matter of fact, though, we don't even require Vue experience at the moment. That's why I wasn't sure what the point was about.
A really great conversation here with some super points
While migrating our main Product codebase to TS would bring benefits, the cost to do so largely outweighs any gain we might eventually see from the effort. We have a lot of other areas we can spend that effort in the codebase(performance, GitLab UI, Security, Legacy Code, Vue 3 Migration, Apollo, larger test coverage) and see much better long-term results without raising the bar of entry for new hires and community contributions.
Not to say it wouldn't be amazing and I wouldn't have a personal bias towards us going that way (TS fan boy here ).
IMHO, we should strongly consider using TypeScript for new packages and projects, but it's a whole other discussion when considering this for the main GitLab project. I think there's a way to get there, but we'll need to be really intentional about it.
I think looking for Strongly-Typed solutions like TS for new modules, packages or Projects is a great place to start and helps to create an intentional ecosystem where we are looking to the future and can help diversity the GitLab Group ecosystem in a safe way.
Personally, I'd be more interested that someone understands the gotcha's of Vue reactivity than the syntax of TypeScript (which the compiler will lovingly reprimand you for anyway).
We're a Rails app first with Vue to support our Frontend. These are the skills we look for when hiring for the main repo and keeping this in mind helps us ensure we get the right people contributing to the codebase. Knowing Vue gotchas, security principles, and the core building blocks of how to contribute to a large application without incurring performance costs outweigh an understanding of something like TS.
@oregand I was intentionally not prescriptive regarding adoption strategy in my comment.
I completely agree that migrating the existing codebase to TS for it's own sake would not be an efficient use of time. There are a number of ways we can get there though and I think we should focus the discussion around that:
individually migrate components to Vue 3, refactor them to TS at the same time
refactor standalone scripts to TS and require new ones be written in TS
require new packages and projects be written in TS natively
Knowing Vue gotchas, security principles, and the core building blocks of how to contribute to a large application without incurring performance costs outweigh an understanding of something like TS.
Vue 3 and all of the Apollo GraphQL tooling are written in TS. I have no doubt we will eventually migrate to Vue 3, so TS experience will become increasingly valuable as we look ahead.
individually migrate components to Vue 3, refactor them to TS at the same time
I have to disagree here. Migration to Vue 3 won't be done on per-component basis. Some components will require changes (the most obvious one that comes to mind is v-model syntax update), some will not. At the same time, migration even with the compat build will be complicated enough. Let's not add more complexity to it, if we decide to use TS, it should be brought on iterative basis after the migration is finished.
A more progressive path to adopt strict type checking is using JSDoc for documentation and type annotation. In my previous company we did that and my experience was that for the 80% of cases adding types annotation was simple. We were able to find and resolve a lot of corner cases by adding type checking.
I definitely see value in bringing TS to our codebase, mainly for documentation purposes. The following piece of code does not give you a clue of what object we're dealing with:
props: { filters: { type: Array }}
Usually I have to console.log the object to see what properties it takes. And even in that case, it doesn't tell me what the object may potentially look like because the next line of code can modify it and add a random property. If we add the speed of our GDK to the equation, you can guess that this process becomes a bit tiring.
I think no one can disagree that having a type definition is much more readable and makes the code easier to navigate:
That said, the Vue documentation itself recommends using Typescript with the Composition API. Luckily enough, we have another RFC to introduce the Composition API. Therefore I'm going to suggest to use Typescript with the Composition API only going forward. It means we won't have to go through a huge refactoring effort to introduce Typescript, but rather once we migrate to Vue 3 we can start using both the composition API and Typescript.
That all being said, I think we can keep discussing the pros and cons of Typescript for another 4 years and we won't reach a consensus However, we should not exclude the fact that:
Typescript is way more popular than it was 4 years ago and it's popularity is increasing. As Dave pointed out, there's also an ECMAScript Proposal to bring type annotations.
If @marshall007 were not reopening this discussion, then probably I'd have. If I'd not have reopened this discussion, then probably someone else would have
I personally believe that the introduction of Vue 3 and the composition API is a good chance for us to start using Typescript in our codebase. Since there will be no major refactoring by following this strategy, this can probably mitigate the effort vs gain concern which was brought multiple times during this discussion.
Probably the only concern here can be the intermediary state where both typescript and javascript will be used. I guess that's the nature of migrations and we don't have many options here. Also, personally I don't see a big issue here because Typescript and Javascript can live together fairly well.
PS as suggested by @pslaughter above, I have created an RFC for the new projects: #105
I'm commencing this thread to capture the instances in which Typescript would have caught some issues. It happened multiple times in the last few months so I thought of creating this thread and capturing all the instances. I'll update this thread from time to time.
I hope this thread is the right place, but I have something where I think TS would have helped:
I recently worked on refactoring a large component that accepts a big, nested object via one prop.
That objects (or parts of it) were then passed down the component tree, which also ended up being deep and wide.
Since the refactor included moving from REST to GraphQL, we needed to determine what data we actually render. Plus document the shape that is needed.
To make matter worse, the response we got from the API went through a couple of Vuex modules, which applied transformations, etc.
We ended up having to manually trace the data, through Vuex, following the component hierarchy branch by branch. Needless to say, that was not a pleasant exercise, and I am still not sure if I didn't miss anything.
I believe if we had static typing in place, this would have been a quick an easy task, and we would get instant feedback if data was missing or incorrect.
One issue we had with typescript in vscode extension project is the difficulty testing with mocking when using typescript especially against a 3rd party library.
If you're testing against a deeply nested and typed API, you end up writing a lot of mocking modules/classes/functions, literally duplicating the API implementation with empty mocked typed classes/functions/types etc.
You later should also maintain the test version of the mock structures, painful especially if it's against an API which is maintained in another organization.
Mocking is in general a lot easier with duck typing, but interaction with a third party API is usually done by abstracting that api using a facade pattern, that way you avoid dealing with changing implementations. This is not to say this isn't a problem, this is to say it will require a change is muscle memory, specially for full stack that have one way of doing things in rails and another very different way of doing things in frontend
We're using facade pattern, but it doesn't save you from maintaining mocks of changing internal interfaces. One common pitfall, in type discussion is the assumption that interfaces don't change but only internal implementations change. Yet in most projects, function/module/class interfaces and their nested structure also change a lot. Facades don't help there, as you must maintain your mocked facade implementation too.
doing things in rails and another very different way of doing things in frontend
I doubt it's about frontend vs backend. In JavaScript too, things are easier when mocking.
Yet in most projects, function/module/class interfaces and their nested structure also change a lot.
But in this case, wouldn't you need to maintain the mocks as well? update them along the new changes added by the APIs? Certainly the interfaces will need to be updated over time (less than using the library directly though, since some of the changes can be treated by an middlelayer implementation), but so is the case when duck typing
I doubt it's about frontend vs backend. In JavaScript too, things are easier when mocking.
What I meant is, once (and if) the change applies, working one way in the frontend with typescript and another way on the backend with rails (that have very different ways of doing things) while context switching will make things difficult. We currently follow generally the same patterns, which makes things a bit easier for fullstackers.