The FY'21 Engagement survey ran from November 16, 2020 - December 16, 2020. The purpose of this issue is to share results with the Product division so we can review data and collaborate on actions moving forward. The highlights/lowlights/potential actions are @sfwgitlab's initial thoughts and takeaways from the survey data.
Please review the Product division summary overview in addition to the highlights/lowlights/potential actions below to discuss potential action items during the PLT Meeting on January 20, 2020.
The goal would is to end the PLT meeting on January 20 with 1-2 Product division level action items for FY'22.
Highlights
Product is above the tech benchmark on just about everything
Most leaders / manager satisfaction / team satisfaction / team culture / values scores are solid
There are improvements from the FY'20 engagement survey regarding seeing positive changes from the prior survey (+7 since FY'20 engagement survey Product)
Product confidence scores ("GitLab's products and services are as good as, or better than, our main competitors") rose since the last survey (+7 since FY'20 engagement survey Product)
Overall comments score was 81% favorable (comments mentioning “manager” and “work-life” at the top)
DIB related scores & comments seem generally positive
COVID related scores & comments seem generally positive
Lowlights
The 25-34 age demographic has significantly lower scores for I believe there are good career opportunities for me at GitLab.
Favorability
Age Range
53%
25-34
76%
35-44
80%
45-54
Scores on compensation satisfaction dropped significantly and well below the company average (-20 from FY'20 engagement survey for Product, and -6 from FY'21 engagement survey company-wide)
L&D scores in general very low (company-wide as well)
Retention questions dropped significantly since the last survey
Comments suggest recognition isn’t evenly distributed, and that people just “doing their day job” in more mundane areas don’t feel as appreciated
Comments and scores indicate that engagement has suffered due to flat budgets, manager changes, role changes, etc.
Lack of alignment amongst e-group and in the OKR process showed up in comments
Comments indicated frustration with trying to do too much and taking on too many priorities simultaneously
Recommended Actions
Product Division Actions:
Define a competency for managers to conduct effective 1-2 year career development conversations. Update the product handbook with guidance, and develop a template so we have a consistent approach across the division. Each team member should then feel welcome to fill out the document and schedule time with their manager to discuss. We very much want to be having regular career conversations on a cadence the works for each person.
Invest in a more robust Learning & Development curriculum for Product team members for at least three key competencies. Leverage content from LinkedIn, ProductTalk, and homegrown sources. Link content to competency development handbook descriptions, the Career Development Framework, and onboarding instructions.
Develop a product handbook page outlining the required monthly release tasks for PM's. Make it clear what's required, and what's optional and/or there for best practice guidance. Endeavor to reduce the monthly release task list for PM's so there is more time available for high value activities like customer, market, and competitive research.
Scott W / Anoop actions:
Partner with People Ops and the Total Rewards to evaluate comp philosophy changes for the Product division in light of increasing comp expectations for team members.
Continually focus on driving as much prioritization and e-group alignment as possible
Update the 3-year and 1-year product strategy content and review with the team
Create a survey for sharing with the Product Team to collect more info about the drop in retention metrics - @kencjohnston
Guidance on how managers can access their reportsPlease note that managers who received less than 5 responses from their direct reports will not have access to a report in Culture Amp. This is to ensure we protect anonymity through the process.
Thanks @glucchesi - can we get a similar report - or maybe just this table - broken down by Section? I think that would help give us Section leaders specific insights for our own action plans.
The reason Section is not pulled in is because CultureAmp auto sync with BambooHR, and Section isn't a field we have indicated in BHR, so it's a manual add in.
I'll keep you posted on People Ops updates to see if we can incorporate this in for this years' results! cc: @jkalimon
@kencjohnston we discovered some reporting capabilities after the reports were sent, we are working with CultureAmp to determine how we can get this corrected for this survey. I will keep @glucchesi posted as we are pushing hard to get a positive solution.
@glucchesi@jkalimon - Sorry - I used the word Section but just "Manager below Anoop" would be sufficient. I can see my own individual report and I put it into a spreadsheet here. It would be great to have all the other managers (Eric, David, Josh, Justin) as columns in that table if they had sufficient responses to add to the report.
@kencjohnston ah understood! The people team wouldn't typically share peer reports between peers, so for this case, I'd recommend asking other managers directly if they're comfortable adding their team's responses to the sheet you prepared.
@glucchesi - Ah. OK. @ebrinkman@david@joshlambert@justinfarris - how do you feel about adding your sections/teams to this table in the interest of transparency? I'm interested if there are areas that your teams evaluated a dimension significantly higher to learn what you are doing in those areas so I can replicate them. cc - @adawar
@ebrinkman unfortunately you wouldn't see if you have less than 5 directs who responded to the survey. This is a tradeoff in ensuring we preserve anonymity in responses. This is a limitation @jkalimon is working through with the Ops team (implementing sub-department level reporting would be a workaround here as it would enable you to see everyone in the Dev section).
@glucchesi - Oh, so Eric can't see his org's response (not just filtered for his direct reports)? I thought that was what I was seeing when I looked at the Team Member Engagement Survey Nov 2020 report directly in CultureAmp. I don't have more than 5 direct reports either.
@kencjohnston could you share your report view? In theory, neither one of you would be able to see due to the less than 5 responses limitation. cc: @jkalimon for Ops team visibility on this
I had seven respondents which is the number of indirect reports I have. I figured that was the way we protect anonymity is that if there isn't enough direct reports at one level, the results are available at the next level up.
@kencjohnston yes! This is logical and how it should be working across the board, but we've been struggling with the manager roll-up access piece, which is why we've been trying to get the "sub-department" level (Section) imported so you all have access to this information.
@glucchesi can you please share more details about the companies/the benchmarks we're quoted against? Also, when we set goals of where we want to be for different questions of engagement - where do we want to be in relation to the benchmark and the market in terms of percentile?
It feels like often the benchmarks are quoted and we're higher than them that then gives the impression we're at a desired outcome. I wonder if we want to be at the very top of the market or sector and not necessarily "settle" for beating benchmark scores. Hope I'm coming across clearly :)
@eliran.mesika yes - you're coming across clearly! Here is a bit more information on the New Tech (1000+) 2020 benchmark that CultureAmp (the tool we use for the engagement survey) measures us against.
The incorporation of this benchmark in the tool is a fairly new metric, so I don't believe we've set where we want to be in relation specifically, and for now have been reviewing to ensure we're ideally higher than other similar organizations. It could be interesting to set more defined company-wide goals here - @jkalimon is there a specific percentile over the new tech benchmark we're targeting for this engagement survey company-wide/is this being considered for the future?
@glucchesi and @eliran.mesika no there is no specific % over the tech benchmark that we are tarting company wide for the engagement survey. It is there as a reference point to compare against like companies.
@gl-product at next Tuesday's team meeting, we will be reviewing engagement survey results and proposed action items for the Product division. Please review in advance so we can have a rich discussion next Tuesday. Thanks!
@sfwgitlab@adawar@kencjohnston (I don't want to ping all the product leaders, but feel free to add other or just chime in) As I won't attend the meeting next Tuesday, I'd like to ask a few questions, and perhaps start the discussion here.
My first question is: what were the surprising or unexpected survey results for you? I'm asking this because I presume these might be the areas where you need the most information from us.
@nagyv-gitlab I was surprised at the size of the drop in scores regarding L&D (learning and development) and compensation. Anoop and I do regular skip level meetings and those two topics hadn't come up much. I can understand why comp wouldn't come up in that forum as it's a sensitive topic, and will probably be sensitive on the all PM call as well. So please work directly with your manager if there are topics that are uncomfortable to discuss in the larger forum.
Is the timing of the survey always the same? I recall it being a month or two later this year and I wonder if that put it closer to Annual performance reviews in Product which impacted compensation perceptions
@nagyv-gitlab - Thanks for asking. The retention indicator scores are the most surprising to me - and where I am most interested in hearing more from the team. We'll discuss it transparently on Tuesday, but I'll also be creating an opportunity for team members to contribute privately by creating a survey.
That is a good thought @jreporter - Historically the survey has launched closer to October, so this could be a factor that played in. @jkalimon from the People Ops side, will we plan to launch in October in the future? Or continue with the Nov/Dec launch timeline?
@sfwgitlab my personal PoV on L&D resources: I don't believe that there are any significant L&D resources for Product professionals here (outside from the odd bookclubs). I've taken a single training so far, and it wasn't great. On the other hand, it doesn't really bother me much that these aren't there -- I'm fully capable of going and finding resources to train myself when needed.
My answer to that question was negative because it was asking about availability of resources, but it didn't ask anything about my personal sentiment about this. Could be part of what's driving the low score but the fact that you don't hear much about it?
Thank you @glucchesi for linking me to this issue. @deuley thanks for the discussion around L&D resources for Product Professionals. Not sure if you've had the chance to check out our recent subscription to LinkedIn Learning but there are some Product specific courses in there:
@Josh_Zimmerman I've looked at the available courses there, and over half of it is either irrelevant or table-stakes for anyone who already is working as a Project Manager. Approximately:
7 courses @ 15 hours of content is targeted towards first-time PMs and those looking to move in to the role
5 courses @ 5 hours of content is fairly table-stakes content for intermediate PMs
9 courses @ 5 hours of content (half of which are 30m in length) are more advanced and interesting concepts
The longest and most in-depth course in the whole catalog is Becoming a Product Manager (10h), which I don't think can be considered L&D for this audience. ;-P
L&D: there are ad-hoc possiblities, and as already collected by Patrick, LinkedIn Learning won't solve this. I think that many of us knows its own shortcomings (in my case: English and financials). Still, I'm on my own (not true, @kbychu supports me a lot :)) to come up with trainings in these areas. Thinking about the various initiatives we went through the past year, I can imagine that some PMs might miss data-related trainings, I continuously hear from UX how great my user interviews are, thus it might happen that others would need trainings there too, and in general around discovery, my personal perception is that we could be much-much stronger. In this respect, we've had a single organized Continuous Interview course, and otherwise we don't have much content in the handbook about where to look for trainings. As I wrote these down, I volunteer to put together such a page.
@kencjohnston My 2 cents about retention: seeing the cut-backs that happened this year, the turbulent re-organizations and a halt on R&D spending is pretty demotivating. At the same time, working for GitLab is a signal on the job market that we are great PMs (even if this might not be true), as a result, I personally receive many more offers from all kinds of companies than I did before. I don't know how much this might be true about others, these are just my personal experiences. (and I don't remember my answers neither from this, nor from last year)
personally receive many more offers from all kinds of companies than I did before.
Plus one here, especially in the last few months, I receive several inquiries each week for new opportunities at companies which is up from a handful a month.
@sfwgitlab I am concerned about the unfavorable perception of the PM's role. Even though the unfavorable perception is small, the unfavorable perception seems to be increasing with tenure. While this may not be something to address in a group setting, I would be interested to know if there is something systemic. What used to be better as a PM at GitLab that's no longer true.
I'll +1 @nagyv-gitlab this as well, specifically for:
seeing the cut-backs that happened this year, the turbulent re-organizations and a halt on R&D spending is pretty demotivating
It definitely feels stale and somewhat monotonous when we can't keep up with all of the features we need to ship, or other tasks, and there's seemingly no hope of additional help coming in the foreseeable future due to R&D adjustments. It feels more like a monthly cadence of managing fires rather than roadmaps.
I also relate to @jreporter and her comment above.
The trend of recruiter messages for me has largely remained the same quantity-wise, but quality has shifted positively, I think, due to larger orgs moving to remote structures.
If I'm being completely vulnerable and open: eventually "the right" opportunity will find all of us - and that's a great thing - but it becomes a bit easier each day/week/month to consider other opportunities.
As mentioned above, keeping R&D investment flat may be challenging as this means we need to find other ways of scaling our operation. That is not necessarily a bad thing because it allows us to identify efficiency improvements and different ways of working (e.g. relying more on "self-service" frameworks to scale) but it can put additional strain on parts of the org. In my experience can lead to a dynamic where some groups are mainly busy with "fire fighting".
Limited growth in the org may also limit personal growth opportunities
With regards to compensation, one thing that required more time investment than I had hoped is to actually understand how changes in location factor will affect me. Some of the concerns regarding the implementation of Geo zones are shown out here: https://gitlab.com/gitlab-com/people-group/total-rewards/-/issues/228#note_467977561 This comment is really more about the process than the net compensation.
Personally receive many more offers from all kinds of companies than I did before.
Seeing the cut-backs that happened this year, the turbulent re-organizations and a halt on R&D spending is pretty demotivating
Limited growth in the org may also limit personal growth opportunities
I echo all 3 of these. Product Mgmt is a competitive field, but opportunities seem to be expanding across the industry quickly, especially for those with some tenure and experience is multiple pm verticals. I've had to remove LinkedIn from my phone due to the frequency of messages.
I wont be able to participate in the product weekly however i would like to share my perspective:
COVID had disrupt the market and caused more organizations to become all-remote - which means that GitLab is not unique anymore.
Cut-backs, re-org and inability to hire external candidates is a real concern to me, we are losing valuable people and unable to compensate it with external candidates, in addition our competitors are not doing any hire freeze and my concern is that we will lose our competitive advantage.
The fact that not all of us are GitLab full-time GitLab employees is demotivating - While we all understand that there is no real difference from an organization point of view, there is a difference in terms of state laws, benefits and even mentality.
Our salary is measured in USD, in some countries it hits all time low, while this is not impacting our existing salary, it might impact our next compensation update.
marked the checklist item Create a survey for sharing with the Product Team to collect more info about the drop in retention metrics - @kencjohnston as completed
Kenny Johnstonchanged the descriptionCompare with previous version
There may be ways to make I believe there are good career opportunities for me at GitLab. more fine granularity to help decide on solutions. When I read that statement, there are several things I could see it meaning:
Successful growth of GitLab the company to put on a resumé
Individual advancement opportunities beyond current role
Learn additional skills / expertise
Compensation & benefits
One suggestion I'd have with the skills and learning opportunities, it doesn't seem like everyone is aware of the tuition reimbursement page in the handbook. My read of that section is it could also be applied to individual classes or training, rather than a full degree program. Mentioning this could help encourage more folks to use it. This is a major differentiator for GitLab compared to a lot of other employers and so folks should take advantage of it if possible.
I've had good experiences with classes like Pragmatic Marketing which are far too expensive to pay for out of pocket but seem to be exactly what would fit well in that program. That would also allow individuals to tailor their learning to their own needs, rather than needing to enroll the entire Product org in the same course.
I just wanted to add that I completely agree with the Pragmatic Marketing comment. I was lucky to be able to take some of those courses in my last job and they were the most informative and helpful product management courses I've found. I would love to be able to take the remaining few that I'm missing (and maybe retake the others, since it has been several years) and see more awareness of the Pragmatic Marketing framework across the board in the product org.
I've also created an issue on another survey that was done around the same time: #2045. This survey intended to go deeper on pain points that ICs have been feeling in the Product Department; would love to discuss these live at the Product Meeting and go into deeper detail on how we can help.
Reviewing the survey for next steps and action items, I'll highlight a few themes that emerged from the results in #2045 (commenting here so we keep this issue as the SSOT):
Need for a longer-term vision to help inspire the team and help stages set priorities. Setting a clear and ambitious vision came up as an ask for Product leadership, as well as documenting our priorities so PMs can make tradeoffs (i.e. category maturity vs. PIs vs. ARR vs. usability improvements) and focus effectively.
Process and reactive communication could be taking time away from customer conversations, strategy, and high-leverage activities. Tasks taking the most time could be streamlined to make room for more activities the team wishes they could spend more time on (customer conversations, strategy, discovery/validation work).
Action: @fseifoddini is already working on this, but responding to @ mentions could be something we could add better handbook guidance on. Responding to Todos was mentioned as something that takes up a lot of everyone's time.
Revisiting our product processes to make sure we're being efficient and not overburdening the team with tasks.
Action: Assign a working group or (preferably more informally) organize a small squad to huddle on handbook updates, focusing primarily on what we can remove and make more efficient. @adawar mentioned that he would be looking at the handbook with this lens, but I suggest doing this as a small group with a focus on opening MRs with a streamlined review process.
@fseifoddini: What do you think about #3 as something to consider organizing in the near future? Getting 3-4 PMs to block and tackle on improving the handbook and removing content could result in a bunch of proposed improvements - if we combine it with a quick review process so these MRs don't sit in review limbo, it could result in some nice improvements in short order.
@sfwgitlab Updated 3 actions look great and capture the live feedback in product weekly and as well the product leadership weekly, thank you! I'm happy to own leading our collaboration and will ramp up the week of February 8.
Product Division Actions:
Define a competency for managers to conduct effective 1-2 year career development conversations. Update the product handbook with guidance, and develop a template so we have a consistent approach across the division. Each team member should then feel welcome to fill out the document and schedule time with their manager to discuss. We very much want to be having regular career conversations on a cadence the works for each person.
Invest in a more robust Learning & Development curriculum for Product team members for at least three key competencies. Leverage content from LinkedIn, ProductTalk, and homegrown sources. Link content to competency development handbook descriptions, the Career Development Framework, and onboarding instructions.
Develop a product handbook page outlining the required monthly release tasks for PM's. Make it clear what's required, and what's optional and/or there for best practice guidance. Endeavor to reduce the monthly release task list for PM's so there is more time available for high value activities like customer, market, and competitive research.
@sfwgitlab@adawar The initial iteration of the actions we committed to are complete. I will pull other relevant feedback from this issue and create other issues for the Product Operations board for prioritization. If it makes sense to close this issue given the actions are complete, please do so. Thank you!
thanks @fseifoddini and the WG for the great results on these three actions! I'll close this issue, and close the loop with the PM team on the next all hands call.
@fseifoddini I'd recommend to /promote this issue into an Epic, and add &1386 (closed)&1387 (closed)&1388 (closed) as sub-epics, so it'll be easy to navigate around WG items. As you are the DRI, I did not wanted to hit /promote as it closes this issue (and opens an epic with all the content from here).