2nd delivery

parent e923077c
......@@ -107,9 +107,9 @@ From focused interviews, they analyzed how organizations implemented \devops in
As observed in academic studies, organizations also have distinct perceptions about what \devops is and what problems they try to solve when implementing it. Consequently, its deployment and evaluation differ among organizations.
%
Although both academic studies and organizations stated the benefits of adopting \devops, Erich \textit{et al.} concluded that no quantitative studies support such a claim~\cite{Erich2017SLR}, and reinforce the necessity of experimental studies to verify the effectiveness of \devops.
Fran\c{c}a et al.~\cite{travassos2016SLR} assumed academic studies were insufficient to thoroughly address \devops, so they alternatively conducted a Multivocal Literature Review (MLR),
in which 69\% (30) of the sources are gray literature, represented by websites, books, industry journals, and industry technical reports.
Fran\c{c}a et al.~\cite{travassos2016SLR} assumed academic studies were insufficient to thoroughly address DevOps, so
they alternatively conducted a Multivocal Literature Review (MLR), in which most of the sources were extracted
from gray literature, including books, websites, and industry journals and technical reports.
%
The authors describe \devops as a set of principles, practices, required skills and organizations' motivations to adopt it.
%
......
......@@ -46,7 +46,7 @@ Each programming language has some tools to cope with the build process by suppo
Each programming language also has some unit-test frameworks, which provide vital feedback for developers about the correctness of the software. Some examples are JUnit\swurl{junit.org} and Mockito\swurl{site.mockito.org} for Java, RSpec\swurl{rspec.info} for Ruby, and NUnit\swurl{nunit.org} for .NET. More sophisticated testing which automates the end-user behavior for web applications is possible with browsing automation tools, such as Selenium\swurl{seleniumhq.org}.
Another type of feedback for developers is regarding source code quality, which is provided by source-code static analysis tools, such as SonarQube\swurl{sonarqube.org} and Analizo~\cite{terceiro2010analizo}. SonarQube classifies code problems and evaluates coverage metrics as well as technical debts in several programming languages via its plugins. Analizo supports the extraction and calculation of a fair number of source code metrics for C, C++, C\#, and Java codes. In general, source-code analysis tools can point to issues in source-code, like non-compliance to standard style, problems in maintainability, risk of bugs, and even vulnerability breaches. Source-code static analysis tools vary in supported programming languages and in the way their are delivered. They can be provided as-a-service, Code Climate\swurl{codeclimate.com} is an example, or even within the developer environment through tools such as PMD\swurl{acanda.github.io/eclipse-pmd} for the Elicpse IDE.
Another type of feedback for developers is regarding source code quality, which is provided by source-code static analysis tools, such as SonarQube\swurl{sonarqube.org} and Analizo~\cite{terceiro2010analizo}. SonarQube classifies code problems and evaluates coverage metrics as well as technical debts in several programming languages via its plugins. Analizo supports the extraction of a variety of source code metrics for C, C++, C\#, and Java codes. In general, source-code analysis tools can point to issues in source-code, like non-compliance to standard style, problems in maintainability, risk of bugs, and even vulnerability breaches. Source-code static analysis tools vary in supported programming languages and in the way their are delivered. They can be provided as-a-service, Code Climate\swurl{codeclimate.com} is an example, or even within the developer environment through tools such as PMD\swurl{acanda.github.io/eclipse-pmd} for the Elicpse IDE.
%TO-DO Paulo
%Seperar ferramentas de extração (Analizo, Pylint e Metric_fu) de plataformas (Sonar, Mezuro e CodeClimate)
......
......@@ -18,7 +18,8 @@ In this section, we discuss the implications of \devops adoption for practitione
\myparagraph{Testing} although companies recognize the importance of automated testing, they still struggle to fully implement it~\citesel{olsson2012stairway}. This is especially true for user interface tests automation~\citesel{leppanen2015highways}. Other factors that make automated testing complex are hardware availability for load testing and user experiment assessment~\citesel{leppanen2015highways}. Parallelizing test suites can be necessary to reduce testing time~\citesel{neely2013easy}. Tests that seem to fail at random, called ``flaky tests,'' are not acceptable~\citesel{neely2013easy}.
%Without the necessary care with test automation, investments on continuous integration infrastructure is worthless.
\myparagraph{Quality assurance team} quality assurance skills are necessary for locating specific error scenarios and corner cases. However, preserving the quality assurance team separate from the development team is questionable. \devops and agile practices require a change in the quality assurance team's role, or possibly its elimination~\citesel{bass2018architect}.
\myparagraph{Quality assurance team} quality assurance skills are necessary for locating specific error scenarios and corner cases. However, preserving the quality assurance team separate from the development team is questionable. DevOps and agile practices require a change in the role of quality
assurance teams, or even its elimination~\citesel{bass2018architect}.
\myparagraph{Legacy systems} although it takes a great deal of work to achieve continuous delivery on mainframe platforms, there are successful reports of it~\citesel{humble2017willwork}. Some legacy architectures might not be designed to run automated tests~\citesel{leppanen2015highways}. Nonetheless, teams must be aware that cultural factors, such as managers who say ``This is the way we've always done it''~\citesel{humble2017willwork}, can limit the adoption of continuous delivery more than technical factors.
......@@ -26,7 +27,7 @@ In this section, we discuss the implications of \devops adoption for practitione
\myparagraph{Learning} software professionals must be prepared to learn new tools, focusing on automation~\cite{xebia2016table}. They must also cope with the ever-changing nature of these tools, which implies maintaining heterogeneous environments and migrating technologies~\citesel{shahin2016architecting}.
\myparagraph{Building the deployment pipeline} the benefits delivered by a deployment pipeline are many. However, engineers must be aware that setting up the infrastructure required for continuous deployment could be cumbersome~\citesel{leppanen2015highways, neely2013easy}. Breaking down the system into microservices also requires building multiple pipelines. Engineers should not try to build all the continuous delivery ecosystem in a single step: an approach based on continuous improvement is preferable~\citesel{neely2013easy}.
\myparagraph{Building the deployment pipeline} the benefits delivered by a deployment pipeline are many. However, engineers must be aware that setting up the infrastructure for continuous deployment can demand a considerable effort~\citesel{leppanen2015highways, neely2013easy}. Breaking down the system into microservices also requires building multiple pipelines. Engineers should not try to build all the continuous delivery ecosystem in a single step: an approach based on continuous improvement is preferable~\citesel{neely2013easy}.
\myparagraph{Pipeline maintenance} pipeline execution generates a lot of artifacts, such as build and logs. Artifacts such as production logs, bug tracks, parameters configuration, and temporary files, must be properly archived and removed at some point~\citesel{pang2016maintenance}. Despite its importance to pipeline sustainability, organizations often overlook this maintenance process~\citesel{pang2016maintenance}.
......@@ -38,7 +39,7 @@ In this section, we discuss the implications of \devops adoption for practitione
\subsection{Implications for managers}
\myparagraph{Adoption of lean principles} since \devops is based on lean principles~\citesel{claps2015journey}, organizations eager for \devops adoption should take a step back and learn them~\cite{poppendieck2006lean}. In particular, Kim recommends~\cite{kim2012ways}: 1) mapping the value stream for optimizing the performance of the entire system, as opposed to local optimization; 2) amplifying continuous feedback loops to support necessary corrections; and 3) improving daily work through a culture that fosters continual experimentation, risk-taking, learning from failure, and understanding that repetition and practice are prerequisites to mastery.
\myparagraph{Adoption of lean principles} since \devops is based on lean principles~\citesel{claps2015journey}, organizations eager for \devops adoption should take a step back and learn them~\cite{poppendieck2006lean}. In particular, Kim recommends~\cite{kim2012ways}: 1) mapping the value stream for optimizing the performance of the entire system, as opposed to local optimization; 2) amplifying continuous feedback loops to support necessary corrections; and 3) improving daily work through a culture promoting frequent experimentation, risk-taking, learning from mistakes, and knowing that practice and repetition are prerequisites to mastery.
\myparagraph{\devops adoption} how to deploy \devops in an organization is the critical question for managers. However, the lack of consensus on the meaning of \devops is one of the reasons that it is a hard decision. This subject is explored by many studies, organizations still struggle with it, and we discuss it in more depth in Section~\ref{sec:devosp-adoption-approaches}.
......
......@@ -50,7 +50,8 @@ An extra challenge for cross-functional teams is guaranteeing that specialists i
Assigning a ``\devops team'' as a bridge between developers and operators~\citesel{nybom2016mixing} has become a trend~\cite{puppet2014devops}. However, it is not clear what precisely ``\devops teams'' are, and how they differ from regular operations teams~\citesel{nybom2016mixing}\cite{puppet2014devops}. This approach is criticized by Humble, who considers that potentially creating a new silo is not the best policy to handle the \devops principle of breaking down walls between silos~\cite{humble2012team}.
Skelton and Pais present some variations, called ``\devops topologies,'' and also \devops anti-patterns~\cite{skelton2013topologies}. They argue that, although the \devops team silo is an anti-pattern, it is acceptable when it is temporary and focused on sharing development practices to operations staff and operations concerns to developers. Siqueira~\emph{et al.} report how a \devops team with senior developers and rotating roles among members was a key decision to construct and maintain a continuous delivery process~\citesel{siqueira2018gov}.
Close to ``\devops teams'' is Site Reliability Engineering (SRE) from Google~\cite{beyer2016site}. In addition to product teams, Google has SRE teams responsible for product reliability engineering (availability, latency, performance, efficiency, change management, monitoring, emergency response, and capacity planning). SRE teams have an upper limit of using 50 percent of their time time on operational tasks because they must spend at least 50 percent of their time on development work to increase product reliability.
Close to ``\devops teams'' is Site Reliability Engineering (SRE) from Google~\cite{beyer2016site}. In addition to
product teams, Google has SRE teams responsible for product reliability engineering, what includes performance, availability, monitoring, and emergency response. SRE teams have an upper limit of using 50 percent of their time time on operational tasks because they must spend at least 50 percent of their time on development work to increase product reliability.
One should also question the impact of deployment automation on operators' roles. Chen reported that, after continuous delivery adoption, operations engineers only needed to click a button to release new versions~\citesel{chen2015huge}. One should question the role of an engineer in such a context.
Moreover, if software updates are adequately broken down into small increments, and delivery automation turns deployment in a ``non-event''~\citesel{humble2011devops}, practices such as
......@@ -68,9 +69,11 @@ Some researchers have proposed maturity models for \devops adoption and continuo
Forsgren and Kersten claim that both survey data and system data~\citesel{forsgren2018metrics} should measure \devops transformations. System data can provide a continuous flow of information, although setting up the aggregation of metrics from different sources can be challenging. Survey data provides a holistic view of out-of-the system issues, such as culture, job satisfaction, and burnout, and can even point to problems on system-data collection.
Whether survey or system data, Forsgren and Kersten still make clear that if results are used to punish teams, data collection will be unreliable~\citesel{forsgren2018metrics}, as also reinforced by Brown from Microsoft~\cite{microsoft2018devops}. Kua cautions that inappropriate use of metrics can lead to problematic behavior and ultimately detracts from a broader project and organizational goals~\cite{kua2013metrics}. Kua also compiles guidelines for better usage of metrics, by: explicitly linking metrics to goals, favoring tracking trends over absolute numbers, using shorter tracking periods, and changing metrics when they stop driving change.
Whether survey or system data, Forsgren and Kersten still make clear that if results are used to punish teams, data collection will be unreliable~\citesel{forsgren2018metrics}, as also reinforced by Brown from Microsoft~\cite{microsoft2018devops}. Kua cautions that inappropriate use of metrics can lead to undesired behavior and even divert the organization from its goals~\cite{kua2013metrics}. Kua also compiles guidelines for better usage of metrics by: explicitly linking metrics to goals, favoring trends over absolute numbers, tracking shorter periods, and changing metrics when they do not drive change anymore.
Some examples of metrics based on survey data used on the State of DevOps Reports~\cite{puppet2014devops, puppet2016devops} are: IT performance as a function of deployment frequency, lead time for changes, and mean time to recover; time spent on unplanned work and rework; typology of organizational culture (pathological, bureaucratic or generative) based on job satisfaction, climate for learning, win-win relationship between developers and operators, usage of version control, and usage of automated testing; and employee engagement, based on the Net Promoter Score, indicating how likely employees are to recommend the company products and services.
Some examples of metrics based on survey data used on the State of DevOps Reports~\cite{puppet2014devops, puppet2016devops} are: IT performance as a function of lead time for changes, frequency of deployment, and mean time to recover; time spent on unplanned work and rework; typology of organizational culture (pathological, bureaucratic or generative) based on climate for learning, job satisfaction, developers and operators having win-win relationships, usage of version control, and usage of automated testing; and
employee engagement, based on the Net Promoter Score, indicating how likely employees are to
recommend the company products and services.
Snyder and Curtis use metrics to evaluate \devops in a specific company~\citesel{snyder2018analytics}. Metrics such as productivity, defect ratio, dollars spent, cycle time release, and build count were aggregated in a ``total quality index''. More importantly, the aggregation of data from different silos was used to align organizational efforts. The authors observed a higher increase in the total quality index of agile and \devops teams, which helped executives justify investments in the agile-\devops transformation.
......
......@@ -3,7 +3,10 @@
% based on the paper "A systematic literature review of service
% choreography adaptation" we have published on SOCA
Given the vast amount of work on \devops, we could not conduct an exhaustive search in every source. We have not considered academic workshops, and we mentioned only a few books in the field. It would not be practical to try to cover every existing work on \devops and to condense it all in this single survey. However, by focusing on journal and conference papers, and by applying the snowballing process, we aimed to cover the primary literature in the field. We decided not to expand the query string to avoid artificially favoring some of the \devops concepts over others. However, the appearance of the keyword ``\devops'' was not mandatory in the snowballing process so that we could select vital work tackling highly related subjects such as continuous delivery.
Given the vast amount of work on \devops, it was not feasible to conduct an exhaustive search in all
the available sources. We have not considered academic workshops, and we mentioned only a few books in the field. It would not be practical to try to cover every existing work on \devops and to condense it all in this single survey. However, by focusing on journal and conference papers, and by applying the snowballing process, we aimed to cover the primary literature in the field. We decided not to expand the query string to avoid artificially favoring some of the \devops concepts over others. However, the appearance of the keyword ``\devops'' was not mandatory in the snowballing process so that we could select vital work tackling highly related subjects such as continuous delivery.
The choice of core and relevant papers may suffer from subjective bias. To reduce this bias, at least two authors participated in the selection process and the classification of papers. More experienced researchers also supervised the whole process.
The choice of core and relevant papers may suffer from subjective bias. To reduce this bias,
at least two authors conducted the selection process and the classification of papers.
Senior researchers also oversaw the whole process.
Our study also suffers the publication bias, since positive results are more likely to be published than negative results. We did not find, for example, a paper reporting a case of failed \devops adoption. We did not address this issue in this study.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment