3rd delivery

parent f222e9cb
\section{Introduction}
\label{sec:intro}
% Comentario comittado com git
Even though the DevOps movement has been discussed for nearly a decade, it lacks a widely accepted definition. By consolidating the most cited definitions of DevOps, we crafted our definition, similar to that proposed by Dyck \emph{et al.}~\citesel{dyck2015release}, which we adopt throughout this paper:
\begin{quotation}
......@@ -32,7 +34,7 @@ We close with the limitations of this survey in Section~\ref{sec:limitations} an
\section{DevOps}
\label{sec:devops}
\devops is a natural evolution of the agile movement~\citesel{christensen2016teaching}.
\devops is an evolution of the agile movement~\citesel{christensen2016teaching}.
Agile Software Development advocates small release iterations with customer reviews. It assumes the team can release software frequently in some production-like environment. However, the pioneer Agile literature puts little emphasis on deployment-specific practices.
No Extreme Programming (XP) practice, for example, is about deployment~\cite{beck2004xp}.
The same sparse attention to deployment is evident in traditional software engineering processes~\cite{kroll2003rup} and books~\cite{pressman2005software,sommerville2011software}. Consequently, the transition to production tends to be a stressful process in organizations, containing manual, error-prone activities, and, even, last-minute corrections~\cite{humble2010continuous}.
......
......@@ -38,9 +38,10 @@ This process was supported by tools like Mendeley\swurl{mendeley.com}, Libre Off
\subsection{Producing the conceptual maps}
Conceptual maps are diagrams structured as graphs in which nodes depict concepts and arrows represent relationships among concepts~\cite{hager1997designing}. Our process of building the conceptual maps was based on Grounded Theory, which is a recognized method for conducting emergent qualitative research~\cite{corbin2014slr} and it is also considered an appropriate research method to explore both human and social aspects of Software Engineering~\cite{stol2016grounded}. According to Charmaz, it reduces preconceived ideas about the research problem and helps researchers remain open to different explanations and understandings of the data~\cite{charmaz2008grounded}. Grounded Theory is usually applied to primary sources, such as interviews and observational field-notes~\cite{sbaraini2011grounded}. In this paper, we used Grounded Theory strategies in a review of the literature to handle the intrinsic subjectivity of conceptual analysis.
Conceptual maps are diagrams structured as graphs in which nodes depict concepts and arrows represent relationships among concepts~\cite{hager1997designing}. Our process of building the conceptual maps was based on Grounded Theory, which is a recognized method for conducting emergent qualitative research~\cite{corbin2014slr} and and whose adoption in software engineering has grown considerably in recent years~\cite{stol2016grounded}. According to Charmaz, it reduces preconceived ideas about the research problem and helps researchers remain amenable to alternative interpretations and apprehend the data in different ways~\cite{charmaz2008grounded}. Grounded Theory is usually applied to primary sources, such as interviews and observational field-notes~\cite{sbaraini2011grounded}. In this paper, we used Grounded Theory strategies in a review of the literature to handle the intrinsic subjectivity of conceptual analysis.
A core strategy of Grounded Theory is \emph{coding}~\cite{charmaz2008grounded}, which breaks data down into much smaller components and labels those components~\cite{sbaraini2011grounded}. The codes, arising from different sources, are continuously compared to produce units of a higher level of abstraction called \emph{concepts}. The concepts are in turn compared and grouped to produce a third level of abstraction called \emph{categories}~\cite{stol2016grounded}. In our study, we read the \numberofmainselectedpapers core studies to produce the codes. Next, we analyzed, abstracted, and grouped the codes into concepts. Finally, we derived four categories of concepts, each one with its conceptual map.
A core strategy of Grounded Theory is \emph{coding}~\cite{charmaz2008grounded}, which breaks down and
labels data into smaller components~\cite{sbaraini2011grounded}. The codes, arising from different sources, are put together and correlated to produce \emph{concepts}~\cite{sbaraini2011grounded}. The concepts are in turn compared and grouped into \emph{categories}~\cite{stol2016grounded}. In our study, we read the \numberofmainselectedpapers core studies to produce the codes. Next, we analyzed, abstracted, and grouped the codes into concepts. Finally, we derived four categories of concepts, each one with its conceptual map.
The process of building our conceptual maps obeyed the following rules and restrictions:
......
......@@ -46,15 +46,9 @@ Even though \devops is a recent research topic, several studies have performed a
\input{tabs/tab-research-questions.txt}
Most studies mention the low quality of these initial studies~\cite{Erich2017SLR}.
The initial studies focused on basic research questions that were significant due to the novelty of the research (Table~\ref{tab:research-questions}), such as the significance of the term \devops, and the motivations and challenges to its adoption. They also concluded that the lack of a well-established definition of \devops affected the perception of \devops' benefits, expectations, and adoption challenges \cite{Erich2017SLR}. The studied literature did not try to identify a comprehensive definition of \devops, as we propose in this paper.
The initial studies focused on basic research questions that were significant due to the novelty of the research (Table~\ref{tab:research-questions}), such as the significance of the term \devops, and the motivations and challenges to its adoption. They also concluded that the absence of a well-established DevOps definition affected the perception of its benefits, expectations, and adoption challenges~\cite{Erich2017SLR}. The studied literature did not try to identify a comprehensive definition of \devops, as we propose in this paper.
%
Erich et al. present \devops as the interaction between development and operations at the individual, team and departmental level~\cite{Erich2014SLR,Erich2017SLR}.
%
For Fran\c{c}a at al., \devops is a neologism for a movement of IT professionals taking a new attitude on software delivery by the collaboration between development and operations, and it is based on a set of principles and practices, such as culture, automation, measurement, and sharing~\cite{travassos2016SLR}.
%
Ghantous and Gill summarize \devops as human communication and collaboration, continuous delivery, and an automated pipeline~\cite{gill2017SLR}.
%
Jabbari et al. define \devops as a development methodology aimed at bridging the gap between Development and Operations~\cite{Jabbari2016SLR}.
Erich et al. present DevOps as the ``interaction between development and operations at the individual level, team level and departmental level''~\cite{Erich2017SLR}. For Fran\c{c}a et al., ``DevOps is a neologism representing a movement of ICT professionals addressing a different attitude regarding software delivery through the collaboration between software systems development and operation functions, based on a set of principles and practices, such as culture, automation, measurement and sharing''~\cite{travassos2016SLR}. Ghantous and Gill summarize DevOps as human communication and collaboration, continuous delivery, and an automated pipeline~\cite{gill2017SLR}. Jabbari et al. define DevOps as ``a development methodology aimed at bridging the gap between Development and Operations''~\cite{Jabbari2016SLR}.
......@@ -87,7 +81,8 @@ Jabbari et al. define \devops as a development methodology aimed at bridging the
In general, previous SLR works made clear the necessity for more research on \devops.
%
They outlined the conceptual gap between academic research and professional practices in \devops \cite{travassos2016SLR}.
They outlined the distinct conceptualizations undertaken by
academics and practitioners in DevOps~\cite{travassos2016SLR}.
%
They deployed methods other than Systematic Literature Review to uncover both academic and practitioners perspectives and to address the novelty of DevOps and the aforementioned conceptual gap.
%
......@@ -100,7 +95,7 @@ Erich \textit{et al.} \cite{Erich2014SLR,Erich2017SLR} is the most referenced li
%
The authors performed an SLR~\cite{Erich2014SLR,Erich2017SLR} and conducted focused interviews on \devops principles and practices at six organizations~\cite{Erich2017SLR}.
%
From SLR, they extracted seven areas related to \devops: culture and collaboration, automation, measurements, sharing, services, quality assurance, and governance. They conclude that there was, in general, a low quality on the academic studies on \devops and a lack of studies to evidence DevOps' effectiveness.
From SLR, they extracted seven areas related to \devops: ``culture of collaboration, automation, measurement, sharing, services, quality assurance, and governance''~\cite{Erich2017SLR}. They conclude that there was, in general, a low quality on the academic studies on \devops and a lack of studies to evidence DevOps' effectiveness.
%
From focused interviews, they analyzed how organizations implemented \devops in four dimensions: governance, personal traits, department, and practitioners. They categorized answers from each organization into one of the seven areas extracted from SLR to portray their perceptions.
%
......@@ -111,7 +106,7 @@ Fran\c{c}a et al.~\cite{travassos2016SLR} assumed academic studies were insuffic
they alternatively conducted a Multivocal Literature Review (MLR), in which most of the sources were extracted
from gray literature, including books, websites, and industry journals and technical reports.
%
The authors describe \devops as a set of principles, practices, required skills and organizations' motivations to adopt it.
The authors describe \devops as a collection of principles, practices, required skills and organizations' motivations to adopt it.
%
They identified seven areas related to \devops: social aspects, automation, measurements, sharing, quality assurance, and leanness.
%
......
......@@ -42,7 +42,7 @@ The most traditional tools in this category are SVN\swurl{subversion.apache.org}
Build tools are highly developer-centered. Their goals relate to enabling \emph{continuous delivery} and the \emph{delivery} category of concepts. We have considered not only tools that generate deployable packages, also called \emph{builds}, but also tools that generate essential artifacts and feedback using the source code as input.
Each programming language has some tools to cope with the build process by supporting dependencies resolution, implementation of custom tasks, or generation of deployable packages. Examples of dependency managers, also called package managers, are Pip\swurl{pypi.org/project/pip} for Python, RubyGems\swurl{rubygems.org} for Ruby, NuGet\swurl{nuget.org} for .NET, and Maven\swurl{maven.apache.org} and Gradle\swurl{gradle.org} that compete in the Java landscape. When there is a need for more flexibility to implement arbitrary commands in the build process, it is convenient to use \emph{make tools}, such as the classic GNU Make\swurl{gnu.org/software/make}, that is often used for GNU/Linux packages, or other options like Ant\swurl{ant.apache.org} for Java, and Rake\swurl{ruby.github.io/rake} for Ruby. Some of the cited tools, such as Maven, are also responsible for producing the deployable package, such as WAR files for Java environments. However, for some languages, like Python and Ruby, there is no need for producing a deployable package as a single-file artifact. It is also possible to use more generic package tools coupled to the target environment, such as Debian packages\swurl{debian.org/distrib/packages}.
Each programming language has some tools to cope with the build process by supporting dependencies resolution, implementation of custom tasks, or generation of deployable packages. Examples of dependency managers, also called package managers, are Pip\swurl{pypi.org/project/pip} for Python, RubyGems\swurl{rubygems.org} for Ruby, NuGet\swurl{nuget.org} for .NET, and Maven\swurl{maven.apache.org} and Gradle\swurl{gradle.org} that compete in the Java landscape. Gradle eases the implementation of custom tasks during the build, as also done by GNU Make\swurl{gnu.org/software/make}, that is often used for GNU/Linux packages, or Rake\swurl{ruby.github.io/rake} for Ruby. Some of the cited tools, such as Maven, are also responsible for producing the deployable package, such as WAR files for Java environments. However, for some languages, like Python and Ruby, there is no need for producing a deployable package as a single-file artifact. It is also possible to use more generic package tools coupled to the target environment, such as Debian packages\swurl{debian.org/distrib/packages}.
Each programming language also has some unit-test frameworks, which provide vital feedback for developers about the correctness of the software. Some examples are JUnit\swurl{junit.org} and Mockito\swurl{site.mockito.org} for Java, RSpec\swurl{rspec.info} for Ruby, and NUnit\swurl{nunit.org} for .NET. More sophisticated testing which automates the end-user behavior for web applications is possible with browsing automation tools, such as Selenium\swurl{seleniumhq.org}.
......
......@@ -39,9 +39,9 @@ assurance teams, or even its elimination~\citesel{bass2018architect}.
\subsection{Implications for managers}
\myparagraph{Adoption of lean principles} since \devops is based on lean principles~\citesel{claps2015journey}, organizations eager for \devops adoption should take a step back and learn them~\cite{poppendieck2006lean}. In particular, Kim recommends~\cite{kim2012ways}: 1) mapping the value stream for optimizing the performance of the entire system, as opposed to local optimization; 2) amplifying continuous feedback loops to support necessary corrections; and 3) improving daily work through a culture promoting frequent experimentation, risk-taking, learning from mistakes, and knowing that practice and repetition are prerequisites to mastery.
\myparagraph{Adoption of lean principles} since \devops is based on lean principles~\citesel{claps2015journey}, organizations eager for \devops adoption should take a step back and learn them~\cite{poppendieck2006lean}. In particular, Kim recommends~\cite{kim2012ways}: 1) mapping the value stream for optimizing global system performance, rather than for local optimization; 2) amplifying continuous feedback loops to support necessary corrections; and 3) improving daily work through a culture promoting frequent experimentation, risk-taking, learning from mistakes, and knowing that practice and repetition are prerequisites to mastery.
\myparagraph{\devops adoption} how to deploy \devops in an organization is the critical question for managers. However, the lack of consensus on the meaning of \devops is one of the reasons that it is a hard decision. This subject is explored by many studies, organizations still struggle with it, and we discuss it in more depth in Section~\ref{sec:devosp-adoption-approaches}.
\myparagraph{\devops adoption} how to deploy \devops in an organization is the critical question for managers. However, the lack of a consensual definition of DevOps is a reason for making it a hard decision. This subject is explored by many studies, organizations still struggle with it, and we discuss it in more depth in Section~\ref{sec:devosp-adoption-approaches}.
\myparagraph{Assessment} an organization should be able to measure the success of a \devops adoption process. Nonetheless, any top-down imposition of a metric-based evaluation must be used with care. If personal evaluation depends on such metrics, engineers can focus on producing good numbers rather than improving the software process~\cite{microsoft2018devops}.
......@@ -50,11 +50,11 @@ Developers must acquire skills from operators and vice versa. It is necessary to
\myparagraph{Culture} Humble stated that, typically, the obstacle to continuous delivery adoption is not the skill level of individual employees, but failures at the management and leadership level~\citesel{humble2017willwork}. High-performing organizations strive, at all levels, towards continuous improvement, rather than treating workers as fungible ``resources'' that should merely execute tasks as efficiently as possible~\citesel{humble2017willwork}. Top and low-level management are responsible for creating an environment in which failure is allowed and people seek continuous improvement.
\myparagraph{Increase in delivery throughput} Neely and Stolt reported that the delivery throughput in their company increased per developer over time as efforts toward continuous delivery progressed and that defect reporting rate narrowed and became more predictable~\citesel{neely2013easy}. Siqueira~\emph{et al.} also reported how, after investments in \devops and continuous delivery, the number of releases per semester remained the same even after a considerable decrease in the number of team members~\citesel{siqueira2018gov}.
\myparagraph{Increase in delivery throughput} Neely and Stolt reported that the delivery throughput per developer in their company increased with the adoption of continuous delivery and that the defect rate became more predictable~\citesel{neely2013easy}. Siqueira~\emph{et al.} also reported how, after investments in \devops and continuous delivery, the number of releases per semester remained the same even after a considerable decrease in the number of team members~\citesel{siqueira2018gov}.
\myparagraph{Building trust} when an organization builds software for other large organization, especially the government, it is common for the relation between contractor and contracted to be dominated by mistrust, which leads to heavy development processes. However, Siqueira~\emph{et al.} advocate, based on their experience, that continuous delivery leverages a trust relationship among contractor and contracted in software development, even in governmental projects~\citesel{siqueira2018gov}.
\myparagraph{Building for the government} although building software for government involves more bureaucratic processes, political forces may change requirements and priorities at any time~\citesel{siqueira2018gov}, which leads to the need to shorten the release cycle through the adoption of agile and \devops practices in the governmental scenario.
\myparagraph{Building for the government} although building software for government involves more bureaucratic processes, requirements and prioritization can often change due to political reasons~\citesel{siqueira2018gov}, which leads to the need to shorten the release cycle by adopting agile and DevOps practices in the governmental scenario.
\subsection{Implication for researchers}
......
......@@ -7,12 +7,12 @@ Throughout this survey, we have shown that overviewing \devops concepts and tool
\label{sec:redesign}
Highly coupled monolithic architectures are obstacles to effective continuous delivery. Complex dependency management of software components and teams is imposed to the deployment pipeline~\citesel{shahin2016architecting}.
An essential principle for successfully adopting and implementing continuous delivery is an architecture composed of small and independently deployable units~\citesel{shahin2016architecting}. Lewis and Fowler call this architectural style ``microservices,'' a single application developed as a suite of small distributed services that communicate through the network, built around business capabilities, and independently deployable~\cite{fowler2014microservices}.
An essential principle for successfully adopting and implementing continuous delivery is an architecture composed of small and independently deployable units, also called \emph{microservices} ~\citesel{shahin2016architecting}. In such architectural style, services interact through the network and are built around business capabilities~\cite{fowler2014microservices}.
Conceiving of an application architecture with loosely coupled and well-encapsulated microservices guarantees two architectural attributes required by continuous delivery: \emph{testability} and \emph{deployability}~\citesel{humble2017willwork}. If each microservice has its test suite, this reduces the blocking of deployment due to long-running tests~\citesel{leppanen2015highways}.
Whereas some authors say microservices facilitate effective implementation of \devops~\citesel{balalaie2016microservices}, others say microservices require \devops~\citesel{ebert2016devops}, since deployment automation minimizes the overhead to manage a significant number of microservices.
However, adopting microservices comes with several challenges. First, there is heterogeneity in non-functional patterns such as startup scripts, configuration files, administration endpoints, and logging locations~\citesel{callanan2016right}. Technological heterogeneity can be a productivity barrier for newcomers in the team. Second, microservices must be deployed to production with the same set of versions used for integration tests~\citesel{brown2016microservices}. These challenges can be overcome by adopting a minimum set of microservices standards across the organization~\citesel{callanan2016right}.
However, adopting microservices comes with several challenges. First, there is heterogeneity in non-functional patterns such as ``startup scripts, configuration files, administration endpoints, and logging locations''~\citesel{callanan2016right}. Technological heterogeneity can be a productivity barrier for newcomers in the team. Second, microservices must be deployed to production with the same set of versions used for integration tests~\citesel{brown2016microservices}. These challenges can be overcome by adopting a minimum set of microservices standards across the organization~\citesel{callanan2016right}.
Common microservice management patterns associated with \devops are: one deployment pipeline per microservice; log aggregator; service registry; correlation ID's~\citesel{brown2016microservices}; and segregation of source code, configuration, and environment specification~\citesel{balalaie2016microservices}.
Other complimentary patterns are strangler application~\cite{fowler2004strangler}\citesel{humble2017willwork,brown2016microservices}, load balancer, consumer-driven contracts~\citesel{balalaie2016microservices}, circuit breaker~\cite{nygard2009release}, backwards compatibility, and versioned APIs~\citesel{humble2017willwork}. However, the last one is controversial, and some communities usually do not recommend microservice versioning~\citesel{balalaie2016microservices}.
......@@ -51,7 +51,7 @@ Assigning a ``\devops team'' as a bridge between developers and operators~\cites
Skelton and Pais present some variations, called ``\devops topologies,'' and also \devops anti-patterns~\cite{skelton2013topologies}. They argue that, although the \devops team silo is an anti-pattern, it is acceptable when it is temporary and focused on sharing development practices to operations staff and operations concerns to developers. Siqueira~\emph{et al.} report how a \devops team with senior developers and rotating roles among members was a key decision to construct and maintain a continuous delivery process~\citesel{siqueira2018gov}.
Close to ``\devops teams'' is Site Reliability Engineering (SRE) from Google~\cite{beyer2016site}. In addition to
product teams, Google has SRE teams responsible for product reliability engineering, what includes performance, availability, monitoring, and emergency response. SRE teams have an upper limit of using 50 percent of their time time on operational tasks because they must spend at least 50 percent of their time on development work to increase product reliability.
product teams, Google has SRE teams responsible for product reliability engineering, what includes performance, availability, monitoring, and emergency response. SRE teams have an upper limit of using 50 percent of their time time on operational tasks because they must allocate at least 50 percent of their time increasing product reliability through engineering and development activities.
One should also question the impact of deployment automation on operators' roles. Chen reported that, after continuous delivery adoption, operations engineers only needed to click a button to release new versions~\citesel{chen2015huge}. One should question the role of an engineer in such a context.
Moreover, if software updates are adequately broken down into small increments, and delivery automation turns deployment in a ``non-event''~\citesel{humble2011devops}, practices such as
......@@ -71,7 +71,7 @@ Forsgren and Kersten claim that both survey data and system data~\citesel{forsgr
Whether survey or system data, Forsgren and Kersten still make clear that if results are used to punish teams, data collection will be unreliable~\citesel{forsgren2018metrics}, as also reinforced by Brown from Microsoft~\cite{microsoft2018devops}. Kua cautions that inappropriate use of metrics can lead to undesired behavior and even divert the organization from its goals~\cite{kua2013metrics}. Kua also compiles guidelines for better usage of metrics by: explicitly linking metrics to goals, favoring trends over absolute numbers, tracking shorter periods, and changing metrics when they do not drive change anymore.
Some examples of metrics based on survey data used on the State of DevOps Reports~\cite{puppet2014devops, puppet2016devops} are: IT performance as a function of lead time for changes, frequency of deployment, and mean time to recover; time spent on unplanned work and rework; typology of organizational culture (pathological, bureaucratic or generative) based on climate for learning, job satisfaction, developers and operators having win-win relationships, usage of version control, and usage of automated testing; and
Some examples of metrics based on survey data used on the State of DevOps Reports~\cite{puppet2014devops, puppet2016devops} are: IT performance as a function of time from commit to deployment, frequency of deployment, and recovery time; time spent on unplanned work and rework; typology of organizational culture (pathological, bureaucratic or generative) based on climate for learning, job satisfaction, developers and operators having win-win relationships, usage of version control, and usage of automated testing; and
employee engagement, based on the Net Promoter Score, indicating how likely employees are to
recommend the company products and services.
......
......@@ -9,4 +9,4 @@ the available sources. We have not considered academic workshops, and we mention
The choice of core and relevant papers may suffer from subjective bias. To reduce this bias,
at least two authors conducted the selection process and the classification of papers.
Senior researchers also oversaw the whole process.
Our study also suffers the publication bias, since positive results are more likely to be published than negative results. We did not find, for example, a paper reporting a case of failed \devops adoption. We did not address this issue in this study.
Our study also suffers the publication bias, which is the propensity of researchers and practitioners to report (and for referees to accept) more about positive results than negative results. We did not find, for example, a paper reporting a case of failed DevOps adoption. We did not address this issue in this study.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment