Versão word 2018-10-26

parent a81b95a9
%\devops aims to enable the adoption of continuous delivery process and to facilitate management within software organizations. Key elements of \devops are automation and human collaboration across the organization. These elements combined lead to a continuous stream of software increment deployments, done by the press of a single button~\cite{humble2010continuous}, where each update complies with both functional and non-functional requirements.
Even though the \devops movement has been discussed for nearly a decade now, there is no consensual definition for its term. We consolidated the most cited definitions of \devops and crafted the following definition, very close to the one proposed by Dyck \emph{et al.}~\citesel{dyck2015release}, that we adopt throughout this paper:
\emph{\devops is a collaborative and multidisciplinary effort within the organization
aiming at automating continuous delivery of new software versions,
while guaranteeing their correctness and reliability.}
Yearning for improvements in their delivery process~\cite{puppet2014devops}, enterprises are widely adopting \devops~\citesel{chen2015huge,gray2006conversation,feitelson2013facebook}. The word ``\devops'' has also become a role in the software industry~\citesel{hussain2017zealand}, and a well paid-one~\cite{stackoverflow2017earn}. Becoming a \devops engineer is an attractive opportunity for software professionals. \devops is also an important phenomenon to be studied by software engineering researchers and already a mandatory topic to be taught in software engineering courses.
\devops and its \textbf{challenges} can be discussed from three perspectives: Engineers, Managers, and Researchers. \textbf{Engineers} profit from \textbf{1)} qualifying themselves for a \devops position, a technically hard task, guided by over 200 papers, 140 books, and 100 tools~\cite{xebia2016table}. Engineers also need to \textbf{2)} learn how to re-architect their systems to embrace continuous delivery. \textbf{Managers} want to know mainly \textbf{3)} how to introduce \devops into an organization, and \textbf{4)} how to assess the quality of \devops practices already established. Managers and engineers, also referred to as practitioners, share the necessity of choosing the best automation toolset. Finally, academic \textbf{researchers} \textbf{5)} conduct studies to determine the state of practice in \devops, contributing to open discussions among engineers and managers, and \textbf{6)} educate a new generation of software engineers for \devops principles and practices.
%Nonetheless, the answer to each \devops challenge can involve all three perspectives. Although the adoption of \devops in enterprise concerns mainly managers, for example, it also requires the high involvement of engineers, and it is also a frequent research topic. Teaching \devops, for instance, must take into account the engineer's learning perspective, but it is also a managerial concern as the enterprise should adopt an educational strategy concerning \devops adoption.
There are several studies and a few surveys tackling \devops challenges. However, with few exceptions~\citesel{neely2013easy}, they usually focus on a single perspective when addressing a given problem.
%The primary example is the challenge of studying strategies for \devops adoption, which are focused mainly on the managerial perspective.
In this context, \emph{this survey contributes to the field by investigating and discussing \devops concepts and challenges from multiple perspectives: engineers, managers, and researchers}. Our review also explores a much broader range of sources, and it is more up to date than previous studies.
We surveyed and analyzed mainly academic bibliography. We explain the selection and analysis procedures of these works in Section~\ref{sec:methodology}. The selected studies are categorized and described in Section~\ref{sec:sources}. Since our discussions are complemented by other non-academic sources, such as books and posts from practitioners blogs, we also describe these other sources in Section~\ref{sec:sources}. Moreover, in this same section, we compare our work to other reviews on \devops.
Section~\ref{sec:concepts} presents the construction of conceptual maps based on our main selected studies. We relate these concepts to the engineer and manager perspectives. We also take a look at the practical side of \devops by analyzing the \devops tools under the concepts perspective in Section~\ref{sec:tools}. We categorize these tools, correlate them to concepts, and discuss which roles in the organization should use which tools.
The discussion on \devops concepts and tools lays the basis to raise important implications for engineers, managers, and researchers. Based on our surveyed literature, we present these implications in Section~\ref{sec:implications}. Although most of the implications are quite straightforward, there are some challenges not completely enlightened by the current literature. We summarize and discuss some of the main \devops challenges in Section~\ref{sec:discussion}, debating limitations and even contradictions found in the literature.
We close with the limitations of this survey in Section~\ref{sec:limitations} and with concluding remarks in Section~\ref{sec:conclusions}. In the next section, we briefly introduce \devops history, motivations, and goals.
\devops is a natural evolution of the agile movement~\citesel{christensen2016teaching}.
Agile Software Development upholds small release iterations with customer reviews. It assumes the team can release software frequently in some production-like environment. However, the pioneer Agile literature puts little emphasis on deployment-specific practices.
No Extreme Programming (XP) practice, for example, is about deployment~\cite{beck2004xp}.
The same little attention to deployment is perceived in traditional software engineering processes~\cite{kroll2003rup} and books~\cite{pressman2005software,sommerville2011software}. Consequently, the transition to production tends to be a stressful process in organizations, containing manual, error-prone activities and, even, last-minute corrections~\cite{humble2010continuous}.
\devops proposes a complementary set of agile practices to allow the iterative delivery of software in short cycles effectively.
From the organization perspective, \devops can be seen as the movement to promote closer collaboration between developers and operators. The existence of distinct departments/silos for operations and development is still prevalent, and this movement changes such structure. Operations staff is responsible for managing software modifications in production and for its service levels~\citesel{woods2016view}. Development teams, on the other hand, are accountable for continuously developing new features to meet business requirements. Each one of these departments has its independent processes, tools, and knowledge bases. The interface between them, in the pre-\devops era, is usually a ticket system, in which development teams demand the deployment of new software versions, and the operations staff manage those tickets manually.
In such an arrangement, development teams continuously seek to push new versions into production, while operations staff attempt to block these changes to maintain software stability and other non-functional concerns. This structure theoretically provides higher stability for software in production. However, in practice, it also results in very long delays between code updates and deployment, and it leads to ineffective problem-solving processes, where different organizational silos keep blaming each other for production problems.
Issues like conflicts between developers and operators, significant deployment times, and the need for frequent and reliable releases led to an inefficient agile execution. In this context, developers and operators started a collaboration movement within enterprises to address this gap. This movement was coined as \devops in 2008~\cite{debois2008devops}.
Many authors closely relate \devops to continuous delivery and deployment~\citesel{shahin2016architecting,wettinger2015environments,yasar2016security,wettinger2015knowledge}. In the \emph{Continuous Delivery} book~\cite{humble2010continuous}, Humble advocates the usage of an \emph{automated deployment pipeline}: any software version committed to the repository must be a production-candidate version; but before it is declared ready for production, it must pass through the pipeline stages, such as compilation and automated tests. Then, software is prepared to go to production by the pressing of a button. This process is called \emph{continuous delivery}. A step further is automatically sending to production every version that passes through the pipeline, which is called \emph{continuous deployment}~\cite{humble2010deployment}. What Yasar, for example, calls a ``DevOps platform''~\citesel{yasar2016security}, is the deployment pipeline.
Besides automating the delivery process, \devops initiatives have also focused on the usage of automated runtime monitoring for improving software runtime properties, such as performance, scalability, availability, and resilience~\citesel{christensen2016teaching,basiri2016chaos,balalaie2016microservices,roche2013quality}. From this view, ``Site Reliability Engineering''~\cite{beyer2016site} emerged as a new term related to the \devops work at runtime.
\section{Study design}
The recent growth of the \devops community reflected in a rapid increase on publications which imposes a barrier to a thorough analysis of every published work on the matter. We adopted a qualitative procedure to limit the number of analyzed studies and the information extracted from them. We established a search and selection protocol, inspired by the procedures of systematic literature reviews~\cite{budgen2006review}. We also set procedures based on Grounded Theory strategies~\cite{charmaz2008grounded}. The primary outcome is a set of conceptual maps, which lists the main concepts of \devops and how they are related to each other (Section~\ref{sec:concepts}). In this section, we detail our search and selection protocol, as well our procedures to build our conceptual maps.
\subsection{Search and selection protocol}
We searched for articles on majors scientific papers repositories, namely: ACM Digital Library\swurl{}, IEEE Xplore Digital Library\swurl{}, and the Spring Link\swurl{}. We also explored Google Scholar\swurl{}, but its outcomes did not provide additional relevant results. The search was conducted according to the following criteria:
\item Title or abstract must contain the word ``\devops''.
\item Papers must have at least three pages.
\item Papers must be published in journals, conferences, congresses, or symposiums (workshop papers were not considered).
\item Papers must be written in English.
\item Literature reviews were not included. We present them as related work in Section~\ref{sec:related-surveys}.
We had initially 198 papers, from which we discarded 45 and selected 153 papers to our analysis. From the 153 selected papers, we elected 36 papers to be thoroughly read. We choose the studies by reading their abstract and, in some cases, reading their content too. We reduced subjectiveness of this selection process by making at least two authors agreeing with each outcome based on the following guidelines:
\item If a paper discusses \devops peculiarities or challenges in some specific context, it tends to be fully read.
\item If a paper is too centered on a single tool and do not discusses \devops itself, it tends to be only selected.
\item If a paper has an extended or a more cited version, only the most relevant version is fully read, whereas the other one is just selected.
\item If a paper is about applying \devops in a specific context, but without elaboration beyond the state-of-the-art, it tends to be discarded.
\item If a paper discusses some topic (e.g., big data) and uses \devops only as background, it tends to be discarded.
\item If a paper has less then four pages, it tends to be discarded.
\item If a paper only presents a simple proposal or opinion, without validation, it tends to be discarded.
We also applied a snowballing process~\cite{claes2014snowballing} to cover important work not found with the query string ``devops''. We explored historical, seminal, high-cited, or very recent references. In this phase, we considered 14 more papers to be fully read, obtaining a total of \numberofmainselectedpapers main selected studies.
We list all main selected studies in Page \pageofmainselecetdstudies at the end of this paper. We codified each one of these paper with a unique reference code, composed by sequence number and a letter indicating how we found the article: ``A'' for the ACM Digital Library, ``I'' for the IEEE Xplore Digital Library, ``S'' for Springer Link, and ``B'' for the snowballing process. This reference code is later used to display the papers reference on the conceptual maps.
\subsection{Producing the conceptual maps}
Conceptual maps are diagrams depicting suggested relationships between concepts~\cite{hager1997designing}. Our process of building the conceptual maps was based on Grounded Theory strategies. Grounded Theory is a significant method for conducting emergent qualitative research~\cite{charmaz2008grounded} and it is also considered an appropriate research method to explore both human and social aspects of Software Engineering~\cite{hoda2011grounded}. According to Charmaz~\cite{charmaz2008grounded}, it minimizes preconceived ideas about the research problem, and it helps researchers to remain open to different explanations and understandings of the data~\cite{charmaz2008grounded}. It usually is applied to primary sources, such as interviews and field-notes of observations~\cite{hoda2011grounded, sbaraini2011grounded}, not on literature articles. Even though, in this paper, we used Grounded Theory strategies to handle the intrinsic subjectivity of conceptual analysis.
A core strategy of Grounded Theory is \emph{coding}~\cite{charmaz2008grounded}, which is a process of breaking data down into much smaller components and labeling those components~\cite{sbaraini2011grounded}. The codes, arising out of different sources, are compared continuously among each other to produce units of a higher level of abstraction, called \emph{concepts}. The concepts themselves are compared and grouped to produce a third level of abstraction called \emph{categories}~\cite{hoda2011grounded}. In our study, we carried the full read of \numberofmainselectedpapers studies to produce the codes. Next, we analyzed, abstracted, and grouped the codes into concepts. Finally, we derived four categories of concepts, each one having its conceptual map.
The process of building our conceptual maps obeyed the following rules and restrictions:
\item Each study selected has a unique reference code.
\item Every concept and link must be supported by at least one selected study.
\item The link's label describes relations among concepts and contains reference codes.
\item A link can have different labels, each one with its references.
%\item The labels on a single link can describe both antagonistic and similarity relations.
\item Some concepts, with common links, are grouped to save space in the figure.
\item A concept can belong to multiple categories.
In the next section, we overview the set of articles analyzed to build our conceptual maps, as well other relevant sources of knowledge on \devops. In Section \ref{sec:concepts}, we present the conceptual maps we produced to structure a set of relevant concepts on DevOps grounded in the literature.
This diff is collapsed.
\section{Fundamental concepts}
In this section, we present the \emph{fundamental concepts on \devops} emerged from our literature review.
Our contribution is \emph{to provide a relevant and structured set of concepts on \devops, grounded in the literature, to support the analysis and understanding of \devops challenges}.
The concepts emerged from the protocol presented in Section \ref{sec:methodology}. They are distributed into four major categories: Process, People, Delivery, and Runtime. \emph{Process} category encompasses business-related concepts. \emph{People} category covers skills and concepts regarding the culture of collaboration. \emph{Delivery} category provides the concepts necessary for continuous delivery. Finally, \emph{runtime} category synthesizes concepts necessary to guarantee the stability and reliability of services in a continuous delivery environment.
While Process and People categories are more related to the management perspective, Runtime and Delivery are more related to engineers. Moreover, while \emph{Delivery} concepts are more related to developers, \emph{Runtime} concepts are more related to the traditional operator role. The categories are depicted in Figure~\ref{fig:concepts-families} and described in the sequence.
\caption{\devops overall conceptual map}
\devops enables and motivates a continuous delivery \emph{process} to achieve business benefits. Reducing risk and cost, complying with regulations, improving product quality and customer satisfaction are examples of these benefits. We grouped these concepts into the \emph{process} category of concepts, and they are all associated with the business outcome from \devops. One could argue that rigorous human and hierarchical approval process is an option to reduce risk, comply with regulations, and provide product quality. The difference here is that the \devops practices are aligned with the agile mindset, which promotes the shortening of the feedback cycle through frequent releases. The \emph{Process} category of concepts is presented in Figure \ref{fig:concepts-process}.
\caption{Process conceptual map}
The term ``\devops'' itself centers around the idea of bringing together \emph{people} from development and operations through a culture of collaboration. The intention is avoiding distinct areas of the organization to have different and conflicting goals, which could reduce efficiency and time-to-market. However, breaking down the silos raises a lot of questions: how to perform a cultural shift in the organization that implies more responsibilities for developers? How do developers acquire operations skills? Can developers and operators work in the same team and still keep different job titles? Should ``\devops'' be a role? Should teams be cross-functional and include operators? Should an operator be exclusive to one single team? What does it mean being an operator in the \devops context? Although the concepts of this category are frequent in the literature, the answers for the previous questions are not clear yet, so we debate them in Section~\ref{sec:devosp-adoption-approaches}. We grouped these concepts in the \emph{People} category of concepts, which is presented in Figure~\ref{fig:concepts-people}.
\caption{People conceptual map}
The core strategy for achieving a frequent and reliable delivery process is the automation of the \emph{deployment pipeline}, from which \devops tools and techniques emerge. We grouped the concepts surrounding the deployment pipeline into the \emph{delivery} category, which is a much more technical category than the previous ones. There is still some debate on concrete strategies for tooling, such as the containerized approach leveraged by Docker from one side, and continuous configuration convergence, such as in Chef and Puppet, on the other side. We discuss such debate in Section~\ref{sec:tools}. However, we claim that the \emph{delivery} concepts are much more stable and accepted by the community than the \emph{people} concepts. The \emph{Delivery} category of concepts is presented in Figure~\ref{fig:concepts-delivery}.
\caption{Delivery conceptual map}
\emph{Runtime} concepts are a subsequent and necessary extension in the \devops since it is not enough to continuously deliver new versions, but it is also necessary for each new version to be stable and reliable. The need for such reliability is the main connection between \devops and cloud environments. The \emph{Runtime} category of concepts is presented in Figure~\ref{fig:concepts-runtime}.
\caption{Runtime category of concepts}
We now complement our conceptual analysis taking a look on the practical side of \devops by investigating the \devops tools, their actors, and their relations to concepts. As the presentation of \devops concepts has already raised some concerns demanding more discussion, some issues will also emerge from the discussion on \devops tools.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
\section{Limitations of this study}
% based on the paper "A systematic literature review of service
% choreography adaptation" we have published on SOCA
Given the vast amount of work on \devops, we could not conduct an exhaustive search in every source. We have not considered academic workshops, and we only mentioned a few books in the field. It would not be practical to try to cover every existing work on \devops and to condense it all in this single survey. However, by focusing on journal and conference papers, and by applying the snowballing process, we hope to have covered the primary literature on the field. We decided not to expand the query string to not artificially favor some of the category concepts over others. However, the appearance of the keyword ``\devops'' was not mandatory in the snowballing process so that we could select vital work tackling highly related subjects as, for example, continuous delivery.
The choice of the primary selected studies, as well paper classification, may suffer from subjective bias. To reduce this bias, at least two authors participated in the selection process and the papers classification. More experienced researchers also supervised the whole process.
There is also the publication bias since positive results are more likely to be published than negative results. We did not find, for example, a paper reporting a case of failed \devops adoption. We did not address this issue in this study.
In this survey, we have discussed \devops concepts and challenges presented by the literature. We associated these concepts to tools, contributing to support practitioners in choosing a toolset correctly. Both concepts, tools, and implications were also associated with the professional perspectives of researchers, managers, and engineers, the last ones including developers and operators. In this way, our reader can better understand the impact of \devops on her activities based on her profile.
Two pillars of \devops are 1) human collaboration across departments and 2) automation. We found that technical issues regarding delivery automation are more consolidated, having only a few minor controversies in literature. On the other hand, there is no consensus on how to effectively empower collaboration among departments, as well there are only a few tools for tackling this issue.
A technical topic highly related to \devops revealed by our sources was microservice architecture. Some technical conflicting points are: there is no consensus if microservices should be versioned; some practitioners see configuration management tools and containerization as complementary approaches, but others understand them as competitors; maximizing reusability of software components and services is possibly not the better strategy; rolling back is usually seen as a \devops practice, but some practitioners tend to favor feature toggles as a way to avoid rolling back.
Structuring an organization to introduce \devops is a major concern in both academics and organizations. There are three possible structures: 1) keeping collaborating departments, 2) building cross-functional teams, and 3) having \devops teams. Each one of these strategies has its challenges, and, by following the current literature, it is not possible to define the best strategy for a given scenario. However, whatever the structure adopted, it is clear that the \devops movement has blurred the frontier between developers and operators irreversibly, even for organizations that have not yet fully embraced \devops.
%Finally, we found not sufficient works on the experience of more than a single organization. Similarly, quantitative metrics for assessing \devops adoption. We did not find any articles reporting negative results. These are some aspects to be improved in future research on \devops.
This diff is collapsed.
# remove tags latex
cat *.tex > merge.txt
sed -r -i 's/\\?devops/DevOps/g' merge.txt
detex merge.txt > limpo.txt
# remove citações
sed -r -i 's/[a-zA-Z]+[0-9]{4}[a-zA-Z]+//g' limpo.txt
\documentclass[format=acmsmall, review=false, screen=true]{acmart}
% From sample-acmsmall.tex
\usepackage{booktabs} % For formal tables
\usepackage[ruled]{algorithm2e} % For algorithms
% Pacotes que nós adicionamos:
\newcites{sel}{Main selected studies}
% ---------------------------------------------------------------------------- %
% Macros for proof-reading
\usepackage[normalem]{ulem} % for \sout
\newcommand{\ugh}[1]{\textcolor{red}{\uwave{#1}}} % please rephrase
\newcommand{\ins}[1]{\textcolor{blue}{\uline{#1}}} % please insert
\newcommand{\del}[1]{\textcolor{red}{\sout{#1}}} % please delete
\newcommand{\chg}[2]{\textcolor{red}{\sout{#1}}{$\rightarrow$}\textcolor{blue}{\uline{#2}}} % please change
% Put edit comments in a really ugly standout display
\setboolean{showcomments}{true} % toggle to show or hide comments
% General comment
% Single author comment
% End of macros for proof-reading
% ---------------------------------------------------------------------------- %
% Text macros
% Metadata Information
% \acmJournal{TWEB}
% \acmVolume{9}
% \acmNumber{4}
% \acmArticle{39}
% \acmMonth{3}
% Copyright
% \acmDOI{0000001.0000001}
% Paper history
% \received{February 2007}
% \received[revised]{March 2009}
% \received[accepted]{June 2009}
% Document starts
% Title portion. Note the short title for running heads
\title{A Survey of DevOps Concepts and Challenges}
\author{Leonardo Leite}
\institution{University of S\~ao Paulo}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{S\~ao Paulo}
% \state{SP}
% \postcode{05508-090}
\author{Carla Rocha}
\institution{University of Bras\'ilia}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
% \state{SP}
% \postcode{05508-090}
\author{Paulo Meirelles}
\institution{Federal University of S\~ao Paulo}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{S\~ao Paulo}
% \state{SP}
% \postcode{05508-090}
\author{Fabio Kon}
\institution{University of S\~ao Paulo}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{S\~ao Paulo}
% \state{SP}
% \postcode{05508-090}
\author{Dejan Milojicic}
\institution{Hewlett Packard Labs}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{Palo Alto}
% \state{SP}
% \postcode{05508-090}
\devops is a collaborative and multidisciplinary effort within an organization, aiming at automating continuous delivery of new software updates while guaranteeing their correctness and reliability.
In this context, the present survey investigates and discusses \devops challenges from the multiple perspectives of engineers, managers, and researchers. We review the literature and develop a \devops conceptual map. We correlate the \devops automation tools with these concepts and discuss their practical implications for engineers, managers, and researchers. Finally, we discuss critically some of the most relevant \devops challenges reported by the literature on the topic.
% The code below should be generated by the tool at
% Please copy and paste the code instead of the example below.
% Tree
% Instructions:
% Relevance scores: 500 - high; 300 - medium; 100 - low
% Use any first- and second-level nodes of the classification scheme which are relevant to your paper, then look at the lower levels under them.
% Identify the lowest branches of the tree that seem to apply to your particular paper. Use the lowest node in the tree available.
% It is not uncommon to include 6 or more terms for an article.
<concept_desc>Software and its engineering~Software development process management</concept_desc>
<concept_desc>Software and its engineering~Programming teams</concept_desc>
<concept_desc>Software and its engineering~Software post-development issues</concept_desc>
\ccsdesc[500]{Software and its engineering~Software development process management}
\ccsdesc[500]{Software and its engineering~Programming teams}
\ccsdesc[300]{Software and its engineering~Software post-development issues}
% End generated code
\keywords{DevOps, other keywords}
% The default list of authors is too long for headers.
\renewcommand{\shortauthors}{L. Leite et al.}
% English help:
%\info{Paper length: should not exceed 35 pages}
% Bibliography
This diff is collapsed.
This diff is collapsed.
\caption{Classification of the main selected studies}
\begin{tabular}{l r r} \hline
\itshape{Category} & \itshape{Quantity} & \itshape{Papers} \\
Practices & 15 & \citesel{wettinger2015knowledge, wettinger2015environments, woods2016view, basiri2016chaos, magoutis2015social, jaatun2018incident, rahman2018defective, li2018api, zhu2015frequency, brown2016microservices, cukier2013patterns, ebert2016devops, kang2016docker, kersten2018explosion, forsgren2018metrics} \\
Experience report & 8 & \citesel{balalaie2016microservices, callanan2016right, chen2015huge, feitelson2013facebook, gray2006conversation, neely2013easy, siqueira2018gov, cerqueira2015researchops} \\
Impact & 7 & \citesel{yasar2016security, shahin2016architecting, roche2013quality, jaatun2017security, rajkumar2016culture, sill2014standards, rossem2018virtualized} \\
Concepts & 7 & \citesel{humble2011devops, humble2017willwork, punjabi2016stories, bass2018architect, dyck2015release, debois2011revolution, pang2016maintenance} \\
Challenges & 6 & \citesel{laukkarinen2017medical, lwakatare2016towards, diel2016distributed, olsson2012stairway,leppanen2015highways, claps2015journey} \\
Education & 4 & \citesel{chung2016ksa, hussain2017zealand, bai2018feedback, christensen2016teaching} \\
Adoption & 3 & \citesel{nybom2016mixing, snyder2018analytics, feijter2018maturity} \\
% end small
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment