finalizando

parent bd6e8b26
\section{Introduction}
\label{sec:intro}
Even though the DevOps movement has been discussed for nearly a decade, it lacks a widely accepted definition. By consolidating the most cited definitions of DevOps, we crafted our definition, similar to the one proposed by Dyck \emph{et al.}~\citesel{dyck2015release}, which we adopt throughout this paper:
\begin{quotation}
\emph{DevOps is a collaborative and multidisciplinary effort within an organization
to automate continuous delivery of new software versions,
while guaranteeing their correctness and reliability.}
\end{quotation}
Desiring to improve their delivery process~\cite{puppet2014devops}, enterprises are widely adopting \devops~\citesel{chen2015huge,gray2006conversation,feitelson2013facebook}. \vtwo{Although in discordance with most of the academic definitions, the software industry also uses the word ``DevOps'' to describe a well-paid job title~\citesel{hussain2017zealand}\cite{stackoverflow2017earn}.} Becoming a \devops engineer is an attractive opportunity for software professionals. \devops is also an important phenomenon studied by software engineering researchers and already a mandatory topic in software engineering courses.
\devops and its \textbf{challenges} can be discussed from three perspectives: engineers, managers, and researchers. \textbf{Engineers} benefit from \textbf{(1)} qualifying themselves for a \devops position, a technically hard task guided by over 200 papers, 230 books, and 100 tools~\cite{xebia2016table}. Engineers also need to \textbf{(2)} learn how to re-architect their systems to embrace continuous delivery. \textbf{Managers} want to know \textbf{(3)} how to introduce \devops into an organization and \textbf{(4)} how to assess the quality of already-adopted \devops practices. Managers and engineers, also referred to as \emph{practitioners}, share the necessity of choosing the best automation toolset. Finally, academic \textbf{researchers} \textbf{(5)} conduct studies to determine the state of practice in \devops, thereby contributing to discussions among engineers and managers, and \textbf{(6)} educate a new generation of software engineers on \devops principles and practices.
\begin{versiontwo}
In this context, our \emph{research problem} is devising a conceptual framework to guide engineers, managers, and academics in the exploration of DevOps tools, implications, and challenges.
There are several studies and a few surveys tackling \devops challenges. However, with few exceptions~\citesel{neely2013easy}, they focus on a single perspective to address a given problem.
In this context, this survey contributes to the field by investigating and discussing \devops concepts and challenges from multiple perspectives: engineers, managers, and researchers. Our review also explores a much broader range of sources and is more up-to-date than previous studies. We exploit technical implication and complexity of adopting \devops such as automation, tightly versus loosely coupled architectures, containerization versus configuration management to deployment automation, and toolset management. Previous surveys did not cover these practical aspects and implications of \devops.
\end{versiontwo}
We first survey and analyze an academic bibliography. We explain the selection and analysis procedures of these works in Section~\ref{sec:methodology}. The selected studies are categorized and described in Section~\ref{sec:sources}. Then, we complement our discussions with non-academic sources, such as books and posts from practitioners' blogs; we describe these other sources in Section~\ref{sec:sources}. Also in this section, we compare our work to other reviews on \devops.
\vtwo{Based on the reviewed literature, Section~\ref{sec:concepts} presents the construction of a conceptual framework~\cite{miles1994qualitative} on DevOps composed of five conceptual maps.} We relate these concepts to the engineer and manager perspectives. We also look at the practical side of \devops by analyzing the \devops tools under the perspective of the concepts in Section~\ref{sec:tools}. We categorize these tools, correlate them to concepts, and discuss which roles in the organization should use which tools.
The discussion of \devops concepts and tools lays the basis to raise essential implications for engineers, managers, and researchers, which we present in Section~\ref{sec:implications}. Although most of the implications are straightforward, some challenges are not entirely illuminated by the current literature. We summarize and discuss some of the main \devops challenges in Section~\ref{sec:discussion}, debating limitations and even contradictions found in the literature.
We close with the limitations of this survey in Section~\ref{sec:limitations} and with concluding remarks in Section~\ref{sec:conclusions}. In the next section, we briefly introduce \devops history, motivations, and goals.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{DevOps}
\label{sec:devops}
\devops is an evolution of the agile movement~\citesel{christensen2016teaching}.
Agile Software Development advocates small release iterations with customer reviews. It assumes the team can release software frequently in some production-like environment. However, pioneer Agile literature puts little emphasis on deployment-specific practices.
No Extreme Programming (XP) practice, for example, is about deployment~\cite{beck2004xp}.
The same sparse attention to deployment is evident in traditional software engineering processes~\cite{kroll2003rup} and books~\cite{pressman2005software,sommerville2011software}. Consequently, the transition to production tends to be a stressful process in organizations, containing manual, error-prone activities, and, even, last-minute corrections~\cite{humble2010continuous}.
\devops proposes a complementary set of agile practices to enable the iterative delivery of software in short cycles effectively.
From an organizational perspective, the DevOps movement promotes closer collaboration between developers and operators. The existence of distinct silos for operations and development is still prevalent: operations staff are responsible for managing software modifications in production and for service levels~\citesel{woods2016view}; development teams, on the other hand, are accountable for continuously developing new features to meet business requirements. Each one of these departments has its independent processes, tools, and knowledge bases. The interface between them in the pre-DevOps era was usually a ticket system: development teams demanded the deployment of new software versions, and the operations staff manually managed those tickets.
In such an arrangement, development teams continuously seek to push new versions into production, while operations staff attempt to block these changes to maintain software stability and other non-functional concerns. Theoretically, this structure provides higher stability for software in production. However, in practice, it also results in long delays between code updates and deployment, as well as ineffective problem-solving processes, in which organizational silos blame each other for production problems.
Conflicts between developers and operators, significant deployment times, and the need for frequent and reliable releases led to inefficient execution of agile processes. In this context, developers and operators began collaborating within enterprises to address this gap. This movement was coined ``DevOps'' in 2008~\cite{debois2008devops}.
In the \emph{Continuous Delivery} book~\cite{humble2010continuous}, Humble advocates for an \emph{automated deployment pipeline}, in which any software version committed to the repository must be a production-candidate version. After passing through stages, such as compilation and automated tests, the software is sent to production by the press of a button. This process is called \emph{Continuous Delivery}. A variant is the \emph{continuous deployment}~\cite{humble2010deployment}, which automatically sends to production every version that passes through the pipeline. Many authors closely relate \devops to continuous delivery and deployment~\citesel{shahin2016architecting,wettinger2015environments,yasar2016security,wettinger2015knowledge}. Yasar, for example, calls the deployment pipeline a ``DevOps platform''~\citesel{yasar2016security}.
Besides automating the delivery process, \devops initiatives have also focused on using automated runtime monitoring for improving software runtime properties, such as performance, scalability, availability, and resilience~\citesel{christensen2016teaching,basiri2016chaos,balalaie2016microservices,roche2013quality}. From this perspective, ``Site Reliability Engineering''~\cite{beyer2016site} emerged as a new term related to \devops work at runtime.
\section{Fundamental concepts}
\label{sec:concepts}
\vtwo{In this section, we present a \emph{conceptual framework} of fundamental concepts of \devops.} The concepts emerged from our systematic analysis of the literature, which followed the protocol presented in Section \ref{sec:methodology}.
Our contribution is to provide a relevant and structured set of concepts on \devops, grounded in the literature, to support analysis and understanding of \devops challenges.
\vtwo{Our conceptual framework~\cite{miles1994qualitative} on DevOps is composed of a conceptual map outlining the conceptual categories and of other four conceptual maps that are diagrams structured as graphs in which nodes depict concepts and arrows represent relationships among concepts.} There is always at least one core bibliographic reference supporting a particular relationship. The references are represented by codes that can be identified in the List of Core Papers on page \pageofmainselecetdstudies.
The concepts are distributed into four major categories: process, people, delivery, and runtime. The \emph{process} category encompasses business-related concepts. \emph{People} covers skills and concepts regarding the culture of collaboration. \emph{Delivery} provides the concepts necessary for Continuous Delivery, and, finally, \emph{runtime} synthesizes concepts necessary to guarantee the stability and reliability of services in a continuous delivery environment.
While the process and people categories relate more to the management perspective, runtime and delivery relate more to an engineering perspective. Moreover, while \emph{delivery} concepts relate more to developers, \emph{runtime} concepts relate more to the traditional operator role. The categories are depicted in Figure~\ref{fig:concepts-families} and described in this section.
\begin{figure}[ht]
\includegraphics[scale=0.4]{figs/concepts-categories}
\caption{\devops overall conceptual map}
\label{fig:concepts-families}
\end{figure}
\begin{versiontwo}
Before proceeding to the categories descriptions, we briefly present some reasons to support that these categories are enough to frame the DevOps field: \emph{i)} it seems reasonable to split ``dev'' and ``ops'' concepts as induced by the categories \emph{delivery} and \emph{runtime}; \emph{ii)} it makes sense to separate more technical concepts from less technical concepts, as caused by the separation between the perspectives of engineering and management. \emph{iii)} These categories match our DevOps definition: ``a collaborative and multidisciplinary effort within an organization'' regards \emph{people}; ``continuous delivery'' is a \emph{process} that is ``automated'' by the \emph{delivery} techniques, which also guarantee software ``correctness''; finally, software ``reliability'' is promoted by the \emph{runtime} concepts.
\end{versiontwo}
\subsection{Process}
\devops aims to achieve some business outcomes, such as reducing risk and cost, complying with regulations, and improving product quality and customer satisfaction. We grouped these concepts into the \emph{process} category of concepts, presented in Figure~\ref{fig:concepts-process}, which reveals that \devops achieves such business outcomes through the accomplishment of a process with frequent and reliable releases. In particular, the diagram explicit that continuous delivery leads to product quality and customer satisfaction because of the short feedback cycle it provides.
One could argue that rigorous human and hierarchical approval processes can reduce risk, comply with regulations, and provide product quality. Nevertheless, \devops is different, once its practices are based on agile and lean principles, which embrace change and shorten the feedback cycle, as depicted in the diagram.
\begin{figure}[ht]
\includegraphics[scale=0.25]{figs/concepts-process}
\caption{Process conceptual map}
\label{fig:concepts-process}
\end{figure}
\subsection{People}
The term ``\devops'' centers on the idea of bringing together \emph{people} from development and operations through a culture of collaboration. The concepts around this idea were grouped in the \emph{people} category, which is presented in Figure~\ref{fig:concepts-people}. \vtwo{It depicts that \devops intends to break down the walls among silos, aligning the incentives across the organization.} However, breaking down silos raises many questions concerning concepts depicted in the diagram: how can organizations perform a cultural shift that implies more responsibilities for developers? How do developers acquire operations skills? Can developers and operators work in the same team and still keep different job titles? Should ``\devops'' be a role? Should teams be cross-functional and include operators? Should an operator be exclusive to one team? What does it mean being an operator in the \devops context? Although the concepts of this category are frequent in the literature, the answers to the previous questions are not clear yet, so we debate them in Section~\ref{sec:devosp-adoption-approaches}.
\begin{figure}[ht]
\includegraphics[scale=0.25]{figs/concepts-people}
\caption{People conceptual map}
\label{fig:concepts-people}
\end{figure}
\subsection{Delivery}
The core strategy for achieving a frequent and reliable delivery process is the automation of the \emph{deployment pipeline}, from which \devops tools and techniques emerge. We grouped the concepts surrounding the deployment pipeline into the \emph{delivery} category, which is presented in Figure~\ref{fig:concepts-delivery}. Automation tools, as depicted in the diagram, are usually open source and enable core \devops practices, such as versioning, testing automation, continuous integration, and configuration management. The diagram also shows the interconnection between \devops and microservices: one supports the other, as we explore in Section~\ref{sec:redesign}. Other concepts related to microservices are also depicted, such as backward compatibility and API versioning.
There is still some debate on concrete strategies for tooling, such as the containerized approach leveraged by Docker on the one hand, and continuous configuration convergence, such as in Chef and Puppet, on the other. We discuss such debates in Section~\ref{sec:tools}. However, we claim the \emph{delivery} concepts are much more stable and accepted by the community than the \emph{people} concepts.
\begin{figure}[ht]
\includegraphics[scale=0.25]{figs/concepts-delivery}
\caption{Delivery conceptual map}
\label{fig:concepts-delivery}
\end{figure}
\subsection{Runtime}
It is not enough to continuously deliver new versions, but it is also necessary for each new version to be stable and reliable. Thus, \emph{runtime} concepts are a subsequent and necessary extension of \devops. The \emph{Runtime} category of concepts is presented in Figure~\ref{fig:concepts-runtime}, which shows the desired outcomes, such as performance, availability, scalability, resilience, and reliability. The figure also shows paths to achieve the mentioned outcomes, like the use of infrastructure-as-code, virtualization, containerization, cloud services, and monitoring. As depicted by the diagram, \devops can monitor high-level business metrics or low-level resource metrics. Another topic, also shown in the figure, is to run experiments in the production environment, like injecting failures to ensure software reliability, as advocated by the chaos engineering approach~\citesel{basiri2016chaos}. All of this reduces human intervention to ensure software reliability, which is another factor that challenges the traditional role of operators, as we argue in Section~\ref{sec:devosp-adoption-approaches}.
\begin{figure}[ht]
\includegraphics[scale=0.25]{figs/concepts-runtime}
\caption{Runtime category of concepts}
\label{fig:concepts-runtime}
\end{figure}
\subsection*{}
We complement our conceptual analysis with the practical side of \devops by investigating \devops' tools and actors, and their relations to concepts. As the presentation of \devops' concepts has already raised some concerns that demand further discussion, some issues will also emerge from the discussion on DevOps tools in the next section.
\section{Limitations of this study}
\label{sec:limitations}
Given the vast amount of work on \devops, it was not feasible to conduct an exhaustive search in all available sources. We have not considered academic workshops, and we mentioned only a few books in the field. It would not be practical to try to cover every existing work on \devops and to condense it all in this single survey. However, by focusing on journal and conference papers, and by applying the snowballing process, we aimed to cover the primary literature in the field. We decided not to expand the query string to avoid artificially favoring some of the \devops concepts over others. However, the appearance of the keyword ``\devops'' was not mandatory in the snowballing process so that we could select vital work tackling highly related subjects such as continuous delivery.
The choice of core and relevant papers may suffer from subjective bias. To reduce this bias,
at least two authors conducted the selection process and the classification of papers.
Senior researchers also oversaw the whole process.
Our study also suffers the publication bias, which is the propensity of researchers and practitioners to report (and for referees to accept) more about positive results than negative results. We did not find, for example, a paper reporting a case of failed DevOps adoption. We did not address this issue in this study.
\section{Conclusions}
\label{sec:conclusions}
In this survey, we have discussed \devops concepts and challenges presented in the literature. By associating these concepts with tools, we contributed to supporting practitioners in choosing a proper toolset. This paper also aides IT professionals by presenting in a systematic way the most relevant concepts, tools, and implications, associated with the professional perspectives of researchers, managers, and engineers -- including developers and operators. We hope our reader can now better understand the impact of \devops on daily activities based on each profile.
Two pillars of \devops are 1) human collaboration across departments and 2) automation. We found that technical issues regarding delivery automation are more consolidated, with only a few minor controversies in the literature. On the other hand, there is no consensus on how to effectively empower collaboration among departments, and there are only a few tools for tackling this issue.
A technical topic highly related to \devops revealed by our sources was microservice architecture. Some technical conflicting points are: there is no consensus on whether microservices should be versioned; some practitioners see configuration management tools and containerization as complementary approaches, but others understand them as competitors; maximizing reusability of software components and services may not be the better strategy; rolling back is usually seen as a \devops practice, but some practitioners tend to favor feature toggles as a way to avoid rolling back.
Structuring an organization to introduce \devops is now a major concern. There are three alternative structures: 1) preserving collaborating departments, 2) building cross-functional teams, and 3) having \devops teams. Each one of these strategies has its challenges and, by following the current literature, it is not possible to define the best strategy for a given scenario. \vtwo{Therefore, exploring these and other structures in real organizations and how these organizations handle the ``DevOps role'' are promising topics for future research. However, whatever the structure an organization adopts}, it is clear that the \devops movement has irreversibly blurred the frontier between developers and operators, even for organizations that have not yet fully embraced \devops.
\documentclass[format=acmsmall, review=false, screen=true]{acmart}
% From sample-acmsmall.tex
\pdfoutput=1
\usepackage{booktabs} % For formal tables
\usepackage[ruled]{algorithm2e} % For algorithms
\renewcommand{\algorithmcfname}{ALGORITHM}
\SetAlFnt{\small}
\SetAlCapFnt{\small}
\SetAlCapNameFnt{\small}
\SetAlCapHSkip{0pt}
\IncMargin{-\parindent}
% Pacotes que nós adicionamos:
\usepackage{xspace}
\usepackage{multibib}
\newcites{sel}{List of Core Papers}
% footnotes in two columns
\usepackage{dblfnote}
\DFNalwaysdouble % for this example
% ---------------------------------------------------------------------------- %
% Macros for proof-reading
\usepackage{color}
\usepackage{xcolor}
\usepackage[normalem]{ulem} % for \sout
\newcommand{\ugh}[1]{\textcolor{red}{\uwave{#1}}} % please rephrase
\newcommand{\ins}[1]{\textcolor{blue}{\uline{#1}}} % please insert
\newcommand{\del}[1]{\textcolor{red}{\sout{#1}}} % please delete
\newcommand{\chg}[2]{\textcolor{red}{\sout{#1}}{$\rightarrow$}\textcolor{blue}{\uline{#2}}} % please change
\newenvironment{versiontwo}{}{}
\newcommand{\vtwo}[1]{#1}
% Put edit comments in a really ugly standout display
\usepackage{ifthen}
\usepackage{amssymb}
\usepackage{adjustbox}
\newboolean{showcomments}
\setboolean{showcomments}{true} % toggle to show or hide comments
\ifthenelse{\boolean{showcomments}}
{\newcommand{\nb}[2]{
\fcolorbox{gray}{yellow}{\bfseries\sffamily\scriptsize#1}
{\sf\small$\blacktriangleright$\textit{#2}$\blacktriangleleft$}
}
\newcommand{\version}{\emph{\scriptsize$-$working$-$}}
}
{\newcommand{\nb}[2]{}
\newcommand{\version}{}
}
% General comment
\newcommand\info[1]{\nb{Info}{#1}}
\newcommand\todo[1]{\nb{ToDo}{#1}}
% Single author comment
\newcommand\Fabio[1]{\nb{Fabio}{#1}}
\newcommand\fabio[1]{\nb{Fabio}{#1}}
\newcommand\leo[1]{\nb{Leo}{#1}}
\newcommand\Leo[1]{\nb{Leo}{#1}}
\newcommand\paulo[1]{\nb{Paulo}{#1}}
\newcommand\Paulo[1]{\nb{Paulo}{#1}}
\newcommand\dejan[1]{\nb{Dejan}{#1}}
\newcommand\Dejan[1]{\nb{Dejan}{#1}}
\newcommand\carla[1]{\nb{Carla}{#1}}
\newcommand\Carla[1]{\nb{Carla}{#1}}
\let\oldparagraph\paragraph
\renewcommand{\paragraph}[1]{\textbf{\oldparagraph{#1}}}
% End of macros for proof-reading
% ---------------------------------------------------------------------------- %
% Text macros
\newcommand\devops{DevOps\xspace}
\newcommand\numberofselectedpapers{167\xspace}
\newcommand\numberofmainselectedpapers{50\xspace}
\newcommand\pageofmainselecetdstudies{30\xspace}
\newcommand{\myparagraph}[1]{\vspace{.5em}\noindent\textit{\textbf{#1:}}\hspace{.3em}}
\newcommand{\swurl}[1]{\footnote{\url{#1}}}
% Metadata Information
% TODO
% \acmJournal{TWEB}
% \acmVolume{9}
% \acmNumber{4}
% \acmArticle{39}
\acmJournal{CSUR}
\acmYear{2019}
% \acmMonth{3}
\copyrightyear{2019}
%\acmArticleSeq{9}
% Copyright
%\setcopyright{acmcopyright}
\setcopyright{acmlicensed}
%\setcopyright{rightsretained}
%\setcopyright{usgov}
%\setcopyright{usgovmixed}
%\setcopyright{cagov}
%\setcopyright{cagovmixed}
% DOI
% \acmDOI{0000001.0000001}
% Paper history
% TODO
% \received{February 2007}
% \received[revised]{March 2009}
% \received[accepted]{June 2009}
% Document starts
\begin{document}
% Title portion. Note the short title for running heads
\title{A Survey of DevOps Concepts and Challenges}
\author{Leonardo Leite}
%\orcid{1234-5678-9012-3456}
\affiliation{%
\institution{University of S\~ao Paulo}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{S\~ao Paulo}
% \state{SP}
% \postcode{05508-090}
\country{Brazil}}
\email{leofl@ime.usp.br}
\author{Carla Rocha}
%\orcid{1234-5678-9012-3456}
\affiliation{%
\institution{University of Bras\'ilia}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{Brasilia}
% \state{SP}
% \postcode{05508-090}
\country{Brazil}}
\email{caguiar@unb.br}
\author{Paulo Meirelles}
%\orcid{1234-5678-9012-3456}
\affiliation{%
\institution{Federal University of S\~ao Paulo}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{S\~ao Paulo}
% \state{SP}
% \postcode{05508-090}
\country{Brazil}}
\email{paulo.meirelles@unifesp.br}
\author{Fabio Kon}
%\orcid{1234-5678-9012-3456}
\affiliation{%
\institution{University of S\~ao Paulo}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{S\~ao Paulo}
% \state{SP}
% \postcode{05508-090}
\country{Brazil}}
\email{kon@ime.usp.br}
\author{Dejan Milojicic}
%\orcid{1234-5678-9012-3456}
\affiliation{%
\institution{Hewlett Packard Labs, Palo Alto}
% \department{Department of Computer Science}
% \streetaddress{Rua do Matao 1010}
\city{Palo Alto}
% \state{SP}
% \postcode{05508-090}
\country{USA}}
\email{dejan.milojicic@hpe.com}
\begin{abstract}
\devops is a collaborative and multidisciplinary organizational effort to automate continuous delivery of new software updates while guaranteeing their correctness and reliability.
The present survey investigates and discusses DevOps challenges from the perspective of engineers, managers, and researchers. We review the literature and develop a DevOps conceptual map, correlating the DevOps automation tools with these concepts. We then discuss their practical implications for engineers, managers, and researchers. Finally, we critically explore some of the most relevant DevOps challenges reported by the literature.
\end{abstract}
%
% The code below should be generated by the tool at
% http://dl.acm.org/ccs.cfm
% Please copy and paste the code instead of the example below.
%
% Tree https://dl.acm.org/ccs/ccs.cfm
% Instructions: https://www.acm.org/binaries/content/assets/publications/article-templates/ccs-howto-v6-12jan2015.docx
% Relevance scores: 500 - high; 300 - medium; 100 - low
% Use any first- and second-level nodes of the classification scheme which are relevant to your paper, then look at the lower levels under them.
% Identify the lowest branches of the tree that seem to apply to your particular paper. Use the lowest node in the tree available.
% It is not uncommon to include 6 or more terms for an article.
\begin{CCSXML}
<ccs2012>
<concept>
<concept_id>10011007.10011074.10011081</concept_id>
<concept_desc>Software and its engineering~Software development process management</concept_desc>
<concept_significance>500</concept_significance>
</concept>
<concept>
<concept_id>10011007.10011074.10011134.10011135</concept_id>
<concept_desc>Software and its engineering~Programming teams</concept_desc>
<concept_significance>500</concept_significance>
</concept>
<concept>
<concept_id>10011007.10011074.10011111</concept_id>
<concept_desc>Software and its engineering~Software post-development issues</concept_desc>
<concept_significance>300</concept_significance>
</concept>
</ccs2012>
\end{CCSXML}
\ccsdesc[500]{Software and its engineering~Software development process management}
\ccsdesc[500]{Software and its engineering~Programming teams}
\ccsdesc[300]{Software and its engineering~Software post-development issues}
%
% End generated code
%
\keywords{DevOps, continuous (delivery, deployment, integration), release process, versioning, configuration management, and build process}
\maketitle
% The default list of authors is too long for headers.
\renewcommand{\shortauthors}{L. Leite et al.}
% English help: https://corpus.byu.edu/coca/
%\info{Paper length: should not exceed 35 pages}
\input{1-introduction}
\input{2-methodology}
\input{3-sources-of-knowledge}
\input{4-concepts}
\input{5-toolset}
\input{6-implications}
\input{7-discussion}
\input{8-limitations}
\input{9-conclusions}
% Bibliography
\bibliographystylesel{ACM-Reference-Format}
\bibliographysel{selected-studies}
\bibliographystyle{ACM-Reference-Format}
\bibliography{references}
\end{document}
{
\small
\begin{table}[ht]
\centering
\caption{Classification of core papers}
\label{tab:papers-classification}
\begin{tabular}{l c r} \hline
\itshape{Category} & \itshape{Quantity} & \itshape{Papers} \\
\hline
% KEEP ORDERED
Practices & 15 & \citesel{wettinger2015knowledge, wettinger2015environments, woods2016view, basiri2016chaos, magoutis2015social, jaatun2018incident, rahman2018defective, li2018api, zhu2015frequency, brown2016microservices, cukier2013patterns, ebert2016devops, kang2016docker, kersten2018explosion, forsgren2018metrics} \\
Experience report & 8 & \citesel{balalaie2016microservices, callanan2016right, chen2015huge, feitelson2013facebook, gray2006conversation, neely2013easy, siqueira2018gov, cerqueira2015researchops} \\
Impact & 7 & \citesel{yasar2016security, shahin2016architecting, roche2013quality, jaatun2017security, rajkumar2016culture, sill2014standards, rossem2018virtualized} \\
Concepts & 7 & \citesel{humble2011devops, humble2017willwork, punjabi2016stories, bass2018architect, dyck2015release, debois2011revolution, pang2016maintenance} \\
Challenges & 6 & \citesel{laukkarinen2017medical, lwakatare2016towards, diel2016distributed, olsson2012stairway,leppanen2015highways, claps2015journey} \\
Education & 4 & \citesel{chung2016ksa, hussain2017zealand, bai2018feedback, christensen2016teaching} \\
Adoption & 3 & \citesel{nybom2016mixing, snyder2018analytics, feijter2018maturity} \\
\hline
\end{tabular}
\end{table}
% end small
}
{
\small
\begin{table}[ht]
\centering
\caption{\devops adoption approaches}
\label{tab:adoption-approaches}
\begin{tabular}{l} \hline
\textbf{Collaborating departments} \\
Development and operations departments collaborate closely.\\
It implies overlapping of developers and operators responsibilities.\\
\emph{Downside:} new responsibilities can be unclear for employees.\\
\hline
\textbf{Cross-functional team} \\
The product team is responsible for deploying and operating (\emph{You built it, you run it}).\\
Recommended by Amazon and Facebook.\\
\emph{Downside:} requires more skilled engineers.\\
\hline
\textbf{\devops team} \\
Acts as a bridge between developers and operators. \\
It is better accepted when it is a temporary strategy for cultural transformation.\\
\emph{Downside:} risk of creating a third silo.\\
\hline
\end{tabular}
\end{table}
% end small
}
{
\small
\begin{table}[ht]
\centering
\caption{Sources of \devops surveys}
\label{tab:reseach-sources}
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{lcccccccc} \hline
\itshape{Author} & \itshape{Google Scholar} & \itshape{Springer Link} & \itshape{ACMDigital Library} & \itshape{IEEE Explore} & \itshape{Scopus} & \itshape{Web of Science} & \itshape{Others} & \itshape{Papers} \\ \hline
Fran\c{c}a \cite{travassos2016SLR} & X & & & & & & X & 43 \\
\hline
Erich \cite{Erich2017SLR} & & & X & X & X & X & & 27 \\
\hline
Smeds \cite{Paasivaara2015LSR} & X & X & X & X & & X & X & 27 \\
\hline
Ghantous \cite{gill2017SLR} & X & X & X & X & & & X & 30 \\
\hline
Stahl \cite{Bosch2017SLR} & & & & & X & & & 35 \\
\hline
Jabbari \cite{Jabbari2016SLR} & & & X & X & X & & & 49 \\
\hline
Lwakatare \cite{Lwakatare2015SLR} & X & & X & X & X & X & X & 22 \\
\hline
\textbf{This survey} & X & X & X & X & X & X & X & \textbf{\numberofselectedpapers (\numberofmainselectedpapers)} \\ \hline
\end{tabular}
\end{adjustbox}
\end{table}
% end small
}
{
\small
\begin{table}[ht]
\centering
\caption{Focuses and methodologies of \devops surveys}
\label{tab:research-methodology}
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{llllll}
\hline
Author & \begin{tabular}[c]{@{}l@{}}Multivocal\\ Literature Review\end{tabular} & \begin{tabular}[c]{@{}l@{}}Systematic \\ Literature Review\end{tabular} & \begin{tabular}[c]{@{}l@{}}Systematic\\ Mapping Review\end{tabular} & Interview & Focus \\\hline
Fran\c{c}a \cite{travassos2016SLR} & X & & & & \begin{tabular}[c]{@{}l@{}}Organization\\ devops adoption\end{tabular} \\ \hline
Erich \cite{Erich2017SLR} & & X & & \begin{tabular}[c]{@{}l@{}}Focused interview - \\ 6 organizations\end{tabular} & \begin{tabular}[c]{@{}l@{}}Organization\\ devops adoption\end{tabular} \\ \hline
Ghantous \cite{gill2017SLR} & & X & & & \begin{tabular}[c]{@{}l@{}}Organization\\ devops adoption\end{tabular} \\ \hline
Smeds \cite{Paasivaara2015LSR} & & X & & \begin{tabular}[c]{@{}l@{}}Semi-structure-\\ 1 organization\end{tabular} & \begin{tabular}[c]{@{}l@{}}Organization \\ devops adoption\end{tabular} \\ \hline
Stahl \cite{Bosch2017SLR} & & & X & & \\
\hline
Jabbari \cite{Jabbari2016SLR} & & & X & & \\
\hline
Lwakatare \cite{Lwakatare2015SLR} & & X & & 3 organizations & \begin{tabular}[c]{@{}l@{}}Practioners\\ \hline/Organization\end{tabular}
\end{tabular}
\end{adjustbox}
\end{table}
% end small
}
{
\small
\begin{table}[ht]
\centering
\caption{Research challenges on \devops}
\label{tab:research-challenges}
\begin{tabular}{l} \hline
Architecturing in the \devops context\\
\devops training\\
\devops and embedded systems\\
\devops and compliance\\
\devops and security\\
Testing large-scale distributed systems\\
Ever-changing \devops tools\\
Metrics usage in research on \devops\\
Improving interdepartmental communication\\
\devops adoption strategies\\
\hline
\end{tabular}
\end{table}
% end small
}
{
\small
\begin{table}[ht]
\centering
\caption{Primary research questions from previous \devops SLR works}
\label{tab:research-questions}
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{l r }
\hline
\emph{Research Question} & \emph{References} \\
\hline
RQ1: What is the meaning of the term \devops? &
\begin{tabular}{@{}c@{}}
\cite{travassos2016SLR, Erich2017SLR, gill2017SLR, Jabbari2016SLR, Bosch2017SLR}
\end{tabular} \\ \hline
RQ2: What are the issues motivating the adoption of \devops? &
\begin{tabular}{@{}c@{}}\cite{travassos2016SLR, Erich2017SLR, gill2017SLR} \\ \end{tabular} \\