Methodology

parent 10b379d2
This diff is collapsed.
......@@ -7,7 +7,7 @@ In this section we present the most relevant sources of knowledge on \devops, wh
This survey is based mainly on the evaluation of peer-reviewed papers. This system is applied mainly on academic community and it is intended to guarantee quality and credibility of published works.
Figures~\ref{fig:papers-per-year}~and~\ref{fig:source_types-per-year} depicts the evolution of published works on \devops according to the criteria defined in our methodology\footnote{We collected the 2018 papers in September 2018.}. Figure~\ref{fig:source_types-per-year} details the type of sources. Like in other research areas, roughly speaking, conferences publish preliminary results, while journals publish more mature studies. Magazines are similar to journals but having also practitioners as intended audience, since regular journals are more oriented to academic audience.
Figures~\ref{fig:papers-per-year}~and~\ref{fig:source_types-per-year} depicts the evolution of published works on \devops according to the criteria defined in our study design\footnote{We collected the 2018 papers in September 2018.}. Figure~\ref{fig:source_types-per-year} details the type of sources. Like in other research areas, roughly speaking, conferences publish preliminary results, while journals publish more mature studies. Magazines are similar to journals but having also practitioners as intended audience, since regular journals are more oriented to academic audience.
\begin{figure}[ht]
\centering
......@@ -23,17 +23,15 @@ Figures~\ref{fig:papers-per-year}~and~\ref{fig:source_types-per-year} depicts th
\end{minipage}
\end{figure}
A common source of papers on \devops is the ``IEEE Software'' magazine. Among the 50 main selected studies, 13 were published in this magazine, being mostly short papers describing experiences in real life faced by organizations. Therefore,
it is advisable for practitioners to follow this source to get future updates on \devops since it has become a reference on the subject.
A common source of papers on \devops is the ``IEEE Software'' magazine. Among the 50 main selected studies, 13 were published in this magazine, being mostly short papers describing experiences in real life faced by organizations. Therefore, it is advisable for practitioners to follow this source to get future updates on \devops since it has become a reference on the subject.
We did not find many peer-reviewed publications or events dedicated to the theme of \devops. The closest related events to the subject we found are the ``International Workshop on Release Engineering'' and the ``International Workshop on Quality-Aware DevOps''. For researchers starting their work on \devops, these are two potential targets for publishing their first results. For the academic community
interested in the subject, this shows that it could be valuable to foster the organization of dedicated \devops events or journal special issues.
We did not find many peer-reviewed publications or events dedicated to the theme of \devops. The closest related events to the subject we found are the ``International Workshop on Release Engineering'' and the ``International Workshop on Quality-Aware DevOps''. For researchers starting their work on \devops, these are two potential targets for publishing their first results. For the academic community interested in the subject, this shows that it could be valuable to foster the organization of dedicated \devops events or journal special issues.
We found \numberofselectedpapers papers from peer-reviewed sources, of which we carefully selected \numberofmainselectedpapers papers as our main studies to produce the conceptual maps of Section~\ref{sec:concepts}. Most of the implications of Section~\ref{sec:implications} and discussions in Section~\ref{sec:discussion} were also based on these selected studies. We follow by describing how we classified our main selected studies and Table~\ref{tab:papers-classification} quantifies this classification.
We now follow by describing how we classified our main selected studies, which were retrieved from peer-reviewed sources, and Table~\ref{tab:papers-classification} quantifies this classification.
\input{tabs/papers-classification.tex}
\myparagraph{Practices} provides tools, practices, patterns and approaches that can enhance the \devops experience. Support for selecting deployment technology~\citesel{wettinger2015knowledge}, explanation about the usage of metrics~\citesel{forsgren2018metrics}, communication of global peers through specialized social networks~\citesel{magoutis2015social}, and causing infrastructure faults at production to ensure system reliability~\citesel{basiri2016chaos} are examples.
\myparagraph{Practices} provides tools, practices, patterns and approaches that can enhance the \devops experience. Some examples are support for selecting deployment technology~\citesel{wettinger2015knowledge}, explanation about the usage of metrics~\citesel{forsgren2018metrics}, communication of global peers through specialized social networks~\citesel{magoutis2015social}, and causing infrastructure faults at production to ensure system reliability~\citesel{basiri2016chaos}.
\myparagraph{Experience Report} refers to concrete experience accounts of organizations adopting \devops or continuous delivery/deployment. For example, the report of Siqueira~\emph{et al.} that describes the role of a \devops team in the context of a governmental project~\citesel{siqueira2018gov}.
......@@ -45,9 +43,7 @@ We found \numberofselectedpapers papers from peer-reviewed sources, of which we
\myparagraph{Education} investigates the challenges in teaching \devops. They propose teaching methods for it~\citesel{christensen2016teaching}, as well explore the knowledge, skills, and abilities (KSA) necessary for \devops professionals~\citesel{chung2016ksa,hussain2017zealand}.
\myparagraph{Adoption} deals with the different models of \devops adoption~\citesel{nybom2016mixing}, which is further discussed on Section~\ref{sec:devosp-adoption-approaches}, and describe how a specific organization used metrics to evaluate its \devops adoption process~\citesel{snyder2018analytics}. \\
In the ``Main Selected Studies'' listing (page \pageofmainselecetdstudies), each article is listed with a code, where the letter of the code indicates how we found the article: ``A'' for the ACM Digital Library\swurl{dl.acm.org}, ``I'' for the IEEE Xplore Digital Library\swurl{https://ieeexplore.ieee.org}, ``S'' for Springer Link\swurl{link.springer.com}, and ``B'' for the snowballing process.
\myparagraph{Adoption} deals with the different models of \devops adoption~\citesel{nybom2016mixing}, which is further discussed on Section~\ref{sec:devosp-adoption-approaches}, and describes how a specific organization used metrics to evaluate its \devops adoption process~\citesel{snyder2018analytics}. \\
In the next section, we detail other surveys on \devops and how this work differentiates from them.
......
\section{Fundamental \devops Concepts}
\label{sec:concepts}
In this section, we present the \emph{fundamental concepts on \devops} emerged from our literature review. These fundamental concepts are organized and presented as conceptual maps, which are diagrams depicting suggested relationships between concepts~\cite{hager1997designing}.
The \emph{objective} of our conceptual maps is \emph{to provide a relevant and structured set of concepts on \devops, grounded on the literature, to support the analysis of \devops challenges}.
In this section, we present the \emph{fundamental concepts on \devops} emerged from our literature review.
The \emph{objective} of our conceptual maps is \emph{to provide a relevant and structured set of concepts on \devops, grounded in the literature, to support the analysis of \devops challenges}.
\subsection{Producing the conceptual maps}
The process of building the conceptual maps was based on Grounded Theory strategies. Grounded Theory is a major method for conducting emergent qualitative research~\cite{charmaz2008grounded} and it is also considered an appropriate research method to explore the human and social aspects of Software Engineering~\cite{hoda2011grounded}. According to Charmaz, it minimizes preconceived ideas about the research problem and it helps researchers to remain open to different explanations and understandings of the data~\cite{charmaz2008grounded}. It usually is applied to primary sources, such as interviews and field-notes of observations~\cite{hoda2011grounded, sbaraini2011grounded}, not on literature articles. In the present work, Grounded Theory strategies were only used to support the definition of a rigorous analysis process to handle the intrinsic subjectivity of our conceptual analysis.
%Nonetheless, we must make clear we do not claim to be using Grounded Theory itself in this work since our objective is not to produce a new theory based on our sources. Moreover,
A core strategy of Grounded Theory is \emph{coding}~\cite{charmaz2008grounded}, which is a process of breaking data down into much smaller components and labeling those components~\cite{sbaraini2011grounded}. The codes, arising out of different sources, are compared continuously among each other to produce units of a higher level of abstraction, called \emph{concepts}, and the concepts themselves are compared and grouped to produce a third level of abstraction called \emph{categories}~\cite{hoda2011grounded}. In our study, we carried the full read of \numberofmainselectedpapers selected studies to produce the codes. After this, we analyzed, abstracted and grouped the codes into concepts. Finally, by analyzing the concepts and their relations, we derived four categories of concepts, each one having its own conceptual map.
The process of building our conceptual maps followed the following rules and restrictions:
\begin{itemize}
\item Every concept and link must be supported by at least one selected study.
\item The label on the link corresponds to the \emph{``code''} of the reference. The codes are defined at the Main Selected Studies listing.
\item A link can have different labels, each one with its own references.
\item The labels in a single link can described by both antagonistic and similarity relations.
\item Some concepts, with common links, are grouped to save space in the figure.
\item The references on link labels are selected studies of our reviewed literature, even when the concepts and relations were originated from work cited by our selected studies.
\end{itemize}
The concepts were separated into four major categories: Process, People, Delivery, and Runtime. These categories are depicted on Figure~\ref{fig:concepts-families}.
The concepts were separated into four major categories: Process, People, Delivery, and Runtime. These categories are depicted on Figure~\ref{fig:concepts-families} and the following subsections present one conceptual map for each of the listed categories.
\begin{figure}[ht]
\includegraphics[scale=0.4]{figs/concepts-categories}
......@@ -32,8 +12,6 @@ The concepts were separated into four major categories: Process, People, Deliver
\label{fig:concepts-families}
\end{figure}
The following subsections present one conceptual map for each of the listed categories.
\subsection{Process}
\devops deploys a continuous delivery \emph{process} to achieve business benefits. Reducing risk and cost, complying with regulations, improving product quality and customer satisfaction are samples of these benefits. We grouped those concepts into the \emph{process} category of concepts, and they are all related to the business outcome from \devops. Some concepts are not exclusively achieved by \devops approach. One could argue that rigorous human and hierarchical approval process is an option to reduce risk, comply with regulations, and provide product quality. The difference here is that the \devops approach is based on the agile mindset, which advocates the shortening of feedback cycle through frequent releases. The \emph{Process} category of concepts is presented in Figure \ref{fig:concepts-process}.
......
No preview for this file type
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment