Writing methodology.

parent fe09cbe5
\section{Methodology}
\section{Study design}
\label{sec:methodology}
Although tools and concrete techniques are essential, we focus only on the primary tools leveraged by the \devops movement. We do not discuss every tool or academic work about automated deployment~\cite{wettinger2013tosca,leite2014deploying}. Automated deployment has a long line of works~\cite{magee1996dynamic,magee1994regis} and discussing each approach does not sum up to illuminate the path of a \devops expert.
This work surveys mainly the academic literature on \devops. However, the vast amount of works on the field is a barrier to a thorough read of every published work on the matter. Therefore, we had to define procedures to prioritize which papers to read, defining a ``search and selection protocol''. The authors built this protocol based on the procedures of systematic literature reviews~\cite{budgen2006review}. The output of the process defined by the protocol are our \numberofmainselectedpapers main selected studies, which were fully read by the main author.
From the reading of these papers we derived our conceptual maps following procedures based on Grounded Theory strategies~\cite{charmaz2008grounded}. We used these derived concepts to stablish relevant points of discussion to engineers, managers, and researchers.
This section concentrates in defining our search and selection protocol, as well our procedures to build our conceptual maps.
\subsection{Search and selection protocol}
We searched for \devops-related articles on the main repositories of scientific papers, namely: ACM Digital Library, IEEE Xplore Digital Library, and the Spring Link. We also searched on Google Scholar. However, it did not provided additional relevant studies.
The search was done with the query string ``devops''. \devops has many aspects, such as culture, technology, process, and so on. We decided to not expand the query string to not artificially favor some \devops aspects over others.
The search on the scientific repositories followed some restrictions. Papers should have at least 3 pages. Pages should be published on journals, conferences, congresses, or symposiums, which means workshop papers were not considered. Papers should be written in English. At this point we discarded literature reviews, which are described in Section~\ref{sec:related-surveys} as related work. This search phase was conducted considering only paper title and metadata. It was conducted by the first author and validated by other authors.
After founding 198 papers in the search phase, we proceeded to the selection phase. After reading the abstract, each paper received a classification.
\emph{Three stars} for papers that should be full read.
\emph{Two stars} for papers relevant in the \devops context, but that we judged were not worthy to read entirely.
\emph{One star} for papers that should not be considered relevant for a \devops study.
We classified 36 papers as three-stars, 117 papers as two-stars, and 45 papers as one-star. The found papers about \devops (two- and three-stars) were considered in counting procedures, like in Figures~\ref{fig:papers-per-year}~and~\ref{fig:source_types-per-year} and Table~\ref{tab:reseach-sources}.
Although the criteria to define the star classification is inherently subjective, we sought to reduce subjectiveness by making at least two authors agreeing with each classifications based on the following guidelines:
\begin{itemize}
\item If the paper discuss \devops peculiarities or challenges in some specific context, it tends to receive three stars.
\item If the paper talks about a tool that may be important for \devops practice (e.g.: containers or deployment engines), but the paper is too centered on this single tool and do not discusses \devops itself, it tends to receive two stars.
\item If the authors have a more complete work, the most complete prevails as three stars. The other one receives two stars.
\item If a paper is very similar to other more complete or more cited works, it tends to receive two stars.
\item If the paper only talks about applying \devops in a specific context, but without elaboration beyond state-of-the-art, it tends to receive one star.
\item If the paper talks about some topic (e.g. big data) and uses \devops only for the background contextualization, it tends to receive one star.
\item If the paper has less then four pages, it tends to receive one star.
\item If the paper only presents a simple proposal or opinion, without validation, it tends to receive one star.
\end{itemize}
There are some papers that initially received three stars, but which were downgraded after more clarification from the reading.
We also applied a snowballing process~\cite{claes2014snowballing} to expand our main selected studies with important work not found with the query string ``devops''. As we read the initial main selected papers we looked for cited relevant references, mainly the historical and seminal ones. In this additional selection, we also considered the papers citation count. So we added more 14 papers in our main selected studies group from this snowballing process.
The final list of the 50 main selected studies is presented at page \pageofmainselecetdstudies.
\subsection{Producing the conceptual maps}
......@@ -52,6 +52,7 @@ In the ``Main Selected Studies'' listing (page \pageofmainselecetdstudies), each
In the next section, we detail other surveys on \devops and how this work differentiates from them.
\subsection{Related Surveys}
\label{sec:related-surveys}
Even though \devops is a recent research topic, several studies have performed a Systematic Literature Review (SLR) on the topic. They were pioneers to map the main concepts, the challenges, the research opportunities, and the gaps found about this subject.
......
No preview for this file type
......@@ -527,3 +527,25 @@ isbn="978-3-319-13835-0"
year = {2014},
note={\url{https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1}, accessed on Sep 2018.}
}
@inproceedings{claes2014snowballing,
author = {Wohlin, Claes},
title = {Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering},
booktitle = {Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering},
series = {EASE '14},
year = {2014},
pages = {38:1--38:10},
publisher = {ACM}
}
@inproceedings{budgen2006review,
author = {Budgen, David and Brereton, Pearl},
title = {Performing Systematic Literature Reviews in Software Engineering},
booktitle = {Proceedings of the 28th International Conference on Software Engineering},
series = {ICSE '06},
year = {2006},
pages = {1051--1052},
publisher = {ACM}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment