Merge branch 'devel'HEADmaster

1 files changed, 335 insertions, 51 deletions

diff --git a/doc/tex/implementation.tex b/doc/tex/implementation.tex
index d46e805..d3cd0c3 100644
--- a/doc/tex/implementation.tex
+++ b/doc/tex/implementation.tex
@@ -1,8 +1,11 @@
 \section{Implementation}
 
+This section is about the specific tools and process used to implement
+\program{}.
+
 \subsection{Development Hardware}
 
-The development and evaluation machine is a Satellite L50-C laptop. Its
+The development and evaluation machine is a Toshiba Satellite L50-C laptop. Its
 specifications are shown as a list for readability.
 
 \begin{itemize}
@@ -15,71 +18,71 @@ specifications are shown as a list for readability.
 \subsection{Development Software}
 
 The tools selected for the development of the project and the documentation are
-listed and explained on this section. All the tools used are either
-free\cite{fsf_free} or open source software. The development machine runs 64
-bits Arch Linux as its operating system.
+listed and explained on this section. All the tools used are either free
+\cite{fsf_free} or open source software. The development machine runs 64 bits
+Arch Linux \cite{arch} as its operating system.
 
 \subsubsection{Language}
 
-The programming language of choice is Python\cite{python}. The rationale behind
+The programming language of choice is Python \cite{python}. The rationale behind
 this decision has been stated on Section \ref{sec:programmingLanguage}. It also
 allows easy use of the Keras library for implementing neural networks.
 
-Various python libraries have been used to easy the development process or
+Various Python libraries have been used to easy the development process or
 assist in the analysis of results. These are:
 
-\paragraph{Keras/Tensorflow\cite{keras}}
+\paragraph{Keras/TensorFlow}
 
-Tensorflow is a platform for machine learning which provides a diverse range of
-tools, one of which is a python library for machine learning.
+TensorFlow \cite{tensorflow} is a platform for machine learning which provides a
+diverse range of tools, one of which is a Python library for machine learning.
 
-Keras is a high-level API for Tensorflow allowing for the easy definition of
-neural networks. It permits easily testing and comparing different network
-layouts.
+Keras \cite{keras} is a high-level API for TensorFlow allowing for the easy
+definition of neural networks. It permits easily testing and comparing different
+network layouts.
 
-\paragraph{NumPy\cite{numpy}}
+\paragraph{NumPy}
 
-A scientific package for python providing a lot of mathematical tools. The most
-interesting for this project are its capabilities to create and transform
-matrices.
+A scientific package for Python providing a lot of mathematical tools
+\cite{numpy}. The most interesting for this project are its capabilities to
+create and transform matrices.
 
-\paragraph{Matplotlib\cite{matplotlib}}
+\paragraph{Matplotlib}
 
-A python library for creating graphs and other visualizations. It is used to
-show the likelihood of moves the neural networks of the project create from a
-board configuration.
+A Python library for creating graphs and other visualizations \cite{matplotlib}.
+It is used to show the likelihood of moves the neural networks of the project
+create from a board configuration.
 
-\paragraph{PLY\cite{ply}}
+\paragraph{PLY}
 
-A tool for generating compilers in Python. It is an implementation of the lex
-and yacc utilities, allowing to create lexers and parsers. It is used in the
-project to create the SGF parser which transform SGF files to internal
-representations of Go matches.
+A tool for generating compilers in Python \cite{ply}. It is an implementation of
+the lex and yacc utilities, allowing to create lexers and parsers. It is used in
+the project to create the \acrshort{sgf} parser which transform \acrshort{sgf}
+files to internal representations of Go matches.
 
 \paragraph{Other utility libraries}
 
-These are some utility libraries commonly used for frequent programming tasks:
+These are some utility libraries commonly used for frequent programming tasks.
 
 \begin{itemize}
-	\item \textbf{sys}: to stop the execution of the program or access system info such
+	\item \textbf{sys}: To stop the execution of the program or access system info such
 		as primitives maximum values.
-	\item \textbf{os}: to interact with files.
-	\item \textbf{re}: to check strings with regular expressions.
-	\item \textbf{random}: to get random values, for example to obtain a random
-		item from a list.
-	\item \textbf{copy}: to obtain deep copies of multidimensional arrays.
+	\item \textbf{os}: To interact with files.
+	\item \textbf{re}: To check strings with regular expressions.
+	\item \textbf{random}: For randomness, for example to obtain a random item
+		from a list.
+	\item \textbf{copy}: To obtain deep copies of multidimensional arrays.
 \end{itemize}
 
 \subsubsection{Development Tools}
 
-\paragraph{Neovim\cite{neovim}}
+\paragraph{Neovim}
 
-A text editor based on Vim\cite{vim}, providing its same functionality with
-useful additions and defaults for modern computers and terminal emulators. With
-some extensions and configuration it can become a powerful development
-environment with a very fluid usage experience. That, and the fact that the
-developer considers Vim's modal editing the best writing experience possible on
-a computer, have made Neovim the editor of choice.
+A text editor based on Vim \cite{vim}, providing its same functionality with
+useful additions and defaults for modern computers and terminal emulators
+\cite{neovim}. With some extensions and configuration it becomes a powerful
+development environment with a very fluid usage experience. That, and the
+preference of the developer of Vim's modal editing as the best writing
+experience possible on a computer, have made Neovim the editor of choice.
 
 %TODO: Write about neovim extensions
 %\begin{itemize}
@@ -88,21 +91,31 @@ a computer, have made Neovim the editor of choice.
 %
 %\end{itemize}
 
+\paragraph{Git}
+
+A version control tool widely used in software development environments
+\cite{git}. If the reader is not already familiar with it, it suffices to say it
+allows to store and manage snapshots of a project, navigate through them and
+diverge into different development branches, among other useful features.
+
+The source code of this document and of the rest of the project is publicly
+available at \url{https://git.taamas.xyz/Taamas/imago}.
+
 \subsubsection{Documentation Tools}
 
-\paragraph{\LaTeX\cite{latex}}
+\paragraph{\LaTeX}
 
-A typesetting system widely used in the investigation field, among others. It
-allows for documentation like this text to be written in plain text and then
-compiled to PDF or other formats, which permits keeping the source files of the
-documentation small and organized plus other benefits of plain text such as
-being able to use version control.
+A typesetting system widely used in the investigation field, among others
+\cite{latex}. It allows for documentation like this text to be written in plain
+text and then compiled to PDF or other formats, which permits keeping the source
+files of the documentation small and organized plus other benefits of plain text
+such as being able to be used in conjunction with version control tools.
 
-\paragraph{PlantUML\cite{puml}}
+\paragraph{PlantUML}
 
-A program which creates diagrams from plain text files. PlantUML supports syntax
-for many different sorts of diagrams, mainly but not only UML. It has been used
-to generate the diagrams used in this text.
+A program which creates diagrams from plain text files \cite{puml}. PlantUML
+supports syntax for many different sorts of diagrams, mainly but not only UML.
+It has been used to generate the diagrams used in this text.
 
 \paragraph{Make}
 
@@ -113,9 +126,280 @@ updates them if they already exist but their source files are newer than them.
 
 It has been used to generate this text from \LaTeX{} and PlantUML source files.
 The contents of the Makefile with which this document has been compiled are
-shown in \flist{code:makefile}.
+shown in \lref{code:makefile}.
 
-\begin{listing}[h]
+\begin{listing}[p]
 	\inputminted{make}{Makefile}
 	\caption{Documentation Makefile}\label{code:makefile}
 \end{listing}
+
+\subsection{Execution of the Testing Plan}
+
+Part of the implementation process is to execute the testing plan. The results
+of this execution are provided in this section.
+
+\subsubsection{Execution of the Unitary Testing}
+
+The script used to run the tests is shown on \lref{lst:test} and its output on
+\lref{lst:testOutput}.
+
+\begin{listing}[p]
+	\inputminted{bash}{listings/test.sh}
+	\caption{Script to run the tests and list the result.}
+	\label{lst:test}
+\end{listing}
+
+\begin{listing}[p]
+	\inputminted[fontsize=\footnotesize]{text}{listings/testOutput.txt}
+	\caption{Unitary testing output.}
+	\label{lst:testOutput}
+\end{listing}
+
+\subsubsection{Execution of the Integration Testing}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+	\toprule
+	\multicolumn{3}{c}{\textbf{Engine and Game modules}} \\
+	\midrule
+	\textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+	\midrule
+	The GTP interface of the engine is used to play a match &
+	The module handles the game and can show its state. &
+	The engine runs correctly and is capable of keeping track of and showing the
+	state of a match, which shows it is making good use of the Game module. \\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+	\toprule
+	\multicolumn{3}{c}{\textbf{Training and Engine module}} \\
+	\midrule
+	\textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+	\midrule
+	The training process is started &
+	The training uses the network defined on the Engine module. &
+	The training output shows the network in training follows the design defined
+	on the Engine module. \\
+	\bottomrule
+\end{tabular}
+
+\subsubsection{Execution of the System Testing}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+	\toprule
+	\multicolumn{3}{c}{\textbf{Game interface}} \\
+	\midrule
+	\textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+	\midrule
+	Play a game of Go with no engine &
+	The game can be played until the end. &
+	The interface continues asking for moves until both players pass
+	consecutively, which ends the game and the execution. \\
+	\midrule
+	Provide a wrong move &
+	The interface shows it is wrong and the game continues without a change of
+	state. &
+	As expected, the interface shows a message signaling that the move is wrong
+	and showing an example of a move. \\
+	\midrule
+	Close the game &
+	The interface closes. &
+	A message is shown signaling that the game is ending and the engine closes.
+	\\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+	\toprule
+	\multicolumn{3}{c}{\textbf{Engine interface}} \\
+	\midrule
+	\textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+	\midrule
+	Ask for the available commands &
+	The interface outputs the available commands. &
+	The list of available commands is printed one per line. \\
+	\midrule
+	Provide a move &
+	The state of the engine updates with the new move. &
+	No output is given, but after providing the \texttt{showboard} command it is
+	shown that the move has been registered. \\
+	\midrule
+	Ask for a move &
+	The engine suggests a move without changing the state of the current game. &
+	After the necessary time for generating the move, it is printed. The
+	\texttt{showboard} is then used to check the state of the game has not
+	changed. \\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+	\toprule
+	\multicolumn{3}{c}{\textbf{Training interface}} \\
+	\midrule
+	\textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+	\midrule
+	Provide some games to train on &
+	A neural network model is created. &
+	First, all the training files used are printed, and then, as expected, the
+	training process commences. A model is created or updated in the
+	\texttt{models} folder. \\
+	\midrule
+	Start the training without providing games &
+	An error message is shown and the execution terminated. &
+	As expected, a message is shown asking for SGF files to be provided as
+	arguments and the execution is terminated.\\
+	\bottomrule
+\end{tabular}
+
+\subsubsection{Usability Testing}
+
+Two human users were asked to interact with the interfaces of \program{} and
+presented with a questionary.
+
+The profile of the first user is of someone who has played some Go matches and
+knows the fundamentals of the game but is a beginner, and who has little
+experience with computers outside of their usage as office tools and internet
+browsers. Here are their answers.
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+	\toprule
+	\multicolumn{2}{c}{\textbf{Playing against a human}} \\
+	\midrule
+	\textbf{Question} & \textbf{Answer} \\
+	\midrule
+	Were you able to start the interface? &
+	Yes. \\
+	\midrule
+	How hard was the interface of the game to understand? &
+	It was easy and intuitive because you just entered the command to start and
+	then you only had to tell it where you wanted to play the stones. It felt
+	easy to me.\\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+	\toprule
+	\multicolumn{2}{c}{\textbf{Playing against the engine}} \\
+	\midrule
+	\textbf{Question} & \textbf{Answer} \\
+	\midrule
+	Were you able to start the interface? &
+	Yes. \\
+	\midrule
+	How hard was the interface of the game to understand? &
+	It was easy to understand because it just was enering the commands and the
+	application did what it had to do, but it was more difficult than the
+	previous one because I'm not used to do these things with commands. It is
+	less intuitive than the previous one regarding playing against the machine
+	because you don't see each move as you write it, having to ask it to show
+	them instead.\\
+	\midrule
+	How strong did you find the engine? &
+	It was not so aggresive as playing against a human who knows the game. It
+	doesn't play to harm its opponent.\\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+	\toprule
+	\multicolumn{2}{c}{\textbf{Playing against the interface through a
+	third-party}} \\
+	\midrule
+	\textbf{Question} & \textbf{Answer} \\
+	\midrule
+	Were you able to start the interface? &
+	Yes. \\
+	\midrule
+	Did you find any problems when setting up the engine? &
+	No, it was well explained step by step, although the images could be better
+	lined up with the text. Anyway, the explanations were clear and easy to
+	follow.\\
+	\midrule
+	Do you think this tool has value for studying Go? &
+	Yes.\\
+	\bottomrule
+\end{tabular}
+
+The profile of the second user is of someone who has experience with computers
+and works as a software developer, but who has just the bare minimum knowledge
+of the game of Go. Here are their answers.
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+	\toprule
+	\multicolumn{2}{c}{\textbf{Playing against a human}} \\
+	\midrule
+	\textbf{Question} & \textbf{Answer} \\
+	\midrule
+	Were you able to start the interface? &
+	Yes, I was able to. \\
+	\midrule
+	How hard was the interface of the game to understand? &
+	I think six out of ten. Some people won't understand what a command is and
+	without a visual interface they could feel confused about it.\\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+	\toprule
+	\multicolumn{2}{c}{\textbf{Playing against the engine}} \\
+	\midrule
+	\textbf{Question} & \textbf{Answer} \\
+	\midrule
+	Were you able to start the interface? &
+	Yes, I was. \\
+	\midrule
+	How hard was the interface of the game to understand? &
+	I think seven out of ten. I was expecting to follow some steps so I could
+	execute it at the same time I was reading the manual, but in the end the
+	most practical thing was read everything before execute commands.\\
+	\midrule
+	How strong did you find the engine? &
+	It followed good paths to win the game, I didn't feel random moves during
+	the game by its side.\\
+	\bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+	\toprule
+	\multicolumn{2}{c}{\textbf{Playing against the interface through a
+	third-party}} \\
+	\midrule
+	\textbf{Question} & \textbf{Answer} \\
+	\midrule
+	Were you able to start the interface? &
+	Yes, I was. \\
+	\midrule
+	Did you find any problems when setting up the engine? &
+	No, I didn't.\\
+	\midrule
+	Do you think this tool has value for studying Go? &
+	I think it is a good tool for a group of players, so they can practise and
+	even train the AI if they want to.\\
+	\bottomrule
+\end{tabular}
+
+The results of these usability tests were useful mostly to update the manual and
+make it easier to understand and follow and also to address some problems with
+the interfaces that raised up during the testing.