aboutsummaryrefslogtreecommitdiff
path: root/doc/tex/implementation.tex
diff options
context:
space:
mode:
Diffstat (limited to 'doc/tex/implementation.tex')
-rw-r--r--doc/tex/implementation.tex386
1 files changed, 335 insertions, 51 deletions
diff --git a/doc/tex/implementation.tex b/doc/tex/implementation.tex
index d46e805..d3cd0c3 100644
--- a/doc/tex/implementation.tex
+++ b/doc/tex/implementation.tex
@@ -1,8 +1,11 @@
\section{Implementation}
+This section is about the specific tools and process used to implement
+\program{}.
+
\subsection{Development Hardware}
-The development and evaluation machine is a Satellite L50-C laptop. Its
+The development and evaluation machine is a Toshiba Satellite L50-C laptop. Its
specifications are shown as a list for readability.
\begin{itemize}
@@ -15,71 +18,71 @@ specifications are shown as a list for readability.
\subsection{Development Software}
The tools selected for the development of the project and the documentation are
-listed and explained on this section. All the tools used are either
-free\cite{fsf_free} or open source software. The development machine runs 64
-bits Arch Linux as its operating system.
+listed and explained on this section. All the tools used are either free
+\cite{fsf_free} or open source software. The development machine runs 64 bits
+Arch Linux \cite{arch} as its operating system.
\subsubsection{Language}
-The programming language of choice is Python\cite{python}. The rationale behind
+The programming language of choice is Python \cite{python}. The rationale behind
this decision has been stated on Section \ref{sec:programmingLanguage}. It also
allows easy use of the Keras library for implementing neural networks.
-Various python libraries have been used to easy the development process or
+Various Python libraries have been used to easy the development process or
assist in the analysis of results. These are:
-\paragraph{Keras/Tensorflow\cite{keras}}
+\paragraph{Keras/TensorFlow}
-Tensorflow is a platform for machine learning which provides a diverse range of
-tools, one of which is a python library for machine learning.
+TensorFlow \cite{tensorflow} is a platform for machine learning which provides a
+diverse range of tools, one of which is a Python library for machine learning.
-Keras is a high-level API for Tensorflow allowing for the easy definition of
-neural networks. It permits easily testing and comparing different network
-layouts.
+Keras \cite{keras} is a high-level API for TensorFlow allowing for the easy
+definition of neural networks. It permits easily testing and comparing different
+network layouts.
-\paragraph{NumPy\cite{numpy}}
+\paragraph{NumPy}
-A scientific package for python providing a lot of mathematical tools. The most
-interesting for this project are its capabilities to create and transform
-matrices.
+A scientific package for Python providing a lot of mathematical tools
+\cite{numpy}. The most interesting for this project are its capabilities to
+create and transform matrices.
-\paragraph{Matplotlib\cite{matplotlib}}
+\paragraph{Matplotlib}
-A python library for creating graphs and other visualizations. It is used to
-show the likelihood of moves the neural networks of the project create from a
-board configuration.
+A Python library for creating graphs and other visualizations \cite{matplotlib}.
+It is used to show the likelihood of moves the neural networks of the project
+create from a board configuration.
-\paragraph{PLY\cite{ply}}
+\paragraph{PLY}
-A tool for generating compilers in Python. It is an implementation of the lex
-and yacc utilities, allowing to create lexers and parsers. It is used in the
-project to create the SGF parser which transform SGF files to internal
-representations of Go matches.
+A tool for generating compilers in Python \cite{ply}. It is an implementation of
+the lex and yacc utilities, allowing to create lexers and parsers. It is used in
+the project to create the \acrshort{sgf} parser which transform \acrshort{sgf}
+files to internal representations of Go matches.
\paragraph{Other utility libraries}
-These are some utility libraries commonly used for frequent programming tasks:
+These are some utility libraries commonly used for frequent programming tasks.
\begin{itemize}
- \item \textbf{sys}: to stop the execution of the program or access system info such
+ \item \textbf{sys}: To stop the execution of the program or access system info such
as primitives maximum values.
- \item \textbf{os}: to interact with files.
- \item \textbf{re}: to check strings with regular expressions.
- \item \textbf{random}: to get random values, for example to obtain a random
- item from a list.
- \item \textbf{copy}: to obtain deep copies of multidimensional arrays.
+ \item \textbf{os}: To interact with files.
+ \item \textbf{re}: To check strings with regular expressions.
+ \item \textbf{random}: For randomness, for example to obtain a random item
+ from a list.
+ \item \textbf{copy}: To obtain deep copies of multidimensional arrays.
\end{itemize}
\subsubsection{Development Tools}
-\paragraph{Neovim\cite{neovim}}
+\paragraph{Neovim}
-A text editor based on Vim\cite{vim}, providing its same functionality with
-useful additions and defaults for modern computers and terminal emulators. With
-some extensions and configuration it can become a powerful development
-environment with a very fluid usage experience. That, and the fact that the
-developer considers Vim's modal editing the best writing experience possible on
-a computer, have made Neovim the editor of choice.
+A text editor based on Vim \cite{vim}, providing its same functionality with
+useful additions and defaults for modern computers and terminal emulators
+\cite{neovim}. With some extensions and configuration it becomes a powerful
+development environment with a very fluid usage experience. That, and the
+preference of the developer of Vim's modal editing as the best writing
+experience possible on a computer, have made Neovim the editor of choice.
%TODO: Write about neovim extensions
%\begin{itemize}
@@ -88,21 +91,31 @@ a computer, have made Neovim the editor of choice.
%
%\end{itemize}
+\paragraph{Git}
+
+A version control tool widely used in software development environments
+\cite{git}. If the reader is not already familiar with it, it suffices to say it
+allows to store and manage snapshots of a project, navigate through them and
+diverge into different development branches, among other useful features.
+
+The source code of this document and of the rest of the project is publicly
+available at \url{https://git.taamas.xyz/Taamas/imago}.
+
\subsubsection{Documentation Tools}
-\paragraph{\LaTeX\cite{latex}}
+\paragraph{\LaTeX}
-A typesetting system widely used in the investigation field, among others. It
-allows for documentation like this text to be written in plain text and then
-compiled to PDF or other formats, which permits keeping the source files of the
-documentation small and organized plus other benefits of plain text such as
-being able to use version control.
+A typesetting system widely used in the investigation field, among others
+\cite{latex}. It allows for documentation like this text to be written in plain
+text and then compiled to PDF or other formats, which permits keeping the source
+files of the documentation small and organized plus other benefits of plain text
+such as being able to be used in conjunction with version control tools.
-\paragraph{PlantUML\cite{puml}}
+\paragraph{PlantUML}
-A program which creates diagrams from plain text files. PlantUML supports syntax
-for many different sorts of diagrams, mainly but not only UML. It has been used
-to generate the diagrams used in this text.
+A program which creates diagrams from plain text files \cite{puml}. PlantUML
+supports syntax for many different sorts of diagrams, mainly but not only UML.
+It has been used to generate the diagrams used in this text.
\paragraph{Make}
@@ -113,9 +126,280 @@ updates them if they already exist but their source files are newer than them.
It has been used to generate this text from \LaTeX{} and PlantUML source files.
The contents of the Makefile with which this document has been compiled are
-shown in \flist{code:makefile}.
+shown in \lref{code:makefile}.
-\begin{listing}[h]
+\begin{listing}[p]
\inputminted{make}{Makefile}
\caption{Documentation Makefile}\label{code:makefile}
\end{listing}
+
+\subsection{Execution of the Testing Plan}
+
+Part of the implementation process is to execute the testing plan. The results
+of this execution are provided in this section.
+
+\subsubsection{Execution of the Unitary Testing}
+
+The script used to run the tests is shown on \lref{lst:test} and its output on
+\lref{lst:testOutput}.
+
+\begin{listing}[p]
+ \inputminted{bash}{listings/test.sh}
+ \caption{Script to run the tests and list the result.}
+ \label{lst:test}
+\end{listing}
+
+\begin{listing}[p]
+ \inputminted[fontsize=\footnotesize]{text}{listings/testOutput.txt}
+ \caption{Unitary testing output.}
+ \label{lst:testOutput}
+\end{listing}
+
+\subsubsection{Execution of the Integration Testing}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+ \toprule
+ \multicolumn{3}{c}{\textbf{Engine and Game modules}} \\
+ \midrule
+ \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+ \midrule
+ The GTP interface of the engine is used to play a match &
+ The module handles the game and can show its state. &
+ The engine runs correctly and is capable of keeping track of and showing the
+ state of a match, which shows it is making good use of the Game module. \\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+ \toprule
+ \multicolumn{3}{c}{\textbf{Training and Engine module}} \\
+ \midrule
+ \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+ \midrule
+ The training process is started &
+ The training uses the network defined on the Engine module. &
+ The training output shows the network in training follows the design defined
+ on the Engine module. \\
+ \bottomrule
+\end{tabular}
+
+\subsubsection{Execution of the System Testing}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+ \toprule
+ \multicolumn{3}{c}{\textbf{Game interface}} \\
+ \midrule
+ \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+ \midrule
+ Play a game of Go with no engine &
+ The game can be played until the end. &
+ The interface continues asking for moves until both players pass
+ consecutively, which ends the game and the execution. \\
+ \midrule
+ Provide a wrong move &
+ The interface shows it is wrong and the game continues without a change of
+ state. &
+ As expected, the interface shows a message signaling that the move is wrong
+ and showing an example of a move. \\
+ \midrule
+ Close the game &
+ The interface closes. &
+ A message is shown signaling that the game is ending and the engine closes.
+ \\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+ \toprule
+ \multicolumn{3}{c}{\textbf{Engine interface}} \\
+ \midrule
+ \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+ \midrule
+ Ask for the available commands &
+ The interface outputs the available commands. &
+ The list of available commands is printed one per line. \\
+ \midrule
+ Provide a move &
+ The state of the engine updates with the new move. &
+ No output is given, but after providing the \texttt{showboard} command it is
+ shown that the move has been registered. \\
+ \midrule
+ Ask for a move &
+ The engine suggests a move without changing the state of the current game. &
+ After the necessary time for generating the move, it is printed. The
+ \texttt{showboard} is then used to check the state of the game has not
+ changed. \\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}}
+ \toprule
+ \multicolumn{3}{c}{\textbf{Training interface}} \\
+ \midrule
+ \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\
+ \midrule
+ Provide some games to train on &
+ A neural network model is created. &
+ First, all the training files used are printed, and then, as expected, the
+ training process commences. A model is created or updated in the
+ \texttt{models} folder. \\
+ \midrule
+ Start the training without providing games &
+ An error message is shown and the execution terminated. &
+ As expected, a message is shown asking for SGF files to be provided as
+ arguments and the execution is terminated.\\
+ \bottomrule
+\end{tabular}
+
+\subsubsection{Usability Testing}
+
+Two human users were asked to interact with the interfaces of \program{} and
+presented with a questionary.
+
+The profile of the first user is of someone who has played some Go matches and
+knows the fundamentals of the game but is a beginner, and who has little
+experience with computers outside of their usage as office tools and internet
+browsers. Here are their answers.
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+ \toprule
+ \multicolumn{2}{c}{\textbf{Playing against a human}} \\
+ \midrule
+ \textbf{Question} & \textbf{Answer} \\
+ \midrule
+ Were you able to start the interface? &
+ Yes. \\
+ \midrule
+ How hard was the interface of the game to understand? &
+ It was easy and intuitive because you just entered the command to start and
+ then you only had to tell it where you wanted to play the stones. It felt
+ easy to me.\\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+ \toprule
+ \multicolumn{2}{c}{\textbf{Playing against the engine}} \\
+ \midrule
+ \textbf{Question} & \textbf{Answer} \\
+ \midrule
+ Were you able to start the interface? &
+ Yes. \\
+ \midrule
+ How hard was the interface of the game to understand? &
+ It was easy to understand because it just was enering the commands and the
+ application did what it had to do, but it was more difficult than the
+ previous one because I'm not used to do these things with commands. It is
+ less intuitive than the previous one regarding playing against the machine
+ because you don't see each move as you write it, having to ask it to show
+ them instead.\\
+ \midrule
+ How strong did you find the engine? &
+ It was not so aggresive as playing against a human who knows the game. It
+ doesn't play to harm its opponent.\\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+ \toprule
+ \multicolumn{2}{c}{\textbf{Playing against the interface through a
+ third-party}} \\
+ \midrule
+ \textbf{Question} & \textbf{Answer} \\
+ \midrule
+ Were you able to start the interface? &
+ Yes. \\
+ \midrule
+ Did you find any problems when setting up the engine? &
+ No, it was well explained step by step, although the images could be better
+ lined up with the text. Anyway, the explanations were clear and easy to
+ follow.\\
+ \midrule
+ Do you think this tool has value for studying Go? &
+ Yes.\\
+ \bottomrule
+\end{tabular}
+
+The profile of the second user is of someone who has experience with computers
+and works as a software developer, but who has just the bare minimum knowledge
+of the game of Go. Here are their answers.
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+ \toprule
+ \multicolumn{2}{c}{\textbf{Playing against a human}} \\
+ \midrule
+ \textbf{Question} & \textbf{Answer} \\
+ \midrule
+ Were you able to start the interface? &
+ Yes, I was able to. \\
+ \midrule
+ How hard was the interface of the game to understand? &
+ I think six out of ten. Some people won't understand what a command is and
+ without a visual interface they could feel confused about it.\\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+ \toprule
+ \multicolumn{2}{c}{\textbf{Playing against the engine}} \\
+ \midrule
+ \textbf{Question} & \textbf{Answer} \\
+ \midrule
+ Were you able to start the interface? &
+ Yes, I was. \\
+ \midrule
+ How hard was the interface of the game to understand? &
+ I think seven out of ten. I was expecting to follow some steps so I could
+ execute it at the same time I was reading the manual, but in the end the
+ most practical thing was read everything before execute commands.\\
+ \midrule
+ How strong did you find the engine? &
+ It followed good paths to win the game, I didn't feel random moves during
+ the game by its side.\\
+ \bottomrule
+\end{tabular}
+
+\vspace{\interclassSpace}
+
+\begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}}
+ \toprule
+ \multicolumn{2}{c}{\textbf{Playing against the interface through a
+ third-party}} \\
+ \midrule
+ \textbf{Question} & \textbf{Answer} \\
+ \midrule
+ Were you able to start the interface? &
+ Yes, I was. \\
+ \midrule
+ Did you find any problems when setting up the engine? &
+ No, I didn't.\\
+ \midrule
+ Do you think this tool has value for studying Go? &
+ I think it is a good tool for a group of players, so they can practise and
+ even train the AI if they want to.\\
+ \bottomrule
+\end{tabular}
+
+The results of these usability tests were useful mostly to update the manual and
+make it easier to understand and follow and also to address some problems with
+the interfaces that raised up during the testing.