\section{Implementation} This section is about the specific tools and process used to implement \program{}. \subsection{Development Hardware} The development and evaluation machine is a Toshiba Satellite L50-C laptop. Its specifications are shown as a list for readability. \begin{itemize} \item \textbf{CPU}: Intel i5-6200U, 2.800GHz \item \textbf{Integrated GPU}: Intel Skylake GT2 \item \textbf{Dedicated GPU}: Nvidia Geforce 930M \item \textbf{RAM}: 8 GiB \end{itemize} \subsection{Development Software} The tools selected for the development of the project and the documentation are listed and explained on this section. All the tools used are either free \cite{fsf_free} or open source software. The development machine runs 64 bits Arch Linux \cite{arch} as its operating system. \subsubsection{Language} The programming language of choice is Python \cite{python}. The rationale behind this decision has been stated on Section \ref{sec:programmingLanguage}. It also allows easy use of the Keras library for implementing neural networks. Various Python libraries have been used to easy the development process or assist in the analysis of results. These are: \paragraph{Keras/TensorFlow} TensorFlow \cite{tensorflow} is a platform for machine learning which provides a diverse range of tools, one of which is a Python library for machine learning. Keras \cite{keras} is a high-level API for TensorFlow allowing for the easy definition of neural networks. It permits easily testing and comparing different network layouts. \paragraph{NumPy} A scientific package for Python providing a lot of mathematical tools \cite{numpy}. The most interesting for this project are its capabilities to create and transform matrices. \paragraph{Matplotlib} A Python library for creating graphs and other visualizations \cite{matplotlib}. It is used to show the likelihood of moves the neural networks of the project create from a board configuration. \paragraph{PLY} A tool for generating compilers in Python \cite{ply}. It is an implementation of the lex and yacc utilities, allowing to create lexers and parsers. It is used in the project to create the \acrshort{sgf} parser which transform \acrshort{sgf} files to internal representations of Go matches. \paragraph{Other utility libraries} These are some utility libraries commonly used for frequent programming tasks. \begin{itemize} \item \textbf{sys}: To stop the execution of the program or access system info such as primitives maximum values. \item \textbf{os}: To interact with files. \item \textbf{re}: To check strings with regular expressions. \item \textbf{random}: For randomness, for example to obtain a random item from a list. \item \textbf{copy}: To obtain deep copies of multidimensional arrays. \end{itemize} \subsubsection{Development Tools} \paragraph{Neovim} A text editor based on Vim \cite{vim}, providing its same functionality with useful additions and defaults for modern computers and terminal emulators \cite{neovim}. With some extensions and configuration it becomes a powerful development environment with a very fluid usage experience. That, and the preference of the developer of Vim's modal editing as the best writing experience possible on a computer, have made Neovim the editor of choice. %TODO: Write about neovim extensions %\begin{itemize} % \item FZF % \item Extensions % %\end{itemize} \paragraph{Git} A version control tool widely used in software development environments \cite{git}. If the reader is not already familiar with it, it suffices to say it allows to store and manage snapshots of a project, navigate through them and diverge into different development branches, among other useful features. The source code of this document and of the rest of the project is publicly available at \url{https://git.taamas.xyz/Taamas/imago}. \subsubsection{Documentation Tools} \paragraph{\LaTeX} A typesetting system widely used in the investigation field, among others \cite{latex}. It allows for documentation like this text to be written in plain text and then compiled to PDF or other formats, which permits keeping the source files of the documentation small and organized plus other benefits of plain text such as being able to be used in conjunction with version control tools. \paragraph{PlantUML} A program which creates diagrams from plain text files \cite{puml}. PlantUML supports syntax for many different sorts of diagrams, mainly but not only UML. It has been used to generate the diagrams used in this text. \paragraph{Make} A tool for specifying and handling dependencies on a build system. It reads a file, typically named ``Makefile'', containing which files are needed and on which other files each of them depends, and then generates those files or updates them if they already exist but their source files are newer than them. It has been used to generate this text from \LaTeX{} and PlantUML source files. The contents of the Makefile with which this document has been compiled are shown in \lref{code:makefile}. \begin{listing}[p] \inputminted{make}{Makefile} \caption{Documentation Makefile}\label{code:makefile} \end{listing} \subsection{Execution of the Testing Plan} Part of the implementation process is to execute the testing plan. The results of this execution are provided in this section. \subsubsection{Execution of the Unitary Testing} The script used to run the tests is shown on \lref{lst:test} and its output on \lref{lst:testOutput}. \begin{listing}[p] \inputminted{bash}{listings/test.sh} \caption{Script to run the tests and list the result.} \label{lst:test} \end{listing} \begin{listing}[p] \inputminted[fontsize=\footnotesize]{text}{listings/testOutput.txt} \caption{Unitary testing output.} \label{lst:testOutput} \end{listing} \subsubsection{Execution of the Integration Testing} \vspace{\interclassSpace} \begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}} \toprule \multicolumn{3}{c}{\textbf{Engine and Game modules}} \\ \midrule \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\ \midrule The GTP interface of the engine is used to play a match & The module handles the game and can show its state. & The engine runs correctly and is capable of keeping track of and showing the state of a match, which shows it is making good use of the Game module. \\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}} \toprule \multicolumn{3}{c}{\textbf{Training and Engine module}} \\ \midrule \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\ \midrule The training process is started & The training uses the network defined on the Engine module. & The training output shows the network in training follows the design defined on the Engine module. \\ \bottomrule \end{tabular} \subsubsection{Execution of the System Testing} \vspace{\interclassSpace} \begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}} \toprule \multicolumn{3}{c}{\textbf{Game interface}} \\ \midrule \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\ \midrule Play a game of Go with no engine & The game can be played until the end. & The interface continues asking for moves until both players pass consecutively, which ends the game and the execution. \\ \midrule Provide a wrong move & The interface shows it is wrong and the game continues without a change of state. & As expected, the interface shows a message signaling that the move is wrong and showing an example of a move. \\ \midrule Close the game & The interface closes. & A message is shown signaling that the game is ending and the engine closes. \\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}} \toprule \multicolumn{3}{c}{\textbf{Engine interface}} \\ \midrule \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\ \midrule Ask for the available commands & The interface outputs the available commands. & The list of available commands is printed one per line. \\ \midrule Provide a move & The state of the engine updates with the new move. & No output is given, but after providing the \texttt{showboard} command it is shown that the move has been registered. \\ \midrule Ask for a move & The engine suggests a move without changing the state of the current game. & After the necessary time for generating the move, it is printed. The \texttt{showboard} is then used to check the state of the game has not changed. \\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.2\linewidth}p{0.3\linewidth}p{0.4\linewidth}} \toprule \multicolumn{3}{c}{\textbf{Training interface}} \\ \midrule \textbf{Test} & \textbf{Expected behaviour} & \textbf{Results} \\ \midrule Provide some games to train on & A neural network model is created. & First, all the training files used are printed, and then, as expected, the training process commences. A model is created or updated in the \texttt{models} folder. \\ \midrule Start the training without providing games & An error message is shown and the execution terminated. & As expected, a message is shown asking for SGF files to be provided as arguments and the execution is terminated.\\ \bottomrule \end{tabular} \subsubsection{Usability Testing} Two human users were asked to interact with the interfaces of \program{} and presented with a questionary. The profile of the first user is of someone who has played some Go matches and knows the fundamentals of the game but is a beginner, and who has little experience with computers outside of their usage as office tools and internet browsers. Here are their answers. \vspace{\interclassSpace} \begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Playing against a human}} \\ \midrule \textbf{Question} & \textbf{Answer} \\ \midrule Were you able to start the interface? & Yes. \\ \midrule How hard was the interface of the game to understand? & It was easy and intuitive because you just entered the command to start and then you only had to tell it where you wanted to play the stones. It felt easy to me.\\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Playing against the engine}} \\ \midrule \textbf{Question} & \textbf{Answer} \\ \midrule Were you able to start the interface? & Yes. \\ \midrule How hard was the interface of the game to understand? & It was easy to understand because it just was enering the commands and the application did what it had to do, but it was more difficult than the previous one because I'm not used to do these things with commands. It is less intuitive than the previous one regarding playing against the machine because you don't see each move as you write it, having to ask it to show them instead.\\ \midrule How strong did you find the engine? & It was not so aggresive as playing against a human who knows the game. It doesn't play to harm its opponent.\\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Playing against the interface through a third-party}} \\ \midrule \textbf{Question} & \textbf{Answer} \\ \midrule Were you able to start the interface? & Yes. \\ \midrule Did you find any problems when setting up the engine? & No, it was well explained step by step, although the images could be better lined up with the text. Anyway, the explanations were clear and easy to follow.\\ \midrule Do you think this tool has value for studying Go? & Yes.\\ \bottomrule \end{tabular} The profile of the second user is of someone who has experience with computers and works as a software developer, but who has just the bare minimum knowledge of the game of Go. Here are their answers. \vspace{\interclassSpace} \begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Playing against a human}} \\ \midrule \textbf{Question} & \textbf{Answer} \\ \midrule Were you able to start the interface? & Yes, I was able to. \\ \midrule How hard was the interface of the game to understand? & I think six out of ten. Some people won't understand what a command is and without a visual interface they could feel confused about it.\\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Playing against the engine}} \\ \midrule \textbf{Question} & \textbf{Answer} \\ \midrule Were you able to start the interface? & Yes, I was. \\ \midrule How hard was the interface of the game to understand? & I think seven out of ten. I was expecting to follow some steps so I could execute it at the same time I was reading the manual, but in the end the most practical thing was read everything before execute commands.\\ \midrule How strong did you find the engine? & It followed good paths to win the game, I didn't feel random moves during the game by its side.\\ \bottomrule \end{tabular} \vspace{\interclassSpace} \begin{tabular}{p{0.4\linewidth}p{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Playing against the interface through a third-party}} \\ \midrule \textbf{Question} & \textbf{Answer} \\ \midrule Were you able to start the interface? & Yes, I was. \\ \midrule Did you find any problems when setting up the engine? & No, I didn't.\\ \midrule Do you think this tool has value for studying Go? & I think it is a good tool for a group of players, so they can practise and even train the AI if they want to.\\ \bottomrule \end{tabular} The results of these usability tests were useful mostly to update the manual and make it easier to understand and follow and also to address some problems with the interfaces that raised up during the testing.