diff options
Diffstat (limited to 'doc/tex/systemAnalysis.tex')
| -rw-r--r-- | doc/tex/systemAnalysis.tex | 246 |
1 files changed, 170 insertions, 76 deletions
diff --git a/doc/tex/systemAnalysis.tex b/doc/tex/systemAnalysis.tex index c3c6aa8..1f0c997 100644 --- a/doc/tex/systemAnalysis.tex +++ b/doc/tex/systemAnalysis.tex @@ -1,5 +1,9 @@ \section{System Analysis} +This section provides a preliminary analysis of the needs of the project, such +as its requirements, components and use cases. It also includes the +specification of the testing plan. + \subsection{System Reach Determination} These are the main goals the final product must reach. @@ -7,19 +11,19 @@ These are the main goals the final product must reach. \begin{enumerate} \item The implementation, analysis and comparison of different decision - algorithms for genarating moves. This is the main goal and the following + algorithms for generating moves. This is the main goal and the following ones are derived from the need of reaching it. \item A library for representing the game of Go. It can be used for the decision algorithms to keep the state of the game and can also be used - in an interactive application for a user to play the game and test the - code. + in an interactive application for a human user to play the game and test + the code. \item An engine program as a way of presenting an interface for using these - algorithms. The engine will use the GTP so it can be used with an - existing GUI or other tools. + algorithms. The engine will use the \acrshort{gtp} so it can be used with an + existing \acrshort{gui} or other tools. - \item A parser for SGF files so they can be processed in the training of + \item A parser for \acrshort{sgf} files so they can be processed in the training of neural networks. \end{enumerate} @@ -50,7 +54,7 @@ requisites needed for the system. coordinates of the vertex to play on or as ``pass''. \begin{enumerate} \item The text introduced for the move must follow the - regular expression \texttt{([A-Z][0-9]+|pass)} + regular expression \texttt{([A-HJ-Z][0-9]+)|(pass)} \item If the move is not valid it must be notified to the user and another move asked for. \end{enumerate} @@ -84,15 +88,14 @@ requisites needed for the system. \item The engine program is interactive. - \item The engine implements the GTP (\textit{Go Text Protocol}) for its - interface. + \item The engine implements the \acrfull{gtp} for its interface. \begin{enumerate} \item Commands are read from standard input. \item Responses are provided via standard output. \item There exist commands to set up the conditions of the match. \begin{enumerate} \item The size of the board can be set. - \item The komi can be set. + \item The \gls{komi} can be set. \end{enumerate} \item There exist commands to manipulate the internal representation of the match. @@ -149,9 +152,9 @@ requisites needed for the system. \item The trainer can import existing games. \begin{enumerate} - \item Records of games stored as SGF can be imported. - \item Files containing records of games are provided as arguments to - the trainer. + \item Records of games stored as \acrshort{sgf} can be imported. + \item Files containing records of games can be provided as arguments + to the trainer. \end{enumerate} \end{enumerate} @@ -179,13 +182,13 @@ requisites needed for the system. \begin{enumerate} \item For understanding the workings of the application the user needs to be - familiar with the basics of the game of Go. + familiar with the rules of the game of Go. \item For directly using the engine the user needs to be familiar with command line interfaces. - \item For directly using the trainer the user needs to know the different - network models available. + \item For directly using the trainer the user needs to know about the + different network models available. \end{enumerate} @@ -195,8 +198,8 @@ requisites needed for the system. \begin{enumerate} - \item The game program will be a python file able to be executed by the - python interpreter. + \item The game program will be a Python script able to be executed by the + Python interpreter. \item The game program will make use of standard input and standard output for communication. @@ -206,8 +209,8 @@ requisites needed for the system. \item Standard output will be used for messages directed to the user. \end{enumerate} - \item The engine program will be a python file able to be executed by the - python interpreter. + \item The engine program will be a Python script able to be executed by the + Python interpreter. \item The engine program will make use of standard input and standard output for communication. @@ -217,11 +220,11 @@ requisites needed for the system. commands. \end{enumerate} - \item The trainer program will be a python file able to be executed by the - python interpreter. + \item The trainer program will be a Python script able to be executed by the + Python interpreter. - \item The engine program will make use of standard input and standard output - for communication. + \item The trainer program will make use of standard input and standard + output for communication. \begin{enumerate} \item Standard input will be used for reading commands. \item Standard output will be used for showing the result of @@ -264,37 +267,41 @@ between human players to show and test its capabilities. \subsubsection{Engine System} -The Engine System will implement the GTP interface and use the Game System to +The Engine System will implement the \acrshort{gtp} interface and use the Game System to analyse positions and generate moves via decision algorithms. This system can be directly called to manually set up game states and ask for -moves or can be called by other programs which use GTP to be used as backend for +moves or can be called by other programs which use \acrshort{gtp} to be used as backend for playing matches against a computer player. \subsubsection{Training System} -The Training System will process SGF files storing records of games, train the +The Training System will process \acrshort{sgf} files storing records of games, train the neural network models over those games and store the result. These models can then be imported by the engine and be used to generate moves. \subsubsection{Interface Between Subsystems} -The Training System depends on the NeuralNetwork interface of the Engine System -and uses it to train and store the neural network models. +The Training System depends on the \texttt{NeuralNetwork} interface of the +Engine System and uses it to train and store the neural network models. -Both the Engine and Training systems depend on the GameMove class of the Game -System. The Engine System uses it to store the state of a game and provide it -to the decision algorithms. The Training System uses it to create the internal -representation of a game resulting from the processing of an SGF file. +Both the Engine and Training systems depend on the \texttt{GameMove} class of +the Game System. The Engine System uses it to store the state of a game and +provide it to the decision algorithms. The Training System uses it to create the +internal representation of a game resulting from the processing of an +\acrshort{sgf} file. \subsection{Class Analysis} +This section describes the needed classes expected as a result of the analysis +of the project. + \subsubsection{Class Diagram} The classes resulting from the analysis phase are shown in \fref{fig:analysisClasses}. -\begin{figure}[h] +\begin{figure}[p] \begin{center} \includegraphics[width=\textwidth]{diagrams/analysisClasses.png} \caption{General classes obtained from the analysis @@ -416,7 +423,7 @@ The classes resulting from the analysis phase are shown in \tabitem{\textbf{visits}: How many times the node has been visited.} \\ \tabitem{\textbf{score}: The number of explorations of the node resulting in victory.} \\ - \tabitem{\textbf{move}: A GameMove for accessing game state and logic.} \\ + \tabitem{\textbf{move}: A \texttt{GameMove} for accessing game state and logic.} \\ \tabitem{\textbf{parent}: This node's parent in the tree.} \\ \tabitem{\textbf{children}: The nodes following from this node in the tree.} \\ @@ -431,7 +438,7 @@ The classes resulting from the analysis phase are shown in Selects the most promising node which still has some unexplored children.} \\ \tabitem{\textbf{expansion()}: Monte Carlo Tree Search expansion step. Picks - an unexplored vertex from the node and adds it as a new MCTSNode.} \\ + an unexplored vertex from the node and adds it as a new \texttt{MCTSNode}.} \\ \tabitem{\textbf{expansionForCoords()}: Performs an expansion for the given coordinates. This represents forcing a move on the algorithm.} \\ \tabitem{\textbf{simulation()}: Play random matches to accumulate reward @@ -446,17 +453,17 @@ The classes resulting from the analysis phase are shown in \textbf{Keras} \\ \midrule \textbf{Description} \\ - Implements the DecisionAlgorithm interface to give access to a neural - network. \\ + Implements the \texttt{DecisionAlgorithm} interface to give access to a + neural network. \\ \midrule \textbf{Responsibilities} \\ \tabitem{Analyzing game states and generating moves.} \\ \midrule \textbf{Proposed attributes} \\ - \tabitem{\textbf{currentMove}: A GameMove for accessing game state and + \tabitem{\textbf{currentMove}: A \texttt{GameMove} for accessing game state and logic.} \\ - \tabitem{\textbf{neuralNetwork}: A NeuralNetwork instance for generating - moves.} \\ + \tabitem{\textbf{neuralNetwork}: A \texttt{NeuralNetwork} instance for + generating moves.} \\ \midrule \textbf{Proposed methods} \\ \decisionAlgorithmMethods @@ -478,10 +485,10 @@ The classes resulting from the analysis phase are shown in \tabitem{Loading a model file to use an existing trained neural network.} \\ \midrule \textbf{Proposed attributes} \\ - \tabitem{\textbf{currentMove}: A GameMove for accessing game state and + \tabitem{\textbf{currentMove}: A \texttt{GameMove} for accessing game state and logic.} \\ - \tabitem{\textbf{neuralNetwork}: A NeuralNetwork instance for generating - moves.} \\ + \tabitem{\textbf{neuralNetwork}: A \texttt{NeuralNetwork} instance for + generating moves.} \\ \midrule \textbf{Proposed methods} \\ \tabitem{\textbf{pickMove()}: Uses the current internal model to pick a move @@ -654,8 +661,8 @@ The classes resulting from the analysis phase are shown in %TODO: Explain why this is empty \midrule \textbf{Proposed methods} \\ - \tabitem{\textbf{loadGameTree()}: Reads a file and generates a GameMove tree - from its contents.} \\ + \tabitem{\textbf{loadGameTree()}: Reads a file and generates a + \texttt{GameMove} tree from its contents.} \\ \bottomrule \end{tabular} @@ -666,12 +673,13 @@ The classes resulting from the analysis phase are shown in \textbf{Parser} \\ \midrule \textbf{Description} \\ - Reads SGF files and converts them to a tree of GameMove from the Game - System. \\ + Reads \acrshort{sgf} files and converts them to a tree of \texttt{GameMove} + from the Game System. \\ \midrule \textbf{Responsibilities} \\ - \tabitem{Read SGF files.} \\ - \tabitem{Convert the content of the SGF files to a tree of GameMove.} \\ + \tabitem{Read \acrshort{sgf} files.} \\ + \tabitem{Convert the content of the \acrshort{sgf} files to a tree of + \texttt{GameMove}.} \\ \midrule \textbf{Proposed attributes} \\ %TODO: Explain why this is empty @@ -688,19 +696,19 @@ The classes resulting from the analysis phase are shown in \textbf{ASTNode} \\ \midrule \textbf{Description} \\ - Makes up the tree resulting from the parsing of an SGF file.\\ + Makes up the tree resulting from the parsing of an \acrshort{sgf} file.\\ \midrule \textbf{Responsibilities} \\ - \tabitem{Obtain a GameMove tree from itself and its children.} \\ + \tabitem{Obtain a \texttt{GameMove} tree from itself and its children.} \\ \midrule \textbf{Proposed attributes} \\ \tabitem{\textbf{children}: The nodes following from itself.} \\ - \tabitem{\textbf{props}: The properties of the tree read from an SGF file.} + \tabitem{\textbf{props}: The properties of the tree read from an \acrshort{sgf} file.} \\ \midrule \textbf{Proposed methods} \\ - \tabitem{\textbf{toGameTree()}: Returns a GameMove tree corresponding to the - tree following from this node.} \\ + \tabitem{\textbf{toGameTree()}: Returns a \texttt{GameMove} tree + corresponding to the tree following from this node.} \\ \bottomrule \end{tabular} @@ -715,13 +723,14 @@ non-human. \item The human player who interacts with the playing interface. \item The human user who interacts with the engine. - \item A GUI software which uses the engine to generate moves. + \item The human user who starts the training. + \item A \acrshort{gui} software which uses the engine to generate moves. \end{itemize} \subsection{Use Cases} -\begin{figure}[h] +\begin{figure}[p] \begin{center} \includegraphics[width=\textwidth]{diagrams/useCases.png} \caption{Use cases.} @@ -737,20 +746,25 @@ use case is explained next. The game interface reads the moves presented by the player and shows their result on the board. +\paragraph{Generate a move} + +The engine interface reads the input for generating a move as stated by the +\acrshort{gtp} protocol and outputs the coordinates of the board to play. + +\paragraph{Train the engine} + +The training system is started to process matches stored as \acrshort{sgf} files +and to train a neural network. + \paragraph{Use as backend for machine player} The engine is used as the backend for generating moves for a machine player, -this is, for automated play, either against a human who is using the GUI or +this is, for automated play, either against a human who is using the \acrshort{gui} or against another machine player. -\paragraph{Generate a move} - -The engine interface reads the input for generating a move as stated by the -GTP protocol and outputs the coordinates of the board to play. - \subsection{Use Case Analysis and Scenarios} -\begin{figure}[h] +\begin{figure}[p] \begin{center} \includegraphics[width=0.8\textwidth]{diagrams/useCase_playAMatch.png} \caption{Use case: Play a match.} @@ -758,6 +772,8 @@ GTP protocol and outputs the coordinates of the board to play. \end{center} \end{figure} +\vspace{\interclassSpace} + \begin{tabular}{lp{0.6\linewidth}} \toprule \multicolumn{2}{c}{\textbf{Play a match}} \\ @@ -783,7 +799,7 @@ GTP protocol and outputs the coordinates of the board to play. main scenario. \\ \midrule \textbf{Notes} & - This scenario does not pretend to be a complete recreation of a go match. It + This scenario does not pretend to be a complete recreation of a Go match. It will be playable, but its main purpose is to see the Game implementation in action.\newline A robustness diagram for this scenario is shown in @@ -793,7 +809,7 @@ GTP protocol and outputs the coordinates of the board to play. \vspace{\interclassSpace} -\begin{figure}[h] +\begin{figure}[p] \begin{center} \includegraphics[width=\textwidth]{diagrams/useCase_generateAMove.png} \caption{Use case: Generate a move.} @@ -810,7 +826,7 @@ GTP protocol and outputs the coordinates of the board to play. \midrule \textbf{Postconditions} & A move is suggested via the engine output. \\ \midrule - \textbf{Actors} & Human user and GUI program. \\ + \textbf{Actors} & Human user and \acrshort{gui} program. \\ \midrule \textbf{Description} & 1. The user or program enters the player to generate the move @@ -835,7 +851,49 @@ GTP protocol and outputs the coordinates of the board to play. \vspace{\interclassSpace} -\begin{figure}[h] +\begin{figure}[p] + \begin{center} + \includegraphics[width=\textwidth]{diagrams/useCase_trainTheEngine.png} + \caption{Use case: Train the engine.} + \label{fig:useCase_trainTheEngine} + \end{center} +\end{figure} + +\begin{tabular}{lp{0.6\linewidth}} + \toprule + \multicolumn{2}{c}{\textbf{Train the engine}} \\ + \midrule + \textbf{Preconditions} & Some SGF files have been acquired to be used as + training games. \\ + \midrule + \textbf{Postconditions} & A neural network model has been created or updated. \\ + \midrule + \textbf{Actors} & Human user. \\ + \midrule + \textbf{Description} & + 1. The user starts the trainer.\newline + 2. The trainer processes the input files.\newline + 3. The trainer trains a model with the games described in the input + files. \\ + \midrule + \textbf{Secondary scenarios} & + --- \\ + \midrule + \textbf{Exceptions} & + \textbf{No input provided}: An error message is shown. Go back to step 1 of + main scenario. \newline + \textbf{The input is not formatted properly}: An error message is shown. Go + back to step 1 of main scenario. \\ + \midrule + \textbf{Notes} & + A robustness diagram for this scenario is shown in + \fref{fig:useCase_trainTheEngine}.\\ + \bottomrule +\end{tabular} + +\vspace{\interclassSpace} + +\begin{figure}[p] \begin{center} \includegraphics[width=\textwidth]{diagrams/useCase_useAsBackend.png} \caption{Use case: Use as backend for machine player.} @@ -852,7 +910,7 @@ GTP protocol and outputs the coordinates of the board to play. \midrule \textbf{Postconditions} & A match has been played against the engine. \\ \midrule - \textbf{Actors} & GUI program. \\ + \textbf{Actors} & \acrshort{gui} program. \\ \midrule \textbf{Description} & 1. The program gives commands to the engine to set up the game. The @@ -879,14 +937,50 @@ GTP protocol and outputs the coordinates of the board to play. \subsection{Testing Plan Specification} +The Testing Plan will include four types of tests: + +\begin{itemize} + + \item Unitary Testing: for isolated code elements. + \item Integration Testing: for the collaboration between components. + \item System Testing: for the product as a whole. + \item Usability Testing: for the experience of users with the product. + +\end{itemize} + \subsubsection{Unitary Testing} -Tests for the python code are developed using the unittest\cite{python_unittest} -testing framework. It has been chosen by virtue of being thoroughly documented -and widely used. +A unitary testing suite provides test coverage at a base level, checking the +correct behaviour of each specific code piece. + +\subsubsection{Integration Testing} + +The interaction between components will be tested to check their correct +behaviour on the various situations where they have to work together: + +\begin{itemize} +\item The Engine module is able to use the Game module to handle a match. +\item The Training module is able to use the Engine module and its definitions + of neural networks. +\end{itemize} + +\subsubsection{System Testing} + +The working of the system as a whole has to be tested. Mainly: + +\begin{itemize} + + \item A game of Go can be played by a human by using the Game module. + \item A game of Go can be played against the computer by using the + \acrshort{gtp} interface of the Engine module. + \item The \acrshort{gtp} interface is compatible with at least one existing + \acrshort{gui} that claims compatibility with this protocol. + \item A training session can be started and it produces some results to be + evaluated independent of this test step. + +\end{itemize} -The coverage of unit testing is checked with Coverage.py\cite{python_coverage}, -which can by itself run the unittest tests and generate coverage reports based -on the results. +\subsubsection{Usability Testing} -% Maybe put an example report here? +Some people representing the final user will be confronted with the interfaces +of the system to test its usability. |