Added library dependancies and first sequence diagrams.

The doc compiles with a recent TeXlive, but probably needs importation of packages to compile in SLC5/6...

Documentation/ProgrammersManual.tex

@@ -28,14 +28,60 @@ plan is being laid out.
 
 CERN SLC5 and SLC6, 64bits. Although it should compile in theory, the 32 bits version is not tested. The unit test purposely returns an error when run on non-64 bits architecture.
 
-\subsection{User requirements}
+\subsection{Pre-existing requirements}
 
 The new tape server (software) will have to replace the software running on a tape server
-(computer). Specifically, it will have to interface with VDQM, VMGR and tape gateway.
-
+(computer). A previous analysis describes the current software stack of the tape servers
+	\footnote{ \href{http://svn.cern.ch/guest/CASTOR/CASTOR2/trunk/castor/tape/doc/TapeBridge.pdf}
+	{http://svn.cern.ch/guest/CASTOR/CASTOR2/trunk/castor/tape/doc/TapeBridge.pdf}}.
+This new tape server will retain the same external interfaces as the old tape server, replacing 
+the stack of daemons from the tape bridge down to the tape drive hardware.
+
+The tape server will interface with the Volume and Drive Queue Manager daemon (vdqmd), the Volume
+manager daemon (vmgrd), the Castor User Privilege Validation daemon (cupvd) and tape gateway 
+daemon (tapegatewayd) for data transfer management and access control.
+
+It will connect to the CAStor disk servers to transfer the data itself, using one of the supported
+protocols (current candidates are rfio, xroot and ceph) and it will use the services of the Remote 
+Media Changer daemon (rmcd) to mount and unmount tapes.
+
+\subsection{Collaboration diagrams and sequences}
+The collaboration diagrams of the previous version of the tape server (with all its sub components)
+can be found in dot format
+	\footnote{ \href{http://svn.cern.ch/guest/CASTOR/CASTOR2/trunk/castor/tape/doc/collaboration\_diagrams/}
+	{http://svn.cern.ch/guest/CASTOR/CASTOR2/trunk/castor/tape/doc/collaboration\_diagrams/}}.
+	
+The new sequencing of a session start, simplified from the internal component communication is shown here:
+
+\begin{sequencediagram}
+	\newthread{gw}{tapegatewayd}
+	\newthread{vm}{vmgrd}
+	\newthread{vd}{vdqmd}
+	\newthread{ts}{tapeserverd(main)}
+	
+	\begin{call}{gw}{get volume}{vm}{tape VID}
+	\end{call}
+	\begin{call}{gw}{get drive}{vd}{VDQM request ID}
+	\end{call}
+	\begin{call}{vd}{schedule}{vd}{}
+	\end{call}
+	\begin{call}{vd}{client info}{ts}{}
+	\end{call}
+\end{sequencediagram}
+
+TODO: VDQM status reporting sequencing
+
+TODO: Mount time checks
+
+TODO: GW protocol
+
+\subsection{Extra requirements}
 Additional requirements, arising from the current practices of operators are:
 
 \begin{itemize}
+\item{}The tape server's session should gracefully handle an unclean situation where the 
+tape is left in the drive by a previously crashed session. The protocol is to clean anything
+left over before proceeding to the new session.
 \item{}A tape sessions should be preemptable by the operator. This is currently achieved 
 by killing the tape process. Closing the session on a kill (-1) could be a solution.
 \item{}The operator should be able to specify values in different SCSI code pages in
@@ -43,6 +89,41 @@ order to setup the tape drive. This setting will be defined differently for each
 drive type.
 \end{itemize}
 
+\section{Tape server architecture}
+To fulfil the requirement for an ability to kill a session, the main tape server daemon
+will be simple, and just report its status to the VDQM and wait for requests from it on
+an open port.
+
+When a session should start, the process will fork a child process, which will reserve the memory
+and instantiate the session machinery.
+
+The layout of the main thread is show in figure \ref{tsParentProcess}. The layout of the child process, which contains all the complexity is shown in figure \ref{tsChildProcess}.
+
+\begin{figure}[h]
+\begin{center}
+\includegraphics{images/TapeServerParentProcess}
+\end{center}
+\caption{\label{tsParentProcess}Tape server parent process: libraries used (purpose built libraries in blue, system libraries in beige, already existing CAStor libraries in green)}
+\end{figure}
+
+
+\begin{figure}[h]
+\begin{center}
+\includegraphics{images/TapeServerChildProcess}
+\end{center}
+\caption{\label{tsChildProcess}Tape server child process: libraries used (purpose built libraries in blue, system libraries in beige, already existing CAStor libraries in green)}
+\end{figure}
+
+The data path will go to/from tape drive, through the generic SCSI interface of st driver (CAStor uses a
+mixture of both in the Tape::Drive class), then through the File structure support class, as controlled by
+the tape thread. The tape thread will communicate the data to (or get from) the disk thread via the 
+data FIFO class. This class will in turn allocate the memory from a preallocated, pool of
+fixed sized blocks. The size of the pool will be controlled by the operators.
+
+Some libraries already exist in CAStor, and will be reused, either by copying or linking from pre-compiled 
+packages. The main parts of the sessions spawner will be taken from the VDQM as well.
+
+
 % Section:  Reference documentations
 % ------------------------------------------------------------------------------
 

Documentation/TapeServer.tex

@@ -3,7 +3,7 @@
 % For clickable links and good readability using dvipdf
 \usepackage{times}
 \usepackage{color}
-\usepackage[ps2pdf,colorlinks=true,linkcolor=webred]{hyperref}
+\usepackage[colorlinks=true,linkcolor=blue,urlcolor=blue,filecolor=blue]{hyperref}
 \usepackage{courier}
 \usepackage{listings}
 \usepackage{float}
@@ -25,6 +25,10 @@
 \usepackage{makeidx}
 \makeindex
 \usepackage{tabularx}
+\usepackage{pgf}
+\usepackage{tikz}
+\usetikzlibrary{arrows,shadows} % for pgf-umlsd
+\usepackage[underline=true,rounded corners=false]{pgf-umlsd}
 
 \begin{document}
 \doctyp{CAStor Tape Server Documentation}

Documentation/TapeServer.toc

+\contentsline {chapter}{Preface}{4}{chapter*.2}
+\contentsline {chapter}{\numberline {1}Developer's manual}{5}{chapter.1}
+\contentsline {section}{\numberline {1.1}Requirements}{5}{section.1.1}
+\contentsline {subsection}{\numberline {1.1.1}Targeted environment}{5}{subsection.1.1.1}
+\contentsline {subsection}{\numberline {1.1.2}Pre-existing requirements}{5}{subsection.1.1.2}
+\contentsline {subsection}{\numberline {1.1.3}Collaboration diagrams and sequences}{5}{subsection.1.1.3}
+\contentsline {subsection}{\numberline {1.1.4}Extra requirements}{6}{subsection.1.1.4}
+\contentsline {section}{\numberline {1.2}Tape server architecture}{6}{section.1.2}
+\contentsline {section}{\numberline {1.3}Reference documentations}{7}{section.1.3}
+\contentsline {subsection}{\numberline {1.3.1}SCSI specifications}{7}{subsection.1.3.1}
+\contentsline {subsubsection}{\numberline {1.3.1.1}Manufacturer's specificities}{7}{subsubsection.1.3.1.1}
+\contentsline {subsection}{\numberline {1.3.2}SCSI support in Linux}{7}{subsection.1.3.2}
+\contentsline {subsection}{\numberline {1.3.3}Unsorted CAStor docs}{8}{subsection.1.3.3}
+\contentsline {subsection}{\numberline {1.3.4}SCSI tape support in Linux (st driver)}{8}{subsection.1.3.4}
+\contentsline {section}{\numberline {1.4}Tools used during development}{9}{section.1.4}
+\contentsline {subsection}{\numberline {1.4.1}Required tools for build}{9}{subsection.1.4.1}
+\contentsline {subsection}{\numberline {1.4.2}Tools used during development}{9}{subsection.1.4.2}
+\contentsline {subsection}{\numberline {1.4.3}Code coverage using lcov}{9}{subsection.1.4.3}
+\contentsline {section}{\numberline {1.5}Software layout}{10}{section.1.5}
+\contentsline {subsection}{\numberline {1.5.1}SCSI structures, constants and endianness}{10}{subsection.1.5.1}
+\contentsline {subsection}{\numberline {1.5.2}Exceptions hierarchy and error handling strategy}{11}{subsection.1.5.2}
+\contentsline {subsection}{\numberline {1.5.3}Non-fatal warnings strategy}{11}{subsection.1.5.3}
+\contentsline {subsection}{\numberline {1.5.4}The Tape::Drive object}{13}{subsection.1.5.4}
+\contentsline {subsection}{\numberline {1.5.5}The Tape::File class}{14}{subsection.1.5.5}
+\contentsline {subsubsection}{\numberline {1.5.5.1}CAStor file format}{14}{subsubsection.1.5.5.1}
+\contentsline {subsubsection}{\numberline {1.5.5.2}File block management}{19}{subsubsection.1.5.5.2}
+\contentsline {subsubsection}{\numberline {1.5.5.3}Responsibilities of the class}{19}{subsubsection.1.5.5.3}
+\contentsline {subsubsection}{\numberline {1.5.5.4}Checksums}{19}{subsubsection.1.5.5.4}
+\contentsline {subsubsection}{\numberline {1.5.5.5}Tape::File API}{20}{subsubsection.1.5.5.5}
+\contentsline {subsection}{\numberline {1.5.6}Memory chunk manager}{20}{subsection.1.5.6}
+\contentsline {subsection}{\numberline {1.5.7}FIFOs}{20}{subsection.1.5.7}
+\contentsline {subsection}{\numberline {1.5.8}Disk client library}{20}{subsection.1.5.8}
+\contentsline {subsection}{\numberline {1.5.9}VDQM client library}{20}{subsection.1.5.9}
+\contentsline {subsection}{\numberline {1.5.10}VMGR client library}{20}{subsection.1.5.10}
+\contentsline {subsection}{\numberline {1.5.11}Stager/TapeGateway client library}{20}{subsection.1.5.11}
+\contentsline {subsection}{\numberline {1.5.12}Logging system client library}{20}{subsection.1.5.12}
+\contentsline {subsection}{\numberline {1.5.13}Application architecture}{20}{subsection.1.5.13}
+\contentsline {subsubsection}{\numberline {1.5.13.1}Session spawner}{20}{subsubsection.1.5.13.1}
+\contentsline {subsubsection}{\numberline {1.5.13.2}Session process}{20}{subsubsection.1.5.13.2}
+\contentsline {subsubsection}{\numberline {1.5.13.3}Tape read thread}{20}{subsubsection.1.5.13.3}
+\contentsline {subsubsection}{\numberline {1.5.13.4}Tape write thread}{20}{subsubsection.1.5.13.4}
+\contentsline {subsubsection}{\numberline {1.5.13.5}Disk read thread}{20}{subsubsection.1.5.13.5}
+\contentsline {subsubsection}{\numberline {1.5.13.6}Disk write thread}{20}{subsubsection.1.5.13.6}
+\contentsline {chapter}{\numberline {2}Administrator's manual}{21}{chapter.2}
+\contentsline {section}{\numberline {2.1}Pending questions}{21}{section.2.1}