Reoganises some documentation

doc/latex/.gitignore

 *.aux
+*.glg
 *.glo
 *.gls
 *.glsdefs

doc/latex/CTA_Admin.tex

@@ -26,15 +26,15 @@
 
 % Chapters
 
-\input{CTA_Admin_Introduction.tex}
+\input{CTA_Admin_Intro.tex}
 \input{CTA_Admin_Install.tex}
-\input{CTA_Admin_Configure.tex}
+\input{CTA_Admin_Start.tex}
 \input{CTA_Admin_Admin_Commands.tex}
 \input{CTA_Admin_User_Commands.tex}
 
 \appendix
 
 \input{CTA_Admin_EOS.tex}
-\input{CTA_Admin_MHVTL.tex}
+\input{CTA_Admin_mhVTL.tex}
 
 \end{document}

doc/latex/CTA_Admin_EOS.tex

@@ -39,7 +39,7 @@ xrootd-server-libs-4.4.1-1.el7.x86_64
 \begin{alertbox}
 The instructions below are for machines on structured cabling which have a fixed IP address. Machines without a fixed IP
 address---including virtual machines or Docker pods---are not recognised by the CERN Kerberos servers. To install a local
-EOS instance in this case, we need to run our own KDC server. See Section~\ref{eos-docker} for more information on how to
+EOS instance in this case, we need to run our own KDC server. See Section~\ref{install_eos_in_docker} for more information on how to
 do this.
 \end{alertbox}
 
@@ -286,7 +286,7 @@ Perform a simple write and read test of the new EOS installation:
 \end{lstlisting}
 
 \section{Installing EOS in Docker}
-\label{eos-docker}
+\label{install_eos_in_docker}
 
 The instructions above are for installing EOS on real hardware. To install EOS in a Docker pod, note that you must:
 \begin{enumerate}

doc/latex/CTA_Admin_Install.tex

@@ -2,26 +2,19 @@
 \label{install_cta}
 
 This chapter describes how to install CTA together with a local EOS instance. It is assumed that the target system is
-CERN CentOS 7 (CC7). Installation on SLC6 will not be supported.
-
-Installation follows these steps:
+CERN CentOS 7 (CC7). Installation on SLC6 will not be supported. Installation follows these steps:
 \begin{enumerate}
-\item Configure yum [done]
-\item Install XRoot and EOS [done]
-\item Install the CTA front-end
-\item Install the Tape Server
-\item Configure Objectstore
-\item Configure an Oracle database account
+\item Configure yum (Section~\ref{install_cta_configure_yum})
+\item Install XRoot and EOS (Section~\ref{install_cta_install_xroot_and_eos})
+\item Install CTA RPMs (Section~\ref{install_cta_rpms})
+\item Configure the ObjectStore (Section~\ref{install_cta_configure_objectstore})
+\item Configure the Catalogue (Section~\ref{install_cta_configure_catalogue})
+\item Configure the CTA Front End (Section~\ref{install_cta_configure_front_end})
+\item Install the Tape Server (Section~\ref{install_cta_install_tape_server})
 \end{enumerate}
 
-\begin{alertbox}
-\begin{itemize}
-\item Ceph---how much space?
-\end{itemize}
-\end{alertbox}
-
 \section{Get the CERN Tape Archive Repository}
-\label{get_cta_repo}
+\label{install_cta_get_cta_repo}
 
 \begin{alertbox}
 Some of the files required for installation are held in the CTA git respository. It is necessary to install the CTA repo
@@ -89,14 +82,47 @@ Now install the RPMs:
 \end{lstlisting}
 If EOS has not been installed, follow the instructions in Appendix~\ref{install_eos} to install a local EOS instance.
 
+\section{Install CTA RPMs}
+\label{install_cta_rpms}
+
+To install CTA from previously-built artifacts:
+\begin{lstlisting}
+# yum-config-manager --enable cta-artifacts
+# yum -y install cta-catalogueutils cta-debuginfo cta-doc cta-frontend \
+    cta-objectstore-tools mt-st mtx lsscsi sg3_utils
+# yum clean packages
+\end{lstlisting}
+
+\begin{alertbox}
+When I follow the above instructions installed from pre-built artifacts, some packages were missing. This needs to be
+fixed. For now the workaround is to build these packages from source as described in
+Section~\ref{install_cta_compile_cta_from_source}
+
+\begin{lstlisting}
+# yum-config-manager --enable cta-artifacts
+Loaded plugins: fastestmirror, kernel-module, priorities, protectbase, versionlock
+# yum -y install cta-catalogueutils cta-debuginfo cta-doc cta-frontend \
+    cta-objectstore-tools mt-st mtx lsscsi sg3_utils
+Loaded plugins: changelog, fastestmirror, kernel-module, priorities, protectbase, tsflags, versionlock
+Loading mirror speeds from cached hostfile
+461 packages excluded due to repository priority protections
+0 packages excluded due to repository protections
+No package cta-catalogueutils available.
+No package cta-debuginfo available.
+No package cta-doc available.
+No package cta-frontend available.
+No package cta-objectstore-tools available.
+\end{lstlisting}
+\end{alertbox}
+
 \section{[Optional] Compile CTA from Source}
-\label{compile_cta_from_source}
+\label{install_cta_compile_cta_from_source}
 
-Usually the CTA RPMs will be pre-built as artifacts (skip to Section~\ref{install_cta_rpms}).
+Usually the CTA RPMs will be pre-built as artifacts (see Section~\ref{install_cta_rpms}). If the correct version of the
+required CTA RPM artifacts is not available, they can be built from source as described in this section.
 
-If the correct version of the required CTA RPM artifacts is not available, they can be built from source as described in
-this section. First, build the source RPM of the CTA project by running \texttt{cmake} against the CTA sources that were
-installed in Section~\ref{get_cta_repo}:
+First, build the source RPM of the CTA project by running \texttt{cmake} against the CTA sources that were
+installed in Section~\ref{install_cta_get_cta_repo}:
 \begin{lstlisting}
 # cd ~
 # mkdir CTA_build
@@ -130,30 +156,13 @@ Build the CTA RPMs:
 \begin{lstlisting}
 # make cta_rpm
 \end{lstlisting}
-
-\section{Install CTA RPMs}
-\label{install_cta_rpms}
-
-\begin{alertbox}
-When I installed from pre-built artifacts, some packages were missing (\texttt{cta-objectstore-tools} and
-\texttt{cta-doc}). This needs to be fixed. For now the workaround is to build these packages from source as described in
-Section~\ref{compile_cta_from_source}
-\end{alertbox}
-
-To install CTA from previously-built artifacts:
-\begin{lstlisting}
-# yum-config-manager --enable cta-artifacts
-# yum -y install cta-catalogueutils cta-debuginfo cta-doc cta-frontend \
-    cta-objectstore-tools mt-st mtx lsscsi sg3_utils
-# yum clean packages
-\end{lstlisting}
-
-To install CTA from RPMs built in Section~\ref{compile_cta_from_source}:
+Install the CTA RPMs:
 \begin{lstlisting}
 # yum -y install RPM/RPMS/x86_64/cta-*
 \end{lstlisting}
 
-\section{Configure ObjectStore}
+\section{Configure the ObjectStore}
+\label{install_cta_configure_objectstore}
 
 \subsection{Configuration for a test system}
 In a test system, configure the ObjectStore as a Virtual File System (VFS). Initialise the new ObjectStore VFS and set
@@ -175,6 +184,11 @@ Then create environment variables to store the type and location of the new Obje
 %\end{lstlisting}
 
 \subsection{Configuration for a production system}
+\begin{alertbox}
+\begin{itemize}
+\item Ceph---how much space?
+\end{itemize}
+\end{alertbox}
 In a production system, configure the ObjectStore to use Ceph. To install Ceph:
 \begin{lstlisting}
 # yum-config-manager --enable ceph
@@ -217,6 +231,12 @@ Delete the ObjectStore content:
 \end{lstlisting}
 
 \section{Configure the Catalogue}
+\label{install_cta_configure_catalogue}
+
+Configure an Oracle database account
+
+\subsection{11/ (DOC) Describe what Oracle credentials are needed.}
+Where to get them and is there an alternative (SQLite)?
 
 \subsection{2/ Config file for the DB:}
 \begin{verbatim}
@@ -246,7 +266,9 @@ oracle:cta_devdb5/PASSWORD@devdb12
   cta-catalogue-schema-create /etc/cta/cta_catalogue_db.conf
 \end{verbatim}
 
-\section{Configure CTA Front-end}
+
+\section{Configure the CTA Front End}
+\label{install_cta_configure_front_end}
 
 Create the \texttt{/etc/cta/cta-frontend.conf} file with the following contents:
 
@@ -261,15 +283,18 @@ Create the CTA user:
 # useradd cta
 \end{lstlisting}
 
-\section{Configure Authentication}
+\subsection{Configure Authentication}
 
 Then there is the voodoo dance with SSS, which should be explained.
 
-\section{Install CTA Front-end}
+\subsection{Install CTA Front End}
 
 Installing RPMs doesn't install cta-frontend. That is done in the script:
 
-https://gitlab.cern.ch/cta/CTA/blob/master/continuousintegration/docker/ctafrontend/cc7/opt/run/bin/ctafrontend.sh
+\begin{lstlisting}
+https://gitlab.cern.ch/cta/CTA/blob/master/continuousintegration/docker/
+ctafrontend/cc7/opt/run/bin/ctafrontend.sh
+\end{lstlisting}
 
 \subsection{Kubernetes ctafrontend.sh}
 
@@ -327,6 +352,12 @@ echo "frontend died"
 sleep infinity
 \end{lstlisting}
 
+\section{Install the Tape Server}
+\label{install_cta_install_tape_server}
+
+If the CTA instance is not using real tape hardware, follow the directions in Appendix~\ref{install_mhvtl} to install the
+mhVTL Virtual Tape Library.
+
 \section{Removing CTA}
 
 To remove the CTA Front-end and associated packages:
@@ -354,60 +385,53 @@ CTA configuration files sometimes use underscore\\
 and sometimes use hyphen (\texttt{/etc/cta/cta-frontend.conf}).
 \end{alertbox}
 
+\subsection{Describe properly RPM dependencies (in the .spec files).}
+\begin{alertbox}
+\begin{itemize}
+\item protobuf3 package is needed
+\item Ceph packages are needed
+\item oracle-instantclient12.1-meta is needed (has Provides declarations)
+\end{itemize}
+\end{alertbox}
 
-\subsection{2/ Describe properly RPM dependencies (in the .spec files).}
- - protobuf3 package is needed
- - Ceph packages are needed
- - oracle-instantclient12.1-meta is needed (has Provides declarations)
+\subsection{ctacli.sh contains comment: <FQDN>:<TCPPort> yet the value mentioned does not have the domain specified.}
 
-\subsection{4/ ctacli.sh contains comment: <FQDN>:<TCPPort> yet the value mentioned does not have the domain specified.}
+\begin{alertbox}
 Can aliases (in general) be used (such as: ctafrontend.cern.ch)? In which circumstances YES and NO?
+\end{alertbox}
+
+\subsection{Unify/Agree on the name of XROOTD configuration files.}
 
-\subsection{7/ Unify/Agree on the name of XROOTD configuration files.}
+\begin{alertbox}
 Either call them /etc/xrd.cf.*
 or /etc/xrootd/<function>.cfg (I prefer /etc/xrootd structure even if different from
 IT-DSS-FDO).
+\end{alertbox}
 
-\subsection{8/ PLEASE make sure the 'status=' string is ALWAYS on the same position in the cta-taped logs.}
+\subsection{PLEASE make sure the 'status=' string is ALWAYS on the same position in the cta-taped logs.}
+
+\begin{alertbox}
 Either at the end or immediately after the 'MSG=' key. Example of 3 (!!!) different positions
 within ~2 hours:
 
-\begin{verbatim}
+\begin{lstlisting}
 2017-04-13T14:32:59.548438+02:00 cta-taped[250]: LVL="Info" TID="250" MSG="Tape session finished" TPVID="I08155" mountType="Retrieve" mountTime="21.423454" positionTime="8.541020" waitInstructionsTime="0.141429" waitFreeMemoryTime="0.000004" waitDataTime="0.000000" waitReportingTime="0.000427" checksumingTime="0.000000" readWriteTime="0.010911" flushTime="0.000000" unloadTime="23.826163" unmountTime="5.488744" encryptionControlTime="0.003776" transferTime="0.152771" totalTime="59.295118" deliveryTime="30.183826" drainingTime="0.000000" dataVolume="420" filesCount="1" headerVolume="480" payloadTransferSpeedMBps="0.000007" driveTransferSpeedMBps="0.000015" status="success"
 2017-04-13T14:43:56.399355+02:00 cta-taped[250]: LVL="Info" TID="250" MSG="Tape session finished" TPVID="I08155" mountType="Archive" status="success" mountTime="15.523610" positionTime="8.509446" waitInstructionsTime="0.151270" waitFreeMemoryTime="0.000000" waitDataTime="0.003547" waitReportingTime="0.000239" checksumingTime="0.912911" readWriteTime="7.656642" flushTime="10.106426" unloadTime="34.299974" unmountTime="5.494463" encryptionControlTime="0.004026" transferTime="18.831035" totalTime="82.511620" deliveryTime="82.511620" drainingTime="0.000000" dataVolume="1048576000" filesCount="1" headerVolume="480" payloadTransferSpeedMBps="12.708222" driveTransferSpeedMBps="12.708228"
 2017-04-13T15:56:48.411636+02:00 cta-taped[250]: LVL="Info" TID="250" MSG="Tape session finished" TPVID="I08155" mountType="Archive" Error_tapeUnload="1" Error_tapeMountForWrite="1" status="failure" mountTime="0.000000" positionTime="0.000000" waitInstructionsTime="0.152993" waitFreeMemoryTime="0.000000" waitDataTime="0.000000" waitReportingTime="0.000000" checksumingTime="0.000000" readWriteTime="0.000000" flushTime="0.000000" unloadTime="0.000000" unmountTime="0.000000" encryptionControlTime="0.593565" transferTime="0.152993" totalTime="61.499234" deliveryTime="61.499234" drainingTime="0.000000" dataVolume="0" filesCount="0" headerVolume="0" payloadTransferSpeedMBps="0.000000" driveTransferSpeedMBps="0.000000"
-\end{verbatim}
+\end{lstlisting}
 
 This REALLY helps a human to read the logs! I would propose to simply order
 all keys alphabetically and that's it.
+\end{alertbox}
 
-\subsection{11/ (DOC) Describe what Oracle credentials are needed.}
-Where to get them and is there an alternative (SQLite)?
-
-\subsection{12/ (DOC) What users should be created on:}
- a/ EOS machines (stage:st ?)
- b/ CTA frontend machine (eos:XX)?
- c/ tape servers (nothing special?)
+\subsection{What users should be created?}
+\begin{alertbox}
+What users should be created on:
+\begin{enumerate}
+\item EOS machines (stage:st ?)
+\item CTA frontend machine (eos:XX)?
+\item tape servers (nothing special?)
+\end{enumerate}
 Please describe.
+\end{alertbox}
 
-\subsection{13/ (DOC) Documentation should clearly separate various functions of individual servers.}
-
-\subsection{14/ (DOC) Documentation should explain why three SSS are needed. Picture explaining where and how they are used would be nice.}
-
-\subsection{17/ (DOC) How to setup an MHVTL library should be an appendinx for those who do not have a real one.}
-Otherwise it is confusing as it is not explain what is part of CTA and what is external.
-
-\subsection{18/ (DOC) Documentation should contain "Hello world" example.}
-That is how to migrate one files to tape and how to get it back.
-This should include:
- - declaration of a library in CTA catalogue
- - declaration of tape pools and a tape
- - puting file into EOS
- - checking that the file is there
- - list pending archives
- - see tape mounted (like castor "tpstat" command)
- - show the "m" bit to confirm the file is on tape
- - remove disk copy on EOS
- - retrieve file from tape
- - see tape mounted (like castor "tpstat" command)
- - show a disk copy of a file somewhere on disk (example: <FST server:/<FST path>/00000000/00000007)

doc/latex/CTA_Admin_Intro.tex

+\chapter{Introduction}
+
+The CERN Tape Archive (CTA) is a tape storage back-end to EOS.
+
+This document is a guide for administrators to install and administer a CTA system.
+
+\section{CTA Overview}
+
+\begin{alertbox}
+Documentation should explain and separate the various functions of individual servers:
+\begin{itemize}
+\item EOS
+\item CTA FrontEnd
+\item ObjectStore
+\item Database
+\item Tape Server
+\end{itemize}
+\end{alertbox}
+
+\section{Authentication between CTA Servers}
+
+\begin{alertbox}
+Documentation should explain why three SSS are needed. Picture explaining where and how they are used would be nice.
+\end{alertbox}
+

doc/latex/CTA_Admin_Introduction.tex

-\chapter{Introduction}
-
-The CERN Tape Archive (CTA) is a tape storage back-end to EOS.
-
-This document is a guide for administrators to install and administer a CTA system.
-

doc/latex/CTA_Admin_Configure.tex → doc/latex/CTA_Admin_Start.tex

-\chapter{Configure CTA}
-\label{configure_cta}
+\chapter{Start CTA}
+\label{start_cta}
+
+\begin{alertbox}
+Documentation should contain "Hello world" example:
+
+That is how to migrate one files to tape and how to get it back.
+This should include:
+\begin{itemize}
+\item declaration of a library in CTA catalogue
+\item declaration of tape pools and a tape
+\item puting file into EOS
+\item checking that the file is there
+\item list pending archives
+\item see tape mounted (like castor "tpstat" command)
+\item show the "m" bit to confirm the file is on tape
+\item remove disk copy on EOS
+\item retrieve file from tape
+\item see tape mounted (like castor "tpstat" command)
+\item show a disk copy of a file somewhere on disk (example: <FST server:/<FST path>/00000000/00000007)
+\end{itemize}
+\end{alertbox}
 
 \begin{enumerate}
 \item Configure an EOS workflow to allow EOS to talk to the CTA front-end

doc/latex/CTA_Admin_MHVTL.tex → doc/latex/CTA_Admin_mhVTL.tex

-\chapter{Install the Virtual Tape Library (MHVTL)}
+\chapter{Install the Virtual Tape Library (mhVTL)}
 \label{install_mhvtl}
 
+Normally CTA will be used with physical Tape Libraries. If a physical Tape Library is not available---for example during
+development and testing---a Virtual Tape Library (VTL) can be used instead. A VTL is a data storage virtualization
+technology which presents a storage component (usually hard disk storage) as tape libraries or tape drives.
+
+This Appendix describes how to install Mark Harvey's Virtual Tape Library \href{http://www.mhvtl.com/}{mhVTL}.
+
+\section{Installing mhVTL}
+
 \begin{lstlisting}
-echo Installing mhvtl
 sudo yum install -y mhvtl-utils kmod-mhvtl mhvtl-utils --enablerepo=castor
 \end{lstlisting}