Skip to content
Snippets Groups Projects
Commit d1212389 authored by mdavis's avatar mdavis
Browse files

Adds proof of concept for generating PDF+HTML documentation

parent 929faa29
No related branches found
No related tags found
No related merge requests found
html/
latex/
*.md
This diff is collapsed.
#!/bin/sh
#
# Build PDF + HTML documentation from doxygen sources
#
# Proof-of-concept: For production, convert it to CMake
#
# Required packages:
#
# doxygen-latex
# graphviz
# Convert .md for Doxygen use
for f in $(find .. -type f -name '*.md')
do
cp $f z_${RANDOM}_$(basename $f)
done
doxygen
cd latex
pdflatex refman.tex
pdflatex refman.tex
mv refman.pdf ../cta.pdf
File added
/*!
\author German Cancio, Eric Cano, Daniele Kruse and Steven Murray
\mainpage Introduction
The main objective of the CTA project is to develop a prototype tape archive system that transfers files directly between remote disk storage systems and tape drives. The concrete remote storage system of choice is EOS.
The data and storage services (DSS) group of the CERN IT department currently provides a tape archive service. This service is implemented by the hierarchical storage management (HSM) system named the CERN advanced storage manager (CASTOR). This HSM has an internal disk-based storage area that acts as a staging area for tape drives. Until now this staging area has been a vital component of CASTOR. It has provided the necessary buffer between the multi-stream, block-oriented disk drives of end users and the single-stream, file-oriented tape drives of the central tape system. Assuming the absence of a sophisticated disk to tape scheduling system, at any single point in time a disk drive will be required to service multiple data streams whereas a tape drive will only ever have to handle a single stream. This means that a tape stream will be at least one order of magnitude faster than a disk stream. With the advent of disk storage solutions that stripe single files over multiple disk servers, the need for a tape archive system to have an internal disk-based staging area has become redundant. Having a file striped over multiple disk servers means that all of these disk-servers can used in parallel to transfer that file to a tape drive, hence using multiple disk-drive streams to service a single tape stream.
The CTA project is a prototype for a very good reason. The DSS group needs to investigate and learn what it means to provide a tape archive service that does not have its own internal disk-based staging area. The project also needs to keep its options open in order to give the DSS group the best opportunities to identify the best ways forward for reducing application complexity, easing code maintenance, reducing operation overheads and improving tape efficiency.
The CTA project currently has no constraints that go against collecting a global view of all tape , drive and user request states. This means the CTA project should be able to implement intuitive and effective tape scheduling policies. For example it should be possible to schedule a tape archive mount at the point in time when there is both a free drive and a free tape. The architecture of the CASTOR system does not facilitate such simple solutions due to its history of having separate staging areas per experiment and dividing the mount scheduling problem between these separate staging areas and the central tape system responsible for issuing tape mount requests for all experiments.
*/
/*!
\page CTA basic concepts
CTA is operated by authorized administrators (AdminUsers) who issue CTA commands from authorized machines (AdminHosts), using the CTA command line interface. All administrative metadata (such as tape, tape pools, storage classes, etc..) is tagged with a "creationLog" and a "lastModificationLog" which say who/when/where created them and last modified them. An administrator may create ("add"), delete ("rm"), change ("ch") or list ("ls") any of the administrative metadata.
Tape pools are logical groupings of tapes that are used by operators to separate data belonging to different VOs. They are also used to categorize types of data and to separate multiple copies of files so that they end up in different buildings. Each tape belongs to one and only one tape pool.
Logical libraries are the concept that is used to link tapes and drives together. We use logical libraries to specify which tapes are mountable into which drives, and normally this mountability criteria is based on location, that is the tape has to be in the same physical library as the drive, and on read/write compatibility. Each tape and each drive has one and only one logical library.
The storage class is what we assign to each archive file to specify how many tape copies the file is expected to have. Archive routes link storage classes to tape pools. An archive route specifies onto which set of tapes the copies will be written. There is an archive route for each copy in each storage class, and normally there should be a single archive route per tape pool.
So to summarize, an archive file has a storage class that determines how many copies on tape that file should have. A storage class has an archive route per tape copy to specify into which tape pool each copy goes. Each tape tool is made of a disjoint set of tapes. And tapes can be mounted on drives that are in their same logical library.
\subsection{Archiving a file with CTA}
CTA has a CLI for archiving and retrieving files to/from tape, that is meant to be used by an external disk-based storage system with an archiving workflow engine such as EOS. A non-administrative "User" in CTA is an EOS user which triggers the need for archiving or retrieving a file to/from tape. A user normally belongs to a specific CTA "mount group", which specifies the mount criteria and limitations (together called "mount policy") that trigger a tape mount. Here we offer a simplified description of the archive process:
\begin{enumerate}
\item EOS issues an archive command for a specific file, providing its source path, its storage class (see above), and the user requesting the archival
\item CTA returns immediately an "ArchiveFileID" which is used by CTA to uniquely identify files archived on tape. This ID will be kept by EOS for any operations on this file (such as retrieval)
\item Asynchronosly, CTA carries out the archival of the file to tape, in the following steps:
\begin{itemize}
\item CTA looks up the storage class provided by EOS and makes sure it has correct routings to one or more tape pools (more than one when multiple copies are required by the storage class)
\item CTA queues the corresponding archive job(s) to the proper tape pool(s)
\item in the meantime each free tape drive queries the central "scheduler" for work to be done, by communicating its name and its logical library
\item for each work request CTA checks whether there is a free tape in the required pool (as specified in b.), that belongs to the desired logical library (as specified in c.)
\item if that is the case, CTA checks whether the work queued for that tape pool is worth a mount, i.e. if it meets the archive criteria specified in the mount group to which the user (as specified in 1.) belongs
\item if that is the case, the tape is mounted in the drive and the file gets written from the source path specified in 1. to the tape
\item after a successful archival CTA notifies EOS through an asynchronous callback
\end{itemize}
\end{enumerate}
An archival process can be canceled at any moment (even after correct archival, but in this case it's a "delete") through the "delete archive" command
\subsection{Retrieving a file with CTA}
Here we offer a simplified description of the retrieve process:
\begin{enumerate}
\item EOS issues a retrieve command for a specific file, providing its ArchiveFileID and desired destination path, and the user requesting the retrieval
\item CTA returns immediately
\item Asynchronosly, CTA carries out the retrieval of the file from tape, in the following steps:
\begin{itemize}
\item CTA queues the corresponding retrieve job(s) to the proper tape(s) (depending on where the tape copies are located)
\item in the meantime each free tape drive queries the central "scheduler" for work to be done, by communicating its name and its logical library
\item for each work request CTA checks whether the logical library (as specified in b.) is the same of (one of) the tape(s) (as specified in a.)
\item if that is the case, CTA checks whether the work queued for that tape is worth the mount, i.e. if it meets the retrieve criteria specified in the mount group to which the user (as specified in 1.) belongs
\item if that is the case, the tape is mounted in the drive and the file gets read from tape to the destination specified in 1.
\item after a successful retrieval CTA notifies EOS through an asynchronous callback
\end{itemize}
\end{enumerate}
A retrieval process can be canceled at any moment prior to correct retrieval through the "cancel retrieve" command
*/
/*!
\page EOS-CTA Authorization Guidelines
One of the requirements of CTA is to limit the crosstalk among different EOS
instances. In more detail:
\begin{enumerate}
\item A listStorageClass command should return the list of storage classes
belonging to the instance from where the command was executed only
\item A queueArchive command should be authorized only if:
\begin{itemize}
\item the instance provided in the command line coincides with the instance from
where the command was executed
\item the storage class provided in the command line belongs to the instance from
where the command was executed
\item the EOS username and/or group (of the original archive requester) provided
in the command line belongs to the instance from where the command was
executed
\end{itemize}
\item A queueRetrieve command should be authorized only if:
\begin{itemize}
\item the instance of the requested file coincides with the instance from where
the command was executed
\item the EOS username and/or group (of the original retrieve requester) provided
in the command line belongs to the instance from where the command was
executed
\end{itemize}
\item A deleteArchive command should be authorized only if:
\begin{itemize}
\item the instance of the file to be deleted coincides with the instance from
where the command was executed
\item the EOS username and/or group (of the original delete requester) provided
in the command line belongs to the instance from where the command was
executed
\end{itemize}
\item A cancelRetrieve command should be authorized only if:
\begin{itemize}
\item the instance of the file to be canceled coincides with the instance from
where the command was executed
\item the EOS username and/or group (of the original cancel requester) provided
in the command line belongs to the instance from where the command was
executed
\end{itemize}
\item An updateFileStorageClass command should be authorized only if:
\begin{itemize}
\item the instance of the file to be updated coincides with the instance from
where the command was executed
\item the storage class provided in the command line belongs to the instance from
where the command was executed
\item the EOS username and/or group (of the original update requester) provided
in the command line belongs to the instance from where the command was
executed
\end{itemize}
\item An updateFileInfo command should be authorized only if:
\begin{itemize}
\item the instance of the file to be updated coincides with the instance from
where the command was executed
\end{itemize}
\end{enumerate}
*/
/*!
\page CTA-EOS Reconciliation Strategy
\subsection{Reconciling EOS file info and CTA disk file info}
This should be the most common scenario causing discrepancies between the EOS namespace and the disk file info within
the CTA catalogue. The proposal is to attack this in two ways: first (already done) we piggyback disk file info on most
commands acting on CTA Archive files ("archive", "retrieve", "cancelretrieve", etc.), second (to be agreed with Andreas)
EOS could have a trigger on file renames or other file information changes (owner, group, path, etc.) that calls our
updatefileinfo command with the updated fields. In addition (also to be agreed with Andreas) there should also be a
separate low priority process (a sort of EOS-side reconciliation process) going through the entire EOS namespace
periodically calling updatefileinfo on each of the known files, we would also store the date when this update function
was called (see below to know why).
\subsection{Reconciling EOS deletes which haven't been propagated to CTA}
Say that the above EOS-side low-priority reconciliation process takes on average 3 months and it is run continuously. We
could use the last reconciliation date to determine the list of possible candidates of files which EOS does not know
about anymore, by just taking the ones which haven't been updated say in the last 6 months. Since we have the EOS
instance name and EOS file id for each file (and Andreas confirmed that IDs are unique and never reused within a single
instance), we can then automatically check (through our own CTA-side reconciliation process) whether indeed these files
exist or not. For the ones that still exist we notify EOS admins for a possible bug in their reconciliation process and
we ask them to issue the updatefileinfo command, for the ones which don't exist anymore we double check with their
owners before deleting them from CTA.
Note: It's important to note that we do not reconcile storage class information. Any storage class change is triggered
by the EOS user and it is synchronous: once we successfully record the change our command returns.
*/
/*!
\page CTA-EOS command line interface
EOS communicates with CTA by issuing commands on trusted hosts. EOS can archive a file, retrieve it, update its
information/storage class, delete it or simply list the available storage classes. See the LimitingInstanceCrosstalk.txt
file for more details on how these commands are authorized by CTA.
\subsection{ARCHIVING from EOS to CTA}
\begin{verbatim}
1) EOS REQUEST: cta a/archive
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--user <user> // string name of the requester of the action (archival),
// used for SLAs and logging,
// not kept by CTA after successful operation
--group <group> // string group of the requester of the action (archival),
// used for SLAs and logging,
// not kept by CTA after successful operation
--diskid <disk_id> // string disk id of the file to be archived,
// kept by CTA for reconciliation purposes
--instance <instance> // string kept by CTA for authorizing the request
// and for disaster recovery
--srcurl <src_URL> // string source URL of the file to archive of
// the form scheme://host:port/opaque_part,
// not kept by CTA after successful archival
--size <size> // uint64_t size in bytes kept by CTA for
// correct archival and disaster recovery
--checksumtype <checksum_type> // string checksum type (ex. ADLER32) kept by CTA
// for correct archival and disaster recovery
--checksumvalue <checksum_value> // string checksum value kept by CTA for correct
// archival and disaster recovery
--storageclass <storage_class> // string that determines how many copies and
// which tape pools will be used for archival
// kept by CTA for routing and authorization
--diskfilepath <disk_filepath> // string the disk logical path kept by CTA
// for disaster recovery and for logging
--diskfileowner <disk_fileowner> // string owner username kept by CTA
// for disaster recovery and for logging
--diskfilegroup <disk_filegroup> // string owner group kept by CTA
// for disaster recovery and for logging
--recoveryblob <recovery_blob> // 2KB string kept by CTA for disaster recovery
// (opaque string controlled by EOS)
--diskpool <diskpool_name> // string used (and possibly kept)
// by CTA for proper drive allocation
--throughput <diskpool_throughput> // uint64_t (in bytes) used (and possibly kept)
// by CTA for proper drive allocation
2) CTA IMMEDIATE REPLY: CTA_ArchiveFileID or Error
CTA_ArchiveFileID: string which is the unique ID of the CTA file to be kept by EOS
while file exists (for future retrievals). In case of retries,
a new ID will be given by CTA (as if it was a new file),
the old one can be discarded by EOS.
3) CTA CALLBACK WHEN ARCHIVED SUCCESSFULLY: src_URL and copy_number with or without Error
src_URL: this is the same string provided in the EOS archival request
copy_number: indicates which copy number was archived
note: if multiple copies are archived there will be one callback per copy
\end{verbatim}
\subsection{RETRIEVING from CTA to EOS}
\begin{verbatim}
1) EOS REQUEST: cta r/retrieve
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--user <user> // string name of the requester of the action (retrieval),
// used for SLAs and logging,
// not kept by CTA after successful operation
--group <group> // string group of the requester of the action (retrieval),
// used for SLAs and logging,
// not kept by CTA after successful operation
--id <CTA_ArchiveFileID> // uint64_t which is the unique ID of the CTA file
--dsturl <dst_URL> // string of the form scheme://host:port/opaque_part,
// not kept by CTA after successful operation
--diskfilepath <disk_filepath> // string the disk logical path kept by CTA
// for disaster recovery and for logging
--diskfileowner <disk_fileowner> // string owner username kept by CTA for
// disaster recovery and for logging
--diskfilegroup <disk_filegroup> // string owner group kept by CTA for disaster
// recovery and for logging
--recoveryblob <recovery_blob> // 2KB string kept by CTA for disaster recovery
// (opaque string controlled by EOS)
--diskpool <diskpool_name> // string used (and possibly kept) by CTA for
// proper drive allocation
--throughput <diskpool_throughput> // uint64_t (in bytes) used (and possibly kept)
// by CTA for proper drive allocation
Note: disk info is piggybacked
2) CTA IMMEDIATE REPLY: Empty or Error
3) CTA CALLBACK WHEN RETRIEVED SUCCESSFULLY: dst_URL with or without Error
dst_URL: this is the same string provided in the EOS retrieval request
\end{verbatim}
\subsection{DELETING an ARCHIVE FILE}
\begin{verbatim}
1) EOS REQUEST: cta da/deletearchive
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--user <user> // string name of the requester of the action (deletion),
// used for SLAs and logging,
// not kept by CTA after successful operation
--group <group> // string group of the requester of the action (deletion),
// used for SLAs and logging,
// not kept by CTA after successful operation
--id <CTA_ArchiveFileID> // uint64_t which is the unique ID of the CTA file
Note: This command may be issued even before the actual archival process has begun
2) CTA IMMEDIATE REPLY: Empty or Error
\end{verbatim}
\subsection{CANCELING a SCHEDULED RETRIEVAL}
\begin{verbatim}
1) EOS REQUEST: cta cr/cancelretrieve
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--user <user> // string name of the requester of the action (cancel),
// used for SLAs and logging,
// not kept by CTA after successful operation
--group <group> // string group of the requester of the action (cancel),
// used for SLAs and logging,
// not kept by CTA after successful operation
--id <CTA_ArchiveFileID> // uint64_t which is the unique ID of the CTA file
--dsturl <dst_URL> // this is the same string provided in the EOS
// retrieval request
--diskfilepath <disk_filepath> // string the disk logical path kept by CTA for
// disaster recovery and for logging
--diskfileowner <disk_fileowner> // string owner username kept by CTA for disaster
// recovery and for logging
--diskfilegroup <disk_filegroup> // string owner group kept by CTA for disaster
// recovery and for logging
--recoveryblob <recovery_blob> // 2KB string kept by CTA for disaster recovery
// (opaque string controlled by EOS)
Note: This command will succeed ONLY before the actual retrieval process has begun
Note: disk info is piggybacked
2) CTA IMMEDIATE REPLY: Empty or Error
\end{verbatim}
\subsection{UPDATE the STORAGE CLASS of a FILE}
\begin{verbatim}
1) EOS REQUEST: cta ufsc/updatefilestorageclass
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--user <user> // string name of the requester of the action (update),
// used for SLAs and logging,
// not kept by CTA after successful operation
--group <group> // string group of the requester of the action (update),
// used for SLAs and logging,
// not kept by CTA after successful operation
--id <CTA_ArchiveFileID> // uint64_t which is the unique ID of the CTA file
--storageclass <storage_class> // updated storage class which may or may not have
// a different routing
--diskfilepath <disk_filepath> // string the disk logical path kept by CTA for
// disaster recovery and for logging
--diskfileowner <disk_fileowner> // string owner username kept by CTA for disaster
// recovery and for logging
--diskfilegroup <disk_filegroup> // string owner group kept by CTA for disaster
// recovery and for logging
--recoveryblob <recovery_blob> // 2KB string kept by CTA for disaster recovery
// (opaque string controlled by EOS)
Note: This command DOES NOT change the number of tape copies! The number will change
asynchronously (next repack or "reconciliation").
Note: disk info is piggybacked
2) CTA IMMEDIATE REPLY: Empty or Error
\end{verbatim}
\subsection{UPDATE INFO of a FILE}
\begin{verbatim}
1) EOS REQUEST: cta ufi/updatefileinfo
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--id <CTA_ArchiveFileID> // uint64_t which is the unique ID of the CTA file
--diskfilepath <disk_filepath> // string the disk logical path kept by CTA for
// disaster recovery and for logging
--diskfileowner <disk_fileowner> // string owner username kept by CTA for disaster
// recovery and for logging
--diskfilegroup <disk_filegroup> // string owner group kept by CTA for disaster
// recovery and for logging
--recoveryblob <recovery_blob> // 2KB string kept by CTA for disaster recovery
// (opaque string controlled by EOS)
Note: This command is not executed on behalf of an EOS user. Instead it is part of a
resynchronization process initiated by EOS.
2) CTA IMMEDIATE REPLY: Empty or Error
\end{verbatim}
\subsection{LISTING all STORAGE CLASSES available}
\begin{verbatim}
1) EOS REQUEST: cta lsc/liststorageclass
--encoded <"true" or "false"> // true if all following arguments are base64 encoded,
// false if all following arguments are in clear
// (no mixing of encoded and clear arguments)
--user <user> // string name of the requester of the action (listing),
// used for SLAs and logging,
// not kept by CTA after successful operation
--group <group> // string group of the requester of the action (listing),
// used for SLAs and logging,
// not kept by CTA after successful operation
2) CTA IMMEDIATE REPLY: storage class list
\end{verbatim}
*/
This diff is collapsed.
/*!
\page The Command Line Interface
\subsection{Administrator commands}
These are the commands that only administrators are allowed to use to operate CTA. For each command there is a short version and a long one, example: \texttt{op/operator}. Subcommands (\texttt{add}, \texttt{rm}, \texttt{ls}, \texttt{ch}, \texttt{reclaim}) do not have short versions.
\tt
\begin{itemize}
\item cta ad/admin
\item cta ah/adminhost{\normalfont \footnote{hostgroups also? not for the prototype}}
\item cta us/user{\normalfont \footnote{or maybe power users? or egroups? will see later, for the moment just users}}
\item cta tp/tapepool{\normalfont \footnote{listing the tapepools should include also stats such as total number of tapes and number of free tapes to help operator}}
\item cta ar/archiveroute
\item cta ll/logicallibrary
\item cta ta/tape
\item cta sc/storageclass{\normalfont \footnote{storage classes should be as static as possible, no change nor deletion if there is at least 1 directory using it}}
\item cta lpa/listpendingarchives
\item cta lpr/listpendingretrieves
\item cta lds/listdrivestates{\normalfont \footnote{more or less the ``drive'' part of the current showqueues -x}}
\end{itemize}
\normalfont
The detailed list of the commands with their subcommands and parameters follows.
\subsubsection{\tt cta admin}
This command is used to manage the administrators of the system. These are special users allowed to perform the admin commands described in this section.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{3}{*}{cta ad/admin add} & -{}-uid/-u <\textit{uid}> \\ & -{}-gid/-g <\textit{gid}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{3}{*}{cta ad/admin ch} & -{}-uid/-u <\textit{uid}> \\ & -{}-gid/-g <\textit{gid}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{2}{*}{cta ad/admin rm} & -{}-uid/-u <\textit{uid}> \\ & -{}-gid/-g <\textit{gid}> \\ \hline
cta ad/admin ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta adminhost}
This command is used to manage the administrator hosts. These are specific machines from which administrators may issue the admin commands described in this section.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{2}{*}{cta ah/adminhost add} & -{}-name/-n <\textit{host\_name}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{2}{*}{cta ah/adminhost ch} & -{}-name/-n <\textit{host\_name}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
cta ah/adminhost rm & -{}-name/-n <\textit{host\_name}> \\ \hline
cta ah/adminhost ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta user}
This command is used to manage the ``normal'' users of CTA. These are typically physicists and experiment data storage managers, who are able to perform only the actions described in the next section (\textit{archive}, \textit{retrieve}, etc.).
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{3}{*}{cta us/user add} & -{}-uid/-u <\textit{uid}> \\ & -{}-gid/-g <\textit{gid}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{3}{*}{cta us/user ch} & -{}-uid/-u <\textit{uid}> \\ & -{}-gid/-g <\textit{gid}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{2}{*}{cta us/user rm} & -{}-uid/-u <\textit{uid}> \\ & -{}-gid/-g <\textit{gid}> \\ \hline
cta us/user ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta tapepool}
This command is used to manage the tape pools, which are logical sets of tapes. These are useful to manage the life cycle of tapes tolabel $\rightarrow$ supply $\rightarrow$ user pool $\rightarrow$ erase $\rightarrow$ tolabel.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{3}{*}{cta tp/tapepool add} & -{}-name/-n <\textit{tapepool\_name}> \\ & -{}-partialtapesnumber/-p <\textit{number\_of\_partial\_tapes}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{3}{*}{cta tp/tapepool ch} & -{}-name/-n <\textit{tapepool\_name}> \\ & -{}-partialtapesnumber/-p <\textit{number\_of\_partial\_tapes}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
cta tp/tapepool rm & -{}-name/-n <\textit{tapepool\_name}> \\ \hline
cta tp/tapepool ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta archiveroute}
This command is used to manage the archive routes, which are the policies linking name space entries to tape pools.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{4}{*}{cta ar/archiveroute add} & -{}-storageclass/-s <\textit{storage\_class\_name}> \\ & -{}-copynb/-c <\textit{copy\_number}> \\ & -{}-tapepool/-t <\textit{tapepool\_name}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{4}{*}{cta ar/archiveroute ch} & -{}-storageclass/-s <\textit{storage\_class\_name}> \\ & -{}-copynb/-c <\textit{copy\_number}> \\ & -{}-tapepool/-t <\textit{tapepool\_name}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{2}{*}{cta ar/archiveroute rm} & -{}-storageclass/-s <\textit{storage\_class\_name}> \\ & -{}-copynb/-c <\textit{copy\_number}> \\ \hline
cta ar/archiveroute ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta logicallibrary}
This command is used to manage the logical libraries. These are logical groupings of tapes and drives based on physical location and tape drive capabilities. Basically a tape can be accessed by a drive if it is in the same physical library and if the drive is capable of reading or writing the tape, in that case we typically have that that tape and that drive are in the same logical library.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{2}{*}{cta ll/logicallibrary add} & -{}-name/-n <\textit{logical\_library\_name}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{2}{*}{cta ll/logicallibrary ch} & -{}-name/-n <\textit{logical\_library\_name}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
cta ll/logicallibrary rm & -{}-name/-n <\textit{logical\_library\_name}> \\ \hline
cta ll/logicallibrary ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta tape}
This command is used to manage the tapes. These are the physical containers of data.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{5}{*}{cta ta/tape add} & -{}-vid/-v <\textit{vid}> \\ & -{}-logicallibrary/-l <\textit{logical\_library\_name}> \\ & -{}-tapepool/-t <\textit{tapepool\_name}> \\ & -{}-capacity/-c <\textit{capacity\_in\_bytes}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{5}{*}{cta ta/tape ch} & -{}-vid/-v <\textit{vid}> \\ & -{}-logicallibrary/-l <\textit{logical\_library\_name}> \\ & -{}-tapepool/-t <\textit{tapepool\_name}> \\ & -{}-capacity/-c <\textit{capacity\_in\_bytes}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
cta ta/tape rm & -{}-vid/-v <\textit{vid}> \\ \hline
cta ta/tape reclaim & -{}-vid/-v <\textit{vid}> \\ \hline
cta ta/tape ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta storageclass}
This command is used to manage the storage classes. These can be associated with CTA directories and they determine the number of tape copies the files in the directory should have.
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command \& Subcommand} & {\normalfont\it Parameters} \\ \hline
\multirow{3}{*}{cta sc/storageclass add} & -{}-name/-n <\textit{storage\_class\_name}> \\ & -{}-copynb/-c <\textit{number\_of\_tape\_copies}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
\multirow{3}{*}{cta sc/storageclass ch} & -{}-name/-n <\textit{storage\_class\_name}> \\ & -{}-copynb/-c <\textit{number\_of\_tape\_copies}> \\ & -{}-comment/-m <\textit{``comment''}> \\ \hline
cta sc/storageclass rm & -{}-name/-n <\textit{storage\_class\_name}> \\ \hline
cta sc/storageclass ls & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{\tt cta list\ldots}
This set of commands is used to list the pending archives and retrieves as well as the state of each tape drive (its status --up or down--, its contents, etc.).
\tt
\begin{center}
\begin{tabular}{ l l }
\rowcolor[gray]{0.9}{\normalfont\it Command} & {\normalfont\it Parameters} \\ \hline
cta lpa/listpendingarchives & -{}-tapepool/-t <\textit{tapepool\_name}> \\ \hline
cta lpr/listpendingretrieves & -{}-vid/-v <\textit{vid}> \\ \hline
cta lds/listdrivestates & \\ \hline
\end{tabular}
\end{center}
\normalfont
\subsubsection{Questions administrators need to be able to answer}
Questions that administrators need to be able to answer easily using commands above (not necessarily for the prototype, but to keep in mind):
\begin{enumerate}
\item Why is data not going to tape?
\item Why is data not coming out of tapes?
\item Which user is responsible for system overload?
\end{enumerate}
\subsection{User commands}
For most commands there is a short version and a long one. Due to the limited number of USER commands it is not convenient (nor intuitive) to use subcommands here (anyway it could be applied only to storage classes).
\tt
\begin{itemize}
\item cta lsc/liststorageclass{\normalfont \footnote{this command might seem a duplicate of the corresponding admin command but it actually shows a subset of fields (name and number of copies)}}
\item cta ssc/setstorageclass <dirpath> <storage\_class\_name>
\item cta csc/clearstorageclass <dirpath>
\item cta mkdir <dirpath>
\item cta chown <uid> <gid> <dirpath>{\normalfont \footnote{we may want to add \texttt{chmod} later on}}
\item cta rmdir <dirpath>
\item cta ls <dirpath>
\item cta a/archive <src1> [<src2> [<src3> [...]]] <dst>
\item cta r/retrieve <src1> [<src2> [<src3> [...]]] <dst>
\item cta da/deletearchive <dst>{\normalfont \footnote{this works both on ongoing and finished archives, that is why it's called ``delete''}}
\item cta cr/cancelretrieve <dst>{\normalfont \footnote{this clearly works only on ongoing retrieves, obviously does not delete destination files, that's why it's called ``cancel''}}
\end{itemize}
\normalfont
\input{ObjectStore.tex}
\input{TapeSessions.tex}
*/
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment