Skip to content
Snippets Groups Projects
Commit 13682437 authored by Steven Murray's avatar Steven Murray
Browse files

Final modifications before submission to the CHEP 2016 paper

parent b4251365
Branches
Tags
No related merge requests found
No preview for this file type
......@@ -118,7 +118,7 @@ but not on EOS disk. They are unblocked when the file has been retrieved from
tape.
Moving to the middle of figure \ref{architecture}. The CTA front-end server
provides a networked based interface to EOS and the CTA administration tool used
provides a network based interface to EOS and the CTA administration tool used
by tape operators (not shown). The CTA front-end stores requests from EOS in
the persistent CTA metadata system. The CTA front-end also queries the CTA
metadata system in order to answer the query commands of the CTA administration
......@@ -195,7 +195,7 @@ The following steps describe how a file is retrieved from tape.
from the tape.
\item The tape server starts transferring the file from tape to EOS disk.
\item On the close of the disk file, EOS updates its namespace to reflect the
fact that there is now a copy of the file back on EOS disk.
fact that there is now a replica of the file back on EOS disk.
\end{enumerate}
\section{Configuration and operations concepts} \label{concepts}
......@@ -230,8 +230,7 @@ create two destination tape pools, each one in a different physical location.
\subsection{The concepts behind giving users a tailored quality of service}
An operator needs to understand the following two concepts in order to tailor
the quality of service delivered to individual EOS users and groups of EOS
users:
the quality of service delivered to EOS users and groups:
\begin{itemize}
\item Mount policy.
\item Mount rule.
......@@ -255,24 +254,24 @@ An operator creates named mount policies in order to define the following:
system is under load.
\end{itemize}
An operator creates mount rules in order to assign mount policies to individual
EOS users and groups of EOS users.
An operator creates mount rules in order to assign mount policies to EOS users
and groups.
\section{The pre-emptive tape drive scheduler} \label{scheduler}
CTA does not utilize a central daemon for scheduling. Instead the scheduler is a
shared software component, running as needed on the front ends and in the tape
servers. The scheduler routines store and retrieve data from the two persistent
stores of CTA: the file catalogue for keeping track of tapes, tape pools,
routes, priorities, and tape files and the object store which keeps track of the
queued archive and retrieve jobs, as well as tape drives statuses.
stores of CTA: the tape file catalogue for keeping track of tapes, tape pools,
archive routes, mount policies and tape files, and the object store for keeping
track of the queued archive and retrieve jobs, as well as tape drives statuses.
\subsection{Request queuing}
When a client EOS instance requests a new transfer, the catalogue is queried to get
routing information for archives or tape file location for retrieves. The
jobs are then added to their corresponding queues in the object store. For
archiving, jobs are queued per tape pool, while for retrieves, they are
attached to one tape at a time. Each queue also keeps track of the summary of the
jobs it contains to allow efficient mount scheduling.
When an EOS instance requests a new file transfer, the tape file catalogue is
queried to get routing information for an archival or tape file locations for a
retrieval. A transfer job is then added to the corresponding queue in the object
store. For archiving, jobs are queued per tape pool, while for retrieves, they
are attached to one tape at a time. Each queue also keeps track of the summary
of the jobs it contains to allow efficient mount scheduling.
\subsection{Tape mount scheduling and job handling}
Several competing requirements need to be taken into account when scheduling tape
......@@ -289,11 +288,12 @@ data verification are high bandwidth tasks with very relaxed latency
requirements which could extend to several months. These low priority tasks
should get drive time when user-initiated tasks are not using the drives.
When a drive is idle and ready, the tape daemon process retrieves the summaries of
all the non-empty queues from the object store and picks the highest priority queue
that can be served by the drive. The mounts already in progress in other drives are
also taken into account, in order to implement drive quotas. This scheduling is done
atomically: at any point in time, only one drive takes a scheduling decision.
When a drive is idle and ready, the tape daemon process retrieves the summaries
of all the non-empty queues from the object store and picks the highest priority
queue that can be served by the drive. The mounts already in progress in other
drives are also taken into account, in order to implement the drive quotas of
mount policies. This scheduling is done atomically: at any point in time, only
one drive takes a scheduling decision.
Once a mount is started, the tape daemon just consumes the jobs from the queue and
executes them.
......@@ -393,7 +393,7 @@ ownership of CASTOR tapes.
In addition to the hierarchical namespace of EOS, CTA will have its own flat
catalogue of every file archived to tape. This redundancy in metadata will
provide an additional recovery tool in case of disaster.
provide an additional recovery tool in the event of a disaster.
The architecture of CTA has benefited from a fresh start. Without the need to
preserve the internal interfaces of the CASTOR networked components, CTA has
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment