Added the scheduler section in the CHEP2016 paper.

doc/CHEP2016/CHEP_2016_paper_CTA.tex

@@ -255,12 +255,67 @@ An operator creates named mount policies in order to define the following:
 
 An operator creates mount rules in order to assign mount policies to individual
 EOS users and groups of EOS users. 
-\pagebreak
-\section{The pre-emptive tape drive scheduler} \label{scheduler}
-
-TO BE DONE
 
-\pagebreak
+\section{The pre-emptive tape drive scheduler} \label{scheduler}
+CTA does not utilize a central daemon for scheduling. Instead the scheduler is a
+shared software component, running as needed on the front ends and in the tape
+servers. The scheduler routines store and retrieve data from the two persistent
+stores of CTA: the file catalog for keeping track of tapes, tape pools,
+routes, priorities, and tape files and the object store which keeps track of the
+queued archive and retrieve jobs, as well as tape drives statuses.
+
+\subsection{Request queuing}
+When a client EOS instance requests a new transfer, the catalog is queried to get
+routing information for archives or tape file location for retrieves. The
+jobs are then added to their corresponding queues in the object store. For
+archiving, jobs are queued per tape pool, while for retrieves, they are
+attached to one tape at a time. Each queue also keeps track of the summary of the
+jobs it contains to allow efficient mount scheduling.
+
+\subsection{Tape mount scheduling and job handling}
+The tape drives scheduling has to take into account several competing requirements.
+User-initiated accesses in both directions should be executed within a bound latency
+(measured in the order of hours). As mounting and unmounting tape cartridges to and 
+from the drive costs about a minute each, the execution
+of data accessed is postponed until the amount of data to transfer makes it worthwhile
+or the job age is too high. The user initiated mounts create an irregular
+demand, driven by the accelerator cycles and experiments data taking, as well as
+various analysis patterns.
+
+The maintenance tasks, retrieves and archives for repack, and verifications are
+high bandwidth tasks with very relaxed latency requirements which could extend to
+several month. Those low priority tasks get the drives when the user-initiated tasks
+are not using the drives.
+
+When a drive is idle and ready, the tape daemon process retrieves the summaries of
+all the non-empty queues from the object store and picks the highest priority queue
+that can be served by the drive. The mounts already in progress in other drives are 
+also taken into account, in order to implement drive quotas. This scheduling is done 
+atomically: at any point in time, only one drive takes a scheduling decision.
+
+Once a mount is started, the tape drive just consumes the jobs from the queue and
+executes them.
+
+This single step scheduling allows an flexibility with the scheduling rules
+adjustments are expected as experience with CTA will grow.
+
+\subsection{Tape drive preemption}
+Long mount can last up to eight or more hours, which is greater than the
+acceptable latency for user tasks. In order to not let the long-running
+low priority mounts block the drives from the higher priority ones, the tape
+daemon will keep polling the scheduling information at a low rate and interrupt
+a low mount priority if a higher priority one is available to replace it.
+
+This mixing of high and low priority tasks previously had to be handled by hand or \textit{ad hoc}
+scripts in CASTOR.
+
+\subsection{CTA to EOS communication and other operations}
+At the file level, the scheduler also handles the file lifecycle reporting. It coordinates
+the tape file information recording in the file catalog, and then the reporting of
+successful transfers back to EOS.
+
+Finally, the scheduler component handles queue listing and other house keeping
+tasks initiated through the front end by operators, via a command line tool.
 
 \section{Migrating from CASTOR to EOS and CTA} \label{migrating}