Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
cta
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Harbor Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
dCache
cta
Commits
13682437
Commit
13682437
authored
8 years ago
by
Steven Murray
Browse files
Options
Downloads
Patches
Plain Diff
Final modifications before submission to the CHEP 2016 paper
parent
b4251365
Branches
Branches containing commit
Tags
Tags containing commit
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
doc/CHEP2016/CHEP_2016_paper_CTA.pdf
+0
-0
0 additions, 0 deletions
doc/CHEP2016/CHEP_2016_paper_CTA.pdf
doc/CHEP2016/CHEP_2016_paper_CTA.tex
+21
-21
21 additions, 21 deletions
doc/CHEP2016/CHEP_2016_paper_CTA.tex
with
21 additions
and
21 deletions
doc/CHEP2016/CHEP_2016_paper_CTA.pdf
+
0
−
0
View file @
13682437
No preview for this file type
This diff is collapsed.
Click to expand it.
doc/CHEP2016/CHEP_2016_paper_CTA.tex
+
21
−
21
View file @
13682437
...
...
@@ -118,7 +118,7 @@ but not on EOS disk. They are unblocked when the file has been retrieved from
tape.
Moving to the middle of figure
\ref
{
architecture
}
. The CTA front-end server
provides a network
ed
based interface to EOS and the CTA administration tool used
provides a network based interface to EOS and the CTA administration tool used
by tape operators (not shown). The CTA front-end stores requests from EOS in
the persistent CTA metadata system. The CTA front-end also queries the CTA
metadata system in order to answer the query commands of the CTA administration
...
...
@@ -195,7 +195,7 @@ The following steps describe how a file is retrieved from tape.
from the tape.
\item
The tape server starts transferring the file from tape to EOS disk.
\item
On the close of the disk file, EOS updates its namespace to reflect the
fact that there is now a
copy
of the file back on EOS disk.
fact that there is now a
replica
of the file back on EOS disk.
\end{enumerate}
\section
{
Configuration and operations concepts
}
\label
{
concepts
}
...
...
@@ -230,8 +230,7 @@ create two destination tape pools, each one in a different physical location.
\subsection
{
The concepts behind giving users a tailored quality of service
}
An operator needs to understand the following two concepts in order to tailor
the quality of service delivered to individual EOS users and groups of EOS
users:
the quality of service delivered to EOS users and groups:
\begin{itemize}
\item
Mount policy.
\item
Mount rule.
...
...
@@ -255,24 +254,24 @@ An operator creates named mount policies in order to define the following:
system is under load.
\end{itemize}
An operator creates mount rules in order to assign mount policies to
individual
EOS users and groups of EOS user
s.
An operator creates mount rules in order to assign mount policies to
EOS users
and group
s.
\section
{
The pre-emptive tape drive scheduler
}
\label
{
scheduler
}
CTA does not utilize a central daemon for scheduling. Instead the scheduler is a
shared software component, running as needed on the front ends and in the tape
servers. The scheduler routines store and retrieve data from the two persistent
stores of CTA: the file catalogue for keeping track of tapes, tape pools,
routes, priorit
ies
,
and tape files and the object store
which keeps track of the
queued archive and retrieve jobs, as well as tape drives statuses.
stores of CTA: the
tape
file catalogue for keeping track of tapes, tape pools,
archive routes, mount polic
ies and tape files
,
and the object store
for keeping
track of the
queued archive and retrieve jobs, as well as tape drives statuses.
\subsection
{
Request queuing
}
When a
client
EOS instance requests a new transfer, the catalogue is
queried to get
routing information for archiv
es
or tape file location for
retrieves. The
jobs are
then added to the
ir
corresponding queue
s
in the object
store. For
archiving, jobs are queued per tape pool, while for retrieves, they
are
attached to one tape at a time. Each queue also keeps track of the summary
of the
jobs it contains to allow efficient mount scheduling.
When a
n
EOS instance requests a new
file
transfer, the
tape file
catalogue is
queried to get
routing information for
an
archiv
al
or tape file location
s
for
a
retrieval. A transfer job is
then added to the corresponding queue in the object
store. For
archiving, jobs are queued per tape pool, while for retrieves, they
are
attached to one tape at a time. Each queue also keeps track of the summary
of the
jobs it contains to allow efficient mount scheduling.
\subsection
{
Tape mount scheduling and job handling
}
Several competing requirements need to be taken into account when scheduling tape
...
...
@@ -289,11 +288,12 @@ data verification are high bandwidth tasks with very relaxed latency
requirements which could extend to several months. These low priority tasks
should get drive time when user-initiated tasks are not using the drives.
When a drive is idle and ready, the tape daemon process retrieves the summaries of
all the non-empty queues from the object store and picks the highest priority queue
that can be served by the drive. The mounts already in progress in other drives are
also taken into account, in order to implement drive quotas. This scheduling is done
atomically: at any point in time, only one drive takes a scheduling decision.
When a drive is idle and ready, the tape daemon process retrieves the summaries
of all the non-empty queues from the object store and picks the highest priority
queue that can be served by the drive. The mounts already in progress in other
drives are also taken into account, in order to implement the drive quotas of
mount policies. This scheduling is done atomically: at any point in time, only
one drive takes a scheduling decision.
Once a mount is started, the tape daemon just consumes the jobs from the queue and
executes them.
...
...
@@ -393,7 +393,7 @@ ownership of CASTOR tapes.
In addition to the hierarchical namespace of EOS, CTA will have its own flat
catalogue of every file archived to tape. This redundancy in metadata will
provide an additional recovery tool in
case
of disaster.
provide an additional recovery tool in
the event
of
a
disaster.
The architecture of CTA has benefited from a fresh start. Without the need to
preserve the internal interfaces of the CASTOR networked components, CTA has
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment