Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
cta
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Harbor Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
dCache
cta
Commits
46394b5a
Commit
46394b5a
authored
8 years ago
by
Michael Davis
Browse files
Options
Downloads
Patches
Plain Diff
Minor edits to Chapter 2
parent
f13e81f3
Branches
Branches containing commit
Tags
Tags containing commit
No related merge requests found
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
doc/EOS-CTA_interface.pdf
+0
-0
0 additions, 0 deletions
doc/EOS-CTA_interface.pdf
doc/cta.pdf
+0
-0
0 additions, 0 deletions
doc/cta.pdf
doc/latex/cta_BasicConcepts.tex
+52
-39
52 additions, 39 deletions
doc/latex/cta_BasicConcepts.tex
doc/queueRequestSeqDiags.pdf
+0
-0
0 additions, 0 deletions
doc/queueRequestSeqDiags.pdf
with
52 additions
and
39 deletions
doc/EOS-CTA_interface.pdf
+
0
−
0
View file @
46394b5a
No preview for this file type
This diff is collapsed.
Click to expand it.
doc/cta.pdf
+
0
−
0
View file @
46394b5a
No preview for this file type
This diff is collapsed.
Click to expand it.
doc/latex/cta_BasicConcepts.tex
+
52
−
39
View file @
46394b5a
...
...
@@ -2,21 +2,23 @@
CTA is operated by authorized administrators (AdminUsers) who issue CTA commands from authorized machines
(AdminHosts), using the CTA command line interface. All administrative metadata (such as tape, tape pools, storage
classes, etc.) is tagged with a ``creationLog'' and a ``lastModificationLog'' which say who
/when/where created them
and last modified them. An administrator may create (``add''), delete (``rm''), change (``ch'') or list
(``ls'')
any of the administrative metadata.
classes, etc.) is tagged with a ``creationLog'' and a ``lastModificationLog'' which say who
\slash
when
\slash
where
created them
and last modified them. An administrator may create (``add''), delete (``rm''), change (``ch'') or list
(``ls'')
any of the administrative metadata.
Tape
p
ools are logical groupings of tapes that are used by operators to separate data belonging to different
\textbf
{
Tape
P
ools
}
are logical groupings of tapes that are used by operators to separate data belonging to different
\glspl
{
vo
}
. They are also used to categorize types of data and to separate multiple copies of files
so that they end up in different buildings. Each tape belongs to one and only one tape pool.
Logical
l
ibraries are the concept that is used to link tapes and drives together. We use logical libraries to
\textbf
{
Logical
L
ibraries
}
are the concept that is used to link tapes and drives together. We use logical libraries to
specify which tapes are mountable into which drives, and normally this mountability criteria is based on location,
that is the tape has to be in the same physical library as the drive, and on read
/
write compatibility. Each tape
that is the tape has to be in the same physical library as the drive, and on read
\slash
write compatibility. Each tape
and each drive has one and only one logical library.
The storage class is what we assign to each archive file to specify how many tape copies the file is expected to
have. Archive routes link storage classes to tape pools. An archive route specifies onto which set of tapes the
A
\textbf
{
Storage Class
}
is assigned to each archive file to specify how many tape copies the file is expected
to have.
\textbf
{
Archive Routes
}
link storage classes to tape pools. An archive route specifies onto which set of tapes the
copies will be written. There is an archive route for each copy in each storage class, and normally there should
be a single archive route per tape pool.
...
...
@@ -32,43 +34,54 @@ which triggers the need for archiving or retrieving a file to/from tape. A user
``mount group'', which specifies the mount criteria and limitations (together called ``mount policy'') that trigger
a tape mount. Here we offer a simplified description of the archive process:
\begin{enumerate}
\item
EOS issues an archive command for a specific file, providing its source path, its storage
class (see above), and the user requesting the archival
\item
CTA returns immediately an ``ArchiveFileID'' which is
used by CTA to uniquely identify files archived on tape. This ID will be kept by EOS for any operations on this
file (such as retrieval)
\item
Asynchronosly, CTA carries out the archival of the file to tape, in the following
steps:
\begin{itemize}
\item
CTA looks up the storage class provided by EOS and makes sure it has correct routings to one or more
tape pools (more than one when multiple copies are required by the storage class)
\item
CTA queues the
corresponding archive job(s) to the proper tape pool(s)
\item
in the meantime each free tape drive queries
the central ``scheduler'' for work to be done, by communicating its name and its logical library
\item
for
each work request CTA checks whether there is a free tape in the required pool (as specified in b.), that
belongs to the desired logical library (as specified in c.)
\item
if that is the case, CTA checks whether
the work queued for that tape pool is worth a mount, i.e. if it meets the archive criteria specified in the
mount group to which the user (as specified in 1.) belongs
\item
if that is the case, the tape is mounted in
the drive and the file gets written from the source path specified in 1. to the tape
\item
after a successful
archival CTA notifies EOS through an asynchronous callback
\end{itemize}
\end{enumerate}
\begin{enumerate}
\item
EOS issues an archive command for a specific file, providing its source path, its storage class (see above), and
the user requesting the archival
\label
{
archiveone
}
\item
CTA returns immediately an ``ArchiveFileID'' which is used by CTA to uniquely identify files archived on tape.
This ID will be kept by EOS for any operations on this file (such as retrieval)
\item
Asynchronosly, CTA carries out the archival of the file to tape, in the following steps:
\begin{enumerate}
\item
CTA looks up the storage class provided by EOS and makes sure it has correct routings to one or more tape
pools (more than one when multiple copies are required by the storage class)
\item
CTA queues the corresponding archive job(s) to the proper tape pool(s)
\label
{
archiveb
}
\item
in the meantime each free tape drive queries the central ``scheduler'' for work to be done, by communicating
its name and its logical library
\label
{
archivec
}
\item
for each work request CTA checks whether there is a free tape in the required pool (as specified in~
\ref
{
archiveb
}
), that
belongs to the desired logical library (as specified in~
\ref
{
archivec
}
)
\item
if that is the case, CTA checks whether the work queued for that tape pool is worth a mount, i.e. if it meets
the archive criteria specified in the mount group to which the user (as specified in~
\ref
{
archiveone
}
) belongs
\item
if that is the case, the tape is mounted in the drive and the file gets written from the source path
specified in~
\ref
{
archiveone
}
to the tape
\item
after a successful archival CTA notifies EOS through an asynchronous callback
\end{enumerate}
\end{enumerate}
An archival process can be canceled at any moment (even after correct archival, but in this case it's a ``delete'')
through the ``delete archive'' command
through the ``delete archive'' command.
\section
{
Retrieving a file with CTA
}
Here we offer a simplified description of the retrieve process:
\begin{enumerate}
\item
EOS issues a retrieve command for a specific file, providing its ArchiveFileID and desired
destination path, and the user requesting the retrieval
\item
CTA returns immediately
\item
Asynchronosly, CTA
carries out the retrieval of the file from tape, in the following steps:
\begin{itemize}
\item
CTA queues the corresponding retrieve job(s) to the proper tape(s) (depending on where the tape copies
are located)
\item
in the meantime each free tape drive queries the central ``scheduler'' for work to be done,
by communicating its name and its logical library
\item
for each work request CTA checks whether the logical
library (as specified in b.) is the same of (one of) the tape(s) (as specified in a.)
\item
if that is the
case, CTA checks whether the work queued for that tape is worth the mount, i.e. if it meets the retrieve
criteria specified in the mount group to which the user (as specified in 1.) belongs
\item
if that is the
case, the tape is mounted in the drive and the file gets read from tape to the destination specified in 1.
\item
after a successful retrieval CTA notifies EOS through an asynchronous callback
\end{itemize}
\end{enumerate}
\begin{enumerate}
\item
EOS issues a retrieve command for a specific file, providing its ArchiveFileID and desired destination path, and
the user requesting the retrieval
\label
{
retrieveone
}
\item
CTA returns immediately
\item
Asynchronosly, CTA carries out the retrieval of the file from tape, in the following steps:
\begin{enumerate}
\item
CTA queues the corresponding retrieve job(s) to the proper tape(s) (depending on where the tape copies are
located)
\label
{
retrievea
}
\item
in the meantime each free tape drive queries the central ``scheduler'' for work to be done, by communicating its
name and its logical library
\label
{
retrieveb
}
\item
for each work request CTA checks whether the logical library (as specified in~
\ref
{
retrieveb
}
) is the same of
(one of) the tape(s) (as specified in~
\ref
{
retrievea
}
)
\item
if that is the case, CTA checks whether the work queued for that tape is worth the mount, i.e. if it meets the
retrieve criteria specified in the mount group to which the user (as specified in~
\ref
{
retrieveone
}
) belongs
\item
if that is the case, the tape is mounted in the drive and the file gets read from tape to the destination
specified in~
\ref
{
retrieveone
}
\item
after a successful retrieval CTA notifies EOS through an asynchronous callback
\end{enumerate}
\end{enumerate}
A retrieval process can be canceled at any moment prior to correct retrieval through the ``cancel retrieve'' command
A retrieval process can be canceled at any moment prior to correct retrieval through the ``cancel retrieve'' command
.
This diff is collapsed.
Click to expand it.
doc/queueRequestSeqDiags.pdf
+
0
−
0
View file @
46394b5a
No preview for this file type
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment