Skip to content
Snippets Groups Projects
Commit 9035196e authored by Michael Davis's avatar Michael Davis
Browse files

Consolidates last sections of CTA_Admin_Install.tex

parent 06dce56c
No related branches found
No related tags found
No related merge requests found
......@@ -335,89 +335,133 @@ XRoot client always uses the last key added the keytab, so the entire keytab can
long as the correct key for that instance is the last one that was added.
\end{alertbox}
\noindent The CTA Front End needs both Kerberos and SSS authentication. Kerberos is used to authenticate user
archive/retrieve commands and admin commands to the Front End. SSS is used to authenticate communication between
the Front End and CTA/EOS \texttt{mgm}.
Edit \texttt{/etc/xrootd/xrootd-cta.cfg} and ensure that both Kerberos and SSS are enabled and using the correct keys.
This is specified using the \texttt{sec.protocol} and \texttt{sec.protbind} configuration lines. The file should finish
looking something like this:
\begin{lstlisting}
sed -i 's|.*sec.protocol sss.*|sec.protocol sss -s /etc/ctafrontend_SSS_s.keytab -c /etc/ctafrontend_SSS_c.keytab|' /etc/xrootd/xrootd-cta.cfg
sed -i 's|.*sec.protocol unix.*|#sec.protocol unix|' /etc/xrootd/xrootd-cta.cfg
# Load the plugin XrdCtaOfs
xrootd.fslib libXrdCtaOfs.so
# Hack the default xrootd-cta.cfg provided by the sources
sed -i 's|.*sec.protocol krb5.*|sec.protocol krb5 /etc/cta-frontend.keytab cta/cta-frontend@TEST.CTA|' /etc/xrootd/xrootd-cta.cfg
# Use the security module
xrootd.seclib libXrdSec.so
# Allow only SSS and krb5 for frontend
sed -i 's|^sec.protbind .*|sec.protbind * only sss krb5|' /etc/xrootd/xrootd-cta.cfg
# Protocol specification
# The xroot server process needs to be able to read the keytab file
sec.protocol krb5 /etc/cta-frontend.keytab cta/cta-frontend@CERN.CH
sec.protocol sss -s /etc/ctafrontend_server_sss.keytab -c /etc/ctafrontend_client_sss.keytab
# Only Kerberos 5 and sss are allowed
sec.protbind * only sss krb5
sec.protbind * only sss krb5
# Wait for the keytab file to be pushed in by the creation script.
echo -n "Waiting for /etc/cta-frontend.keytab"
for ((;;)); do test -e /etc/cta-frontend.keytab && break; sleep 1; echo -n .; done
echo OK
# Allow copying from absolute paths
all.export /
# Turn off asynchronous i/o
xrootd.async off
# Use a port other than 1094, already used by EOS xroot server
xrd.port 10955
\end{lstlisting}
\begin{alertbox}
Then there is the voodoo dance with SSS, which should be explained.
How is /etc/cta-frontend.keytab created?
In one of the other scripts, this keytab has a different name: /etc/krb5.keytab.cta
\end{alertbox}
\begin{alertbox}
Unify/Agree on the name of XROOTD configuration files:
The CTA Command Line Interface needs to speak to a \emph{xroot} server which uses the CTA plugin to interpret and execute
the commands sent by the user. All the information needed by the \emph{xroot} server is contained within a configuration
file which is currently called \texttt{xrd.cf.cta}.
All this will work fine provided that you have a file called \texttt{/etc/krb5.keytab.cta} which is a copy of your keytab
file (\texttt{/etc/krb5.keytab}). \texttt{/etc/krb5.keytab.cta} must be owned by the user.group that will run the
\emph{xroot} server process. If you don't have a keytab file you may get one using \texttt{cern-get-keytab} (for CERN
supported Linux systems) or \texttt{msktutil} (for other systems), as explained in:
Either call them /etc/xrd.cf.* or /etc/xrootd/<function>.cfg
(I prefer /etc/xrootd structure even if different from IT-DSS-FDO).
\begin{verbatim}
http://linux.web.cern.ch/linux/docs/kerberos-access.shtml
\end{verbatim}
\end{alertbox}
\begin{alertbox}
Why do we have some XRoot configuration files under \texttt{/etc/xrd.cf.<daemon>} and some under \texttt{/etc/xrootd/<daemon>.cfg}?
\end{alertbox}
\section{Install the Tape Server}
\label{install_cta_install_tape_server}
\subsection{Start CTA Front End}
If the CTA instance is not using real tape hardware, follow the directions in Appendix~\ref{install_mhvtl} to install the
mhVTL Virtual Tape Library.
Add the following lines to \texttt{/etc/cta/cta.conf}:
\begin{lstlisting}
# cd ~cta
# xrootd -n cta -c /etc/xrootd/xrootd-cta.cfg -I v4
taped BufferCount 10
general ObjectStoreURL rados://cta-id@cta-tapepool:cta-ns
\end{lstlisting}
\begin{alertbox}
BufferCount specifies the size of the ObjectStore in blocks. But what is the size of a block?
\end{alertbox}
\section{Configure an EOS workflow to allow EOS to talk to the CTA front-end}
\subsection{tapeserverd configuration}
\section{Rmcd configuration}
Rmcd is the Remote Media Changer daemon, a TCP/IP server (which you must run locally for security reasons: it only listens to the localhost network). Has a device driver for the drive and one for the library.
The tape drives can now be indicated to tapeserverd through the \texttt{/etc/castor/TPCONFIG} file:
The file \texttt{/etc/sysconfig/rmcd} contains the following parameters:
\begin{lstlisting}
DAEMON_COREFILE_LIMIT=unlimited
RUN_RMCD=yes
RMCD_OPTIONS=/dev/smc
\end{lstlisting}
\begin{boxedverbatim}
VDSTK1 VLSTK /dev/nst0 smc0
VDSTK2 VLSTK /dev/nst1 smc1
\end{boxedverbatim}
\section{Install the Tape Server}
\label{install_cta_install_tape_server}
The memory blocks size and number shall also be added in \texttt{/etc/castor/castor.conf}:
If the CTA instance is not using real tape hardware, follow the directions in Appendix~\ref{install_mhvtl} to install the
mhVTL Virtual Tape Library.
\begin{boxedverbatim}
TapeServer BufSize 5242880
TapeServer NbBufs 200
\end{boxedverbatim}
\section{Starting up the xroot server with the CTA plugin}
Julien and Eric's scripts create the following \texttt{/etc/xrootd/xrootd-cta.cfg}:
\begin{lstlisting}
# Load the plugin XrdCtaOfs
xrootd.fslib libXrdCtaOfs.so
Finally we start the CTA version of \textbf{tapeserverd}:
# Use the security module
xrootd.seclib libXrdSec.so
\begin{boxedverbatim}
# service rmcd start
Starting rmcd: [ OK ]
# <Build directory>/tapeserver/castor/tape/tapeserver/daemon/tapeserverd
# tpstat
userid jid dgn stat dvn rl vsn vid
VLSTK down VDSTK1@devbox
VLSTK down VDSTK2@devbox
\end{boxedverbatim}
# Protocol specification
# The xroot server process needs to be able to read the keytab file
#sec.protocol krb5 /etc/krb5.keytab.cta host/<host>@CERN.CH
#sec.protocol sss -s /var/spool/xrootd/.xrd/sss.keytab -c /tmp/sss.keytab
sec.protocol unix
Using \texttt{tpstat} and \texttt{tpconfig}, one can put the drive up and see it attempt to schedule itself
in the logs (either \texttt{/var/log/messages} or \texttt{/var/log/castor/tapeserverd.log} depending on the
installation and leftovers from previous castor tape server.
# Only Kerberos 5 and sss are allowed
#sec.protbind * only sss
sec.protbind 172.17.0* only unix
The log typically look like (simplified) in a loop:
# Allow copying from absolute paths
all.export /
\begin{boxedverbatim}
MSG="ProcessForker handling a ProcessForker message" type="ForkDataTransfer" len="47"
MSG="ProcessForker handling ForkDataTransfer message" unitName="VDSTK1"
MSG="ProcessForker forked data-transfer session" pid="32453"
MSG="Closed command receiver socket" cmdSocket="7"
MSG="Data-transfer child-process started" unitName="VDSTK1"
MSG="Got process ID of the data-transfer session from the ProcessForker" pid="32453"
MSG="Child of ProcessForker instantiated a ZMQ context" contextAddr="0x17494b0"
MSG="ProcessForker child process terminated" terminatedPid="32453" WEXITSTATUS="0" WCOREDUMP="false" WIFCONTINUED="false"
MSG="ProcessForker notifying TapeDaemon of process exit" pid="32453" exitCode="0" payloadLen="6"
MSG="ProcessForkerConnectionHandler handling a message" type="ProcessExited" payloadLen="6"
MSG="ProcessForkerConnectionHandler handling ProcessExited message" pid="32453" exitCode="0"
MSG="Tape session finished"
\end{boxedverbatim}
# Turn off asynchronous i/o
xrootd.async off
\section{Start CTA Front End}
# Use a port other than 1094, already used by EOS xroot server
xrd.port 10955
\begin{lstlisting}
# su - cta
[~]$ xrootd -n cta -c /etc/xrootd/xrootd-cta.cfg -I v4
\end{lstlisting}
\subsection{Starting up the xroot server with the CTA plugin}
Before starting the xroot server we need to change the VFS backend permissions again (because new files have been created with the \texttt{stage:st} user by the tapeserver daemon). Example:
......@@ -432,34 +476,6 @@ We now create some useful soft links to simplify the xrootd configuration:
$ ln -s <CTA src directory>/xroot_plugins/xrd.cf.cta /etc/xrd.cf.cta
\end{verbatim}
The CTA Command Line Interface needs to speak to a \emph{xroot} server which uses the CTA plugin to interpret and execute the commands sent by the user. All the information needed by the \emph{xroot} server is contained within a configuration file which is currently called \texttt{xrd.cf.cta}. Its contents is the following:
\begin{verbatim}
# Load the plugin XrdCtaOfs
xrootd.fslib /usr/lib64/libXrdCtaOfs.so
# Use the Kerberos 5 security module
xrootd.seclib libXrdSec.so
# The xroot server process needs to be able to read the keytab file
sec.protocol krb5 /etc/krb5.keytab.cta host/<host>@CERN.CH
# Only Kerberos 5 is allowed
sec.protbind * only krb5
# Allow copying from absolute paths
all.export /
# Turn off asynchronous i/o
xrootd.async off
\end{verbatim}
All this will work fine provided that you have a file called \texttt{/etc/krb5.keytab.cta} which is a copy of your keytab file (\texttt{/etc/krb5.keytab}). \texttt{/etc/krb5.keytab.cta} must be owned by the user.group that will run the \emph{xroot} server process. If you don't have a keytab file you may get one using \texttt{cern-get-keytab} (for CERN supported Linux systems) or \texttt{msktutil} (for other systems), as explained in:
\begin{verbatim}
http://linux.web.cern.ch/linux/docs/kerberos-access.shtml
\end{verbatim}
The \emph{xroot} server then can be launched with the following command:
\begin{verbatim}
......@@ -507,6 +523,8 @@ Where the first parameter is the name of the xroot instance, the second one is t
The \emph{xroot} server can be stopped by pressing \texttt{Ctrl+c}. Once the server is running, the CTA CLI will be able to send commands and receive results.
\section{Configure an EOS workflow to allow EOS to talk to the CTA front-end}
\subsection{Preparing the EOS workflows }
The closew,retrive and deletearchive EOS workflows need to be registered to work with CTA. The following EOS command have to be applied for CTA directory:
......@@ -559,54 +577,15 @@ To trigger manually EOS workflows just use the following eos commands:
file workflow <filename> default delete
\end{verbatim}
\section{tapeserverd configuration}
The tape drives can now be indicated to tapeserverd through the \texttt{/etc/castor/TPCONFIG} file:
\begin{boxedverbatim}
VDSTK1 VLSTK /dev/nst0 smc0
VDSTK2 VLSTK /dev/nst1 smc1
\end{boxedverbatim}
The memory blocks size and number shall also be added in \texttt{/etc/castor/castor.conf}:
\begin{boxedverbatim}
TapeServer BufSize 5242880
TapeServer NbBufs 200
\end{boxedverbatim}
Finally we start the CTA version of \textbf{tapeserverd}:
\begin{boxedverbatim}
# service rmcd start
Starting rmcd: [ OK ]
# <Build directory>/tapeserver/castor/tape/tapeserver/daemon/tapeserverd
# tpstat
userid jid dgn stat dvn rl vsn vid
VLSTK down VDSTK1@devbox
VLSTK down VDSTK2@devbox
\end{boxedverbatim}
Using \texttt{tpstat} and \texttt{tpconfig}, one can put the drive up and see it attempt to schedule itself
in the logs (either \texttt{/var/log/messages} or \texttt{/var/log/castor/tapeserverd.log} depending on the
installation and leftovers from previous castor tape server.
The log typically look like (simplified) in a loop:
\section{Rmcd configuration}
Rmcd is the Remote Media Changer daemon, a TCP/IP server (which you must run locally for security reasons: it only listens to the localhost network). Has a device driver for the drive and one for the library.
\begin{boxedverbatim}
MSG="ProcessForker handling a ProcessForker message" type="ForkDataTransfer" len="47"
MSG="ProcessForker handling ForkDataTransfer message" unitName="VDSTK1"
MSG="ProcessForker forked data-transfer session" pid="32453"
MSG="Closed command receiver socket" cmdSocket="7"
MSG="Data-transfer child-process started" unitName="VDSTK1"
MSG="Got process ID of the data-transfer session from the ProcessForker" pid="32453"
MSG="Child of ProcessForker instantiated a ZMQ context" contextAddr="0x17494b0"
MSG="ProcessForker child process terminated" terminatedPid="32453" WEXITSTATUS="0" WCOREDUMP="false" WIFCONTINUED="false"
MSG="ProcessForker notifying TapeDaemon of process exit" pid="32453" exitCode="0" payloadLen="6"
MSG="ProcessForkerConnectionHandler handling a message" type="ProcessExited" payloadLen="6"
MSG="ProcessForkerConnectionHandler handling ProcessExited message" pid="32453" exitCode="0"
MSG="Tape session finished"
\end{boxedverbatim}
The file \texttt{/etc/sysconfig/rmcd} contains the following parameters:
\begin{lstlisting}
DAEMON_COREFILE_LIMIT=unlimited
RUN_RMCD=yes
RMCD_OPTIONS=/dev/smc
\end{lstlisting}
\section{Removing CTA}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment