Skip to content
Snippets Groups Projects
Commit 99cb28da authored by Michael Davis's avatar Michael Davis
Browse files

Documents configuring the Catalogue

parent 144804bd
Branches
Tags
No related merge requests found
No preview for this file type
No preview for this file type
\chapter{Install CTA}
\label{install_cta}
This chapter describes how to install CTA together with a local EOS instance. It is assumed that the target system is
CERN CentOS 7 (CC7). Installation on SLC6 will not be supported. Installation follows these steps:
This chapter describes how to install CTA together with a local EOS instance. These instructions assume that the target
system is CERN CentOS 7 (CC7). In principle CTA can be installed on SLC6 and the steps should be more or less the same.
Installation follows these steps:
\begin{enumerate}
\item Configure yum (Section~\ref{install_cta_configure_yum})
\item Install XRoot and EOS (Section~\ref{install_cta_install_xroot_and_eos})
......@@ -168,14 +169,11 @@ Install the CTA RPMs:
In a test system, configure the ObjectStore as a Virtual File System (VFS). Initialise the new ObjectStore VFS and set
global \texttt{rwx} permissions:
\begin{lstlisting}
# ~/CTA_build/objectstore/cta-objectstore-initialize
New object store path: file:///tmp/jobStoreVFSUaHSAk
# chmod -R 0777 /tmp/jobStoreVFSUaHSAk/
\end{lstlisting}
Then create environment variables to store the type and location of the new ObjectStore:
\begin{lstlisting}
# export OBJECTSTORETYPE=file
# export OBJECTSTOREURL=file:///tmp/jobStoreVFSUaHSAk
# export OBJECTSTOREURL=$(cta-objectstore-initialize | sed 's/^.*file:\/\///')
# chmod -R 0777 $OBJECTSTOREURL
# ls -ld $OBJECTSTOREURL
drwxrwxrwx. 2 root root 240 May 9 11:22 /tmp/jobStoreVFSg4Mqz4/
\end{lstlisting}
%Now we need to add it as a configuration parameter in the \texttt{castor.conf}, as in the following example:
......@@ -230,6 +228,67 @@ Delete the ObjectStore content:
$OBJECTSTORENAMESPACE rm toto
\end{lstlisting}
Julien/Eric's script to create a Ceph ObjectStore:
\begin{lstlisting}
#!/bin/bash
OBJECTSTORE_CONFIG_DIR=/etc/config/objectstore
function get_conf {
test -r ${OBJECTSTORE_CONFIG_DIR}/$1 && cat ${OBJECTSTORE_CONFIG_DIR}/$1 || echo -n UNDEF
}
OBJECTSTORETYPE=UNDEF
OBJECTSTOREURL=UNDEF
rm -f /tmp/objectstore-rc.sh
case "$(get_conf objectstore.type)" in
"UNDEF")
echo "objectstore configmap is not defined"
ls ${OBJECTSTORE_CONFIG_DIR}
exit 1
;;
"ceph")
echo "Configuring ceph objectstore"
cat <<EOF >/etc/ceph/ceph.conf
[global]
mon host = $(get_conf objectstore.ceph.mon):$(get_conf objectstore.ceph.monport)
EOF
cat <<EOF >/etc/ceph/ceph.client.$(get_conf objectstore.ceph.id).keyring
[client.$(get_conf objectstore.ceph.id)]
key = $(get_conf objectstore.ceph.key)
caps mon = "allow r"
caps osd = "allow rwx pool=$(get_conf objectstore.ceph.pool) namespace=$(get_conf objectstore.ceph.namespace)"
EOF
OBJECTSTORETYPE=ceph
OBJECTSTOREURL="rados://$(get_conf objectstore.ceph.id)@$(get_conf objectstore.ceph.pool):$(get_conf objectstore.ceph.namespace)"
echo "export OBJECTSTORENAMESPACE=$(get_conf objectstore.ceph.namespace)" >> /tmp/objectstore-rc.sh
echo "export OBJECTSTOREID=$(get_conf objectstore.ceph.id)" >> /tmp/objectstore-rc.sh
echo "export OBJECTSTOREPOOL=$(get_conf objectstore.ceph.pool)" >> /tmp/objectstore-rc.sh
;;
"file")
echo "Configuring file objectstore"
OBJECTSTORETYPE=file
OBJECTSTOREURL=$(echo $(get_conf objectstore.file.path) | sed -e "s#%NAMESPACE#${MY_NAMESPACE}#")
;;
*)
echo "Error unknown objectstore type: $(get_conf objectstore.type)"
exit 1
;;
esac
cat <<EOF >>/tmp/objectstore-rc.sh
export OBJECTSTORETYPE=${OBJECTSTORETYPE}
export OBJECTSTOREURL=${OBJECTSTOREURL}
EOF
exit 0
\end{lstlisting}
\section{Configure the Catalogue}
\label{install_cta_configure_catalogue}
......@@ -264,54 +323,36 @@ To delete an existing database schema:
\section{Configure the CTA Front End}
\label{install_cta_configure_front_end}
Create the \texttt{/etc/cta/cta-frontend.conf} file with the following contents:
Create the CTA Front End configuration file \texttt{/etc/cta/cta-frontend.conf}. The ObjectStore Backend should point to
the \texttt{\$OBJECTSTOREURL} specified in Section~\ref{install_cta_configure_objectstore}:
\begin{lstlisting}
ObjectStore BackendPath rados://cta-id@cta-tapepool:cta-ns
Catalogue NumberOfConnections 1
Log URL file:/cta-frontend.log
\end{lstlisting}
Create the corresponding logfile:
\begin{lstlisting}
# touch /cta-frontend.log
# chmod a+w /cta-frontend.log
\end{lstlisting}
Create the CTA user:
\begin{lstlisting}
# useradd cta
\end{lstlisting}
\subsection{Configure Authentication}
\subsection{Configure CTA Front End Authentication}
\begin{alertbox}
Then there is the voodoo dance with SSS, which should be explained.
\end{alertbox}
\begin{alertbox}
Unify/Agree on the name of XROOTD configuration files:
\subsection{Install CTA Front End}
Installing RPMs doesn't install cta-frontend. That is done in the script:
\begin{lstlisting}
https://gitlab.cern.ch/cta/CTA/blob/master/continuousintegration/docker/
ctafrontend/cc7/opt/run/bin/ctafrontend.sh
\end{lstlisting}
\subsection{Kubernetes ctafrontend.sh}
Either call them /etc/xrd.cf.* or /etc/xrootd/<function>.cfg
(I prefer /etc/xrootd structure even if different from IT-DSS-FDO).
\end{alertbox}
\begin{lstlisting}
yes | cp -r /opt/ci/ctafrontend/etc /
/opt/run/bin/init_objectstore.sh
. /tmp/objectstore-rc.sh
echo "ObjectStore BackendPath $OBJECTSTOREURL" > /etc/cta/cta-frontend.conf
echo "Catalogue NumberOfConnections 1" >>/etc/cta/cta-frontend.conf
echo "Log URL file:/cta-frontend.log" >>/etc/cta/cta-frontend.conf
/opt/run/bin/init_database.sh
. /tmp/database-rc.sh
echo ${DATABASEURL} >/etc/cta/cta_catalogue_db.conf
# Create user cta early so that we can set file ownership correctly
useradd cta
# EOS INSTANCE NAME used as username for SSS key
EOSINSTANCE=ctaeos
......@@ -334,24 +375,21 @@ sed -i 's|^sec.protbind .*|sec.protbind * only sss krb5|' /etc/xrootd/xrootd-cta
echo -n "Waiting for /etc/cta-frontend.keytab"
for ((;;)); do test -e /etc/cta-frontend.keytab && break; sleep 1; echo -n .; done
echo OK
\end{lstlisting}
touch /cta-frontend.log
chmod a+w /cta-frontend.log
tail -F /cta-frontend.log &
echo "Launching frontend"
runuser --shell='/bin/bash' --session-command='cd ~cta; xrootd -n cta -c /etc/xrootd/xrootd-cta.cfg -I v4' cta
\subsection{Start CTA Front End}
echo "frontend died"
sleep infinity
\begin{lstlisting}
# cd ~cta
# xrootd -n cta -c /etc/xrootd/xrootd-cta.cfg -I v4
\end{lstlisting}
\section{Configure an EOS workflow to allow EOS to talk to the CTA front-end}
\section{Rmcd configuration}
Rmcd is the Remote Media Changer daemon, a TCP/IP server (which you must run locally for security reasons: it only listens to the localhost network). Has a device driver for the drive and one for the library.
The file \texttt{/etc/sysconfig/rmcd} contains the following parameters:
\begin{lstlisting}
DAEMON_COREFILE_LIMIT=unlimited
RUN_RMCD=yes
......@@ -365,6 +403,36 @@ If the CTA instance is not using real tape hardware, follow the directions in Ap
mhVTL Virtual Tape Library.
\section{Starting up the xroot server with the CTA plugin}
Julien and Eric's scripts create the following \texttt{/etc/xrootd/xrootd-cta.cfg}:
\begin{lstlisting}
# Load the plugin XrdCtaOfs
xrootd.fslib libXrdCtaOfs.so
# Use the security module
xrootd.seclib libXrdSec.so
# Protocol specification
# The xroot server process needs to be able to read the keytab file
#sec.protocol krb5 /etc/krb5.keytab.cta host/<host>@CERN.CH
#sec.protocol sss -s /var/spool/xrootd/.xrd/sss.keytab -c /tmp/sss.keytab
sec.protocol unix
# Only Kerberos 5 and sss are allowed
#sec.protbind * only sss
sec.protbind 172.17.0* only unix
# Allow copying from absolute paths
all.export /
# Turn off asynchronous i/o
xrootd.async off
# Use a port other than 1094, already used by EOS xroot server
xrd.port 10955
\end{lstlisting}
Before starting the xroot server we need to change the VFS backend permissions again (because new files have been created with the \texttt{stage:st} user by the tapeserver daemon). Example:
\begin{verbatim}
......@@ -554,7 +622,6 @@ MSG="ProcessForkerConnectionHandler handling ProcessExited message" pid="32453"
MSG="Tape session finished"
\end{boxedverbatim}
\section{Removing CTA}
To remove the CTA Front-end and associated packages:
......@@ -597,14 +664,6 @@ and sometimes use hyphen (\texttt{/etc/cta/cta-frontend.conf}).
Can aliases (in general) be used (such as: ctafrontend.cern.ch)? In which circumstances YES and NO?
\end{alertbox}
\subsection{Unify/Agree on the name of XROOTD configuration files.}
\begin{alertbox}
Either call them /etc/xrd.cf.*
or /etc/xrootd/<function>.cfg (I prefer /etc/xrootd structure even if different from
IT-DSS-FDO).
\end{alertbox}
\subsection{PLEASE make sure the 'status=' string is ALWAYS on the same position in the cta-taped logs.}
\begin{alertbox}
......
......@@ -36,6 +36,8 @@
% Listings
\usepackage{upquote} % Don't use curly quotes in verbatim text
\usepackage{listings}
\usepackage{courier}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment