Commit 3d372c82 authored by Giuseppe Lo Presti's avatar Giuseppe Lo Presti
Browse files

Added more notes

parent 37a8e072
......@@ -18,6 +18,8 @@ The tools to perform the metadata migration from CASTOR to CTA are as follows:
* `complete_cta_export.py`: terminates an ongoing export, cleaning up the previously created intermediate tables and flagging all files as 'on CTA' and tapes as `EXPORTED`.
* `zerolen_castor_to_cta.py`: similarly to `tapepool_castor_to_cta.py`, migrates all 0-byte files belonging to a given VO to CTA. The criteria to infer whether a 0-byte file belongs to a VO is purely path-based: all and only files whose path starts with `/castor/cern.ch/<VO>` or `/castor/cern.ch/grid/<VO>` are taken into account. If the specified VO is `'ALL'`, then any non-migrated 0-byte files are taken into account.
## Procedure
The tools are designed to work as follows, for any given VO:
0. Make sure all tools are installed and operational in the **VO's CASTOR head nodes**. The tools require access to the CASTOR Nameserver and Stager databases, as well as to the CTA catalogue and the VO's CTA EOS instance. In particular, a `/etc/castor/CTACONFIG` file is required, with a similar format as the `/etc/castor/NSCONFIG` file, and a `/etc/cta/castor-migration.conf` file is required with the options detailed in the corresponding `.example` file.
......@@ -49,15 +51,20 @@ The tools are designed to work as follows, for any given VO:
* `eos-import-files`
* `complete_cta_export.py`
4. After all VOs have been successfully exported, a final run of the step 3. is required in order to export the remaining 0-byte files from the CASTOR Namespace:
4. After all VOs have been successfully exported, a final run of step 3. is required in order to export the remaining 0-byte files from the CASTOR Namespace:
* `zerolen_castor_to_cta.py --vo ALL --instance eosctapublic --doit`
* `eos-import-dirs --delta / # if needed`
* `eos-import-files`
* `complete_cta_export.py`
In case of errors, the tools abort and the operator is expected to fix the issue and rerun the export, possibly re-executing by hand one of the commands documented above. Errors are accumulated in suitable Oracle tables both for the database migration and the EOS namespace injection tools. For the latter, please refer to their specific instructions.
## Further Notes
The database migration phases are based on either idempotent operations or single transactions, in order to minimize the disruption in case of failures. In particular, `fileclass` and `tapepool` entities are imported into the `STORAGE_CLASS` and `TAPE_POOL` CTA tables via idempotent statements, potentially overwriting what is pre-existing in the CTA catalogue. Similarly, `LOGICAL_LIBRARY` entities are populated on the very first import, but never updated again. On the contrary, `ARCHIVE_FILE` and `TAPE_FILE` entities are inserted in bulk in a single non-idempotent transaction.
Nevertheless, in case of errors, the tools abort and the operator is expected to fix the issue and rerun the export, possibly re-executing by hand one of the commands documented above. Errors are accumulated in suitable Oracle tables both for the database migration and the EOS namespace injection tools. For the latter, please refer to their specific instructions.
In addition, the following tools are provided, which can be used as part of a restore/recovery procedure. Such procedure has deliberately **not** been automated and will have to be dealt with on a case by case basis.
* `vmgr_reenable_tapepool.sh`: reverts in CASTOR the export of a tapepool, which had been successfully exported to CTA. All related tapes are marked `FULL`.
* `cta-catalogue-remove-castor-tapes.py`: removes in the CTA catalogue all metadata related to CASTOR tapes for a given tapepool. If additional CTA tapes were added to the tapepool, they are left in the catalogue. If the tapepool concerns dual tape copies, metadata related to the other copies is **also removed**.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment