ReleaseNotes.md



v4.Next

Features

cta/CTA#213 - Improve error messages for cta-eos-namespace-inject

cta/CTA#308 - Remove catalogue autogenerated files
cta/CTA#294 - Improve error messages for 'Exec' in the gRPC client


Bug Fixes

cta/CTA#305 - AllCatalogueSchema file not created when updating schema


v4.8.5-1

Summary

Features

cta/CTA#166 - Refactor Catalogue and RdbmsCatalogue classes
cta/CTA#211 - Fix issues with cta-readtp encryption
cta/CTA#213 - Add tool for injecting file into eos
cta/CTA#213 - Improve test for eos injection tool
cta/CTA#218 - Do not retry during repack requests
cta/CTA#219 - Update Ceph repo public key in dev env and buildtree
cta/CTA#222 - Review what gets logged in CTA
cta/CTA#224 - Improve error message for cta-verify-file whn VID does not exist
cta/CTA#230 - Modify CTA code to enforce VID uppercase
cta/CTA#239 - Add improvments to the cta-change-storage-class tool
cta/CTA#241 - Add missing forward declarations to standalone cli tools
cta/CTA#248 - Clean up output from cta-readtp
cta/CTA#250 - Trim SQL query length in catalogue DB failure reason
cta/CTA#252 - Update cta-change-storage-class to accept json file as input
cta/CTA#254 - Create git submodule for CTA catalogue schema
cta/CTA#284 - Add kerberos authentication for standalone cli tool tests
cta/CTA#295 - Update standalone cli tool tests, remove sudo and use more generated tmp folders
cta/CTA#301 - Switch from new to unique pointer in CtaReadTp to fix memory leak


Bug Fixes

cta/CTA#174 - Improving error description in cta-tape-label in case of wrong volume label format
cta/CTA#181 - cta-statistics-update can fail for catalogues in postgres
cta/CTA#189 - Avoid postgres logging frequent warnings about no transaction in progress
cta/CTA#234 - Replace stoi with toUint64 in standalone cli tool
cta/CTA#237 - Fix misplaced ERROR/WARNING messages triggered by the queue cleanup runner
cta/CTA#238 - Compilation fails when using cta::common::Configuration::getConfEntInt(...)
cta/CTA#242 - cta-frontend-grpc - problem with loading pem_root_certs
cta/CTA#273 - Fix tape state change command idempotency when resetting REPACKING/BROKEN/PENDING
cta/CTA#280 - Add a timeout to tape server global lock on the object store
cta/CTA#288 - Do not allow tape server to transition from REPACKING_DISABLED to DISABLED
cta/CTA#289 - Avoid a DB query and improve filtering time in sortAndGetTapesForMountInfo
cta/CTA#290 - Remove temporary counters used to track single-vid-GetTapesByVid calls
cta/CTA#292 - Problem with cppcheck


Continuous Integration

cta/CTA#253 - Allow Failure for cta_valgrind tests
cta/CTA#255 - Updating CI to run with old and latest kubernetes versions
cta/CTA#286 - Upgrade eos to 4.8.98/5.1.9 fixing operation critical eosreport see EOS-5367
cta/CTA#298 - Remove docs CI stage, moved to submodule


v4.8.4-1

Summary
This release reduces the number of DB queries issued to the CTA catalogue.

Bug Fixes

cta/CTA#275 - Avoid DB queries via getTapesByVid in OStoreDB::fetchMountInfo
cta/CTA#274 - Remove unnecessary catalogue DB queries from QueueCleanupRunner


v4.8.3-1

Summary
This release is the same as '4.8.2-1', except for the addition of some DB query counters which will be used for profiling purposes. 

The counters are expected to be removed in a future release, when the analysis is no longer necessary.

Bug Fixes

cta/CTA#155 - Tape server querying DB at very high rate (log query-count patch)


v4.8.2-1

Summary

Features

cta/CTA#155 - Tape server querying DB at very high rate


v4.8.1-1

Summary

Bug fixes

cta/CTA#243 - Modify new queue cleanup protobuf fields from 'required' to 'optional'


v4.8.0-1

Summary
This CTA release contains significant changes related to repacking, including the addition of new final and temporary states.
It may make be incompatible with pre-existing operational tools that relied on the old repacking behaviour.

Features

cta/CTA#83 - Setup new tape state REPACKING
cta/CTA#226 - Setup new tape state EXPORTED
cta/CTA#77 - Add maintenance runner for cleaning-up (retrieve) queue requests and managing new internal states
cta/CTA#211 - Add functionality for reading encrypted tapes with cta-readtp
cta/CTA#214 - Update manual page for cta-admin to include info about user defined config files.


Bug fixes

cta/CTA#93 - Refactor Frontend code to allow code sharing between SSI and gRPC implementations
cta/CTA#221 - Change option in cta-send-event from instance to eos.instance
cta/CTA#223 - Remove vid check to improve run time of cta-verify-file, fix possible _S_construct null not valid error
cta/CTA#13 - Fix cta-catalogue-schema-verify checking of NOT NULL constraints in Postgres


v4.7.14-1

Summary

Features

cta/CTA#133 - Change severity from ERROR to INFO for "reported a batch of retrieve jobs" log message in Scheduler.cpp
cta/CTA#201 - Improve error message when oracle configured without oracle support
cta/CTA#203 - Refactor cta-restore-deletes-files by using the connection configuration class in standalone_cli_tools/common


Bug fixes

cta/CTA#209 - handle if $HOME is not defined when choosing config file for cta-admin


v4.7.13-3

Summary

Building and Packaging

cta/CTA#15 - Repackaging CTA for easy installation of public RPMs


v4.7.13-2

Summary

Building and Packaging

cta/CTA#207 - Fix tag pipeline


v4.7.13-1

Summary

Features

cta/CTA#16 - Add option for a user config file
cta/CTA#23 - Change Owner Identifier String and System Code of Creating System values in tape labels
cta/CTA#41 - Delete verification_status of tape when tape is reclaimed
cta/CTA#78 - Tool to update the storage class
cta/CTA#80 - Revise tape thread complete success/failure logic
cta/CTA#94 - Remove tape session error codes
cta/CTA#106 - Investigate cta-taped Watchdog timer configuration
cta/CTA#153 - Allow verification status to be cleared with cta-admin
cta/CTA#156 - Change handeling of multilple fxids as input to the cta-restore-files tool
cta/CTA#166 - Removing Catalogue headers from hpp files and small refactoring of Catalogue classes.
cta/CTA#173 - Update release notes and small changes to refactoring of operation tools cmd line parsing - Compatible with operations 0.4-95 or later
cta/CTA#180 - Allow to submit multiple files for verification
cta/CTA#198 - Add vid existence check and update usage message for cta-verify-file
cta/CTA#199 - Add static library ctaCmdlineToolsCommon


Continuous Integration

cta/CTA#118 - Add unit tests for OSM label
cta/CTA#191 - Block merge until cta_valgrind success


Bug fixes

cta/CTA#48 - Catch tape server exception and log an error instead
cta/CTA#80 - Fix tape thread complete success/failure message parameter
cta/CTA#123 - Change some tape server errors into warnings
cta/CTA#150 - Fix job fetching logic in RecallTaskInjector
cta/CTA#161 - Fix bug when using temporary tables with PostgreSQL
cta/CTA#175 - Consistency issue in volume names format
cta/CTA#182 - Fix cta_valgrind error
cta/CTA#197 - Include order in XrdSsiCtaRequestMessage.cpp


v4.7.12-2

Summary

Building and Packaging

cta/CTA#15 - Repackaging CTA for easy installation of public RPMs


v4.7.12-1

Summary

Features

cta/CTA#144 - frontend-grpc: use provided archiveReportURL AS-IS
cta/CTA#146 - Refactoring of operation tools cmd line parsing


Continuous Integration

cta/CTA#157 - Upgrade eos5 to 5.1.1-1
cta/CTA#158 - Install 'libisa-l_crypto' in 'doublebuildtree-stage2b-scripts/Dockerfile'
cta/CTA#15 - Configuring CI for public binary RPM distribution


Bug fixes

cta/CTA#122 - Problem with handling of the non-native formats by the CleanerSession
cta/CTA#160 - Improve DB access to get all the Tape Drive States
cta/CTA#164 - Cppcheck job is not failing when there are errors
cta/CTA#165 - Fix oracle dbunittests
cta/CTA#169 - Fix misconfigured rules in .gitlab-ci.yml files
cta/CTA#171 - CI runner randomly crashing in cta-tape-label


Building and Packaging

cta/CTA#107 - Check latest version of gtest suite


v4.7.11-1

Summary

Note: When using Spectra Logic libraries, this release requires Spectra Logic firmware version >= BlueScale12.8.08.01


Features

cta/CTA#89 - Create stubs for Enstore tape label format
cta/CTA#126 - Remove cta-admin schedulinginfo subcommand


Continuous Integration

cta/CTA#7  - Use same versionlock.list file for xrootd4 and 5
cta/CTA#18 - CI - Testing of DB schema upgrade script
cta/CTA#49 - Clean up orchestration test scripts
cta/CTA#101 - CI should cancel running jobs for a branch after a force pushi
cta/CTA#130 - Do not fail liquibase-update test when CTA is updated without oracle support
cta/CTA#112 - Add postgres 10.0 to 11.0 liquibase Schema migration script
cta/CTA#132 - Support postgres migrations


Building and Packaging

cta/CTA#92 - Refactor CTA code so that it can be build without Oracle dependencies
cta/CTA#63 - CTA Frontend protobuf changes to support dCache


Bug fixes

cta/CTA#30 - Check size of comments before commit them to the Catalogue
cta/CTA#142 - Save logs in liquibase-update test
cta/CTA#127 - DataTransferSession keeps busy while waiting for a mount to be required
cta/CTA#130 - Liquibase-update test fails when CTA is update without oracle support
cta/CTA#127 - DataTransferSession keeps busy while waiting for a mount to be required
cta/CTA#125 - Problem with reading OSM tapes using position by block id


v4.7.10-1

Summary

This is a schema upgrade release for catalogue version 12.0.
For details on how to do the upgrade check: https://eoscta.docs.cern.ch/catalogue/upgrading_the_schema/


Catalogue Schema

cta/CTA#135 - Added EXPORTED and EXPORTED_PENDING states to catalogue
cta/CTA#131 - Fix indexes on columns in CTA Catalogue which use LOWER()
cta/CTA#140 - Added REPACKING_DISABLED state to catalogue
cta/CTA#137 - Added a new column ENCRYPTION_KEY_NAME in the table TAPE_POOL


v4.7.9-2

Summary
This release fixes some packaging issues from v4.7.9-1: when upgrading from v4.7.8-1 to v4.7.9-1 some configuration
files were renamed to configurationfile.rpmsave and thefore some services and cta-cli lost their configuration.
See all details and fix in cta/CTA#119

Bug fixes

cta/CTA#119 [new repo] - Packaging issue with CTA 4.7.8-1/4.7.9-1


v4.7.9-1

Summary
Important NOTE:

This is the first release using the new repo: https://gitlab.cern.ch/cta/CTA

Some issues were already closed, so they can only be found on the old repo: https://gitlab.cern.ch/cta/CTA-old


To make the distinction clear, [old repo] and [new repo] will be used here to distinguish between
issues on both repositories.

Changes to options and default values for cta-taped
This release makes some minor changes to options and default values for cta-taped. All options are
detailed in the cta-taped man page and the provided example files. The changes are outlined below:
Options which have been renamed:

"general FileCatalogConfigFile" was renamed to "taped CatalogueConfigFile"
to be consistent with other options.
"DisableRepackManagement" and "DisableMaintenanceProcess" were
renamed to "UseRepackManagement" and "UseMaintenanceProcess" with default changed from "no" to "yes"
(default semantics are not changed).
"FetchEosFreeSpaceScript" renamed to "externalFreeDiskSpaceScript".

Options for which default values have changed:

LogMask, MountCriteria, BufferCount, TapeLoadTimeout,
UseRAO, RAOLTOAlgorithm, RAOLTOAlgorithmOptions.

The following manual pages have been updated:

cta-admin(1cta)
cta-catalogue-admin-user-create(1cta)
cta-catalogue-schema-create(1cta)
cta-catalogue-schema-drop(1cta)
cta-catalogue-schema-set-production(1cta)
cta-catalogue-schema-verify(1cta)
cta-database-poll(1cta)
cta-fst-gcd(1cta)
cta-readtp(1cta)
cta-restore-deleted-files(1cta)
cta-rmcd(1cta)
cta-smc(1cta)
cta-taped(1cta)
cta-tape-label(1cta)


Features

cta/CTA-old#979  [old repo] - Document configuration options of daemons and command-line tools
cta/CTA-old#1252 [old repo] - Adds Liquibase changelog files for PostgreSQL
cta/CTA#3        [new repo] - Implement listing of DISABLED libraries
cta/CTA#64       [new repo] - Adds support for the OSM Tape Label format to the CTA
cta/CTA-old#1278 [old repo] - Add support for reading multiple tape formats by the ReadtpCmd command
cta/CTA-old#1278 [old repo] - Support multiple tape formats in ReadtpCmd command


Bug fixes

cta/CTA-old#947  [old repo] - cta-taped should log the FST being used for a data transfer
cta/CTA-old#1093 [old repo] - The cta-taped manpage showed outdated config options
cta/CTA-old#1106 [old repo] - "cta-admin schedulinginfo ls" returns records larger than SSI buffer size limit
cta/CTA-old#1269 [old repo] - cta-restore-deleted-files injects wrong diskFileId in CTA Catalogue
cta/CTA-old#1276 [old repo] - Some configuration files should be renamed from .conf to .conf.example

cta/CTA#28       [new repo] - Do NOT allow capacity change of a cartridge if there a still files registered on that tape
cta/CTA#29       [new repo] - cta-taped should log the FST being used for a data transfer
cta/CTA#108      [new repo] - Fix broken CTA containerised installation
cta/CTA#109      [new repo] - Move logic to minimize mounts for multi-copy tape pool recalls out of scheduler logic


Continuous Integration

cta/CTA#88       [new repo] - xrootd 5 pipelines failing in CI
cta/CTA-old#1187 [old repo] - Run EOS5 CI every night
cta/CTA-old#1280 [old repo] - Refactor .gitlab-ci.yml using templates


Building and Packaging

cta/CTA-old#1229 [old repo] - Introduce build of a new taped, using new type of SchedulerDatabase
cta/CTA#111      [new repo] - Remove daemon/LabelSession
cta/CTA-old#1143 [old repo] - HTTP REST API transition to CTA
cta/CTA-old#1224 [old repo] - Remove CASTOR to CTA migration tools RPM


v4.7.8-1

Summary

This is a catalogue schema upgrade release
For details on how to do the upgrade check: https://eoscta.docs.cern.ch/catalogue/upgrading_the_schema/


Catalogue Schema

cta/CTA#1213 - New internal states REPACKING_PENDING & BROKEN_PENDING
cta/CTA#1226 - Remove deprecated catalogue columns


v4.7.7-1

Summary

Features

cta/CTA#1239 - Add support to CTA for multiple tape label formats
cta/CTA#1257 - Refactor CTA/tapeserver/castor/tape/tapeserver/file/File.cpp
cta/CTA#1263 - Abstract ReadSession and FileReader
cta/CTA#1265 - Create base of the dCache OSM label format


Bug fixes

cta/CTA#1267 - Fix scheduling bug introduced in 4.7.6


Continuous Integration

cta/CTA#1266 - Fix eos5 currently failing CI schedules
cta/CTA#1262 - Fix unnecessary log dump in unit-tests


Other

cta/CTA#1251 - Remove dcache leftovers on the grpc-based frontend


v4.7.6-1

Summary

Features

cta/CTA#1238 - All drive down reasons set by cta-taped should start with [cta-taped]
cta/CTA#1254 - Remove tape label option from cta-admin
cta/CTA#1253 - add disk instance to cta-admin vo ls output and fix handling of case sensitive names in cta-admin vo add
cta/CTA#1159 - All drive statuses must be set only in data transfer session and read/write threads


Bug fixes

cta/CTA#1255 - Resolve "VID is missing in DriveHandler when session is killed"
cta/CTA#1256 - Catch and fix EOS regressions in tape specific xrootd API introduced in eos 4.8.79-1 with eos 4.8.87-1
cta/CTA#1247 - Fix improper initialization of the variable m_lastFseq of type uint64_t with -1 value in the constructor of castor::tape::tapeserver::daemon::TapeWriteSingleThread


Building and Packaging

cta/CTA#1224 - Remove CASTOR to CTA migration tools RPM


v4.7.5-1

Summary
This release introduced an additional gRPC based frontend for storage backends. This is still a work in progress and is not ready for use.
The command cta-verify-file now requires the options eos.instance, eos.request.user and eos.request.group to be configured in /etc/cta/cta-cli.conf.

Features

cta/CTA#1222 - Add minimal gRPC based frOntend for integration with dCache
cta/CTA#1241 - make cta-verify-file get instance, request user and group otions from ctacli config file


Bug fixes

cta/CTA#1225 - Fix bug causing tapeserver to sometimes pop the entire archive queue at the end of the mount


Building and Packaging

cta/CTA#1224 - Removes CASTOR to CTA migration tools RPM and references to CASTOR repo