ReleaseNotes.md



v4.NEXT

Summary

Upgrade Instructions

Features

cta/CTA#1205 - Fail pipeline if cppcheck detects errors
cta/CTA#1206 - Change NULL for nullptr
cta/CTA#1217 - Schema verification should just issue an warning if there are extra indexes in the db that are not in the catalogue
cta/CTA#1220 - Improve queued retrieve logging message
cta/CTA#1152 - Reduce eos free space query load


Bug fixes

cta/CTA#1120 - Fix negative disk space reservation content


Building and Packaging

Catalogue Schema

v4.7.3-1

Summary

Upgrade Instructions

Features

cta/CTA#1161 - Tape server refactoring, "Decide where m_reportPacker.setTapeDone() should be called"
cta/CTA#1195 - cta-catalogue-schema-drop should drop the CTA_CATALOGUE table last


Bug fixes

cta/CTA#950 - Eliminate race condition preventing the drive to go down on failure
cta/CTA#1160 - Fix DrainingToDisk stale status in case if there is DiskWriteThreadPool thread left
cta/operations#708 - Fix "Should run cleaner but VID is missing. Putting the drive down"
cta/CTA#1197 - Return code from cta-catalogue-schema-verify should indicate if the schema is in UPGRADING state
cta/CTA#1206 - Change NULL for nullptr


v4.7.2-1

Summary

Deprecated: Replaced by release v4.7.3-1


v4.7.1-1

Summary

Upgrade Instructions

Features

cta/CTA#1179 - Use std::optional instead of cta::optional
cta/CTA#1190 - Use std::make_unique instead of cta::make_unique
cta/CTA#1198 - Use hardcoded mount policy for verification requests
Add verification flag to queued retrieve request log message
propagate labelFormat from TAPE catalog to VolumeInfo
cta/CTA#1200 - Remove range class


v4.7.0-1

Summary

Upgrade Instructions
This CTA release requires a non-backwards compatible database schema upgrade to CTA catalogue schema v10.0.
Please consult the database upgrade documentation.

Features

cta/CTA#1163 - cta-admin now prefixes the drivename with a '!' if the respective logical library is disabled
cta/CTA#1168 - Add configuration option for scheduler stack size to /etc/cta/cta-frontend-xrootd.conf
cta/CTA#1151 - Update cta-admin ds add for catalogue schema version 10


Bug fixes

cta/CTA#1156 - failed to instantiate the RAO algorithm


Building and Packaging

cta/CTA#823  - Ensure unit tests CANNOT be executed against a production database
cta/CTA#1082 - Review software license text in CTA
cta/CTA#1153 - Change compiler FLAGS in cmake files
cta/CTA#1155 - Remove GCC suppresions in DriveGeneric and file/Structures
cta/CTA#1173 - Remove GCC '-Wno-unused-function' suppresion
cta/CTA#1166 - Add SCL-RH Repo to CTA CI boot scripts
cta/CTA#1185 - Make expectedLabelStandard default argument (for Enstore reads)


Catalogue Schema

cta/CTA#1043 - Add verification column to tape table
cta/CTA#1158 - Check all foreign key references have a full index on both sides of the constraint
cta/CTA#1171 - Add LABEL_FORMAT column to TAPE table
cta/CTA#1172 - Add disabled reason column to logical library table
cta/CTA#1177 - Fix CTA catalogue schema verify index checker for PostgreSQL
cta/CTA#1147 - Add foreign key constraint between the disk instance name of a vo and the disk instance table
cta/CTA#1151 - Drop ACTIVITY_WEIGHTS and TAPE_DRIVE tables


v4.6.1-1

Summary

Upgrade Instructions
This CTA release requires a database schema upgrade to CTA catalogue schema v4.6.
Please consult the database upgrade documentation.

Features

cta/CTA#1137 - Stop deletion of failed retrieve/archive requests
cta/CTA#1150 - Add option to pass desired catalogue version into cta-catalogue-schema-create
cta/CTA#1147 - Add Disk Instance Column to VO table
cta/CTA#1119 - Remove support of manual mode for loading tapes
cta/CTA#1123 - Add mount id to disk space reservations, prevent tape servers from releasing disk space from a previous mount


Bug fixes

cta/CTA#1138 - sortAndGetTapesForMountInfo only queries tapes in the current logical library
cta/CTA#1117 - Update masterDataInBytes when writing files to tape
cta/CTA#1125 - cta-admin dr ls should show '-' instead of "NO_MOUNT" for Mount Type


Continuous Integration

cta/CTA#1131 - Compile CTA using devtoolset-8 in CI
cta/CTA#1126 - Create tests for TapeDrivesCatalogueState


v4.6.0-1

Summary
Updates EOS version in CI to 4.8.75.

Upgrade Instructions
This CTA release requires a database schema upgrade to CTA catalogue schema v4.5.
Please consult the database upgrade documentation.

Features

cta/CTA#999  - Add a default mount rule for recalls
cta/CTA#1109 - Add --dirtybit option to cta-admin ta ch and show dirty bit value in cta-admin --json ta ls
cta/CTA#1107 - add mountpolicyname to request schema in objectstore
cta/CTA#1111 - Add disk instance and disk instance tables to catalogue and respective cta-admin diskinstance/diskinstancespace add/ls/ch/rm commands
cta/CTA#1114 - Log SSI events in ctafrontend
cta/CTA#1108 - Make cta-admin --json sq show mount policy with highest priority and mount policy with lowest request age for each queue.


Bug fixes

cta/CTA#1102 - Make requeued jobs retain their original creation time
cta/CTA#1091 - Move TAPE_DRIVE table to DRIVE_STATE and refactor code to update drive states
cta/CTA#1110 - Moves disk space reservations to the DRIVE_STATE table, uses atomic updates
Don't throw an exception if DiskSpaceReservationRequest brings reservation < 0, just reset and log the error


Continuous Integration

cta/CTA#734  - Adds test for FST delete-on-close behaviour
Adds system tests for 'prepare' and 'query prepare' (idempotent prepare)


v4.5.1-2

Summary
No code changes. Updates EOS version in CI to 4.8.74.

v4.5.1-1

Summary

Bug fixes

cta/CTA#1101 - Fix disk space reservation logic adding all existing disk space reservations for all disk systems
cta/CTA#1023 - Retrieve puts the queue to sleep if the eos disk instance is not reachable


v4.5.0-1

Summary

Features

Improve cta-versionlock script
cta/CTA#1091 - New Drive State table in CTA Catalogue
cta/CTA#1054 - Fix filing of disk buffer when recalling from tapeservers with RAO
cta/CTA#1076 - Retrieve fails if disk system configuration is removed
cta/CTA#1087 - Add new tapeserver config option UseEncryption
cta/CTA#1096 - Better handling of bad checksums in archive/retrieve sessions


Bug fixes

cta/CTA#1092 - Fix overflow error with drive state latestBandwith causing cta frontend crash
cta/CTA#501 - Fix disappearing reason when TapeDrive is reading or writing


v4.4.1-1

Bug fixes

cta/CTA#1092 Fix overflow with drive bandwith causing frontend to crash


v4.4.0-1

Upgrade Instructions
This CTA release requires a database schema upgrade to CTA catalogue schema v4.3. Please consult the database upgrade documentation.

Features

Upgraded EOS to 4.8.67 in CI versionlock.list file

EOS/EOS-4976 Fix activity field passed from EOS to CTA


cta/CTA#607 - Add client host and username in cta-frontend logs
cta/CTA#777 - Minimize mounts for dual copy tape pool recalls
cta/CTA#928 - Add youngest request age to cta-admin sq
cta/CTA#1020 - cta-restore-deleted-files command for restoring deleted files
cta/CTA#1026 - Add activity Mount Policy resolution to CTA
cta/CTA#1057 - Remove support for MySQL
cta/CTA#1069 - Open BackendVFS ObjectStore files in R/W mode when obtaining exclusive locks
cta/CTA#1070 - Update eos to version 4.8.67
cta/CTA#1074 - Improve error reporting when retrieving archive
cta/CTA#1077 - Remove activity fair scheduling logic
cta/CTA#1083 - Upgrade ceph to version 15.2.15
cta/CTA#1068 - Build cta with gcc 7.x(C++17) and fix deprecated code


Bug fixes

cta/CTA#1059 - Clean up warnings reported by static analysis
cta/CTA#1062 - cta-admin tf rm should store the diskFilePath when deleting the tape file copy
cta/CTA#1073 - Retry failed reporting for archive jobs
cta/CTA#1078 - fix STALE message when cta-taped is restarting
cta/CTA#1081 - refactor database queries for drive ls


v4.3-3

Summary

Features

cta/CTA#1053 Remove the objectstore presence in another classes
cta/CTA#501 cta-taped should set the state to DOWN when machine rebooting


Bug fixes

cta/CTA#1056 Fix bugs in cta 4.3-2
cta/CTA#1058 Remove helgrind_annotator from production


v4.3-2

Summary

Features

cta-admin: All short options with more than one character now require two dashes


Bug fixes

cta/CTA#1013 reportType uninitialized
cta/CTA#1044 Fix segmentation fault due to uninitialized optional value and remove diskSpaceReservations from cta-admin dr ls


v4.3-1

Summary

Features

cta/CTA#976 Add logical part of Drive Status using Catalogue
cta/CTA#988 Add diskSpaceReservations map in cta-admin --json dr ls output.
cta/CTA#983

Add cta-versionlock helper script to cta-release package
Update cta repo file to use the new public repo


cta/CTA#1036 Better error reporting in cta-admin tools
cta/CTA#1039 Improve logging of cta admin commands in cta frontend
cta/CTA#1041 Fix host values in cta-admin commands


Bug fixes

cta/CTA#501 cta-taped should set the state to DOWN when machine rebooting
cta/CTA#955 cta-taped daemon should stop "immediately" and cleanly when systemctl stop/restart is executed
cta/CTA#991  Drive is not put down if the device file is removed while cta-taped is running and before a data transfer
cta/CTA#996  Removes PARALLEL from migration scripts
cta/CTA#1029 Fix segmentatin fault in frontend when list repacks of a tape that has been deleted in the catalogue
cta/CTA#1031 Fix Warning in updateDriveStatus
cta/CTA#1032 cta-admin dr ls crashes the frontend if executed during an archive/retrieve


v4.2-3

Summary

Features

cta/CTA#983

Add cta-versionlock helper script to cta-release package
Update cta repo file to use the new public repo


cta/CTA#1036 Better error reporting in cta-admin tools
cta/CTA#1039 Improve logging of cta admin commands in cta frontend
cta/CTA#1041 Fix host values in cta-admin commands
cta/CTA#1026 cta-admin sq now shows the mount policies of potential mounts


Bug fixes

cta/CTA#1029 Fix segmentatin fault in frontend when list repacks of a tape that has been deleted in the catalogue
cta/CTA#1032 cta-admin dr ls crashes the frontend if executed during an archive/retrieve
cta/CTA#996  Removes PARALLEL from migration scripts
cta/CTA#1035 log configuration options on frontend startup
cta/CTA#1042 Do not iterate over retrieve queues when holding global scheduler lock

Repacks on a disabled tape must now use a mount policy whose name starts with repack
There is no longer an empty mount when a disabled/broken tape queue is full of deleted requests


cta/CTA#1027 Mitigate popNextBatch bad behaviour in archive queues
Migration tools: fixes filemode of files imported from CASTOR to EOS


v4.2-2

Summary

Bug fixes

cta/CTA#1029 Fix segmentatin fault in frontend when list repacks of a tape that has been deleted in the catalogue


v4.2-1

Summary

Features

Catalogue schema version 4.2
cta/CTA#1001 Maximum file size is now defined by VO instead of globally.
cta/CTA#1019 New command cta-readtp allows reading files from tape and verifying their checksum


v4.1-1

Summary

Features

Catalogue schema version 4.1
cta/CTA#1016 New options for filtering deleted files using cta-admin rtf ls command.
cta/CTA#983 Add cta-release package for public binary rpm distribution.
cta/CTA#980 Add external encryption script option
cta/CTA#976 Define a new table in the DB schema to contain the drive status.
cta/CTA#834 New command "recycletf restore" allows undeleting a copy of a file from tape deleted using tapefile rm
[frontend] New command "tapefile rm" allows deleting a copy of a file from tape


Bug fixes

cta/CTA#1014 Fix last column alignment when more than 1000 items are listed.


v4.0-5

Summary

Features

[frontend] Add options to "tapepool ls" to filter tapepools on their name, vo and encryption
cta/CTA#898 cta-send-event now gets the requester id and eos instance as command line arguments
cta/CTA#1005 "tape ls" now can filter tapes on wether they were imported from CASTOR
cta/CTA#1006 "repack ls" now shows the tapepool of the tape being repacked


v4.0-5

Summary

Features

[frontend] Add options to "tapepool ls" to filter tapepools on their name, vo and encryption
cta/CTA#898 cta-send-event now gets the requester id and eos instance as command line arguments
cta/CTA#1005 "tape ls" now can filter tapes on wether they were imported from CASTOR
cta/CTA#1006 "repack ls" now shows the tapepool of the tape being repacked


Bug fixes

[frontend] Adds missing break after "schedulinginfo ls" command
cta/CTA#999 Adds a default mount rule
cta/CTA#1003 The expansion of a repack request now fails if the archive route for archiving the repacked files is missing


v4.0-4

Bug fixes

cta/CTA#1002 Do not requeue report jobs when reportType is NoReportRequired


v4.0-3

Features

Upgraded ceph to version 14.2.20
Adds cta-verify-file to cta-cli RPM


v4.0-2

Summary

Features

Upgraded EOS to 4.8.45 in CI versionlock.list file

EOS/EOS-4658 EOS workflow engine should not insist on the W_OK mode bit (for prepare ACL)
EOS/EOS-4684 Make the "file archived" GC aware of different EOS spaces


Upgraded eos-xrootd to 4.12.8 in CI versionlock.list file
cta/CTA#966 Unable to distinguish empty and wrong tape pool
cta/CTA#926 Improve MigrationReportPacker::ReportSkipped::execute() exception message
cta/CTA#584 Validate checksum when recalling from tape


Bug fixes

cta/CTA#582 Changes cta-admin storageclass add --copynb to --numberofcopies
cta/CTA#987 Do not import metadata for zero-length files into CTA Catalogue
cta/CTA#984 Removes diskFileGid == 0 check
cta/CTA#669 cta-taped now display the correct error message when the drive device does not exist
cta/CTA#777 Minimize mounts for dual copy tape pool recalls - Temporary fix
cta/CTA#927 Bad message for archive route pointing non-existing pool
cta/CTA#930 Batched the queueing and the deleting of the repack subrequests
cta/CTA#969 xrdfs query prepare malformed JSON output
cta/CTA#972 Updates cta-taped and rmcd man pages
cta/CTA#967 CI DB cleanup issues
cta/CTA#982 cta-fst-gcd ignored eos files with fid>4294967295
cta/operations#352 Unable to delete empty tape pool


v4.0-1

Summary
This version contains the last and clean version of the CTA catalogue schema. This CTA version can not run anymore with the
CTA v3.1-14.

Features

Catalogue schema version 4.0
cta/CTA#964 Adds failure log messages to processCLOSEW in CTA Frontend
When the operator submits a tape to repack, a check is done about the tape state before queueing the repack request to ensure it can be repacked
Oracle catalogue migration scripts 3.1to3.2.sql: replaced DELETE FROM table_name by TRUNCATE TABLE table_name


v3.2-1

Summary
This version is a transition version between CTA v3.1-14 and CTA v4.0. The functionalities of the v4.0 are implemented,
but at the database level, the columns to be deleted have been put as NULLABLE and the NOT NULL columns to be added as NULLABLE.
The xrootd-ssi-protobuf-interface is not up-to-date with the CTA v4.0: deprecated fields have not been removed.

Features

Catalogue schema version 3.2
Upgraded EOS to 4.8.37-1
cta/CTA#922 The superseded concept has been removed and replaced by a new recycle bin
cta/CTA#943 A new tape lifecycle logic has been implemented
cta/CTA#948 The max drives allowed for reading and writing are now set per virtual organization and not per mount policy anymore
cta/CTA#952 Reclaiming a tape resets the IS_FROM_CASTOR flag to 0
cta/CTA#951 The query used by RdbmsCatalogueTapeContentsItor has been put back to the state it was in v3.1-13
cta/CTA#883 Tape verification tool


v3.1-14

Summary

Features

Upgraded EOS to 4.8.35-1
cta/CTA#954 The drive is put down if the CleanerSession fails to eject the tape from it after a shutdown
cta/CTA#945 The timeout for tape load action is now configurable


Bug fixes

cta/CTA#957 The state of the successful Archive for repack jobs is now changed before being queued for reporting
cta/CTA#958 The RepackRequest garbage collection now changes the owner of the RepackRequest


v3.1-13

Summary

Features

Upgraded EOS to 4.8.34-1
Upgraded xrootd to 4.12.6-1


Bug fixes

cta/CTA#941 Slow cta-admin sq even when there is very little activity
cta/CTA#951 Improve the performance of RdbmsCatalogueTapeContentsItor
cta/CTA#939 cta-objectstore-dereference-removed-queue removes all kind of manually deleted queues from the RootEntry


v3.1-12

Features

Upgraded EOS to 4.8.30-1


v3.1-11

Features

cta/CTA#932 Add environment file for cta-frontend service: frontend configured to use 10 XRootD polling threads by default
cta/CTA#292 Allow non interactive usages of cta-admin with sss authentication
Upgraded EOS to 4.8.29-1

cta/operations#155 Fix for conversion issues
cta/operations#154 Improve sys.retrieve.req_id to allow to cancel retrieves on a running instance: adding epoch timestamp in ids
EOS-4505 Separate archive and retrieve ACLs in EOS: only needs p ACL for prepare


Bug fixes

cta/operations#150 high priority Archive job not scheduled when Repack is running: fixed