GitLab

Ported DataTransferSessionTest unit test  DataTransferSessionGooddayMigration and DataTransferSessionEmptyOnVolReq.

Completed changes so that archive and retrieves can be queued, mounts created and jobs reported done.

Removed deprecated structures from archiveNS and nameserver namespaces. Removed RetrieveToFileRequest.

This implied removing tpconfig, tplabel and tpstat.

Added protection agains creation of checksums of the wrong size and added casts where needed.

Added url style conversion of checksums
Added support for checksums in mockNs
Fixed support for checksums in the scheduler
Re-instated several ASSERT_NO_THROW which were commented out during debugging.

Fixed confusion between archive and retieve in logging "path" and creation of file object by file factory.

Fixed 4 bugs that are preventing DataTransferSessionGooddayRecall test to succeed. One more left: next commit will do

Adjusted unit test to accomodate the new structure (SchedulerDatabase::RetrieveMount is now a pure virtual class).

Reworked the queueing of retrieve jobs in the OStoreDB.
Improved some variable names.

Please note that you will have to install the following CASTOR rpms
in order to compile:

  castor
  castor-devel
  castor-lib
  castor-lib-server
  castor-lib-tape

Renamed CTA/tapeserver/castor to CTA/tapeserver/castor_checkout in preparation for the moving the sub-directory CTA/tapeserver/castor/castor up the directory tree

Fixed.

Removed "h/" from all #includes.

Propagated references to the watchdog to disk and tape threads and tasks.
Added a maps for storing the count of all errors that occured during the session.
Added propagation of the counts to the initial process.
Added error reporting in disk and tapes threads and tasks by using a string marker
allowing to know which part went wrong in high level exception. Exceptions then
bumps up the count in the watchdog (synchronously), and the watchdog sends the
new count to the initial thread later (in the watchdog's thread).
Plus some missing logs fixed when an exception is thrown in a disk write task.

Added interface to reference the recall watchdog to the recall disk thread pool.
Next: Do same for migration. Actually store the reference. Add new error map storing in watchdog. Add error reporting in tape and disk threads.

WIP: Added missing file ID for disk write thread. Added missing error log when an exception is thrown in a disk write task.

WIP: switching to CASTOR-4839 tapeserverd: task injector should decide on closing the session earlier

WIP. Next: log unmount errors (in the RAII)

Finished error counting in data threads.

Removed leftovers of special case snprintf call for double, now that snprintf is the default for doubles.

CASTOR-4823: tapeserverd's main process should log statistics and status of the closing session in all cases

Added new messages between the session process and the main process: addition and removal of log parameters.
Added storage of the log parameters in the catalogueTransferSession.
Added logging of the tape session at the end in the initial process, allowing statistics gathering even in
case of crash. This standard log message is intended to be used as an input for the tape log system.
Changed the default off the formatting of doubles in the log so that we don't end up with hard to parse
scientific notation in the logs.

Added the open and close time in the disk/non-disk time ratio calculation in disk tasks (close is the most important).

Added better measurement of the disk performance to locate slow disk servers during tests in production.

The offending message has been changed in a previous case: CASTOR-4749

Reviewed all bandwidth calculations to homogenize the statistics printing.
Fixed a bug where the total time for a session was not properly retrieved before logging, leading to NaN printouts.

Created a new unit test to try and reproduce the problem seen here. It did not.
The best explanation we have is a stuck file client. As we totally fail to read data
from the tape, it makes no sense to open a file for which we have no data. So we deferred
the file opening when the first memory block arrives from the tape thread. The outputs of the
unit test showed that the file opening has been successfully deferred.

I believe the Timer class is generic enough to be used by any
part of CASTOR and not just tape.