CASTOR-4889: tapeserverd's end of session summary could indicate the
files still open

Fixed.

Add reports for still open files in the "Tape session finished"
log message to indicate problematic disk servers. It should be
only visible in case of kill of the data transfer session.

Fixed 32-bit limit bug in DiskWriteTask::checkErrors.
Refs: commit e13f841a8c21d35f1bf75dc63d8c98b21d8dd8cb in castor/master

CASTOR-5419: Ceph aborts when writing a 0-sized buffer

Prevented writes of zero size in DiskWriteTask. They happen
when we read from tape a file aligned with the memory block size.

Replaced castor file header with CTA file header

Moved castor::server::
  BlockingQueue
  Semaphores
  AtomicCounter
  AtomicFlag
into cta::threading::

Ported DataTransferSessionTest unit test  DataTransferSessionGooddayMigration and DataTransferSessionEmptyOnVolReq.

Completed changes so that archive and retrieves can be queued, mounts created and jobs reported done.

Removed deprecated structures from archiveNS and nameserver namespaces. Removed RetrieveToFileRequest.

This implied removing tpconfig, tplabel and tpstat.

Added protection agains creation of checksums of the wrong size and added casts where needed.

Added url style conversion of checksums
Added support for checksums in mockNs
Fixed support for checksums in the scheduler
Re-instated several ASSERT_NO_THROW which were commented out during debugging.

Fixed confusion between archive and retieve in logging "path" and creation of file object by file factory.

Fixed 4 bugs that are preventing DataTransferSessionGooddayRecall test to succeed. One more left: next commit will do

Adjusted unit test to accomodate the new structure (SchedulerDatabase::RetrieveMount is now a pure virtual class).

Reworked the queueing of retrieve jobs in the OStoreDB.
Improved some variable names.

Please note that you will have to install the following CASTOR rpms
in order to compile:

  castor
  castor-devel
  castor-lib
  castor-lib-server
  castor-lib-tape

Renamed CTA/tapeserver/castor to CTA/tapeserver/castor_checkout in preparation for the moving the sub-directory CTA/tapeserver/castor/castor up the directory tree

Fixed.

Removed "h/" from all #includes.

Propagated references to the watchdog to disk and tape threads and tasks.
Added a maps for storing the count of all errors that occured during the session.
Added propagation of the counts to the initial process.
Added error reporting in disk and tapes threads and tasks by using a string marker
allowing to know which part went wrong in high level exception. Exceptions then
bumps up the count in the watchdog (synchronously), and the watchdog sends the
new count to the initial thread later (in the watchdog's thread).
Plus some missing logs fixed when an exception is thrown in a disk write task.

Added interface to reference the recall watchdog to the recall disk thread pool.
Next: Do same for migration. Actually store the reference. Add new error map storing in watchdog. Add error reporting in tape and disk threads.

WIP: Added missing file ID for disk write thread. Added missing error log when an exception is thrown in a disk write task.

WIP: switching to CASTOR-4839 tapeserverd: task injector should decide on closing the session earlier

WIP. Next: log unmount errors (in the RAII)

Finished error counting in data threads.

Removed leftovers of special case snprintf call for double, now that snprintf is the default for doubles.

CASTOR-4823: tapeserverd's main process should log statistics and status of the closing session in all cases

Added new messages between the session process and the main process: addition and removal of log parameters.
Added storage of the log parameters in the catalogueTransferSession.
Added logging of the tape session at the end in the initial process, allowing statistics gathering even in
case of crash. This standard log message is intended to be used as an input for the tape log system.
Changed the default off the formatting of doubles in the log so that we don't end up with hard to parse
scientific notation in the logs.

Added the open and close time in the disk/non-disk time ratio calculation in disk tasks (close is the most important).

Added better measurement of the disk performance to locate slow disk servers during tests in production.