when dealing with double copy recalls.

Proper fix by defining a new RECALLJOB_SELECTED2NDCOPY state instead
of hijacking the (deprecated) fileTransactionId.

diskserver.
- Improved logging of the drainRunner job, and cleaned up old unused
messages.

  - avoid looping too much in the tapegateway when some file is locked for long
  - improved handling of log messages when the DLFLogs table is too full :
    + do not take all of it in one go
    + use an index as the table may be scattered

Some fixes + extar logging on rebalancing. In particular, rebalance only online hardware that is in production or readonly

Improved draining by implementing back pressure on the scheduling (no more than 1000 jobs pending per drainingJob) and ordering of files by importance (no tape and fewer copies on disk first)

depending on the job type.
On the way, fixed an incorrect subrequest restart statement.

This includes :
  - merge of d2dtransfer into diskmanagerd
  - drop of the WAITDISKTODISKCOPY state in DiskCopy, and of the diskcopies for ongoing replication in general. Now DiskCopies are only created at the end of the replication, when they actually exist on disk
  - drop of the StageReplicaRequest table and concept
  - introduction of the DiskToDiskCopyJob table, holding the list of ongoing disk to disk copies
  - a split of transferToSchedule into userTransferToSchedule and D2dTransferToSchedule
  - an according split in the dispatcher part of the transfer manager to call both methods. Note that with this commit, both methods fill the same FIFO queue of transfers and both take everything they can find without any throttling.

At this stage, the draining facility is broken.

Merged STAGED and CANBEMIGR into VALID in DiskCopy statuses and create a tapeStatus on the CastorFile with possible values NOTONTAPE, ONTAPE and DISKONLY.
Note that user interface is not touched and still displays CANBEMIGR and STAGED. Only internals (particularly the DB) are concerned.

Fixed bug #101344: CASTOR should not try to create new copies of ARCHIVED, DISABLED and EXPORTED tape copies

to make the creation script work again.

Implemented a new tool, deletediskcopy, to replace cleanLostFiles
and provide all features described in the above bug and its related ones.

- Introduced support for a READONLY state for Diskservers and Filesystems.
Hardware in such a state will accept migrations, [PrepareTo]Get and
DiskCopyReplica requests, and will not accept recalls or [PrepareTo]Put/Update
ones when they switch to write mode.
- Dropped the {DiskServer,FileSystem}.adminStatus fields and the obsoleted
code around them and introduced a new DiskServer.hwOnline boolean field, which
is updated by the heartbeat reports and displayed by printdiskserver.
In turn, modifydiskserver can not change the online flag but only the status.
For backward compatibility, on stager_qry -s a Diskserver will be reported
as DISABLED if hwOnline = 0 regardless its status.

integration of the monitoring of diskserver statuses in the diskmanagerd/transfermanagerd infrastrucutre. In particular rmmasterd, rmnoded and all the shared memory infrastructure have now disappeared

used for long, this is only to cleanup and make the code clear.

Fixed bug #98151: Do not try to recall a diskonly file. Also improved logging of cases where we do not find any segment for a file that should be recalled. Distinguish the case with no segment at all (logged a critical, with note that the file is lost) and the case where all segments are disables (warning only). Also list all disabled segments in the log with the reason for their unavailability (segment status and tape status)

Plus added some code in abortRepack to catch exceptions and reset the state
before raising them to the user.

Added new GCTYPE_OVERWRITTEN constant and used it in recreateCastorFile
to catch when files get overwritten.
Side refactoring: all GCTYPE constants were renamed and added to the dconst package,
as well as all draining processes specific constants.

Fixed bug #97529: "File disappeared from namespace" (during recall or migration) should not be an error

- Fail outstanding requests on recreateCastorFile

table, so to simplify other actions such as aborting or listing (abort
was broken for SUBMITTED tapes)

- Changed the db job interval to 2 minutes

As part of this implementation, a repackd.log log file will be created
with pure DB entries driven by the repackManager() DB job.

Fixed creation of missing segments.

dropped useless status NEW for RecallJobs. This also fixes the issue that new recall jobs could not be handled on the fly by a running recallMount

Introduced a generic procedure to run DB jobs and log any exception they would throw with its full stack trace.

to the schema this time after Sebastien's big refactoring,
only some improvements in the logging (now both migrations and recalls
have a request UUID for each bulk update).

marshalling of arrays or custom types over a db link, hence temporary
tables are used to transfer the data and read back the results.
Also introduced a serrno PL/SQL package in both the NS and Stager
databases.
Adapted the OCCI interface to match the PL/SQL procedure for migration,
still some modifications to be done (the bulk methods don't return anything)
prior to start testing.