It was wrongly added to OStoreDB::ArchiveMount::getNextJob() instead in a05ad490.

Timeouting full runs after 50 minutes: 10 minutes for namespace creation and 40 minutes for the test, so that gitlab does not times it out and leaves a dirty CI runner.

Performing 100 rm in parallel for rados, this should not be painful as those synchronous rm are mostly waiting

XRD_TIMEOUTRESOLUTION=600   # increased from 15s

replaced eosh script with ls -y except after retrieval as archived and
retrieved are the same status regarding eos...
Looks like sometime ls -y determined archived files is not a growing
function...

client_ar.sh can now write to /eos/ctaeos/preprod with -d option, just complains: Could not remove disk replica for /eos/ctaeos/preprod/ as drop is already done in the wfe script. Should test for disk replica before trying to drop with ls -y on directory.

In the MemQueue, the promise for the next batch was set after the queue was committed, but
before the lock was released (by the last user of the queue, through a shared pointer). This
would lead to an uselessly early start of the next queue batch for writing an avoidable contention
on the object store lock. This would no lead to a pile up though as only 2 thread would be contended
(previous and early starting next).

... in preparation for replacement of RetrieveMount::getNextJob().

The retrieve request now gets properly queued in case of retrieve error.
The errors are counted and the request gets deleted eventually.
A new field was added to the retrive request in object store. This commit
will fail on upgrade if there are retrieve requests still queued at update time.
Cleaned up some unused structures in cta.proto
Minor modifications to ArchiveJobs.

Converted all bytes to Mbytes. Removed extra space in the output.
Reordered fields.

This is a stop gap solution while we wait for efficient archive/retrieve reporting.

This affects only unit tests as taped already relied on getNextJobBatch().

As rados re-creates an object when trying to lock it, we tested for presence before locking.
This is racy as object could be deleted in the mean time.
Instead, we now lock blindly and delete the object if we find it having a zero-size.
As we own the lock, this is safe.

This problem led to issues in garbage colector, where agent gets polled while it could disappear.

This fixes frontend failing to start on file permission error.

Set default to 8k as seen in this documentation:
http://www.rsyslog.com/doc/v8-stable/configuration/global/index.html.

Added the vid in the name of the retrieve queue object.

Also fixed flaws in the cache update logic.

Some further adaptation was needed on the templates.