- Oct 25, 2016
-
-
Steven Murray authored
This reverts commit 341d1c7f. After much discussion the commit being reverted has and would have generated more work than problems solved.
-
- Oct 21, 2016
-
-
Steven Murray authored
Before this commit, Daniele had hardcoded into the CTA front-end the constraint that CTA admins must be authenticated by krb5. This commit drops this contraint for two reasons: 1. The xrootd/EOS configuration files already require operators to specify which authentication methods can be used. Daniele’s constraint is not needed to further enforce security, it only serves to constrain admin authentication to krb5 which I personally do not see as providing any added value. 2. The CERN infrastructure uses krb5, other sites like RAL do not.
-
- Oct 20, 2016
-
-
Eric Cano authored
The destructor of the child process was wrongly de-registering the signal handler from epoll. This is now prevented by calling the extra post-fork cleanup function for handlers not involved in a child process before actually running the child process itself.
-
Eric Cano authored
-
Eric Cano authored
-
Eric Cano authored
-
Victor Kotlyar authored
1d92302d5304266ee14a86b8d9fcbd671b567e5c Fix for drive::xx::clearCompressionStats - For DriveT10000, DriveMHVTL: stateful accumulator - For DriveIBM3592, DriveLTO: log select on specific log page e6780a8f0a5ed6366ae750eaecc189ac7d0d07fe Fix overflowing mountTotal{Read,Write}BytesProcessed by bumping up to 64-bit container 12c31edf5193dae90ef65eaa6c3d0eb81a7c6927 Refactor fix for drive::xx::clearCompressionStats f626a773aae60b36146d3170e4eb2ef5fd13db35 Merge branch 'fix_clear_compression_stats' into 'master' Fix clear compression stats ## Description The aim of this merge request is to fix the `drive::clearCompressionStats()` method which deleted all log metrics with log select on page 0x00 (aka reset all metrics). The following changes are introduced: * For **IBM and LTO drives**: We are selectively clearing only the * compression statistics (pages 0x38, 0x32 equivalently) * For **Oracle and MHVTL drives**: Wa are making compression * statistics stateful, by accumulating values and subtracting on * each flush Moreover, we are piggybacking a fix for overflowing mountTotal{Read,Write}BytesProcessed (drive::xx::getTape{Read,Write}Errors) by bumping up the container map in 64-bit value ## Testing The results of the changes were tested against IBM and Oracle drives. More specifically, we attempted to write 40 randomly generated files of 2GB in size with the previous CASTOR version (2.1.16-9), extracted the compression metrics and compared ### IBM **before:** ``` Oct 6 12:23:09 tpsrv240 tapeserverd[14257]: LVL="Info" TID="14268" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tps Oct 6 12:24:53 tpsrv240 tapeserverd[14257]: LVL="Info" TID="14268" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tps Oct 6 12:25:45 tpsrv240 tapeserverd[14257]: LVL="Info" TID="14268" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tps ``` **after: ** ``` Oct 6 12:34:28 tpsrv240 tapeserverd[14836]: LVL="Info" TID="14847" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tps Oct 6 12:36:12 tpsrv240 tapeserverd[14836]: LVL="Info" TID="14847" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tps Oct 6 12:37:04 tpsrv240 tapeserverd[14836]: LVL="Info" TID="14847" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tps ``` ### Oracle **before:** ``` Oct 6 12:21:44 tpsrv614 tapeserverd[3062]: LVL="Info" TID="3073" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv Oct 6 12:24:02 tpsrv614 tapeserverd[3062]: LVL="Info" TID="3073" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv Oct 6 12:25:12 tpsrv614 tapeserverd[3062]: LVL="Info" TID="3073" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv ``` **after:** ``` Oct 6 12:34:58 tpsrv614 tapeserverd[3691]: LVL="Info" TID="3702" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv Oct 6 12:37:16 tpsrv614 tapeserverd[3691]: LVL="Info" TID="3702" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv Oct 6 12:38:26 tpsrv614 tapeserverd[3691]: LVL="Info" TID="3702" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv ``` See merge request !11
-
- Oct 19, 2016
-
-
Eric Cano authored
Remove debug printouts as the previous commit solves the race condition in CI we were investigating.
-
Eric Cano authored
In recalls, failing any step does not require stopping the mount nor end the reporting so we can try and report everything until the session is ended (by an expicit end session report).
-
- Oct 11, 2016
-
-
Steven Murray authored
-
- Oct 17, 2016
-
-
Eric Cano authored
-
- Oct 11, 2016
-
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
Steven Murray authored
-
- Oct 12, 2016
-
-
Eric Cano authored
Enabled string logger printout to better understand intermittent failures in continuous integration.
-
Victor Kotlyar authored
-
- Oct 10, 2016
-
-
Steven Murray authored
-
- Oct 11, 2016
-
-
Victor Kotlyar authored
6b6374d6c2e209c98c0d4d7aa665e1df83d71aaa CASTOR-5350: Introduce encryption SCSI commands in tape drive backend Implementation of two methods: * setEncryptionKey(key): Sets encryption params to drive. * clearEncryptionKey: Clears encryption params from drive. 3cf91d48f5c7b0cb563c3037aee69ec769f5ab94 Added support for an interface script that will setup drive encryption per tape e9ca601687508de20fab7154e63bb0dbd1b25a8a Migrate TapeWriteSingleThread::TapeCleaning::~TapeCleaning() body to .cpp 266b02d8175b5cfc0b688135cbdc335e93060b CASTOR-5350: Refactor support for only external key management script 789b26a0bc69053ff1ab792a02676a7753f093ed Merge branch 'encryption_backend' into 'master' CASTOR-5350: Encryption backend ## Description The aim of this merge request is to incorporate encryption support into CASTOR. The proposed changes are to be used in conjunction with the operators' [ExternalEncryptionKeyScript](https://gitlab.cern.ch/slaskari/castor-get-encryption-key). The **aim** is to enable encryption in specific tape pools of CASTOR. ## Changes * Introduce encryption SCSI backend to DriveGeneric. * Introduce encryption control wrapper * (`castor/tape/tapeserver/daemon/EncryptionControl`) for * abstracting the two sub-components of: * Calling the `ExternalEncryptionKeyScript`, * Calling the equivalent DriveGeneric function for * passing/clearing the encryption parameters to/from the drive. * Add new configuration option in `castor.conf` for the external key * management script. * Create a Subprocess wrapper for executing external commands as * CASTOR children (`castor/server/Subprocess.{h,c}pp`). * Incorporate encryption handling in the: * DataTransferSession * LabelSession * CleanerSession * Add encryption control timer in the task Watchdog. See merge request !1 fa550707c42d80466bbd448e355aaf9be5ea8e04 Clear encryption key only when encryption enabled Changes include: - Making EncryptionControl stateful - Calling clearEncryptionKey on the drive only when encryption is on. Also includes a minor duplicate code fix on DriveGeneric. cf4eb9f3ae36c9cfc9c40349d69ab6642020e81e Merge branch 'encryption_changes' into 'master' Clear encryption key only when encryption enabled ## Description Changes include: - Making EncryptionControl stateful - Calling `clearEncryptionKey()` on the drive only when encryption is on. Also includes a minor duplicate code fix on **DriveGeneric.cpp**. See merge request !2 40366d963ee33ca081df6c991189b21369e461fd Check if the drive has encryption capability enabled: * Add isEncryptionCapEnabled() vendor-specific function * Check isEncryptionCapEnabled() before passing encryption params * Check isEncryptionCapEnabled() before clearing encryption params * Clear encryption key before unencrypted I/O 4ccc661d01eccfc3fdfb9ee2578d15a147a0c55a Merge branch 'encryption_capability_enabled' into 'master' Drive encryption capabilities inclusion ## Description The aim of this merge request is to address issues related to encryption on drive without the encryption capability enabled. More specifically: * It introduces a vendor-specific way of identifying if the drive * has encryption capability enabled * **IBM**: Through the SPIN index SCSI page * **Oracle**: Through the general INQUIRY SCSI page * If the data to be written are to be encrypted, an additional check * of the encryption capability of the drive is made. In case of * encrypted data, but no encryption capability, the session fails. In essence, all encryption related operations are made modulo the encryption capability of the drive. Last, in case of unencrypted I/O, we clear the keys of the drive (if encryption capable) to avoid encrypted data with previous keys on CASTOR's system failure. ## Testing Before the merge request's submission, the following tests were passed: On drives with **encryption capability enabled**: * Label session * Label with previously set encryption key * Write without encryption * Read without encryption * Write with encryption * Read with encryption * Write with previously set encryption key * Read with previously set encryption key On drive with **encryption cabability disabled**: * Label session * Write without encryption * Read without encryption * Write with encryption - session **should** fail * Read with encryption - session **should** fail See merge request !3 55b85a2cb4681d697565116c00ff98c6becea4fb Secure session against invalid encryption script output 3a54875c680fe6c1c9d5cf25cf98d2780196e0d1 Changes in encryption workflow - VMGR tag is updated only on write operations - Empty key signifies no encryption f5408cf0ccbae9a4ab94a533f3b6d7be323f72fb Minor encryption log enhancements * Error line in Read/Write session with ErrorMesage key * Fix for delimiter in the end of arguments in argsToString() 2e7204fb0dd24b472a959fa5e13320c34df4f017 Merging in improvements on tape encryption support. 92533a1746d0744ee528781558a720c63ca3c4d1 Removed nullptr which is not supported in SLC6's gcc. Added automatic deletion of json objects in EncryptionControl::parse_json_script_output. fca3bb9e7fce364b429fc0b5c036fb752fd67ff1 Fix log typo
-
- Oct 07, 2016
-
-
Eric Cano authored
-
Victor Kotlyar authored
CASTOR-4889: tapeserverd's end of session summary could indicate the files still open Fixed. Add reports for still open files in the "Tape session finished" log message to indicate problematic disk servers. It should be only visible in case of kill of the data transfer session.
-
Eric Cano authored
-
Eric Cano authored
-
Eric Cano authored
-
Victor Kotlyar authored
3a67b08bd5b0e0cf4ef03de8185630fb0bcc5e80 CASTOR-5275 RFE: Log the tapeserverd tape-file descriptor if it becomes bad whilst waiting for a tape drive to become ready Fixed. b5d562503b6f954cfc6be09ef742474c685e6d60 CASTOR-5275 RFE: Log the tapeserverd tape-file descriptor if it becomes bad whilst waiting for a tape drive to become ready Differentiated the log messages, and added the file descriptor logging in all instances of the Could not read drive status errors.
-
- Oct 06, 2016
-
-
Eric Cano authored
Added drive status reporting in scheduler Added drive status reporting in OStoreDB Added support for drive status listing in the front end Removed virtual functions from the Scheduler, which is never overloaded. Added DesiredDriveState structure to drive state. Removed usage of duplicate MountType, DriveStatus and DriveState structures. Created DriveInfo structure to allow recreation of drive register entry in all reporting situation (potentially with partial/assumed info).
-
Victor Kotlyar authored
CASTOR-4836: tapeserverd should have a new time counter: delivery time Fixed. Add two counters: deliveryTime and drainingTime. For the recall session deliveryTime is the total time of the disk threads and drainigTime is the time difference between deliveryTime and the total time of the tape thread or in other words how much time disk threads spent after the tape thread finished. For the migration session deliveryTime is the total time of the tape thread and drainingTime always equal 0.
-
Victor Kotlyar authored
CASTOR-4982: tapeserverd should tolerate some non-fatal tape alerts before writing Fixed. Add logic to the TapeWriteSingleThread to skip not-fatal tape alerts before writing to the tape. Only "Lost statistics" tapeAlertLostStatistics 0x32 tolerated as non-fatal.
-