1. 11 Oct, 2016 5 commits
  2. 12 Oct, 2016 2 commits
  3. 10 Oct, 2016 1 commit
  4. 11 Oct, 2016 1 commit
    • Victor Kotlyar's avatar
      Ported commits from castor/master for Encryption: · 1a3812e0
      Victor Kotlyar authored
      6b6374d6c2e209c98c0d4d7aa665e1df83d71aaa
        CASTOR-5350: Introduce encryption SCSI commands in tape drive
        backend
      
        Implementation of two methods:
          * setEncryptionKey(key): Sets encryption params to drive.
          * clearEncryptionKey: Clears encryption params from drive.
      
      3cf91d48f5c7b0cb563c3037aee69ec769f5ab94
        Added support for an interface script that will setup drive
        encryption per tape
      
      e9ca601687508de20fab7154e63bb0dbd1b25a8a
        Migrate TapeWriteSingleThread::TapeCleaning::~TapeCleaning() body to
        .cpp
      
      266b02d8175b5cfc0b688135cbdc335e93060b
        CASTOR-5350: Refactor support for only external key management script
      
      789b26a0bc69053ff1ab792a02676a7753f093ed
        Merge branch 'encryption_backend' into 'master'
      
        CASTOR-5350: Encryption backend
      
        ## Description
          The aim of this merge request is to incorporate encryption support
          into CASTOR.
          The proposed changes are to be used in conjunction with the
          operators'
          [ExternalEncryptionKeyScript](https://gitlab.cern.ch/slaskari/castor-get-encryption-key).
      
          The **aim** is to enable encryption in specific tape pools of
          CASTOR.
        ## Changes
          * Introduce encryption SCSI backend to DriveGeneric.
          * Introduce encryption control wrapper
          * (`castor/tape/tapeserver/daemon/EncryptionControl`) for
          * abstracting the two sub-components of:
            * Calling the `ExternalEncryptionKeyScript`,
            * Calling the equivalent DriveGeneric function for
            * passing/clearing the encryption parameters to/from the drive.
          * Add new configuration option in `castor.conf` for the external key
          * management script.
          * Create a Subprocess wrapper for executing external commands as
          * CASTOR children (`castor/server/Subprocess.{h,c}pp`).
          * Incorporate encryption handling in the:
            * DataTransferSession
            * LabelSession
            * CleanerSession
          * Add encryption control timer in the task Watchdog.
        See merge request !1
      
      fa550707c42d80466bbd448e355aaf9be5ea8e04
        Clear encryption key only when encryption enabled
        Changes include:
          - Making EncryptionControl stateful
          - Calling clearEncryptionKey on the drive only when encryption is
            on.
        Also includes a minor duplicate code fix on DriveGeneric.
      cf4eb9f3ae36c9cfc9c40349d69ab6642020e81e
        Merge branch 'encryption_changes' into 'master'
        Clear encryption key only when encryption enabled
        ## Description
          Changes include:
          - Making EncryptionControl stateful
          - Calling `clearEncryptionKey()` on the drive only when encryption
            is on.
          Also includes a minor duplicate code fix on **DriveGeneric.cpp**.
        See merge request !2
      
      40366d963ee33ca081df6c991189b21369e461fd
        Check if the drive has encryption capability enabled:
          * Add isEncryptionCapEnabled() vendor-specific function
          * Check isEncryptionCapEnabled() before passing encryption params
          * Check isEncryptionCapEnabled() before clearing encryption params
          * Clear encryption key before unencrypted I/O
      
      4ccc661d01eccfc3fdfb9ee2578d15a147a0c55a
        Merge branch 'encryption_capability_enabled' into 'master'
        Drive encryption capabilities inclusion
        ## Description
          The aim of this merge request is to address issues related to
          encryption on drive without the encryption capability enabled.
          More specifically:
            * It introduces a vendor-specific way of identifying if the drive
            * has encryption capability enabled
            * **IBM**: Through the SPIN index SCSI page
            * **Oracle**: Through the general INQUIRY SCSI page
            * If the data to be written are to be encrypted, an additional check
            * of the encryption capability of the drive is made. In case of
            * encrypted data, but no encryption capability, the session fails.
            In essence, all encryption related operations are made modulo the
          encryption capability of the drive.
            Last, in case of unencrypted I/O, we clear the keys of the drive (if
          encryption capable) to avoid encrypted data with previous keys on
          CASTOR's system failure.
        ## Testing
          Before the merge request's submission, the following tests were
          passed:
            On drives with **encryption capability enabled**:
              * Label session
              * Label with previously set encryption key
              * Write without encryption
              * Read without encryption
              * Write with encryption
              * Read with encryption
              * Write with previously set encryption key
              * Read with previously set encryption key
            On drive with **encryption cabability disabled**:
              * Label session
              * Write without encryption
              * Read without encryption
              * Write with encryption - session **should** fail
              * Read with encryption - session **should** fail
          See merge request !3
      
      55b85a2cb4681d697565116c00ff98c6becea4fb
        Secure session against invalid encryption script output
      
      3a54875c680fe6c1c9d5cf25cf98d2780196e0d1
        Changes in encryption workflow
          - VMGR tag is updated only on write operations
          - Empty key signifies no encryption
      
      f5408cf0ccbae9a4ab94a533f3b6d7be323f72fb
        Minor encryption log enhancements
          * Error line in Read/Write session with ErrorMesage key
          * Fix for delimiter in the end of arguments in argsToString()
      2e7204fb0dd24b472a959fa5e13320c34df4f017
        Merging in improvements on tape encryption support.
      
      92533a1746d0744ee528781558a720c63ca3c4d1
        Removed nullptr which is not supported in SLC6's gcc.
        Added automatic deletion of json objects in
        EncryptionControl::parse_json_script_output.
      
      fca3bb9e7fce364b429fc0b5c036fb752fd67ff1
        Fix log typo
      1a3812e0
  5. 07 Oct, 2016 6 commits
  6. 06 Oct, 2016 4 commits
    • Eric Cano's avatar
      Implemented drive status support in drive register structure and code. · 56c4c332
      Eric Cano authored
      Added drive status reporting in scheduler
      Added drive status reporting in OStoreDB
      Added support for drive status listing in the front end
      Removed virtual functions from the Scheduler, which is never overloaded.
      Added DesiredDriveState structure to drive state.
      Removed usage of duplicate MountType, DriveStatus and DriveState structures.
      Created DriveInfo structure to allow recreation of drive register entry in all
      reporting situation (potentially with partial/assumed info).
      56c4c332
    • Victor Kotlyar's avatar
      Ported commit fab791bc805f95d59c55cbcb554e6f218710b0dd from castor/master · 425af9d0
      Victor Kotlyar authored
      CASTOR-4836: tapeserverd should have a new time counter: delivery time
      
      Fixed.
      
      Add two counters: deliveryTime and drainingTime. For the recall
      session deliveryTime is the total time of the disk threads and
      drainigTime is the time difference between deliveryTime and
      the total time of the tape thread or in other words how much time
      disk threads spent after the tape thread finished. For the
      migration session deliveryTime is the total time of the tape thread
      and drainingTime always equal 0.
      425af9d0
    • Victor Kotlyar's avatar
      Ported commit 2a14c5d7ef7bb395a37454789abcbfd7266edcc2 from castor/master · a4776842
      Victor Kotlyar authored
      CASTOR-4982: tapeserverd should tolerate some non-fatal tape alerts
      before writing
      
      Fixed.
      
      Add logic to the TapeWriteSingleThread to skip not-fatal tape
      alerts before writing to the tape. Only "Lost statistics"
      tapeAlertLostStatistics 0x32 tolerated as non-fatal.
      a4776842
    • Victor Kotlyar's avatar
      Ported commit 01b32f6ffc3d28ec7681eff02973ba91191fcc60 castor/master · adfd407e
      Victor Kotlyar authored
      CASTOR-5042: The SCSI layer of the tape server overlooks SCSI errors
      not coming from the target
      
      Fixed.
      
      Add output for all errors available for SCSI call.
      The logic implemented is to go through all errors in the following
      order: status, host_status, driver_status. In case of particular
      error an appropriate exception generated.
      adfd407e
  7. 05 Oct, 2016 4 commits
    • Eric Cano's avatar
      1311a2bf
    • Victor Kotlyar's avatar
      Ported commits from castor/master: · 9dfbe96d
      Victor Kotlyar authored
      83ab8201100d1bcdd5c6d056fd5a395c24bcb885
        Get detailed Firmware Version
          - Differentiate SCSI Inquiry for Oracle to get firmware version
            minor version
          - Change getDriveFirmwareVersion() to get firmware version from SCSI
            backend
      
      6481bcabc4907c8a1813253fe9a2d40cd67d07fe
        Provide DriveGeneric inquiry implementation for DriveMHVTL
      
      2033932f133b9eb7469190972a4f228dd8b16396
        Move Oracle inquiry structure handling inside
        DriveT10000::getDeviceInfo()
      
      a8be8c16a4e3af2f952676f1138213d7e5584705
        Merge branch 'tape-firmware' into 'master'
          Get detailed Firmware Version
          ## Description
            - Differentiate SCSI Inquiry for Oracle to get firmware version
              minor version
            - Change getDriveFirmwareVersion() to get firmware version from SCSI
              backend
          ## Change in logs
            Old version:
              firmwareVersion="468E" # IBM
              firmwareVersion="4.11" # Oracle
            New version:
              firmwareVersion="468E" # IBM
              firmwareVersion="4.11.111" # Oracle
      9dfbe96d
    • Victor Kotlyar's avatar
    • Victor Kotlyar's avatar
      Ported commits from castor/master: · 1797bcd6
      Victor Kotlyar authored
      c5c8509eaf971b2c37a1b2c333693474e14da31b
        ported only radosstriper part of tapserverd
      
        CASTOR-4739 tapeserverd should support localfile, rfio, xroot and
        rados striper access for disk files
      
        Implemented the support for direct rados striper support in
        tapeserverd.
        The expected URL is currently: radosstriper:///user@pool:filePath
        (with no ':' in user and pool).
      
      078bd4b88b19a5d879c5e92d5a6469e645a2f7a2
        Added missing test for the radosstriper URL parsing regex and fixed
        it.
      
      b797c72352d03db79cf62471b8eeec65c4fa1270
        Fixed call striperPool: we now call the exception throwing version.
      1797bcd6
  8. 04 Oct, 2016 2 commits
  9. 03 Oct, 2016 4 commits
    • Victor Kotlyar's avatar
    • Victor Kotlyar's avatar
      Ported commits from castor/master: · 3c60ee4b
      Victor Kotlyar authored
      d28aa41f02ffe8e944e3981d51a73a261613affa
        Volume SCSI metrics implementation for Oracle T10K
      
        After new update of firmware (4.11), Oracle has introduced an
        implementation for SCSI page 0x17. Integrating metrics to CASTOR.
      
      d140f448f8913f0573940a39d391b384d5296dc7
        Add comment for Oracle specific implementation on Volume SCSI page 0x17
      
      1f7c854f222633ea0d7e2c717dc38e276c899a13
        Volume SCSI metrics implementation for Oracle T10K
      
        ## Description
          After new update of firmware (4.11), Oracle has introduced an implementation
          for SCSI page 0x17 (Volume statistics). Integrating metrics to CASTOR.
        ## Metrics
          * validity
          * lifetimeVolumeMounts
          * lifetimeVolumeRecoveredWriteErrors
          * lifetimeVolumeRecoveredReadErrors
          * volumeManufacturingDate
      
      adb79718f17b668d9ac5d19b26708a1b9975150e
      b51dfaafc546047d580d7d609621f88bc9d40f4b
        Add unimplemented MHVTL Volume Statistics SCSI page
      
        Page 0x17 is not implemented in MHVTL.
        As it inherits from DriveT10000D, it should be overridden in order
        to return empty map of metrics values.
      
      8a810d013ace455722b79125586e9551c73716d8
        Add unimplemented MHVTL Volume Statistics SCSI page
      
        ## Description
          Page 0x17 is not implemented in MHVTL.
          As it inherits from DriveT10000D, it should be overridden in order
          to return empty map of metrics values.
      3c60ee4b
    • Victor Kotlyar's avatar
      Ported commits from castor/master: · 28c7456a
      Victor Kotlyar authored
      da054c343578da49fad9792c32a9ed3068c6b6f8
        Removed redundant setting of the TPVID paramter while mounting the
        tape. This had the nasty side effect of removing the parameter after
        leaving the functions (for all the following logs).
      
      a354b3ee7996494fb78fb0337e91cb567f0ae8f0
        CASTOR-5323 Tapeserver logs inconsistencies
      
        Removed duplicate setting of a scoped TPVID parameter in the logs.
        When going out of scope, the parameter
        made TPVID disappear from following logs, which is not desired.
      28c7456a
    • Victor Kotlyar's avatar
      Ported commits from castor/master: · 2c2f4a27
      Victor Kotlyar authored
      e13f841a8c21d35f1bf75dc63d8c98b21d8dd8cb
      b69feea3520772a7515d3faa4bf0b8f6106c5a37
        Add drive's serial number to logs
          ## Description
            Add the drive's serial number, as read by the SCSI Inquiry command
            to the SCSI metrics log lines.
          ## Changes
            * Add the line to the logs
            * Modify session test to check existence of the property in the log
            * line
      2c2f4a27
  10. 30 Sep, 2016 1 commit
    • Victor Kotlyar's avatar
      Ported commits from castor/master for general,drive,volume SCSI statistics: · e8b4ec34
      Victor Kotlyar authored
      fa889fed2541e22179b5e035d863f87e7be18fb9
        CASTOR-5322 RFE: Enhance tapeserverd logs with SCSI tape drive
          statistics
      
      b13f495e4ee229b2469f9470a2ffa6b4003a29ec
        Fix for mhtvl scsi log sense exceptions
      
      ad71058fbcb6de85e0440797d7ffa5358e26bf89
        CASTOR-5329 Enhance tape statistics
      
      806e48f4285122d8ab9f118364a15e740518028f
        CASTOR-5332 RFE: Reduce log level to INFO with MHVTL - SCSI Statistics
          could not be acquired from drive
      
      79c5a4c2c36b7acc5b10505ca1694fd521fc6832
      c7f6d4d7aaa564b37c2b36c3110dfe2fc96ec970
        Move volume SCSI statistics inside the dtor of TapeCleaningMove volume
          SCSI statistics inside the dtor of TapeCleaning
      
        ## Description
      
        When first introduced volume SCSI Statistics (at the moment
        IBM-specific), we explicitly put the function after the unmount of the
        tape was done due to an invalid file descriptor error occurring during
        the SCSI query.
      
        This bug no longer occurs for IBM drives.
        This may be attributed to the update of firmware of the IBM drives
      
        Apart from the change of the position of the changes, there is no
        alteration in terms of the metrics reported from the drive to the logs.
      
        ## Testing
      
        The tests the new code has been through are:
          * Write/Read file on IBM lib0 drive *(older one)*
          * Write/Read file on IBM lib4 drive *(newer one)*
          * Write/Read file on Oracle T10k drive
      e8b4ec34
  11. 28 Sep, 2016 4 commits
  12. 27 Sep, 2016 3 commits
  13. 26 Sep, 2016 2 commits
  14. 23 Sep, 2016 1 commit