README.md 2.26 KB
Newer Older
Gernot Maier's avatar
Gernot Maier committed
1
# Introduction
Gernot Maier's avatar
Gernot Maier committed
2

Gernot Maier's avatar
Gernot Maier committed
3
4
5
6
Some simple script to prepare and steer file lists from CTA prod5 production and compare it with locally (DESY)
available files

# Usage
Gernot Maier's avatar
Gernot Maier committed
7
8
9

## Prepare list of available data sets

Gernot Maier's avatar
Gernot Maier committed
10
uses DIRAC command
Gernot Maier's avatar
Gernot Maier committed
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

e.g.

```
cta-prod-show-dataset | grep Prod5_Paranal_AdvancedBaseline_NSB5x  | grep DL0 >| Prod5_Paranal_AdvancedBaseline_NSB5x.list
```

This should result in a list of availabe data sets, e.g.,

```
Prod5_Paranal_AdvancedBaseline_NSB5x_electron_North_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_electron_South_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_gamma-diffuse_North_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_gamma-diffuse_South_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_gamma_North_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_gamma_South_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_proton_North_20deg_DL0
Prod5_Paranal_AdvancedBaseline_NSB5x_proton_South_20deg_DL0
```

## Prepare list of data files

Prepare lists of data files into the directory of the corresponding dataset.

e.g. into Prod5_LaPalma_AdvancedBaseline_NSB1x:

```
./getFiles_and_checkifLocal.sh lists Prod5_LaPalma_AdvancedBaseline_NSB1x
```

This queries the CTA production DB and puts file lists into Prod5_LaPalma_AdvancedBaseline_NSB1x/<particle type>

**Note the difference in naming: gamma-diffuse is renamed for Eventdisplay into gamma_cone**

## Check which files are on DESY dCache, lustre, GRID

The following commands queries both dCache and lustre for files.
Gernot Maier's avatar
Gernot Maier committed
48
This command takes usually a long time as it has to query thousands of files.
Gernot Maier's avatar
Gernot Maier committed
49
50
51
52
53

```
./getFiles_and_checkifLocal.sh check Prod5_LaPalma_AdvancedBaseline_NSB1x
```

Gernot Maier's avatar
Gernot Maier committed
54
55
56
57
58
59
60
61
62
This will prepare the following files for each particle type, and azimuth direction:

```
Prod5_LaPalma_AdvancedBaseline_NSB1x_electron_North_20deg_DL0.dCache.list
Prod5_LaPalma_AdvancedBaseline_NSB1x_electron_North_20deg_DL0.GRID.list
Prod5_LaPalma_AdvancedBaseline_NSB1x_electron_North_20deg_DL0.list
Prod5_LaPalma_AdvancedBaseline_NSB1x_electron_North_20deg_DL0.local.list
```

Gernot Maier's avatar
Gernot Maier committed
63
## Downloading files
Gernot Maier's avatar
Gernot Maier committed
64

Gernot Maier's avatar
Gernot Maier committed
65
66
67
68
69
70
71
uses DIRAC command

```
./getRawFilesFromGRID-DIRAC.sh Prod5_LaPalma_AdvancedBaseline_NSB1x/Prod5_LaPalma_AdvancedBaseline_NSB1x_gamma-diffuse_North_20deg_DL0.GRID.list Prod5_LaPalma_AdvancedBaseline_NSB1x/gamma_cone 100
```

(this obviously takes quite a while)
Gernot Maier's avatar
Gernot Maier committed
72