Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
cta
MonteCarlo
ProdX-Download
Commits
f8dd0e7d
Commit
f8dd0e7d
authored
Jul 29, 2020
by
Gernot Maier
Browse files
Added downloading script
parent
d0e95110
Changes
2
Hide whitespace changes
Inline
Side-by-side
README.md
View file @
f8dd0e7d
...
...
@@ -7,7 +7,7 @@ available files
## Prepare list of available data sets
DIRAC command
uses
DIRAC command
e.g.
...
...
@@ -60,5 +60,13 @@ Prod5_LaPalma_AdvancedBaseline_NSB1x_electron_North_20deg_DL0.list
Prod5_LaPalma_AdvancedBaseline_NSB1x_electron_North_20deg_DL0.local.list
```
## Downloading files
uses DIRAC command
```
./getRawFilesFromGRID-DIRAC.sh Prod5_LaPalma_AdvancedBaseline_NSB1x/Prod5_LaPalma_AdvancedBaseline_NSB1x_gamma-diffuse_North_20deg_DL0.GRID.list Prod5_LaPalma_AdvancedBaseline_NSB1x/gamma_cone 100
```
(this obviously takes quite a while)
getRawFilesFromGRID-DIRAC.sh
0 → 100755
View file @
f8dd0e7d
#!/bin/bash
#
# simple script to download raw files from the GRID
# using DIRAC tools
# (adjusted to DESY environment)
#
#
if
[
$#
-lt
3
]
;
then
echo
"
./getRawFilesFromGRID-DIRAC.sh <run list> <target directory> <max files (100)>
download using DIRAC tools
(adjusted to DESY environment)
"
exit
fi
PDIR
=
`
pwd
`
if
[
-e
${
2
}
]
then
mkdir
-p
${
2
}
fi
# temporary directory for file lists
FFN
=
`
basename
$1
`
mkdir
-p
${
2
}
/tmplists
# loop over all files in the list
NTMPLIST
=
`
wc
-l
$1
|
awk
'{print $1}'
`
FILEN
=
${
3
}
for
((
l
=
1
;
l <
$NTMPLIST
;
l+
=
$FILEN
))
do
# create file lists with $FILEN files each
let
"k =
$l
+
$FILEN
- 1"
let
"z =
$z
+ 1"
LLIST
=
$2
/tmplists/
$FFN
.tmplist.d.
$z
.list
echo
$LLIST
sed
-n
"
$l
,
$k
p"
$1
>
$LLIST
# check if files are on disk
FF
=
`
cat
$LLIST
`
LFIST
=
$FFN
.tmplist.f.
$z
.list
touch
$2
/tmplists/
$LFIST
for
F
in
$FF
do
FIL
=
`
basename
$F
`
if
[
!
-e
$2
/
$FIL
]
then
echo
$F
>>
$2
/tmplists/
$LFIST
fi
done
NWC
=
`
wc
-l
$2
/tmplists/
$LFIST
|
awk
{
'print $1'
}
`
echo
$NWC
if
[
"
$NWC
"
-gt
0
]
then
# run dirac get files
echo
"Downloading
$NWC
files"
cd
${
2
}
if
[[
-e
tmplists/
$LFIST
]]
;
then
dirac-dms-get-file tmplists/
$LFIST
else
echo
"File list not found: tmplists/
$LFIST
"
fi
fi
cd
${
PDIR
}
done
cd
${
PDIR
}
exit
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment