FAQ
This section present some commonly encountered use-cases and basic solutions regarding ADES/EMS operation or more specifically related to CWL specification.
How to specify the Docker image reference?
In most situations, the CommandLineTool
process will need to run a Docker image. Doing so is as simple as
adding the DockerRequirement
(reference) as follows to the Application Package definition:
{
"cwlVersion": "v1.0",
"requirements": {
"DockerRequirement": {
"dockerPull": "<docker-url>"
}
},
"inputs": ["<...>"],
"outputs": ["<...>"],
}
Note
The Docker image reference must be publicly accessible to allow CWL to pull it. Alternatively, a private Docker reference can be used if the image is locally available. The process will fail to execute if it cannot resolve the reference.
The Application Package can be provided during process deployment. Please refer to below references for more details.
See also
Supported Application Package definitions in WPS-REST deployment.
Deploy request.
Fixing permission error on input files
Some processes expect their inputs to be writable (e.g.: ZIP files). When running an Application Package based on a Docker image, Weaver mounts the input files as volumes in read-only mode for security reasons. This causes these processes to immediately fail as the running user cannot override nor write temporary files in the same directory (where the volume was mounted to), as it is marked with read permissions.
To resolve this issue, the application developer should add the InitialWorkDirRequirement
(reference, example) to his CWL package definition. This tells CWL to stage the files into the
Docker container into the running directory where the user will be allowed to generate outputs, and therefore,
also allow edition of the inputs or generation of temporary files as when unpacking a compressed file.
As example, the CWL definition could be similar to the following:
{
"cwlVersion": "v1.0",
"class": "CommandLineTool",
"requirements": {
"DockerRequirement": {
"dockerPull": "<docker-url>"
},
"InitialWorkDirRequirement": {
"listing": [{
"entry": "$(inputs.input_file)",
"writable": true
}
]
}
},
"arguments": ["$(runtime.outdir)"],
"inputs": {
"input_file": {
"type": "File"
}
}
Note that $(inputs.input_file)
within InitialWorkDirRequirement
tells which input to resolve for staging using
the "writable": True
parameter. All files listed there will be mounted with write permissions into working runtime
directory of the executed Docker container.
Problem connecting workflow steps together
See also
Where can I find references to CWL specification and examples?
There exist multiple sources, but official ones provided below have a create amount of examples and are being continuously improved by the developers (including being updated according to changes).
See also below.
Where can I find examples of Process Deployment, Execution and Application Package definitions?
Many examples are reported in detail in the Examples chapter.
Is there an easier way to interact with Weaver for its multiple requests and operations?
Yes!
Take a look at the Weaver CLI and Client chapter for available utilities to interact with Weaver using shell or Python scripting.