Weaver CLI and Client¶
Once Weaver package is installed (see Installation), it provides a command line interface (CLI)
as well as a Python weaver.cli.WeaverClient
to allow simplified interactions through shell calls or
Python scripts.
This offers to the user methods to use file references (e.g.: local CWL Application Package definition) to rapidly operate with functionalities such as Deploy, Describe, Execute and any other operation described in Managing processes included in Weaver ADES/EMS section.
Please refer to following sections for more details.
Python Client Commands¶
For details about using the Python weaver.cli.WeaverClient
, please refer directly to its class
documentation and its underlying methods.
Shell CLI Commands¶
Following are the detail for the shell CLI which provides the same features.
Run Weaver operations.
usage: weaver [-h] [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[--version]
{deploy,undeploy,capabilities,processes,describe,execute,monitor,dismiss,status,results,upload}
...
Optional Arguments¶
- --version, -V
Display the version of the package.
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Operations¶
Name of the operation to run.
- operation
Possible choices: deploy, undeploy, capabilities, processes, describe, execute, monitor, dismiss, status, results, upload
Sub-commands:¶
deploy¶
Deploy a process.
weaver deploy [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] -u URL [-p PROCESS_ID] [-b BODY] [--cwl CWL | --wps WPS]
[-t TOKEN] [-U USERNAME] [-P PASSWORD] [-D]
Optional Arguments¶
- -U, --username
Username to compute the authentication token for Docker image retrieval from a private registry.
- -P, --password
Password to compute the authentication token for Docker image retrieval from a private registry.
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -u, --url
URL of the instance to run operations.
Optional Arguments¶
- -p, --id, --process
Process identifier for deployment. If no
--body
is provided, this is required. Otherwise, provided value overrides the corresponding ID in the body.- -b, --body
Deployment body directly provided. Allows both JSON and YAML format when using file reference. If provided in combination with process ID or CWL, they will override the corresponding content. Can be provided either with a local file, an URL or literal string contents formatted as JSON.
- --cwl
Application Package of the process defined using Common Workflow Language (CWL) as JSON or YAML format when provided by file reference. File reference can be a local file or URL location. Can also be provided as literal string contents formatted as JSON. Provided contents will be inserted into an automatically generated request deploy body if none was specified with
--body
option (note:--process
must be specified instead in that case). Otherwise, it will override the appropriate execution unit section within the provided deploy body.- --wps
Reference URL to a specific process under a Web Processing Service (WPS) to package as OGC-API Process.
- -t, --token
Authentication token to retrieve a Docker image reference from a private registry during execution.
- -D, --delete, --undeploy
Perform undeploy step as applicable prior to deployment to avoid conflict with exiting process.
Default: False
undeploy¶
Undeploy an existing process.
weaver undeploy [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] -u URL -p PROCESS_ID
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -u, --url
URL of the instance to run operations.
- -p, --id, --process
Identifier of the process to run undeploy operation.
capabilities (processes)¶
List available processes.
weaver capabilities [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] -u URL
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -u, --url
URL of the instance to run operations.
describe¶
Obtain an existing process description.
weaver describe [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] -u URL -p PROCESS_ID [-S {OLD,OGC}]
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -u, --url
URL of the instance to run operations.
- -p, --id, --process
Identifier of the process to run describe operation.
Optional Arguments¶
- -S, --schema
Possible choices: OLD, OGC
Representation schema of the returned process description.
Default: “OGC”
execute¶
Submit a job execution for an existing process.
weaver execute [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] -u URL -p PROCESS_ID -I INPUTS [-M] [-T TIMEOUT]
[-W INTERVAL]
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -u, --url
URL of the instance to run operations.
- -p, --id, --process
Identifier of the process to run execute operation.
- -I, --inputs
Literal input definitions, or a file path or URL reference to JSON or YAML contents defining job inputs with OGC-API or CWL schema. This parameter is required.
To provide inputs using a file reference, refer to relevant CWL Job schema or API request schema for selected format. Both mapping and listing formats are supported.
To execute a process without any inputs (e.g.: using its defaults), supply an explicit empty input (i.e.:
-I ""
or loaded from file as{}
).To provide inputs using literal command-line definitions, inputs should be specified using
<id>=<value>
convention, with distinct-I
options for each applicable input value.Values that require other type than string to be converted for job submission can include the type following the ID using a colon separator (i.e.:
<id>:<type>=<value>
). For example, an integer could be specified as follows:number:int=1
while a floating point number would be:number:float=1.23
.File references (
href
) should be specified usingFile
as the type (i.e.:input:File=http://...
). Note thatFile
in this case is expected to be an URL location where the file can be download from. When a local file is supplied, Weaver will automatically convert it to a remote Vault File in order to upload it and make it available for the remote process.Array input (
maxOccurs > 1
) should be specified using semicolon (;) separated values. The type of an item of this array can also be provided (i.e.:array:int=1;2;3
).Example:
-I message='Hello Weaver' -I value:int=1234
Optional Arguments¶
- -M, --monitor
Automatically perform the monitoring operation following job submission to retrieve final results. If not requested, the created job status location is directly returned.
Default: False
- -T, --timeout
Wait timeout (seconds) of the maximum monitoring duration of the job execution (default: 60s). If this timeout is reached but job is still running, another call directly to the monitoring operation can be done to resume monitoring. The job execution itself will not stop in case of timeout.
Default: 60
- -W, --wait, --interval
Wait interval (seconds) between each job status polling during monitoring (default: 5s).
Default: 5
monitor¶
Monitor a pending or running job execution until completion or up to a maximum wait time.
weaver monitor [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] [-u URL] -j JOB_REFERENCE [-T TIMEOUT] [-W INTERVAL]
Required Arguments¶
- -j, --job
Job URL or UUID to run monitor operation. If full Job URL is provided, the instance
--url
parameter can be omitted.
Optional Arguments¶
- -u, --url
URL of the instance to run operations.
- -T, --timeout
Wait timeout (seconds) of the maximum monitoring duration of the job execution (default: 60s). If this timeout is reached but job is still running, another call directly to the monitoring operation can be done to resume monitoring. The job execution itself will not stop in case of timeout.
Default: 60
- -W, --wait, --interval
Wait interval (seconds) between each job status polling during monitoring (default: 5s).
Default: 5
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
dismiss¶
Dismiss a pending or running job, or wipe any finished job results.
weaver dismiss [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] [-u URL] -j JOB_REFERENCE
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -j, --job
Job URL or UUID to run dismiss operation. If full Job URL is provided, the instance
--url
parameter can be omitted.
Optional Arguments¶
- -u, --url
URL of the instance to run operations.
status¶
Obtain the status of a job using a reference UUID or URL. This is equivalent to doing a single-shot ‘monitor’ operation without any pooling or retries.
weaver status [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] [-u URL] -j JOB_REFERENCE
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -j, --job
Job URL or UUID to run status operation. If full Job URL is provided, the instance
--url
parameter can be omitted.
Optional Arguments¶
- -u, --url
URL of the instance to run operations.
results¶
Obtain the output results description of a job. This operation can also download them from the remote server if requested.
weaver results [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] [-u URL] -j JOB_REFERENCE [-D] [-O OUT_DIR]
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -j, --job
Job URL or UUID to run results operation. If full Job URL is provided, the instance
--url
parameter can be omitted.
Optional Arguments¶
- -u, --url
URL of the instance to run operations.
- -D, --download
Download all found job results file references to output location. If not requested, the operation simply displays the job results (default: False).
Default: False
- -O, --outdir
Output directory where to store downloaded files from job results if requested (default: ${CURDIR}/{JobID}/<outputs.files>).
upload¶
Upload a local file to the remote server vault for reference in process execution inputs. This operation is accomplished automatically for all execution inputs submitted using local files. [note: feature only available for Weaver instances]
weaver upload [--stdout] [--log LOG]
[--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
[-h] -u URL [-c CONTENT_TYPE] -f FILE
Logging Options¶
Options that configure output logging.
- --stdout
Enforce logging to stdout for display in console.
Default: False
- --log, --log-file
Output file to write generated logs.
- --quiet, -q
Do not output anything else than error.
Default: False
- --debug, -d
Enable extra debug logging.
Default: False
- --verbose, -v
Output informative logging details.
Default: False
- --log-level, -l
Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn
Explicit log level to employ (default: None).
Required Arguments¶
- -u, --url
URL of the instance to run operations.
- -f, --file
Local file path to upload to the vault.
Optional Arguments¶
- -c, --content-type
Content-Type of the file to apply. This should be an IANA Media-Type, optionally with additional parameters such as charset. If not provided, attempts to guess it based on the file extension.
CLI and Client Examples¶
Following sections present different typical usage of the Shell CLI Commands and Python Client Commands. Operations are equivalent between the CLI and Python client.
Note that more operations and option parameters are available, and are not all necessarily represented in below examples.
For each of the following examples, the client is created as follows:
client = WeaverClient(url="{WEAVER_URL}")
Deploy Example¶
Todo
example
Undeploy Example¶
Todo
example
GetCapabilities Example¶
Accomplishes the GetCapabilities request to obtain a list of available Process.
weaver capabilities -u {WEAVER_URL}
client.capabilities()
Sample Output:
{
"description": "Listing of available processes successful.",
"processes": [
"docker-demo-cat",
"docker-python-script",
"Echo",
"file_index_selector",
"file2string_array",
"image-utils",
"jsonarray2netcdf",
"las2tif",
"metalink2netcdf",
"sleep"
],
"page": 0,
"total": 25
}
DescribeProcess Example¶
Accomplishes the DescribeProcess request to obtain the Process definition.
weaver describe -u {WEAVER_URL} -p jsonarray2netcdf
client.describe("jsonarray2netcdf")
Sample Output:
{
"id": "jsonarray2netcdf",
"title": "JSON array to NetCDF",
"version": "1.1",
"abstract": "Extracts and fetches NetCDF files from a JSON file containing an URL string array, and provides them on the output directory.",
"keywords": [],
"metadata": [],
"inputs": {
"input": {
"title": "input",
"formats": [
{
"mimeType": "application/json",
"default": true
}
],
"minOccurs": "1",
"maxOccurs": "1"
}
},
"outputs": {
"output": {
"title": "output",
"formats": [
{
"mimeType": "application/x-netcdf",
"default": true
}
]
}
},
"visibility": "public",
"processEndpointWPS1": "http://localhost:4002/wps",
"processDescriptionURL": "http://localhost:4002/processes/jsonarray2netcdf/jobs",
"executeEndpoint": "http://localhost:4002/processes/jsonarray2netcdf/jobs",
"links": [
{
"type": "application/json",
"title": "Process description.",
"hreflang": "en-CA",
"href": "http://localhost:4002/processes/jsonarray2netcdf",
"rel": "self"
},
{
"type": "application/json",
"title": "Process description.",
"hreflang": "en-CA",
"href": "http://localhost:4002/processes/jsonarray2netcdf",
"rel": "process-desc"
},
{
"type": "application/json",
"title": "Process execution endpoint for job submission.",
"hreflang": "en-CA",
"href": "http://localhost:4002/processes/jsonarray2netcdf/execution",
"rel": "execute"
},
{
"type": "application/json",
"title": "List of registered processes.",
"hreflang": "en-CA",
"href": "http://localhost:4002/processes",
"rel": "collection"
},
{
"type": "application/xml",
"title": "Service definition.",
"hreflang": "en-CA",
"href": "http://localhost:4002/ows/wps?service=WPS&request=GetCapabilities",
"rel": "service-desc"
}
]
}
Execute Example¶
Accomplishes the Execute request to obtain launch a Job with the specified Process and provided inputs.
Todo
example
Dismiss Example¶
Todo
example
GetStatus Example¶
Todo
example
Monitor Example¶
Todo
example
Results Example¶
Todo
example
Upload Example¶
This operation allows manual upload of a local file to the Vault.
Note
When running the execute operation, any detected local file reference will be automatically uploaded as Vault file in order to make it available for the remote Weaver server for Process execution.
See also
File Vault Inputs and Uploading File to the Vault provide more details about this feature.
weaver upload -u {WEAVER_URL} -f /path/to/file.txt
client.upload("/path/to/file.txt")
Sample Output:
{
"description": "File successfully uploaded to vault.",
"access_token": "<ACCESS_TOKEN>",
"file_id": "e337a830-dbd6-4915-90ef-bfd46e237a3e",
"file_href": "vault://e337a830-dbd6-4915-90ef-bfd46e237a3e"
}