unrecognized extension field dct:contributor

[attilagk]

Hi Kenny, I've just run the test workflow with the newest commit 681b015 and the run failed with the messages below.

attila-ThinkS:~/projects/software/bsmn-common-workflow$ cwl-runner workflow-entrypoint.cwl test-workflow-entrypoint.json
/usr/bin/cwl-runner 1.0.20170114120503
Resolved 'workflow-entrypoint.cwl' to 'file:///home/attila/projects/software/bsmn-common-workflow/workflow-entrypoint.cwl'
Tool definition failed initialization:
Tool definition file:///home/attila/projects/software/bsmn-common-workflow/steps/bwa-index-tool.cwl failed validation:
steps/bwa-index-tool.cwl:3:1:   Object `steps/bwa-index-tool.cwl` is not valid because
                                  tried `CommandLineTool` but
steps/bwa-index-tool.cwl:11:1:       * unrecognized extension field `dct:contributor` and strict
                                     is True.  Did you include a $schemas section?
steps/bwa-index-tool.cwl:16:1:       * unrecognized extension field `dct:creator` and strict is
                                     True.  Did you include a $schemas section?
steps/bwa-index-tool.cwl:21:1:       * unrecognized extension field `dct:description` and strict
                                     is True.  Did you include a $schemas section?
attila-ThinkS:~/projects/software/bsmn-common-workflow$ echo $?
1

[kdaily]

Should be fixed by #6 and #7

[attilagk]

Thanks @kdaily
I tested the workflow with the fixes but got new errors related to unrecognized extension fields. Would you mind having a look at those below?

attila-ThinkS:~/projects/software/bsmn-common-workflow$ cwl-runner workflow-entrypoint.cwl test-workflow-entrypoint.json
/usr/bin/cwl-runner 1.0.20170114120503
Resolved 'workflow-entrypoint.cwl' to 'file:///home/attila/projects/software/bsmn-common-workflow/workflow-entrypoint.cwl'
No handlers could be found for logger "rdflib.term"
Tool definition failed initialization:
Tool definition file:///home/attila/projects/software/bsmn-common-workflow/steps/bwa-index-tool.cwl failed validation:
steps/bwa-index-tool.cwl:3:1:   Object `steps/bwa-index-tool.cwl` is not valid because
                                  tried `CommandLineTool` but
steps/bwa-index-tool.cwl:69:1:       * unrecognized extension field `s:contributor` and strict is
                                     True.  Did you include a $schemas section?
steps/bwa-index-tool.cwl:3:1:       * unrecognized extension field `https://schema.org/author`
                                    and strict is True.  Did you include a $schemas section?
attila-ThinkS:~/projects/software/bsmn-common-workflow$ echo $?
1

[kdaily]

I see your version of cwl-runner; I have a newer version and I do not get these errors. Can you try to upgrade?

> cwl-runner --version
/home/kdaily/.virtualenvs/cwl-python3/bin/cwl-runner 1.0.20181118133959

I fixed a few other files that still had the foaf:-style author metadata too.

[attilagk]

@kdaily I went to Ubuntu launchpad and upgraded cwltool to version 1.0.20180302231433-1, which is the current stable release but is still behind the version you have. Launchpad doesn't have your version but has an unstable version 1.0.20181217162649+dfsg-10; I tried to install this but failed because some of the dependencies couldn't be installed. So I stayed with 1.0.20180302231433-1 and got the following error:

attila-ThinkS:~/projects/software/bsmn-common-workflow$ dpkg --list | grep cwltool
ii  cwltool                                   1.0.20180302231433-1                        all          Common Workflow Language reference implementation
attila-ThinkS:~/projects/software/bsmn-common-workflow$ git pull
Already up-to-date.
attila-ThinkS:~/projects/software/bsmn-common-workflow$ cwl-runner workflow-entrypoint.cwl test-workflow-entrypoint.json
/usr/bin/cwl-runner 1.0.20180302231433
Resolved 'workflow-entrypoint.cwl' to 'file:///home/attila/projects/software/bsmn-common-workflow/workflow-entrypoint.cwl'
I'm sorry, I couldn't load this CWL file, try again with --debug for more information.
The error was: __init__() got an unexpected keyword argument 'skip_schemas'

As you see I updated my local bsmn-common-workflow repo from GitHub before running the test. Do you think it's something about the version of cwltool? Where did you get your cwltool from?

[kdaily]

I'm using a Python virtual environment, so I installed with pip:

pip install cwlref-runner

I used the instructions here:

https://github.com/common-workflow-language/cwltool#install

Let me see if I can add a requirements.txt file for adding the required packages. I didn't realize that there were such differences between the cwl runners - I successfully ran this (on my local Fedora linux and a RedHat AWS EC2 instance) using both the reference runner and the Toil runner (https://toil.readthedocs.io/en/latest/gettingStarted/quickStart.html#running-a-basic-cwl-workflow).

Thanks for your patience here!

[attilagk]

Hi @kdaily I installed cwltool 1.0.20190607183319 using pip and ran the test as

cwltool --debug workflow-entrypoint.cwl test-workflow-entrypoint.json

The problem now is the lack of Docker on my system. Below is the error message (the entire output is much longer due to the --debug option).

DEBUG Docker error
Traceback (most recent call last):
  File "/home/attila/.local/lib/python3.5/site-packages/cwltool/job.py", line 634, in run
    getdefault(runtimeContext.tmp_outdir_prefix, DEFAULT_TMP_PREFIX)))
  File "/home/attila/.local/lib/python3.5/site-packages/cwltool/docker.py", line 192, in get_from_requirements
    raise WorkflowException("docker executable is not available")
cwltool.errors.WorkflowException: docker executable is not available
ERROR Workflow error:
Docker is not available for this tool, try --no-container to disable Docker, or install a user space Docker replacement like uDocker with --user-space-docker-cmd.: docker executable is not available

The message suggests either --no-container to disable Docker (which I found also to lead to error because the workflow does need Docker) or --user-space-docker-cmd.

So I tried the latter option but this is what I got

attila-ThinkS:~/projects/software/bsmn-common-workflow$ cwltool --debug --user-space-docker-cmd workflow-entrypoint.cwl test-workflow-entrypoint.json
INFO /home/attila/.local/bin/cwltool 1.0.20190607183319
INFO Resolved 'test-workflow-entrypoint.json' to 'file:///home/attila/projects/software/bsmn-common-workflow/test-workflow-entrypoint.json'
ERROR Tool definition failed validation:
No cwlVersion found. Use the following syntax in your CWL document to declare the version: cwlVersion: <version>.
Note: if this is a CWL draft-2 (pre v1.0) document then it will need to be upgraded first.
Traceback (most recent call last):
  File "/home/attila/.local/lib/python3.5/site-packages/cwltool/main.py", line 664, in main
    skip_schemas=args.skip_schemas)
  File "/home/attila/.local/lib/python3.5/site-packages/cwltool/load_tool.py", line 227, in resolve_and_validate_document
    "No cwlVersion found. "
schema_salad.validate.ValidationException: No cwlVersion found. Use the following syntax in your CWL document to declare the version: cwlVersion: <version>.
Note: if this is a CWL draft-2 (pre v1.0) document then it will need to be upgraded first.

Note that it didn't matter when I edited workflow-entrypoint.cwl and changed cwlVersion: v1.0 to cwlVersion: v1.0.20190607183319.

Can you advise me on this situation? Thanks.

[kdaily]

Hi @attilagk Docker is indeed required.

[attilagk]

Hi @kdaily I installed docker.io and ran the workflow. It produced an indexed BAM successfully. The synapse-annotate step and the subsequent synapse-store-alignment-output step failed, however. Details below:

sudo cwltool workflow-entrypoint.cwl test-workflow-entrypoint.json
...
...
[step synapse-annotate] start
[workflow synapse-annotate] start
[workflow synapse-annotate] starting step synapse-annotate_2
[step synapse-annotate_2] Cannot make job: Expression evaluation error:
Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "annotations_json_string": "{\"specimenID\": \"test\"}",
04     "synapse_config": {
05         "class": "File",
06         "location": "file:///home/attila/.synapseConfig",
07         "size": 58,
08         "basename": ".synapseConfig",
09         "nameroot": ".synapseConfig",
10         "nameext": ""
11     },
12     "synapse_id": null
13 };
14 var self = null;
15 var runtime = {
16     "tmpdir": null,
17     "outdir": null
18 };
19 (function(){return ((inputs.file_to_store.path));})()
stdout was: ''
stderr was: 'evalmachine.<anonymous>:19
(function(){return ((inputs.file_to_store.path));})()
                                          ^

TypeError: Cannot read property 'path' of undefined
    at evalmachine.<anonymous>:19:43
    at evalmachine.<anonymous>:19:52
    at Script.runInContext (vm.js:107:20)
    at Script.runInNewContext (vm.js:113:17)
    at Object.runInNewContext (vm.js:296:38)
    at Socket.<anonymous> ([eval]:11:57)
    at Socket.emit (events.js:189:13)
    at addChunk (_stream_readable.js:284:12)
    at readableAddChunk (_stream_readable.js:261:13)
    at Socket.Readable.push (_stream_readable.js:220:10)'

[workflow synapse-annotate] completed permanentFail
[step synapse-annotate] completed permanentFail
[workflow synapse-store-alignment-output] completed permanentFail
...