README.md 2.11 KB
Newer Older
Jasper Koehorst's avatar
Jasper Koehorst committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84




This document is intended to work on a workflow example from begin to end

### Obtain data from irods

export irodsHost=xxx

export irodsPort=xxx

export irodsUserName=xxx

export irodsZone=xxx

export irodsAuthScheme=password

export irodsHome=xxx

export irodsCwd=xxx

export irodsPassword=xxx

### Run workflow

Preserve environment is needed to have access to the exported variables

```
cwltool --preserve-entire-environment --tmp-outdir-prefix=./BLA/ --provenance PROVENANCE workflow_1.cwl --files /tempZone/Projects/P_IRODSRUNNER/I_Test_investigation/S_BABY_X/Amplicon/A_amp1bx/Unprocessed/G76494_R1_001.fastq.gz --files /tempZone/Projects/P_IRODSRUNNER/I_Test_investigation/S_BABY_X/Amplicon/A_amp1bx/Unprocessed/G76494_R2_001.fastq.gz --destination /tempZone/Projects/P_IRODSRUNNER/I_Test_investigation/S_BABY_X/Amplicon/A_amp1bx/Processed/HASH_MY2
```



### Run workflow within a docker on kubernetes or any other cloud instance which runs docker images

#### Step 1

Transmit arguments to the docker instance

- Path to yaml file + which cwl file to run according to metadata

- **/tempZone/Projects/P_IRODSRUNNER/I_Test_investigation/S_BABY_X/Amplicon/A_amp1bx/Unprocessed/testYAML.yml**

- **Metadata values:**

  attribute: cwl

  value: NGTAX_Workflow.cwl

  units: waiting

#### Step 2

- Docker instance is started

- ``` docker run -it --entrypoint "" -v `pwd`:/test wurssb/unlock_base /bin/bash```

- Bash script or python script is executed which will download the yaml file

  - Idea: python start.py -yml .... -cwl .... -args .... .... ....

- ```java -jar binaries/IRODSTransfer.jar -pull --files /tempZone/Projects/P_IRODSRUNNER/I_Test_investigation/S_BABY_X/Amplicon/A_amp1bx/Unprocessed/testYAML.yml```

  

  executes the workflow

- ```cwltool --preserve-entire-environment --provenance PROVENANCE cwls/workflow_ngtax.cwl testYAML.yml```

  

  Remove the data folder from the provenance?

- ```rm -r PROVENANCE/data/```

  

  uploads provenance

- ```java -jar binaries/IRODSTransfer.jar --push --files PROVENANCE/ --irods /tempZone/Projects/P_IRODSRUNNER/I_Test_investigation/S_BABY_X/Amplicon/A_amp1bx/Processed/HASHSAH```