Ph pd 2514 multiome on terra #1223
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Staging -> master
dev -> staging
Staging -> master
|
Remember to squash merge! |
phendriksen100
commented
Feb 29, 2024
| File azure_gex_whitelist = "https://datasetpublicbroadref.blob.core.windows.net/dataset/RNA/resources/arc-v1/737K-arc-v1_gex.txt" | ||
| File azure_atac_whitelist = "https://datasetpublicbroadref.blob.core.windows.net/dataset/RNA/resources/arc-v1/737K-arc-v1_atac.txt" | ||
|
|
||
| Determine which whitelist files to use based on cloud provider |
There was a problem hiding this comment.
Suggested change
| Determine which whitelist files to use based on cloud provider | |
| # Determine which whitelist files to use based on cloud provider |
There was a problem hiding this comment.
Oof, good catch
pipelines/skylab/multiome/atac.wdl
Outdated
| # Docker image names | ||
| String warp_tools_2_0_0 = "warp-tools:2.0.0" | ||
| String cutadapt_docker = "cutadapt:1.0.0-4.4-1709146458" | ||
| String sam_tools_docker = "samtools-dist-bwa:3.0.0" |
There was a problem hiding this comment.
Suggested change
| String sam_tools_docker = "samtools-dist-bwa:3.0.0" | |
| String samtools_docker = "samtools-dist-bwa:3.0.0" |
pipelines/skylab/multiome/atac.wdl
Outdated
| @@ -91,21 +113,24 @@ workflow ATAC { | |||
| output_base_name = input_id, | |||
| nthreads = num_threads_bwa, | |||
| mem_size = mem_size_bwa, | |||
| cpu_platform = cpu_platform_bwa | |||
| cpu_platform = cpu_platform_bwa, | |||
| docker_path = docker_prefix + sam_tools_docker | |||
There was a problem hiding this comment.
Suggested change
| docker_path = docker_prefix + sam_tools_docker | |
| docker_path = docker_prefix + samtools_docker |
…ute/warp into ph_pd-2514_multiome_on_terra
| String docker_prefix = if cloud_provider == "gcp" then gcr_docker_prefix else acr_docker_prefix | ||
|
|
||
| # Define docker images | ||
| snap_atac_docker_image = "snapatac2:1.0.4-2.3.1-1700590229" |
There was a problem hiding this comment.
Suggested change
| snap_atac_docker_image = "snapatac2:1.0.4-2.3.1-1700590229" | |
| String snap_atac_docker_image = "snapatac2:1.0.4-2.3.1-1700590229" |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
pipelines/skylab/multiome/atac.wdl
Outdated
| @@ -3,6 +3,7 @@ version 1.0 | |||
| import "../../../tasks/skylab/MergeSortBam.wdl" as Merge | |||
| import "../../../tasks/skylab/FastqProcessing.wdl" as FastqProcessing | |||
| import "../../../tasks/skylab/PairedTagUtils.wdl" as AddBB | |||
| "../../../tasks/broad/Utilities.wdl" as utils | |||
There was a problem hiding this comment.
Suggested change
| "../../../tasks/broad/Utilities.wdl" as utils | |
| import "../../../tasks/broad/Utilities.wdl" as utils |
|
Remember to squash merge! |
|
@phendriksen100 this will require minor doc updates |
* add logic to choose which docker * fix param_meta and import * add cloud provider to checkinput * handle hard coded white list paths in CheckInputs.wdl * last few dockers * last few dockers * last few dockers * change error msg * use ubuntu image * use ubuntu image * change whitelists * point to azure public whitelists * add sas token * echo whitelist * echo whitelist * testing for coa * testing for coa * change back to terra buckets for whitelists * change whitelists to point at public azure bucket * files to strings * print statemtns to checkinputs * string to files * change to terra bucket paths * strings not files * append sas token * append sas token * append sas and use strings * back to bucket urls * back to bucket urls * use google cloud urls * using public urls * trying to export sas_token * trying to export sas_token * trying to export sas_token * terra on gcp * update azure whitelist files * changelogs * changelogs * changelogs * changelogs * fix some inputs * fix some inputs * fix some inputs * fix some inputs * update optimus dockers * warp_tools_docker_path for staralign * stop using ice lake as default * update pipeline docs * 2 threads * counting mode * changelogs --------- Co-authored-by: phendriksen100 <[email protected]> Co-authored-by: kayleemathews <[email protected]>
|
closing this as it is now part of this pr: #1239 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Updated Multiome.wdl and atac.wdl to include a
cloud_providerparameter. This enhancement enables us to specify the preferred cloud service provider for workflow execution, determining the appropriate Docker images and storage file locations accordingly.PD-2514
Checklist
If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP documentation team by tagging either @ekiernan or @kayleemathews in a comment on this PR.