-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Openssl is not available in netConnectHttps for importing remote BigWig files #83
Comments
Ahh, note that @ChristopherWilks posted at leekgroup/recount#23 (comment) that the same code works with |
…lks/snaptron#17. Also #23. I tried insulating recount from these tests, so they'll be reported as warnings instead of errors on the BioC build machines for now.
…lks/snaptron#17. Also #23. I tried insulating recount from these tests, so they'll be reported as warnings instead of errors on the BioC build machines for now.
Support for SSL depends on having the openssl library available at build time. It's conceivable that either the user is building the package from source without openssl, or Bioconductor at some point stopped providing Mac binaries with openssl support. |
Hi Michael, The user in this case is me but also, the Bioc machines were reporting the same error. I did notice that the openssl package wasn't being loaded from my R session info above. http://bioconductor.org/checkResults/release/bioc-LATEST/recount/ doesn't show the error anymore, but that's because I turned it into a warning with some edits to the tests at leekgroup/recount@5f2696d that rely on I'll ask on bioc-devel to see if someone else knows about a change in the Best, |
Also, my collaborator @nellore pointed out we had run into a similar issue back in 2016 as noted at https://support.bioconductor.org/p/81267/ |
By "openssl" I mean the C library, not the R package. I'm guessing that the Bioconductor build machine needs to be configured to build openssl support into the Mac binary. This should be as simple as installing openssl with brew. Would you happen to know the right person to contact about that? |
Hi Michael, Jennifer and Hervé replied at https://stat.ethz.ch/pipermail/bioc-devel/2023-March/019503.html. It sounds like BioC is building the packages with Do you have other leads? cc @ChristopherWilks @nellore. Best, |
Hi, I no longer get the The above link (first message on this thread) has changed from Duffel link reproducible exampleHere's the small reproducible code: ## Remotely access from duffel link
library("GenomicRanges")
library("rtracklayer")
range <- GRanges(seqnames = "chrY", ranges = IRanges(1, 57227415))
rtracklayer::import("http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw", selection = reduce(range), as = "RleList")
traceback()
options(width = 120)
sessioninfo::session_info()
curl::curl_version() Here's the R output > ## Remotely access from duffel link
> library("GenomicRanges")
> library("rtracklayer")
> range <- GRanges(seqnames = "chrY", ranges = IRanges(1, 57227415))
> rtracklayer::import("http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw", selection = reduce(range), as = "RleList")
Error in seqinfo(con) : UCSC library operation failed
In addition: Warning message:
In seqinfo(con) :
Couldn't open https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw
> traceback()
7: seqinfo(con)
6: seqinfo(con)
5: .local(con, format, text, ...)
4: import(FileForFormat(con), ...)
3: import(FileForFormat(con), ...)
2: rtracklayer::import("http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw",
selection = reduce(range), as = "RleList")
1: rtracklayer::import("http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw",
selection = reduce(range), as = "RleList")
> options(width = 120)
> sessioninfo::session_info()
─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.4.0 (2024-04-24)
os macOS Sonoma 14.5
system aarch64, darwin20
ui X11
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/New_York
date 2024-05-20
pandoc 3.1.12.1 @ /opt/homebrew/bin/pandoc
─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
package * version date (UTC) lib source
abind 1.4-5 2016-07-21 [1] CRAN (R 4.4.0)
Biobase 2.64.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
BiocGenerics * 0.50.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
BiocIO 1.14.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
BiocParallel 1.38.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
Biostrings 2.72.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
bitops 1.0-7 2021-04-24 [1] CRAN (R 4.4.0)
cli 3.6.2 2023-12-11 [1] CRAN (R 4.4.0)
codetools 0.2-20 2024-03-31 [1] CRAN (R 4.4.0)
crayon 1.5.2 2022-09-29 [1] CRAN (R 4.4.0)
curl 5.2.1 2024-03-01 [1] CRAN (R 4.4.0)
DelayedArray 0.30.1 2024-05-07 [1] Bioconductor 3.19 (R 4.4.0)
GenomeInfoDb * 1.40.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
GenomeInfoDbData 1.2.12 2024-05-03 [1] Bioconductor
GenomicAlignments 1.40.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
GenomicRanges * 1.56.0 2024-05-01 [1] Bioconductor 3.19 (R 4.4.0)
httr 1.4.7 2023-08-15 [1] CRAN (R 4.4.0)
IRanges * 2.38.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
jsonlite 1.8.8 2023-12-04 [1] CRAN (R 4.4.0)
lattice 0.22-6 2024-03-20 [1] CRAN (R 4.4.0)
Matrix 1.7-0 2024-03-22 [1] CRAN (R 4.4.0)
MatrixGenerics 1.16.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
matrixStats 1.3.0 2024-04-11 [1] CRAN (R 4.4.0)
R6 2.5.1 2021-08-19 [1] CRAN (R 4.4.0)
RCurl 1.98-1.14 2024-01-09 [1] CRAN (R 4.4.0)
restfulr 0.0.15 2022-06-16 [1] CRAN (R 4.4.0)
rjson 0.2.21 2022-01-09 [1] CRAN (R 4.4.0)
Rsamtools 2.20.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
rtracklayer * 1.64.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
S4Arrays 1.4.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
S4Vectors * 0.42.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.4.0)
SparseArray 1.4.3 2024-05-07 [1] Bioconductor 3.19 (R 4.4.0)
SummarizedExperiment 1.34.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
UCSC.utils 1.0.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
XML 3.99-0.16.1 2024-01-22 [1] CRAN (R 4.4.0)
XVector 0.44.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
yaml 2.3.8 2023-12-11 [1] CRAN (R 4.4.0)
zlibbioc 1.50.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.0)
[1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> curl::curl_version()
$version
[1] "8.6.0"
$ssl_version
[1] "(SecureTransport) LibreSSL/3.3.6"
$libz_version
[1] "1.2.12"
$libssh_version
[1] NA
$libidn_version
[1] NA
$host
[1] "x86_64-apple-darwin23.0"
$protocols
[1] "dict" "file" "ftp" "ftps" "gopher" "gophers" "http" "https" "imap" "imaps" "ldap"
[12] "ldaps" "mqtt" "pop3" "pop3s" "rtsp" "smb" "smbs" "smtp" "smtps" "telnet" "tftp"
$ipv6
[1] TRUE
$http2
[1] TRUE
$idn
[1] FALSE
IDIES/AWS linksHere's more code for testing with the IDIES or AWS links directly, thus bypassing the redirect service provided by Note that download the file with R code: ## Remotely access from IDIES link
library("GenomicRanges")
library("rtracklayer")
range <- GRanges(seqnames = "chrY", ranges = IRanges(1, 57227415))
rtracklayer::import("http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw", selection = reduce(range), as = "RleList")
traceback()
## Remotely access from AWS link
library("GenomicRanges")
library("rtracklayer")
range <- GRanges(seqnames = "chrY", ranges = IRanges(1, 57227415))
rtracklayer::import("https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw", selection = reduce(range), as = "RleList")
traceback()
## Locally download data from duffel link
temp_duffel <- tempfile("mean_SRP002001_duffel.bw")
download.file("http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw", temp_duffel, mode = "wb")
chr <- "chrY"
rtracklayer::import(BigWigFile(temp_duffel), selection = reduce(range), as = "RleList")[[chr]]
## Locally download data from IDIES link
temp_idies <- tempfile("mean_SRP002001_idies.bw")
download.file("http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw", temp_idies, mode = "wb")
rtracklayer::import(BigWigFile(temp_idies), selection = reduce(range), as = "RleList")[[chr]]
## Locally download data from AWS link
temp_aws <- tempfile("mean_SRP002001_aws.bw")
download.file("https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw", temp_aws, mode = "wb")
rtracklayer::import(BigWigFile(temp_aws), selection = reduce(range), as = "RleList")[[chr]] R output: > ## Remotely access from IDIES link
> library("GenomicRanges")
> library("rtracklayer")
> range <- GRanges(seqnames = "chrY", ranges = IRanges(1, 57227415))
> rtracklayer::import("http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw", selection = reduce(range), as = "RleList")
Error in seqinfo(con) : UCSC library operation failed
In addition: Warning message:
In seqinfo(con) :
Couldn't open https://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw
> traceback()
7: seqinfo(con)
6: seqinfo(con)
5: .local(con, format, text, ...)
4: import(FileForFormat(con), ...)
3: import(FileForFormat(con), ...)
2: rtracklayer::import("http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw",
selection = reduce(range), as = "RleList")
1: rtracklayer::import("http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw",
selection = reduce(range), as = "RleList")
> ## Remotely access from AWS link
> library("GenomicRanges")
> library("rtracklayer")
> range <- GRanges(seqnames = "chrY", ranges = IRanges(1, 57227415))
> rtracklayer::import("https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw", selection = reduce(range), as = "RleList")
Error in seqinfo(con) : UCSC library operation failed
In addition: Warning message:
In seqinfo(con) :
Couldn't open https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw
>
> traceback()
7: seqinfo(con)
6: seqinfo(con)
5: .local(con, format, text, ...)
4: import(FileForFormat(con), ...)
3: import(FileForFormat(con), ...)
2: rtracklayer::import("https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw",
selection = reduce(range), as = "RleList")
1: rtracklayer::import("https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw",
selection = reduce(range), as = "RleList")
>
>
> ## Locally download data from duffel link
> temp_duffel <- tempfile("mean_SRP002001_duffel.bw")
> download.file("http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw", temp_duffel, mode = "wb")
trying URL 'http://duffel.rail.bio/recount/SRP002001/bw/mean_SRP002001.bw'
Content type 'binary/octet-stream' length 50936703 bytes (48.6 MB)
==================================================
downloaded 48.6 MB
> chr <- "chrY"
> rtracklayer::import(BigWigFile(temp_duffel), selection = reduce(range), as = "RleList")[[chr]]
numeric-Rle of length 57227415 with 2855 runs
Lengths: 2838976 36 62104 36 21210 1 ... 573 36 30043 36 341328
Values : 0.0000 25.7608 0.0000 12.8804 0.0000 12.8804 ... 0.0000 12.8804 0.0000 12.8804 0.0000
> ## Locally download data from IDIES link
> temp_idies <- tempfile("mean_SRP002001_idies.bw")
> download.file("http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw", temp_idies, mode = "wb")
trying URL 'http://data.idies.jhu.edu/recount2/data/SRP002001/bw/mean_SRP002001.bw'
Content type 'text/plain' length 50936703 bytes (48.6 MB)
==================================================
downloaded 48.6 MB
> rtracklayer::import(BigWigFile(temp_idies), selection = reduce(range), as = "RleList")[[chr]]
numeric-Rle of length 57227415 with 2855 runs
Lengths: 2838976 36 62104 36 21210 1 ... 573 36 30043 36 341328
Values : 0.0000 25.7608 0.0000 12.8804 0.0000 12.8804 ... 0.0000 12.8804 0.0000 12.8804 0.0000
> ## Locally download data from AWS link
> temp_aws <- tempfile("mean_SRP002001_aws.bw")
> download.file("https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw", temp_aws, mode = "wb")
trying URL 'https://recount-opendata.s3.amazonaws.com/recount2/SRP002001/bw/mean_SRP002001.bw'
Content type 'binary/octet-stream' length 50936703 bytes (48.6 MB)
==================================================
downloaded 48.6 MB
> rtracklayer::import(BigWigFile(temp_aws), selection = reduce(range), as = "RleList")[[chr]]
numeric-Rle of length 57227415 with 2855 runs
Lengths: 2838976 36 62104 36 21210 1 ... 573 36 30043 36 341328
Values : 0.0000 25.7608 0.0000 12.8804 0.0000 12.8804 ... 0.0000 12.8804 0.0000 12.8804 0.0000 recount reprexBuilding library("GenomicRanges")
library("rtracklayer")
files <- "http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw"
chr <- "chrY"
chrlen <- 57227415
bList <- BigWigFileList(files)
which <- GRanges(seqnames = chr, ranges = IRanges(1, chrlen))
x <- import(bList[[1]], selection = reduce(which), as = "RleList")
traceback() Here's the output: > files <- "http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw"
> chr <- "chrY"
> chrlen <- 57227415
> bList <- BigWigFileList(files)
> which <- GRanges(seqnames = chr, ranges = IRanges(1, chrlen))
> x <- import(bList[[1]], selection = reduce(which), as = "RleList")
Error in seqinfo(con) : UCSC library operation failed
In addition: Warning message:
In seqinfo(con) :
Couldn't open https://recount-opendata.s3.amazonaws.com/recount2/SRP009615/bw/mean_SRP009615.bw
> traceback()
5: seqinfo(con)
4: seqinfo(con)
3: .local(con, format, text, ...)
2: import(bList[[1]], selection = reduce(which), as = "RleList")
1: import(bList[[1]], selection = reduce(which), as = "RleList") Note that downloading the file locally does work. But well, connection issues can pop up way more frequently when such a large file is being downloaded. > temp_SRP009615 <- tempfile("mean_SRP009615_duffel.bw")
> download.file(files, temp_SRP009615, mode = "wb")
trying URL 'http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw'
Content type 'binary/octet-stream' length 1392034077 bytes (1327.5 MB)
==================================================
downloaded 1327.5 MB
> rtracklayer::import(BigWigFile(temp_SRP009615), selection = reduce(which), as = "RleList")[[chr]]
numeric-Rle of length 57227415 with 150774 runs
Lengths: 2781486 36 1 36 8 18 ... 57 36 49 36 339535
Values : 0.000000 0.286485 0.000000 0.372865 0.000000 0.323784 ... 0.000000 0.323784 0.000000 0.310604 0.000000 I guess that I could expand Let me know if I can provide any more useful information. Best, |
…erage_matrix() call to run. Related to lawremi/rtracklayer#83
…erage_matrix() call to run. Related to lawremi/rtracklayer#83
…elated to the workaround to lawremi/rtracklayer#83
…elated to the workaround to lawremi/rtracklayer#83
…Instead, use pre-computed files. This shoudl resolve https://github.com/LieberInstitute/recountWorkflow/actions/runs/9174682873/job/25226047568#step:15:347. Related to lawremi/rtracklayer#83
…Instead, use pre-computed files. This shoudl resolve https://github.com/LieberInstitute/recountWorkflow/actions/runs/9174682873/job/25226047568#step:15:347. Related to lawremi/rtracklayer#83
…s. So we can now run all this code on Windows, unlike before. It's not working for remote BigWig files, but I don't know if that's due to lawremi/rtracklayer#83 or something else Windows-specific.
…s. So we can now run all this code on Windows, unlike before. It's not working for remote BigWig files, but I don't know if that's due to lawremi/rtracklayer#83 or something else Windows-specific.
Hi,
I'm having trouble importing remote BigWig files with
derfinder
, which internally usesrtracklayer::import()
. I noticed this when looking atrecount
which is failing on BioC 3.16 and 3.17 (details at leekgroup/recount#23).You can reproduce this issue with:
I noticed issue #63 and saw that PR #68 fixed that issue with
rtracklayer
version 1.55.4 https://github.com/sanchit-saini/rtracklayer/blob/fa2a29d01f4f2975d8e2fe0de5ce4073b4e6b187/DESCRIPTION#L3 (which I'm assuming is already part of version 1.58.0 I'm using).I also saw #73, and can tell that these are different errors since the canonical message here is
No openssl available in netConnectHttps for sciserver.org : 443
.You get a similar error with
duffel
(
duffel
currently points to AWS nellore/digitalocean-duffel@c6e53d5 so these two are the same. IDIES is a different mirror forrecount2
data).Let me know if there's any piece of info that might be helpful to you.
Best,
Leo
The text was updated successfully, but these errors were encountered: