Issues with browserSession() #95
Comments
|
UCSC here, we did a code release on Tuesday (we have one every three weeks). This could be a new rtracklayer bug or it could also related to something that we changed at UCSC, but I have no idea what it could be. We need more details from an rtracklayer developer to say more. |
|
Same error. Here is the session info R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.2 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so; LAPACK version 3.10.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
time zone: Europe/Berlin
tzcode source: system (glibc)
attached base packages:
[1] stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] BSgenome.Hsapiens.UCSC.hg19_1.4.3 BSgenome_1.68.0 Biostrings_2.68.1
[4] XVector_0.40.0 rtracklayer_1.60.1 GenomicRanges_1.52.0
[7] GenomeInfoDb_1.36.4 IRanges_2.34.1 S4Vectors_0.38.1
[10] BiocGenerics_0.46.0
loaded via a namespace (and not attached):
[1] Matrix_1.6-1 compiler_4.3.1 rjson_0.2.21 crayon_1.5.2
[5] SummarizedExperiment_1.30.2 Biobase_2.60.0 GenomicAlignments_1.36.0 Rsamtools_2.16.0
[9] bitops_1.0-7 parallel_4.3.1 BiocParallel_1.34.2 yaml_2.3.7
[13] fastmap_1.1.1 lattice_0.21-8 here_1.0.1 S4Arrays_1.0.5
[17] knitr_1.44 XML_3.99-0.14 DelayedArray_0.26.7 MatrixGenerics_1.12.3
[21] rprojroot_2.0.3 GenomeInfoDbData_1.2.10 rlang_1.1.1 xfun_0.40
[25] cli_3.6.1 magrittr_2.0.3 zlibbioc_1.46.0 digest_0.6.33
[29] grid_4.3.1 rstudioapi_0.15.0 lifecycle_1.0.3 vctrs_0.6.3
[33] evaluate_0.21 codetools_0.2-19 abind_1.4-5 RCurl_1.98-1.12
[37] restfulr_0.0.15 rmarkdown_2.22 purrr_1.0.2 matrixStats_1.0.0
[41] tools_4.3.1 BiocIO_1.10.0 htmltools_0.5.6 |
|
I am also getting similar error when I try to use makeTxDbFromUCSC("hg38"). The error I am getting is: It was working fine couple of days back. |
|
Same problem here. Breaks rtracklayer This started a couple of days ago so could be related with your latest code release @maximilianh. FWIW this also breaks the following rtracklayer's rev deps: CNVRanger, coMET, customProDB, DMRcate, ELMER, epigraHMM, epimutacions, GenomicFeatures, GenomicRanges, goseq, IdeoViz, MEAL, methylPipe, MethylSeekR, missMethyl, OGRE, segmenter. |
|
Could it be that a redirect was recently implemented from HTTP://genome.ucsc.edu/cgi-bin/hgGateway to HTTPS://genome.ucsc.edu/cgi-bin/hgGateway @maximilianh? To my surprise everything seems to work as usual. @lawremi @sanchit-saini Do you guys want me to submit a PR? Thanks |
|
Why are you closing the issue @muqiaowhale? |
|
Thanks for reopening. Also what error you got when doing |
|
@hpages yes, the same error |
|
Hmm.. I just tried this on 6 different machines (2 Ubuntu Linux, 2 Intel Mac, 2 arm64 Mac), in addition to my laptop, and it works on all of them. Maybe it's time for you guys to disclose your sessionInfo(): |
Matrix products: default locale: attached base packages: other attached packages: loaded via a namespace (and not attached): |
|
Thanks @muqiaowhale, that's what I suspected i.e. you're using Bioconductor 3.15 which is old, no longer supported, and archived. So you're on your own. Note that current version is 3.17, and 3.18 will be released next week. In the meantime I went ahead with a Pull Request (see PR #97). @lawremi @sanchit-saini Would be greatly appreciated if you could take a look at it. Thanks! |
|
I am able to clear the error with the following code However, when I actually go to pull and plot anything, the error persists: Info |
|
@slsevilla That's because there are many other places in rtracklayer internal code where HTTP needs to be replaced with HTTPS. PR #97 (rtracklayer 1.61.2) should take care of all of them. FWIW your example above works for me with rtracklayer 1.61.2. If you want to give it a try, easiest way is to install rtracklayer 1.61.2 with |
|
Hi @hpages Thanks for looking into it. As you have pointed out, the HTTP needs to be replaced with HTTPS, and you have already fixed it in almost all the places in PR. I was able to replicate it and the following also needs to be replaced, and I think you were not able to replicate it because it only triggers at the time of mirror redirection, which happens based on IP and server region proximity. I think your IP is close to "https://genome.ucsc.edu" server. Line 20 in 5a681a2 url <- sub(".*?a href=\"http([^[:space:]]+cgi-bin/).*", "https\\1", gw)Can you please also update it in the PR ? |
|
Done: b5d2e0f. Thanks for the feeback @sanchit-saini |
|
Hi rtracklayer developers,
yes! We did add this HTTP -> HTTPS redirect, after a ton of testing and
external feedback. We didn't think of reaching out to you guys, sorry. We
can take back and delay the redirect for a few weeks, but maybe it's
easiest if you can update rtracklayer now. Maybe we could also skip the
redirect if the request comes from rtracklayer - is there a special user
agent?
…On Fri, Oct 20, 2023 at 8:17 AM Hervé Pagès ***@***.***> wrote:
Done: b5d2e0f
<b5d2e0f>.
Thanks for the feeback @sanchit-saini <https://github.com/sanchit-saini>
—
Reply to this email directly, view it on GitHub
<#95 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACL4TK7M3HJEJ3NSGHEN6LYAIJOBAVCNFSM6AAAAAA6GVNIGGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZSGE2DMNJXHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Thanks for confirming @maximilianh. Let's hope PR #97 can be quickly merged (only 3 days left for that @lawremi). |
|
Hi there, |
|
@ukesh-sys Issue is closed because it is fixed in the |
|
Thank you so much @hpages for the clarification. I will wait till tomorrow and continue my analysis. Thank you everyone in the team for quick responses and the fix. |
|
Hey @hpages, does rtracklayer send a special user agent string or is there
some other way that we could exclude rtracklayer requests from the redirect
rule ?
…On Sat, Oct 21, 2023 at 04:35 Ukesh Karki ***@***.***> wrote:
Thank you so much @hpages <https://github.com/hpages> for the
clarification. I will wait till tomorrow and continue my analysis. Thank
you everyone in the team for quick responses and the fix.
—
Reply to this email directly, view it on GitHub
<#95 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACL4TIXYOVFD5PTKP3EAKTYAMYFXAVCNFSM6AAAAAA6GVNIGGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTGYYTKNRTGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
hi @maximilianh Line 27 in 07ed980 |
|
I've updated my rtracklayer, and am now getting a different error: Session info: |
|
@slsevilla Please make sure to report the full output of |
|
@slsevilla Also it seems that you are still running BioC 3.17. As explained earlier, you first need to upgrade your installation to BioC 3.18 to benefit from the fixes. Furthermore, there are other important fixes that were applied to the latest GenomicFeatures that you will need but won't get if you don't upgrade your installation to BioC 3.18 first. That was also explained earlier. |
|
@hpages - Same outcome even after updating Initial Setting session does not fix the issue Session info |
|
Thank you @slsevilla At least now As for the If this fails, no need to go further, please report the problem in a new issue. If it works, then keep reading. Now does the following code work for you?
If the Thanks, |
|
See issue #101. |
|
OK we've excluded this user agent from the redirect. Can you test this and tell us if this works with rtracklayer for the machine genome-test.gi.ucsc.edu: $ curl -v http://genome-test.gi.ucsc.edu/ 2>&1 | grep Location -i Redirect is not done if the useragent is "rtracklayer": $ curl --user-agent=rtracklayer -v http://genome-test.gi.ucsc.edu/ 2>&1 | grep Location -i If you can confirm that this solves the problem for older rtracklayer versions, we can roll out this change to the redirect rule to all servers. Sorry for the delay, I sent my emails to the wrong person, otherwise this could have been applied right away. |
|
Ignore my previous message, this is not working yet. I'll investigate and get back to you. |
|
OK, we made this change on a test site. Can you try an old version of rtracklayer against http://genome-test.gi.ucsc.edu and see if that fixes it? This command does not bring up a HTTPS redirect anymore: curl -A rtracklayer -v http://genome-test.gi.ucsc.edu/ > /dev/null Once you confirm that it works, we can roll out the change to the public servers. |
|
FWIW I attempted to implement this with an older version of rtracklayer (rtracklayer_1.56.1) and still ran into errors. Running old version of rtracklayer doesn't produce an error at I tried setting the session to this test site, but now I'm getting the Session info |
|
Hi @slsevilla , I can't help with this. I confirmed that with the user agent "rtracklayer" genome-test.gi.ucsc.edu doesn't redirect anymore, but beyond that, I can't do anything. I can't debug rtracklayer... Here is the command that shows that this server does not redirect to https anymore: $ curl -A rtracklayer -v http://genome-test.gi.ucsc.edu/ > /dev/null |
|
Any news from the rtracklayer developers? Or should we just close this? |
|
I haven't had any update. They are sort of saying the same thing - they dont see the issue, although there are several other people who are in the same situation as me. |
|
But the redirect is still in place for http://genome.ucsc.edu/cgi-bin/hgGateway @maximilianh: As long as the redirect is active, old versions of rtracklayer will remain broken. |
|
@maximilianh Any chance the redirect can be removed for http://genome.ucsc.edu/cgi-bin/hgGateway when user agent is "rtracklayer"? |
|
This has been rolled out to genome.ucsc.edu, genome-asia.ucsc.edu and genome-euro.ucsc.edu. Can you try again? Sorry for the long delay, we had some issues with our apache configs (and sorted them out now) |
|
ok great, thanks @maximilianh @slsevilla Can you test with rtracklayer 1.56.1 again? |
|
Hi! I don't have access to R 4.2 anymore... will it work with any later version? |
Hi,
thanks for your awesome package! I’m trying to extract UCSC data using your package. I successfully ran the command below last week but got this error today. Is there anything wrong with the function or UCSC? Could you please help me to figure it out? Thank you so much. Look forward to your reply!
Jingyu
The text was updated successfully, but these errors were encountered: