How to upload to embargoed dandiset?

[haesemeyer]

Hi,

I have created a dandiset and set an embargo. It appears that I cannot download the dandiset (and therefore cannot upload any data). Specifically it seems as though it can't be found (see what appears to be a 404 error in the log if I interpret it correctly). Is this related to the embargo? If yes, how do I upload data to an embargoed dandiset? I pasted the command line output as well as the logs below (note that the dandi upload command should be correct as it was copied from the website).

Thanks,
Martin

(dandi) DHLRI-M151:dandi_test haes02$ dandi download https://dandiarchive.org/dandiset/000234/draft
2022-04-16 08:55:05,715 [    INFO] Logs saved in /Users/haes02/Library/Logs/dandi-cli/20220416125503Z-80070.log
Error: https://dandiarchive.org/dandiset/000234/draft

(dandi) DHLRI-M151:dandi_test haes02$ cat /Users/haes02/Library/Logs/dandi-cli/20220416125503Z-80070.log 
2022-04-16T08:55:03-0400 [INFO    ] dandi 80070:4527453632 dandi v0.39.1, hdmf v3.2.1, pynwb v2.0.1, h5py v3.6.0
2022-04-16T08:55:03-0400 [INFO    ] dandi 80070:4527453632 sys.argv = ['/opt/anaconda3/envs/dandi/bin/dandi', 'download', 'https://dandiarchive.org/dandiset/000234/draft']
2022-04-16T08:55:03-0400 [INFO    ] dandi 80070:4527453632 os.getcwd() = /Users/haes02/dandi_test
2022-04-16T08:55:03-0400 [DEBUG   ] urllib3.connectionpool 80070:4527453632 Starting new HTTPS connection (1): rig.mit.edu:443
2022-04-16T08:55:04-0400 [DEBUG   ] urllib3.connectionpool 80070:4527453632 https://rig.mit.edu:443 "GET /et/projects/dandi/dandi-cli HTTP/1.1" 200 579
2022-04-16T08:55:04-0400 [DEBUG   ] dandi 80070:4527453632 No newer (than 0.39.1) version of dandi/dandi-cli found available
2022-04-16T08:55:04-0400 [DEBUG   ] h5py._conv 80070:4527453632 Creating converter from 7 to 5
2022-04-16T08:55:04-0400 [DEBUG   ] h5py._conv 80070:4527453632 Creating converter from 5 to 7
2022-04-16T08:55:04-0400 [DEBUG   ] h5py._conv 80070:4527453632 Creating converter from 7 to 5
2022-04-16T08:55:04-0400 [DEBUG   ] h5py._conv 80070:4527453632 Creating converter from 5 to 7
2022-04-16T08:55:05-0400 [DEBUG   ] dandi 80070:4527453632 Parsing url https://dandiarchive.org/dandiset/000234/draft
2022-04-16T08:55:05-0400 [DEBUG   ] urllib3.connectionpool 80070:4527453632 Starting new HTTPS connection (1): dandiarchive.org:443
2022-04-16T08:55:05-0400 [DEBUG   ] urllib3.connectionpool 80070:4527453632 https://dandiarchive.org:443 "HEAD /dandiset/000234/draft HTTP/1.1" 404 0
2022-04-16T08:55:05-0400 [DEBUG   ] dandi 80070:4527453632 Caught exception https://dandiarchive.org/dandiset/000234/draft
2022-04-16T08:55:05-0400 [INFO    ] dandi 80070:4527453632 Logs saved in /Users/haes02/Library/Logs/dandi-cli/20220416125503Z-80070.log

[satra]

@haesemeyer - thanks for reporting this. this is a bug, but one that should go away after a set of changes scheduled to deploy on monday. in the meantime you can use this: dandi download https://gui.dandiarchive.org/#/dandiset/000234/draft

since it is an embargoed dandiset, both upload and download will require your api key, which you should be able to retrieve from clicking your user initials (top right corner) after logging on to dandiarchive.org.

cc/ @dchiquito @AlmightyYakob @yarikoptic @jwodder

[haesemeyer]

@satra thanks so much for the quick reply! That worked perfectly. I had already set the API key but with the other URL that didn't do me any good ;-). If the URL does stay different it would be good to add this to the docs which currently don't go over the implications of the embargo.

Have a great rest of the weekend,
Martin