Use blobfuse2 for streamable TesInputs #692
Labels
enhancement
New feature or request
Performance
Enable users can run task as cheap and as fast as possible
Scalability
Enable users can scale TES workloads
Problem:
Customers need the ability to perform random reads using a file system for large genomics reference files without downloading the entire file, which costs more and puts pressure on the storage account.
Solution:
TesInput.Streamable
is set totrue
, the TES runner should download and install blobfuse2blobfuse2 mount
path
specified for the TesInput.path worksI confirmed that random reads in blobfuse2 work as expected:
blobfuse2 mount /ref --config-file=./b2.yaml
dd if=stLFR.split_read.1.fq.gz skip=50000000000 bs=1 count=128 iflag=skip_bytes 2>/dev/null | xxd
The text was updated successfully, but these errors were encountered: