FileWriter from Chrome extension

[nuthinking]

I'm trying to figure out if I can use webm-writer with a FileWriter using a chrome extension and puppeteer, but it looks like only Chrome apps can in fact use the Storage API, not an extension as mentioned in the Readme.md.

I currently have a web app that when running in Electron it leverages the filesystem to record videos. I'm trying to figure out how I could bring the rendering to the cloud. Thanks!

[guest271314]

You can use File System Access API https://wicg.github.io/file-system-access/ in an extension.

What are you trying to achieve?

[nuthinking]

@guest271314 Can I generate a fileWriter or a fd (file handler) with the File System Access API to be passed to the WebMWriter constructor?

Do I have to pass a LockedFile.fileHandle as fd?

I want WebMWrite to save the file as it has been recorded to optimize RAM consumption. Imagine a very long recording.

[guest271314]

I want WebMWrite to save the file as it has been recorded to optimize RAM consumption. Imagine a very long recording.

I did a storage quota error message before running bleachbit. To limit RAM instead of awaiting the entire file to be written with pipeTo() we can create a new FileSystemWritableFileStream instance then make sure to call close() after every write().

User action is required for showDirectoryPicker(), AFAIK there is no currently existing Chromium flag to disable internal Chromium "User Activation" implmentation.

In pertinent part

  var BlobBuffer = function(fs) {
    console.log(fs);
    return function(destination) {
      console.log(destination);
      var
        buffer = [],
        writePromise = Promise.resolve(),
        fileWriter = null,
        fd = null,
        fileHandle = null;

if (destination instanceof FileSystemFileHandle) {
     fileHandle = destination;
   }

if (fileHandle) {
  console.log(fileHandle);
  return fileHandle.createWritable({keepExistingData: true})
  .then(writer => {
    return Promise.all([writer.seek(newEntry.offset), writer]);
  })
  .then(([,writer]) => {
    return Promise.all([writer.write(new Blob([newEntry.data])), writer]);
  })
  .then(([,writer]) => {
    return writer.close();
  });
}

optionDefaults = {
   // ..
   fileHandle: null, // File System Access API FileSystemFileHandle

const dir = await showDirectoryPicker();
const status = await dir.requestPermission({mode: 'readwrite'});
const fileHandle = await dir.getFileHandle('file.webm', {create: true});
// ..
const videoWriter = new WebMWriter({
  fileHandle: fileHandle,

$ mkvmerge -J file.webm
{
  "attachments": [],
  "chapters": [],
  "container": {
    "properties": {
      "container_type": 17,
      "duration": 4828000000,
      "is_providing_timestamps": true,
      "muxing_application": "webm-writer-js",
      "writing_application": "webm-writer-js"
    },
    "recognized": true,
    "supported": true,
    "type": "Matroska"
  },
  "errors": [],
  "file_name": "file.webm",
  "global_tags": [],
  "identification_format_version": 12,
  "track_tags": [],
  "tracks": [
    {
      "codec": "VP8",
      "id": 0,
      "properties": {
        "codec_id": "V_VP8",
        "codec_name": "VP8",
        "codec_private_length": 0,
        "default_track": true,
        "display_dimensions": "768x576",
        "display_unit": 0,
        "enabled_track": true,
        "forced_track": false,
        "language": "und",
        "minimum_timestamp": 0,
        "number": 1,
        "pixel_dimensions": "768x576",
        "uid": 1
      },
      "type": "video"
    }
  ],
  "warnings": []
}

null
webm-writer.js:227 FileSystemFileHandle {kind: "file", name: "file.webm"}
webm-writer-file-handle.html:93 
webm-writer-file-handle.html:39 4.96
webm-writer-file-handle.html:42 768 576
favicon.ico:1 GET http://localhost:8000/favicon.ico 404 (Not Found)
webm-writer-file-handle.html:115 [Array(35)]
webm-writer-file-handle.html:120 8.01818181818182
webm-writer.js:367 FileSystemFileHandle {kind: "file", name: "file.webm"}
webm-writer.js:367 FileSystemFileHandle {kind: "file", name: "file.webm"}
webm-writer.js:367 FileSystemFileHandle {kind: "file", name: "file.webm"}
webm-writer.js:367 FileSystemFileHandle {kind: "file", name: "file.webm"}
webm-writer.js:367 FileSystemFileHandle {kind: "file", name: "file.webm"}
webm-writer-file-handle.html:139 File {name: "file.webm",..}
webm-writer-file-handle.html:134 4.828
webm-writer-file-handle.html:135 4.828 4.828

webm-writer.js.tar.gz
file.webm.tar.gz

[guest271314]

Instead of creating a new Blob using the constructor, which can reside in memory (https://stackoverflow.com/a/56419176), we can write newEntry.data directly

return Promise.all([writer.write(newEntry.data), writer]);

[guest271314]

Tested running this code https://plnkr.co/edit/Inb676?preview at localhost, with only 4 seconds of data.

[nuthinking]

User action is required for showDirectoryPicker()

Maybe puppeteer can handle it?

[guest271314]

You can try. If not file an issue for that capability.

Note, technically it is possible to read files written to disk written by FileSystem API https://stackoverflow.com/a/36098618.

An alterative approach would be to take a series of screenshot in headless mode (Puppeteer) and when each screen shot is saved locally use inotify-tools to execute a command that adds the frame to the track being written.

Are you essentially trying to capture videos of remote sites programmatically at command line, without a GUI?

What is the goal of the project?

[nuthinking]

Cloud rendering for a video creator app. Sure, image sequence to video is an option, but not very optimized in terms of hard drive usage.

[guest271314]

webm-writer is not "optimized" for hard drive usage either. Compare resulting file size of 5 seconds of recordings from MediaRecorder and webm-writer. What webm-writer offers is complete control over the procedure.

Since the application is online you can just use MediaRecorder. At Chromium you can use ImageCapture on the MediaStreamTrack as well, if necessary, to create a video from multiple images, e.g., see https://github.com/guest271314/MediaFragmentRecorder/tree/imagecapture-audiocontext-readablestream-writablestream and the other branches at that repository. Or, use the modified webm-writer code to use FileSystemFileHandle to write directly to user filesystem.

Why is a Chrome extension necessary to run create a video using multiple images?

[guest271314]

Note, MediaRecorder implementation at Chromium does not set duration of resulting WebM file, which can cause issues downstream. You can use ts-ebml guest271314/MediaFragmentRecorder#8 (comment) to set duration of WebM or Matroska file produced by Chromium MediaRecorder implementation.

Otherwise, as stated above, the prompt user for filesystem access, then write directly to the file at the website, no cloud necessary.

[guest271314]

@nuthinking In this case, where the browser is Chrome, File System Access API can be utilized directly in the browser to write one or more files or directories as webm-writer dynamically produces the output data without need for an extension or puppeteer. Any working code that you have can be included in a Chrome extension. An HTML page can be created for the extension, e.g., https://github.com/guest271314/native-messaging-mkvmerge. I am not sure why the Storage API is necessary? While Chrome Apps are deprecated, AFAIK Chrome extensions and Native Messaging are not slated for deprecation, where you can run nodejs, puppeteer, shell scripts, native applications. In this case an extension does not appear to be strictly necessary to achieve the requirement, though you can automate the process.

[nuthinking]

@guest271314 I didn't know MediaRecorder can be used to record a video asynchronously with optimal disk and RAM usage. Cloud recording would be useful if one day I decide to support iOS devices. Out of curiosity, do you freelance? Maybe you can give us a hand?

[guest271314]

@nuthinking Willing to help where able when have the time.

[nuthinking]

Will write to your gmail, thanks!

[guest271314]

Does the code at #19 (comment) resolve the issue?

[nuthinking]

@guest271314 not yet, I will try and let you know. Thanks! 🙌

[guest271314]

@nuthinking No worries.