Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Toon vrms #76

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 130 additions & 5 deletions type_templates/vrm.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import * as THREE from 'three';
import metaversefile from 'metaversefile';
import { VRMMaterialImporter } from '@pixiv/three-vrm/lib/three-vrm.module';
const { useApp, useLoaders, usePhysics, useCleanup, useActivate, useLocalPlayer } = metaversefile;
const { useApp, useLoaders, usePhysics, useCleanup, useActivate, useLocalPlayer, useSettingsManager } = metaversefile;

const localVector = new THREE.Vector3();
const localVector2 = new THREE.Vector3();
Expand Down Expand Up @@ -77,12 +77,99 @@ const _addAnisotropy = (o, anisotropyLevel) => {
});
};

const _setQuality = async (quality, app) => {
const skinnedVrms = app.skinnedVrms;
const baseVrm = skinnedVrms.base;

const _swapMaterials = async (type) => {

//actually do the swap
switch (type) {
case "toon": {

let update = Object.keys(app.materials.toon).length > 0;

skinnedVrms.base.scene.traverse((object) => {
if (object.material && app.isBasic(object.material)) {
const name = object.material.name;
app.setMaterial(name, 'base', object.material);
update && (object.material = [app.materials.toon[name]]);
}

});
break;

}
default: {
let update = false;
const target = skinnedVrms.base;
target.scene.traverse((object) => {
if (!update && object.material && app.isToon(object.material)) {
update = true;
}
});

target.scene.traverse((object) => {
if (object.material && app.isToon(object.material)) {
const name = object.material[0].name;
update && app.setMaterial(name, 'toon', object.material[0]);

object.material = app.materials.base[name];
}
});
}
}
}

switch (quality ?? 'MEDIUM') {
case 'LOW':
case 'MEDIUM':
case 'HIGH': {
await _swapMaterials();
baseVrm.scene.name = "base mesh"
app.setActive('base');
break;
}
case 'ULTRA': {
if (skinnedVrms.toon) {

} else {
if (!skinnedVrms.toon) {

await _toonShaderify(baseVrm);
skinnedVrms['toon'] = baseVrm;
skinnedVrms.toon.scene.name = 'base-tooned';
}
}

await _swapMaterials("toon");
app.setActive('toon');
break;
}
default: {
throw new Error('unknown avatar quality: ' + quality);
}
}
return app.getActive();
}

export default e => {
const physics = usePhysics();

const app = useApp();
app.appType = 'vrm';
app.active = 'base';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Active sounds like a boolean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Active is the mesh that corresponds to the users performance settings. this only has support for base and toon, but there will also be sprite and crunched. I can rename if you'd prefer something else

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The answer to "What is the app's .active?" sounds like a boolean to me.


//make sure we have materials to work with
app.materials = {
toon: {},
base: {}
};
app.isToon = material => material[0] && material[0].isMToonMaterial;
app.isBasic = material => material.type == "MeshBasicMaterial" && material.name; //we're only changing named materials
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try not to base anything on THREE.js name, that is mostly for user convenience.

app.setMaterial = (name, type, material) => app.materials[type][name] = material;

app.skinnedVrms = {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't expose this, it's internal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure


const srcUrl = ${ this.srcUrl };
for (const { key, value } of components) {
Expand All @@ -93,25 +180,53 @@ export default e => {
const _cloneVrm = async () => {
const vrm = await parseVrm(arrayBuffer, srcUrl);
vrm.cloneVrm = _cloneVrm;
vrm.toonShaderify = _toonShaderify;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't expose this, it's internal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

return vrm;
};

const _prepVrm = (vrm) => {
//vrm.visible = false; //will need later
vrm.visible = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do this. If the VRM is invisible it is probably for good reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there will always be at least 2 vrms loaded, the base(high quality) vrm, and the crunched vrm. this defaults all the meshs to not visible to start, and the required meshes will be toggles accordingly.

app.add(vrm);
vrm.updateMatrixWorld();
_addAnisotropy(vrm, 16);
}

app.getActive = (_app = false) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a good function.

First of all it it's a getter that takes a parameter, which is bad.
Second, that parameter is not even consistent with the parameter of setActive.
Third, getActive sounds like a boolean. Is it active? But that is not what is returned here.
Fourth, the method is polymorphic and will return many different types.

Can we delete this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getActive is going to return the mesh that's corresponded with the current quality setting. this only has support for base(high quality) and toon(ultra quality), but there will also be sprite and crunched. I can rename for clarity, and remove the _app parameter so that only only 1 type is returned, and have the developer simply access .scene on their own.

//return scene if we have an active vrm and we're not requesting the scene. else return the(possibly) active vrm
return app.skinnedVrms[app.active] && !_app ? app.skinnedVrms[app.active].scene : app.skinnedVrms[app.active];
}

// use this to change which mesh we're using
app.setActive = (target) => {
for (const key in app.skinnedVrms) {
if (Object.hasOwnProperty.call(app.skinnedVrms, key)) {
app.skinnedVrms[key].scene.visible = false
}
}

app.active = target;
!app.getActive().parent && _prepVrm(app.getActive());
app.getActive().visible = true;
}

let physicsIds = [];
let activateCb = null;
e.waitUntil((async () => {
arrayBuffer = await _fetchArrayBuffer(srcUrl);

const skinnedVrmBase = await _cloneVrm();
app.skinnedVrm = skinnedVrmBase;
await _toonShaderify(skinnedVrmBase);
app.skinnedVrms['base'] = skinnedVrmBase;
app.skinnedVrm = skinnedVrmBase; //temporary support for webaverse code base until it's updated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I delete this, the webaverse app will break because it's currently referencing skinnedVrm when making avatars. We'll need to update the webaverse app to reference skinnedVrms, which will also cause it to break if this has not been merged yet.

_prepVrm(skinnedVrmBase.scene);
app.skinnedVrms.base.scene.name = 'base mesh';
//collect basic materials for reuse
app.skinnedVrms.base.scene.traverse((o) => {
if (o.material && app.isBasic(o.material)) {
const name = o.material.name;
app.setMaterial(name, 'base', o.material);
}
});


const _addPhysics = () => {
const fakeHeight = 1.5;
Expand Down Expand Up @@ -144,8 +259,18 @@ export default e => {
localPlayer.setAvatarApp(app);
};

app.updateQuality = async () => {
const gfxSettings = useSettingsManager().getSettingsJson('GfxSettings');
const quality = gfxSettings.character.details;
return await _setQuality(quality, app)
}

await app.updateQuality();


})());


useActivate(() => {
activateCb && activateCb();
});
Expand Down Expand Up @@ -189,4 +314,4 @@ export default e => {
export const contentId = ${ this.contentId };
export const name = ${ this.name };
export const description = ${ this.description };
export const components = ${ this.components };
export const components = ${ this.components };