Restructuring shader generation in ofMaterial

[neilmendoza]

I’m super psyched that PBR materials with shadows are part of OF now. Thanks for all the hard work that went into this @NickHardeman! After having played around, I have a few ideas that could make it more usable and offer something unique that’s not available in any other coding framework or game engine.

The thing that’s always drawn me back to OF is its hackability and that it doesn’t add unnecessary complexity on top of the underlying APIs and libraries it uses. I felt like with the shadow/PBR (and to some extent the renderer abstraction) code, there was a slight shift towards hiding complexity at the expense of flexibility (in day to day use rather than writing a new renderer). When doing anything out of the ordinary with custom shaders and shadows, you need a lot of knowledge about what OF is doing under the hood to avoid gotchas. For instance…

• The generated shaders would always use GL_TEXTURE_RECTANGLE or GL_TEXTURE_2D, you couldn’t mix in a texture of the other type.
• There were OF specific placeholders, variable names, function names and naming conventions that weren’t documented.
• Shaders would get disabled and enabled in unexpected places, e.g. you could enable a depth shader and then the renderer would disable it before any rendering happened.
• Automatic binding could be confusing, for example…
  • I’d have to call mat.setDiffuseTexture(dummyTexture) before mat.setShaderMain(x,x,x) to be able to bind a dynamic texture later on.
  • You’d have to call setCustomUniform(x,x) before the material was bound rather than just being able to bind uniforms directly to the shader after it was bound

It’s been a couple of months since I was deep in openFrameworks land so the details are a little hazy but it felt like there was an extra level of indirection that we could get rid of with a little restructuring. Maybe that could be a combination of any of the following…

• Use a full on template engine for the shader generation, e.g. https://github.com/pantor/inja.
• Take shader building out of ofMaterial into a separate class which allows for more fine grained control modular control of how the shaders are generated.
• Have some functionality for “manual mode” where you use the generated shaders like any other shader.

[NickHardeman]

@neilmendoza thank you for your thoughts on this! It has been an interesting task tying in the complexity that comes with shadows and PBR rendering with the goal of creating something that is flexible to use, fast (enough ;) and runs on all of the supported platforms and GL versions. There is definitely room for improvement as it feels a bit hard to read and follow as the generated shaders are scattered throughout the OF project.

• The generated shaders would always use GL_TEXTURE_RECTANGLE or GL_TEXTURE_2D, you couldn’t mix in a texture of the other type.

Ah yes, with the current setup, a shader per texture type, this might be tricky but the ability should be supported.

• There were OF specific placeholders, variable names, function names and naming conventions that weren’t documented.

There needs to be some documentation on this, down to the OF shader level, ie. the uniforms that are set by OF. The shadow frag shader needs certain vars to write proper depth to the output fbo. There are a lot of functions available for the material shaders that need to be documented ( outside of just the examples ) to properly understand the structure of the shader.

• Shaders would get disabled and enabled in unexpected places, e.g. you could enable a depth shader and then the renderer would disable it before any rendering happened.

This should be fixed now, thank you for mentioning it at an earlier time :)

• Automatic binding could be confusing, for example…

  • I’d have to call mat.setDiffuseTexture(dummyTexture) before mat.setShaderMain(x,x,x) to be able to bind a dynamic texture later on.

That is strange and confusing and should be remedied!

• You’d have to call setCustomUniform(x,x) before the material was bound rather than just being able to bind uniforms directly to the shader after it was bound.

Hmm, I think you need to call setCustomUniform(x,x) at least once so the uniforms are exposed since ofMaterial stores the uniforms internally. Currently if the shader was bound, every time that setCustomUniform(x,x) was called and the uniform not present, then it would have to unbind, reconstruct, load and then bind the shader.
The material could just try to set the uniform and not try to check if it's present in its internal list.
Would it make sense to have an addCustomUniform(x, x) in addition to the setCustomUniform(x,x)?

You should be able to set uniforms while the shader is bound, but not create new ones ( maybe not textures though :0 Some gpus did not like if a texture was not set) The materialPBRExample should demonstrate this:

openFrameworks/examples/gl/materialPBRExample/src/ofApp.cpp

Line 57 in cb8d7af

material.begin();

• Take shader building out of ofMaterial into a separate class which allows for more fine grained control modular control of how the shaders are generated.

Yes, I love this idea and have been thinking about something similar for a while but am unsure about the best approach. Decoupling renderer specific code ( ie glsl shader code, GL fbo code, etc. ) from classes to hold data only would help with future renderers as well.
Thinking of ofMaterial, ofLight, ofCubeMap and ofShadow. But unsure where the renderer specific code would live per class? Any thoughts on this?
A lot of code is GL dependent in ofShadow and ofCubeMap as they require certain GL textures and fbos to function properly. Ideally they would be renderer agnostic.
I added a function to configure a shader to be compatible with ofShadow, but is confusing that it lives inside the ofShadow class.

openFrameworks/libs/openFrameworks/gl/ofShadow.cpp

Line 1144 in cb8d7af

const bool ofShadow::setupShadowDepthShader(ofShader& ashader, const std::string aShaderMain) const {

It seems like it would be easier to include shaders/shadow/shadowFunctions.glsl or something similar in your glsl shader, but then those shaders would have to be exposed somehow. They are in various places throughout the project and mostly dynamically generated or configured, so it's hard to decipher and understand.

A few ways that shaders are currently being constructed or dynamically created:

• ofShader: Using a string that gets replaced.
  https://github.com/openframeworks/openFrameworks/blob/7d10a735c40c34e58f9cc453e7363b87e4a5ca3b/libs/openFrameworks/gl/ofShader.cpp#L364C38-L369

• ofShadow: function to create the vert shader.
  

  openFrameworks/libs/openFrameworks/gl/ofShadow.cpp

  Lines 1144 to 1146 in 7d10a73

  	const bool ofShadow::setupShadowDepthShader(ofShader& ashader, const std::string aShaderMain) const {
  	return setupShadowDepthShader( ashader, data->lightType, aShaderMain, isSingleOmniPass() );
  	}

• ofMaterial: dynamic defines, custom uniforms and string replacement based on GL capabilities.
  Stores at least 6 different shaders based on what is set, which is why it could be difficult to have both texture2D and textueRect with the current setup.
  

  openFrameworks/libs/openFrameworks/gl/ofMaterial.h

  Lines 413 to 422 in 7d10a73

  	struct Shaders {
  	ofShader noTexture;
  	ofShader color;
  	ofShader texture2DColor;
  	ofShader textureRectColor;
  	ofShader texture2D;
  	ofShader textureRect;
  	size_t numLights;
  	size_t numCubeMaps;
  	std::string shaderId;

Wondering if a large shader for ofMaterial that activates pieces of code via defines would make the code more readable or just too overwhelming. The pieces of code could be separated into files and included based on what is needed. So the code would be in the separated files and not spread throughout ofMaterial.

• Have some functionality for “manual mode” where you use the generated shaders like any other shader.

Would love a way to have a shader as you would normally use it, but include bits of shader code regarding PBR, etc. The shader files for PBR are broken up to be included in such a manner. It also includes functions to create and configure materials, etc. However, the only way to access it easily is via ofMaterial. And then you must set a shader main on ofMaterial, which can be confusing, as you mentioned.