Introduction

This is a R&D project for personal learning purpose, potentially can also be a viewer for cg dev tool in the future.

Setup Environment

Here is a suggested way to setup the dev environment, to prepare:

• python (MacOS: 2.7.16 64bit, Windows: 2.7.18 32bit)
• pip

First to get the virtualenv.

pip install virtualenv

Then choose a the dev folder that contains two sub-folders, one for git repository where stores the source code, the other for the virutal python environment. The structure looks like this:

+ poly3d
  + poly3d
  + poly3d_venv

Goto poly3d root folder, enter the command line to initialize the virtual environment, this might take a while to get virtual pip/setuptools installed depends on the network speed.

virtualenv -p <python_dir>/python.exe poly3d_venv

Once gets everything done, then install poly3d dependencies:

poly3d_venv\Scripts\pip.exe install -r poly3d\requirements.txt

After installing dependencies, you could start project

poly3d_venv\Scripts\python.exe poly3d\app\main.py

troubleshooting

Unable to load OpenGL Library

If you're trying to run PyOpenGL on a MacOS that is newer than BigSur, then you would probably get this error when launching

raise ImportError("Unable to load OpenGL library", *err.args)
ImportError: ('Unable to load OpenGL library', 'dlopen(OpenGL, 10): image not found', 'OpenGL', None)

Solution:

To fix it now you can edit PyOpenGL file OpenGL/platform/ctypesloader.py changing line fullName = util.find_library( name ) to fullName = '/System/Library/Frameworks/OpenGL.framework/OpenGL'

source: https://stackoverflow.com/questions/63475461/unable-to-import-opengl-gl-in-python-on-macos

Dependencies Introduction

• pyopengl is the python-wrapped opengl api set
• pyopengl-accelerate set of C(Cython) extensions provides acceleration of common operations for slow points in PyOpenGL 3.x
• PyOpenGL-Demo contains some demo to get start with
• numpy is the c based math library
• pillow used to be PIL, an image library
• freetype-py the font module
• glfw is the basic ui solution for the app

You can also download pyopengl and its accelerate x86/x64 from this link:

https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopengl

Tips:

• If pip get error with vc++9 issue, you can download whl file, and use pip install *.whl
• If your OS is win10, use 64bit installer

Branch strategy

• master - stable version (with tag)
• dev - feature development.

Architecture and configuration

Only high-level architecture will be mentioned, more details can be found from the source code.

The entire program has 2 parts: app and data. App is where the program itself is while data is where those resources are in terms of objects, meshes, textures and so on, it's also called workspace. Once app is installed, you could have multiple workspace to switch. The configure file workspace.cfg file under the app folder does the configuration.

let's have a look at the workspace.cfg file, by default it looks like this:

{
    "root": "sample_data"
}

root is the name of the root path, you could optionally use one of these method to config it:

• Only provide a folder name (eg: sample_data), then the app will be looking for a folder from its neighbours.
• Otherwise a full path of the workspace needs to be given.

The workspace structure needs to have proper structure:

+ workspace
    + meshes
        *.obj/fbx/..   # only support obj for now
    + textures
        *.png/tga/..
    + model
        *.model
    + shaders
        *.vert
        *.frag

Files under meshes/textures/shades folders are resources while models are basically assets configure files, it uses resources from others. this is an example of a model file:

{
    "model":"pbrspheres.obj",
    "vx_shader":"base.vert",
    "px_shader":"test.frag",
    "shader_sampler":
    {
        "albedoMap":"pbr_c.tga",
        "metallicMap":"pbr_m.tga",
        "roughnessMap":"pbr_r.tga",
        "normalMap":"pbr_n.tga"
    },
    "shader_attrib":
    {
        "r":1.0
    }
}

Note: the shder could be from workspace/shaders or app/poly3d/shaders/*, shaders under app are supposed to be built in and shaders under workspace are customized.

Explore the scene.

Command line is the only way of interacting with the scene currently. There are two ways of accessing objects

• OA (object access) to manipulate object(s)
• CA (command access) to deal with the scene or settings and so on.

These are public objects in the memory that you can access when start the program.

• ctrl. is the instance object of /control/control.py -> Control
• gl. is the instance object of /graphics/scene.py -> Scene
• wld. is the instance object of /world/scene.py -> MainScene
• fbo. is the instance object of /world/fbomgr.py -> FramebufferMgr
• .$obj. is the instance object of /world/entity.py -> Entity
  • ($obj is the current selected object)

Examples of how to use (please check the API from source code):

> gl.eyepos  # showing cam position
> scr.draw_profile=1 # turn on profile info
> wld.draw_cubemap_bg=0 # turn off cubemap display from the background
> fbo.do_blur=1  # turn on blur rendering framebuffer object
> .sphere.mesh.set_pos(0,1,0)  # set sphere pos to 0,1,0

Load assets

You can load an asset by doing this

.objname.load

it's equal to

wld.load_entity(objname)

this command reads .model file, or you could load multiple .model files by open a scene

wld.open_scene(scenename)

Load Cubemaps

by calling this command to load a hdr cubemap

gl.load_cubemap($index)

Load Scene

wld.open_scene($scene_name)

Search methods and attributes

If you want to lookup available memebers, press '' after the last dot, for example, you type:

> wld.    # press '\', all the available attributes and methods would be listed.

You could also put some characters after the dot, press 'tab', it helps you to search the member starts with these charaters.

> .sphere.mesh.se    # press 'tab', to complete all the se* words.

Switch cameras

Press '`' key to switch camera mode, you have 2 types of camera to work with

• Object camera, which looks at center 0,0,0 at begining, you could offset it with cmd
• WASD camera, I'm pretty sure you know what it is! :)

Note: in the WASD mode, you can't use command. WASD for moving, while mouse left button for the orientation.

Using Debugging Features

Debugging Frame Buffers

gl.g_debug=0

• 0: disable debugging.
• 1: position
• 2: sil
• 3: normal
• 4: roughness
• 5: albedo
• 6: metallic
• 7: depth
• 8: shadow map
• 9: ssao
• 10: screen space position
• 11: screen space normal

How to debug textures

You could debug a selected texture from an Entity. Make sure you have loaded a model(eg: pbrspheres), then:

.pbrspheres.debug_texture(albedoMap)

xx to kill all the debugging fbo

Sometimes the debugging buffer is too bright to see the command text, in this case, you could blindly press Esc for couple of times (to clear commands), then press xx to clear fbo, eg:

> xx

FBO Feature turned on/off

fbo.use_hdr
fbo.use_gamma
fbo.use_bloom
fbo.use_autoexp
fbo.use_blur
fbo.use_dof
fbo.use_lighting_model
fbo.ssao

Above commands can be directly queried, eg: fbo.use_hdr will return 0 or 1 to judge if it's turned on or not. (excepting fbo.use_lighting_model use 1 for pbr, 2 for lambert and 0 for unlit)

some special cases:

• fbo.use_dof relied on fbo.use_blur is on, because dof needs a blur buffer
• fbo.use_pbr=0 means using lambert lighting model.

Turn on stencil buffer

load object and set it selected.

.pbrspheres.load
.pbrspheres.set_selected(1)

Debug Compute Shader

To debug compute shader:

fbo.use_compute_shader=1

More details for implementation: see this link

Extend Shader

The program has its own shading pipeline and internal shader managerment system, while you could still extend it.

Built-in shaders are placed under /app/poly3d/shaders/* folder, most of them are core lighting model, you would only use shaders under /surface/* folder. If you want to have some customized shader rather than using or touching shaders in the surface folder, you could make them under /{data}/shaders/*, data is where your meshes and textures are.

So all *.vert and *.frag file placed under /{data}/shaders folder and /poly3d/shaders/surface folder can be recognized when loading object.

Build your shader

The shader language is GLSL, but it has tiny little difference from the standard one. You can use #include command on top of the file to include some functions. Below is a typical exmaple of how it works:

#include /fbo/g_buffer.frag;
#include /func/normal.frag;

uniform sampler2D albedoMap;
uniform sampler2D normalMap;
uniform sampler2D metallicMap;
uniform sampler2D roughnessMap;
uniform sampler2D aoMap;
uniform sampler2D alphaMap;

void main() {
    float alpha = texture(alphaMap, TexCoords).r;
    vec3 albedo = texture(albedoMap, TexCoords).rgb;
    float metallic = texture(metallicMap, TexCoords).r;
    float roughness = texture(roughnessMap, TexCoords).r;
    float ao = 1.0;//texture(aoMap, TexCoords).r;
    vec3 N = normal_ws(normalMap, TexCoords, TBN);

    deferred_render(albedo, N, roughness, metallic);
}

• g_buffer.frag is a fragment shader that extract info to the multiple-render-target(MRT) also called g buffer, we use deferred_render function to the extraction.
• normal.frag has some functions dealing with normal vector.

Mesh Instancing

Instancing mesh requires a specific shader (a vertex shader in this case) to identify a mesh to be instanced.

On the CPU side, using glDrawElementsInstanced() to send the vertex data to GPU, and specify the number of instance to be drawed in the parameters.

In the shader (GPU) using gl_InstanceID to represent the current instanced mesh, by using this ID, you could define different position for the instanced mesh. Below is sample code for instancing mesh.

#include /lightmodel/pbr_core.vert;

void main() {
    vec3 modelSpacePos = vec3(0, gl_InstanceID*2, 0);
    gl_Position = pbr(modelSpacePos);
}