About dataset

[LeeHanmin]

Hi~  It's a really fascinating achievement！
I have some questions.After I preprocess facescape with python data_preprocess/preprocess.py in imface,do I get imface_image and image_model?
Should I use python run/train.py [--config file's name] to train imface in the next step to get imface_mask?
Because after I run train_pl.py he reports AttributeError: 'NoneType' object has no attribute 'shape'

[aejion]

Thanks for your attention!

First, we use the data preprocessing code to crop the high-quality mesh, which is necessary for subsequent rendering mask. So, when using ImFace's preprocessing code, you cannot obtain "imface_image" and "imface_model", and "imface_model" denotes the "tu_model" provided by Facescape, which is used for correcting camera poses.

Secondly, I recommend using the rendering code provided by the official Facescape repo to render the mask. However, please note that you need to use the cropped mesh and corrected camera poses to render the mask. And please refer to #2 for more details.

[aejion]

Hi, we have updated the data preprocessing code in this repo. @LeeHanmin

[LeeHanmin]

Thanks!

[LeeHanmin]

What is the “high_mesh_path = '' # origin mesh path (high quality)” in cut_mesh.py file?
and
Is "tu_mesh_path = '' # tu model path " the obj file in models_reg in the TU_model folder？

When I use the new preprocessed file, I often get the following error: ValueError: string is not a file.
So I wrote a section to automatically read the files in models_reg under each id as trimesh.load objects
models_reg_dir = './ImFace/dataset/FacescapeOriginData'
id_folders = [folder for folder in os.listdir(models_reg_dir) if os.path.isdir(os.path.join(models_reg_dir, folder))]
for id_folder in id_folders:
id_path = os.path.join(models_reg_dir, id_folder)
models_reg_path = os.path.join(id_path, 'models_reg')

 if os.path.exists(models_reg_path):
     obj_files = [file for file in os.listdir(models_reg_path) if file.lower().endswith('.obj')]
    
     for obj_files in obj_files:
         model_path = os.path.join(models_reg_path, obj_files)

tu_mesh_path = '' # tu model path
high_mesh_path = '' # origin mesh path (high quality)
output_mesh_path = '' # output mesh path

mview_mesh = trimesh.load(model_path, process=False, maintain_order=True)
print(f"Loaded model '{obj_files}' with {len(mview_mesh.vertices)} vertices.")

Can you show me your dataset folder structure like you did in imface？

[aejion]

First of all, high_mesh_path denotes the ply file of the mesh in Facescpae origin dataset (named fsmview_trainset_images_xxx-xxx.zip) and tu_mesh_path denotes the obj file in the TU_model folder. And I recommend you to check whether 'id_index' and 'exp_index' is correct.

[LeeHanmin]

Thanks a lot.

[LeeHanmin]

I found a problem. A series of obj files generated by cut_mesh.py are point clouds instead of mesh, but these obj files cannot be read in render_mask.py. Do I need to convert all point cloud files into mesh files before proceeding with the render_mask step?

[aejion]

Could you please provide an example? I have not encountered this issue in my use.

[aejion]

Could you give me the origin mesh, which you have used in the cut_mesh.py?

[LeeHanmin]

Do you mean the high_mesh_path in cut_mesh.py?

[aejion]

Yes, and have you changed the 'id_index' and 'exp_index' in cut_mesh.py when you deal with different mesh?

[aejion]

You should modify the 'id_index' and 'exp_index' variables in the cut_mesh.py file to align with the specific mesh you are processing. If the issue persists, please email me with the problematic mesh.

[aejion]

The term 'tu_mesh_path' refers to the TU mesh obtained from FaceScape. Each TU mesh is associated with a corresponding high quality mesh. Additionally, it's important to note that the location of the nose point varies ('point') for each mesh. Therefore, you should determine the nose point based on the specific mesh you process.

[LeeHanmin]

So what you mean is that the .obj you get using cut_mesh.py is a mesh and not a point cloud file, right?