[BUG] Un-necessary assertion ntypes < len(pps) when making siesta input files

[AsymmetryChou]

Bug summary

Hi!
I'm running DPGEN for systems containing element 'C' and 'H' with SIESTA as the DFT engine.
In this task, 'sys_configs' includes pure 'C' systems and systems with both 'C' and 'H'. The 'fp_pp_files = ["C.psf", "H.psf"]'. However, an unexpected error occurs:

INFO:dpgen:-------------------------iter.000015 task 06--------------------------
INFO:dpgen:system 015 candidate :      1 in     11   9.09 %
INFO:dpgen:system 015 failed    :     10 in     11  90.91 %
INFO:dpgen:system 015 accurate  :      0 in     11   0.00 %
INFO:dpgen:system 015 accurate_ratio:   0.0000    thresholds: 1.0000 and 1.0000   eff. task min and max   -1   20   number of fp tasks:      1
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/dpgen/bin/dpgen", line 7, in <module>
    sys.exit(main())
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/main.py", line 255, in main
    args.func(args)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/run.py", line 5721, in gen_run
    run_iter(args.PARAM, args.MACHINE)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/run.py", line 5070, in run_iter
    make_fp(ii, jdata, mdata)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/run.py", line 4011, in make_fp
    make_fp_calculation(iter_index, jdata, mdata)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/run.py", line 4034, in make_fp_calculation
    make_fp_siesta(iter_index, jdata)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/run.py", line 3723, in make_fp_siesta
    ret = make_siesta_input(sys_data, fp_pp_files, fp_params)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/lib/siesta.py", line 137, in make_siesta_input
    ret += _make_siesta_02_species(sys_data, fp_pp_files)
  File "/home/ubuntu/miniconda3/envs/dpgen/lib/python3.10/site-packages/dpgen/generator/lib/siesta.py", line 50, in _make_siesta_02_species
    assert ntypes == len(pps)
AssertionError

The root reason seems to be a unnecessary assert check in dpgen/generator/lib/siesta.py:_make_siesta_02_species. The 'ntypes' is just necessary to be a subset of 'pps'.

DP-GEN Version

Version: 0.13.2.dev62+g037d6b819

Platform, Python Version, Remote Platform, etc

No response

Input Files, Running Commands, Error Log, etc.

'param.json'

{
     "type_map": ["C","H"],
     "mass_map": [12,1],
     "init_data_prefix": "../init/deepmd_data",
     "init_data_sys": [
  "C110H0", "C112H0", "C123H3", "C126H0", "C128H0",
  "C55H3", "C62H0", "C64H0", "C71H3", "C82H0",
  "C84H0", "C87H3", "C111H3", "C122H0", "C124H0",
  "C127H3", "C54H0", "C56H0", "C63H3", "C70H0",
  "C72H0", "C83H3", "C86H0", "C88H0"
],
     "sys_configs_prefix": "/home/ubuntu/run/CNT_single_1560/init",
     "sys_configs": [
  ["POSCARS_OUT/1000k/6_6/5775/POSCAR"],
  ["POSCARS_OUT/1000k/6_6/DV/POSCAR"],
  ["POSCARS_OUT/1000k/6_6/MVH/POSCAR"],
  ["POSCARS_OUT/1000k/6_6/P/POSCAR"],
  ["POSCARS_OUT/1000k/7_1/5775/POSCAR"],
  ["POSCARS_OUT/1000k/7_1/DV/POSCAR"],
  ["POSCARS_OUT/1000k/7_1/MVH/POSCAR"],
  ["POSCARS_OUT/1000k/7_1/P/POSCAR"],
  ["POSCARS_OUT/1000k/9_0/5775/POSCAR"],
  ["POSCARS_OUT/1000k/9_0/DV/POSCAR"],
  ["POSCARS_OUT/1000k/9_0/MVH/POSCAR"],
  ["POSCARS_OUT/1000k/9_0/P/POSCAR"],
  ["POSCARS_OUT/300k/6_6/5775/POSCAR"],
  ["POSCARS_OUT/300k/6_6/DV/POSCAR"],
  ["POSCARS_OUT/300k/6_6/MVH/POSCAR"],
  ["POSCARS_OUT/300k/6_6/P/POSCAR"],
  ["POSCARS_OUT/300k/7_1/5775/POSCAR"],
  ["POSCARS_OUT/300k/7_1/DV/POSCAR"],
  ["POSCARS_OUT/300k/7_1/MVH/POSCAR"],
  ["POSCARS_OUT/300k/7_1/P/POSCAR"],
  ["POSCARS_OUT/300k/9_0/5775/POSCAR"],
  ["POSCARS_OUT/300k/9_0/DV/POSCAR"],
  ["POSCARS_OUT/300k/9_0/MVH/POSCAR"],
  ["POSCARS_OUT/300k/9_0/P/POSCAR"],
  ["POSCARS_OUT/50k/6_6/5775/POSCAR"],
  ["POSCARS_OUT/50k/6_6/DV/POSCAR"],
  ["POSCARS_OUT/50k/6_6/MVH/POSCAR"],
  ["POSCARS_OUT/50k/6_6/P/POSCAR"],
  ["POSCARS_OUT/50k/7_1/5775/POSCAR"],
  ["POSCARS_OUT/50k/7_1/DV/POSCAR"],
  ["POSCARS_OUT/50k/7_1/MVH/POSCAR"],
  ["POSCARS_OUT/50k/7_1/P/POSCAR"],
  ["POSCARS_OUT/50k/9_0/5775/POSCAR"],
  ["POSCARS_OUT/50k/9_0/DV/POSCAR"],
  ["POSCARS_OUT/50k/9_0/MVH/POSCAR"],
  ["POSCARS_OUT/50k/9_0/P/POSCAR"],
  ["POSCARS_OUT/700k/6_6/5775/POSCAR"],
  ["POSCARS_OUT/700k/6_6/DV/POSCAR"],
  ["POSCARS_OUT/700k/6_6/MVH/POSCAR"],
  ["POSCARS_OUT/700k/6_6/P/POSCAR"],
  ["POSCARS_OUT/700k/7_1/5775/POSCAR"],
  ["POSCARS_OUT/700k/7_1/DV/POSCAR"],
  ["POSCARS_OUT/700k/7_1/MVH/POSCAR"],
  ["POSCARS_OUT/700k/7_1/P/POSCAR"],
  ["POSCARS_OUT/700k/9_0/5775/POSCAR"],
  ["POSCARS_OUT/700k/9_0/DV/POSCAR"],
  ["POSCARS_OUT/700k/9_0/MVH/POSCAR"],
  ["POSCARS_OUT/700k/9_0/P/POSCAR"]
],
     "_comment": " that's all ",
     "numb_models": 4,
     "train_backend": "pytorch",
     "training_iter0_model_path":[
        "./",
        "./",
        "./",
        "./"
     ],
     
     "default_training_param": {
        "model": {
     "descriptor": {
      "type": "dpa3",
      "repflow": {
        "n_dim": 128,
        "e_dim": 64,
        "a_dim": 32,
        "nlayers": 6,
        "e_rcut": 6.0,
        "e_rcut_smth": 5.3,
        "e_sel": 120,
        "a_rcut": 4.0,
        "a_rcut_smth": 3.5,
        "a_sel": 30,
        "axis_neuron": 4,
        "fix_stat_std": 0.3,
        "a_compress_rate": 1,
        "a_compress_e_rate": 2,
        "a_compress_use_split": true,
        "update_angle": true,
        "smooth_edge_update": true,
        "use_dynamic_sel": true,
        "use_exp_switch": true,
        "update_style": "res_residual",
        "update_residual": 0.1,
        "update_residual_init": "const"
      },
      "activation_function": "silut:3.0",
      "use_tebd_bias": false,
      "precision": "float32",
      "concat_output_tebd": false
    },
    "fitting_net": {
      "neuron": [
        240,
        240,
        240
      ],
      "resnet_dt": true,
      "precision": "float32",
      "activation_function": "silut:3.0",
      "seed": 1,
      "_comment": " that's all"
    },
    "_comment": " that's all"
  },
         "learning_rate": {
             "type": "exp",
             "decay_steps": 3000,
             "start_lr": 0.00003,
             "stop_lr": 3e-06,
             "_comment": "that's all"
         },
         "loss": {
             "start_pref_e": 0.2,
             "limit_pref_e": 2,
             "start_pref_f": 100,
             "limit_pref_f": 100,
             "start_pref_v": 0.0,
             "limit_pref_v": 0.0
         },
         "training": {
             "stop_batch": 20000,
             "disp_file": "lcurve.out",
             "disp_freq": 1000,
             "numb_test": 4,
             "save_freq": 5000,
             "save_ckpt": "model.ckpt",
             "disp_training": true,
             "time_training": true,
             "profiling": false,
             "profiling_file": "timeline.json",
             "_comment": "that's all"
         }
     },
     "model_devi_dt": 0.002,
     "model_devi_skip": 0,
     "model_devi_f_trust_lo": 0.05,
     "model_devi_f_trust_hi": 0.2,
     "model_devi_e_trust_lo": 10000000000.0,
     "model_devi_e_trust_hi": 10000000000.0,
     "model_devi_clean_traj": true,
     "model_devi_jobs": [
  {"sys_idx":[0],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"00"},
  {"sys_idx":[1],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"01"},
  {"sys_idx":[2],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"02"},
  {"sys_idx":[3],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"03"},
  {"sys_idx":[4],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"04"},
  {"sys_idx":[5],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"05"},
  {"sys_idx":[6],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"06"},
  {"sys_idx":[7],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"07"},
  {"sys_idx":[8],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"08"},
  {"sys_idx":[9],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"09"},
  {"sys_idx":[10],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"10"},
  {"sys_idx":[11],"temps":[1000],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"11"},
  {"sys_idx":[12],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"12"},
  {"sys_idx":[13],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"13"},
  {"sys_idx":[14],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"14"},
  {"sys_idx":[15],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"15"},
  {"sys_idx":[16],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"16"},
  {"sys_idx":[17],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"17"},
  {"sys_idx":[18],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"18"},
  {"sys_idx":[19],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"19"},
  {"sys_idx":[20],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"20"},
  {"sys_idx":[21],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"21"},
  {"sys_idx":[22],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"22"},
  {"sys_idx":[23],"temps":[300],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"23"},
  {"sys_idx":[24],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"24"},
  {"sys_idx":[25],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"25"},
  {"sys_idx":[26],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"26"},
  {"sys_idx":[27],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"27"},
  {"sys_idx":[28],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"28"},
  {"sys_idx":[29],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"29"},
  {"sys_idx":[30],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"30"},
  {"sys_idx":[31],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"31"},
  {"sys_idx":[32],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"32"},
  {"sys_idx":[33],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"33"},
  {"sys_idx":[34],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"34"},
  {"sys_idx":[35],"temps":[50],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"35"},
  {"sys_idx":[36],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"36"},
  {"sys_idx":[37],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"37"},
  {"sys_idx":[38],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"38"},
  {"sys_idx":[39],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"39"},
  {"sys_idx":[40],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"40"},
  {"sys_idx":[41],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"41"},
  {"sys_idx":[42],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"42"},
  {"sys_idx":[43],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"43"},
  {"sys_idx":[44],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"44"},
  {"sys_idx":[45],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"45"},
  {"sys_idx":[46],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"46"},
  {"sys_idx":[47],"temps":[700],"press":[1.0],"trj_freq":1000,"nsteps":10000,"ensemble":"nvt","_idx":"47"}
],

     "fp_style": "siesta",
     "fp_task_max": 20,
     "fp_task_min": 1,
     "use_clusters": false,

     "fp_pp_path": "./",
     "fp_pp_files": ["C.psf","H.psf"],
     "fp_params": {
       "ecut": 400,
       "ediff": 1.0e-6,
       "mixingWeight": 0.02,
       "NumberPulay": 10,
       "kspacing": 0.1
    }
}

Steps to Reproduce

Just run dpgen with siesta as DFT engine

Further Information, Files, and Links

No response

[dosubot]

Hi @AsymmetryChou! I'm Dosu and I’m helping the dpgen team.

The AssertionError happens because the code in _make_siesta_02_species expects the number of atomic types in your system to exactly match the number of pseudopotential files (assert ntypes == len(pps)), which isn't always true if some systems only contain a subset of elements from your type_map and fp_pp_files (like pure C systems in a C/H workflow). This is a known issue and not required for correct SIESTA input generation—ntypes just needs to be a subset of the elements for which you provide pseudopotentials.

A practical workaround is to modify the code so that, before calling _make_siesta_02_species, you build the pps list dynamically to include only the pseudopotential files for the atomic types actually present in the current system. This approach is already used for similar issues in other DFT engines in dpgen and has been validated by maintainers. The key is to map each element in sys_data['atom_names'] to its corresponding pseudopotential file from the full fp_pp_files list, using the order in type_map as the reference. This way, the assertion will always pass, and SIESTA input files will be generated correctly for systems with any subset of your elements. For reference and a code example, see this discussion and fix.

The dpgen documentation confirms that the order of fp_pp_files should match type_map, but doesn't address this subset issue or the assertion error directly (docs).

In summary: you can safely remove or adapt the assertion and dynamically select the relevant pseudopotentials for each system to resolve this error.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}