Issues with flash-attn and running on python 3.13

[nathanchenseanwalter]

I installed versa successfully but when I ran the test I got

I'm not sure if it's because flash attention is a requirement since I saw this before the run failed. But at least with my setup I'm not able to get flash-attn. It seems like there are other depracations.

  import pkg_resources
[nltk_data] Error loading punkt: <urlopen error [SSL:
[nltk_data]     CERTIFICATE_VERIFY_FAILED] certificate verify failed:
[nltk_data]     unable to get local issuer certificate (_ssl.c:1028)>
Failed to import Flash Attention, using ESPnet default: No module named 'flash_attn'
^CTraceback (most recent call last):
  File "/scratch/gpfs/nc1514/specscrub/versa/test/test_pipeline/test_general.py", line 7, in <module>
    from versa.scorer_shared import (
    ...<4 lines>...
    )
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/versa/__init__.py", line 35, in <module>
    from versa.utterance_metrics.speaker import speaker_metric, speaker_model_setup
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/versa/utterance_metrics/speaker.py", line 10, in <module>
    from espnet2.bin.spk_inference import Speech2Embedding
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/espnet2/bin/spk_inference.py", line 13, in <module>
    from espnet2.tasks.spk import SpeakerTask
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/espnet2/tasks/spk.py", line 40, in <module>
    from espnet2.tasks.abs_task import AbsTask
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/espnet2/tasks/abs_task.py", line 88, in <module>
    import wandb
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/wandb/__init__.py", line 22, in <module>
    from wandb.sdk.lib import wb_logging as _wb_logging
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/wandb/sdk/__init__.py", line 24, in <module>
    from . import wandb_helper as helper
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/wandb/sdk/wandb_helper.py", line 6, in <module>
    from .lib import config_util
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/wandb/sdk/lib/config_util.py", line 10, in <module>
    from wandb.util import load_yaml
  File "/scratch/gpfs/nc1514/specscrub/.venv/lib/python3.13/site-packages/wandb/util.py", line 56, in <module>
    import wandb.env
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec```

[nathanchenseanwalter]

For minimal reproducible, I used uv to make a venv. then installed on that.

So after git clone, and cd

uv venv --seed --python 3.13.5
uv pip install .
python test/...

[nathanchenseanwalter]

Ok so an update to this:

1. I manually installed flash-attn through the prebuilt. However I still get that flash-attn can't be imported

2. I needed to install fadtk manually

3. there was an issue with certification, so i first had to run
   python -c "import certifi; print(certifi.where())"

then
export SSL_CERT_FILE=$(python -c "import certifi; print(certifi.where())")

4. and had to add in these keys to test_general.py

"dnsmos_pro_bvcc": 1.1717286109924316, # added in manually
"dnsmos_pro_nisqa": 1.4733699560165405, # added in manually
"dnsmos_pro_vcc2018": 1.930935263633728, # added in manually
"dpam_distance": 0.15004253387451172, # added in manually
"cdpam_distance": 0.05146043747663498, # added in manually

Then the test passed. However there are still a lot of depracation warnings. I'll post below

[nathanchenseanwalter]

/scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/pysptk/util.py:32: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
[nltk_data] Downloading package punkt to /home/nc1514/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
Failed to import Flash Attention, using ESPnet default: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/flash_attn_2_cuda.cpython-313-x86_64-linux-gnu.so)
WARNING:root:FADTK is not installed. Please install it following `tools/install_fadtk.sh`
{'name': 'mcd_f0', 'f0min': 40, 'f0max': 800, 'mcep_shift': 5, 'mcep_fftl': 1024, 'mcep_dim': 39, 'mcep_alpha': 0.466, 'seq_mismatch_tolerance': 0.1, 'power_threshold': -20, 'dtw': False}
{'name': 'signal_metric'}
{'name': 'pesq'}
{'name': 'stoi'}
{'name': 'discrete_speech'}
Using a cache at /scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/discrete_speech_metrics/km/km200.bin
Using a cache at /scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/discrete_speech_metrics/km/km200.bin
{'name': 'pseudo_mos', 'predictor_types': ['utmos', 'dnsmos', 'plcmos', 'dnsmos_pro_bvcc', 'dnsmos_pro_nisqa', 'dnsmos_pro_vcc2018'], 'predictor_args': {'utmos': {'fs': 16000}, 'dnsmos': {'fs': 16000}, 'plcmos': {'fs': 16000}}}
Using cache found in versa_cache/ftshijt_SpeechMOS_main
/scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/torch/nn/utils/weight_norm.py:144: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
  WeightNorm.apply(module, name, dim)
{'name': 'speaker', 'model_tag': 'default'}
Fetching 24 files: 100%|██████████████████████████████| 24/24 [00:00<00:00, 16067.57it/s]
{'name': 'squim_ref'}
{'name': 'squim_no_ref'}
{'name': 'sheet_ssqa'}
Using cache found in versa_cache/unilight_sheet_v0.1.0
100%|███████████████████████████████████████████████| 1.63k/1.63k [00:00<00:00, 11.6MB/s]
/scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/torch/nn/utils/weight_norm.py:144: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
  WeightNorm.apply(module, name, dim)
{'name': 'se_snr', 'model_tag': 'default'}
Fetching 32 files: 100%|██████████████████████████████| 32/32 [00:00<00:00, 11234.43it/s]
WARNING:root:Eps is deprecated in si_snr loss, set clamp_db instead.
{'name': 'dpam'}
{'name': 'cdpam'}
  0%|                                                              | 0/1 [00:00<?, ?it/s]/scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/versa/sequence_metrics/signal_metric.py:50: FutureWarning: mir_eval.separation.bss_eval_sources
        Deprecated as of mir_eval version 0.8.
        It will be removed in mir_eval version 0.9.
  sdr, sir, sar, _ = bss_eval_sources(gt_x, pred_x, compute_permutation=False)
/scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/espnet2/asr/frontend/asteroid_frontend.py:89: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
/scratch/gpfs/nc1514/versa/.venv/lib/python3.13/site-packages/versa/sequence_metrics/signal_metric.py:50: FutureWarning: mir_eval.separation.bss_eval_sources
        Deprecated as of mir_eval version 0.8.
        It will be removed in mir_eval version 0.9.
  sdr, sir, sar, _ = bss_eval_sources(gt_x, pred_x, compute_permutation=False)
100%|██████████████████████████████████████████████████████| 1/1 [00:11<00:00, 11.32s/it]
Summary: {'mcd': 5.045226506332897, 'f0rmse': 20.281004489942774, 'f0corr': -0.07540903652440147, 'sdr': 4.873952979593651, 'sir': inf, 'sar': 4.873952979593651, 'si_snr': 1.0702767372131348, 'ci_sdr': 4.8739519119262695, 'pesq': 1.5722705125808716, 'stoi': 0.007625108859647413, 'speech_bert': 0.9727544784545898, 'speech_bleu': 0.6699938983346256, 'speech_token_distance': 0.850506056080969, 'utmos': 1.9074389934539795, 'dns_overall': 1.45260584658217, 'dns_p808': 2.09430193901062, 'plcmos': 3.1869560718536376, 'dnsmos_pro_bvcc': 1.1717288494110107, 'dnsmos_pro_nisqa': 1.4733701944351196, 'dnsmos_pro_vcc2018': 1.9309346675872803, 'spk_similarity': 0.895361065864563, 'torch_squim_mos': 3.9482548236846924, 'torch_squim_stoi': 0.6027804017066956, 'torch_squim_pesq': 1.1683127880096436, 'torch_squim_si_sdr': -11.109053611755371, 'sheet_ssqa': 1.5056077241897583, 'se_sdr': -10.220606003834323, 'se_sar': -10.220606003834323, 'se_si_snr': -16.837072372436523, 'se_ci_sdr': -10.220602989196777, 'dpam_distance': 0.1500423550605774, 'cdpam_distance': 0.05146058648824692}

[ftshijt]

Hi thanks for your careful check and interests in using the toolkit.

Regarding your questions and comments:

• flash-attn is not necessary (the error msg was from ESPnet, not VERSA). But we do find that there is usually strict dependency on flash-attn in recent model release, and are looking for stable way of handling it. But in short, you can ignore the warning message now.
• fadtk is not required also, unless certain metrics are in interest.
• The major issue that blocks you seems to be the certification process, which is however, from nltk, so we cannot make dedicate adjustment, but will try to see if we can share your experiences in the installation doc (Many thanks for catching this issue!)
• Many thanks for sharing that the keys are not included! We will add that accordingly.

As you may notice, we are working towards a refactoring of the whole project with complete support in automating test cases, trying to include all metrics in CI tests and support more stable inference experiences (#37). While the interface would be exactly the same as for now, we expect the whole using experiences will be improved significantly. Please stay tuned!