Towards the Explainability of Multimodal Speech Emotion Recognition

Implementation for the paper (Interspeech 2021). The paper has been accepted, its full-text will be shared after publication.
Towards the Explainability of Multimodal Speech Emotion Recognition
Puneet Kumar, Vishesh Kaushik, and Balasubramanian Raman

Setup and Dependencies

1. Install Anaconda or Miniconda distribution and create a conda environment with Python 3.6+.
2. Install the requirements using the following command:

pip install -r Requirements.txt

3. Download glove.6B.zip, unzip and keep in glove.6B folder.
4. Download the required datasets.

Steps to run the Code

1. For IEMOCAP Dataset:
   Run Data_Preprocess(IEMOCAP).ipynb in Jupyter Notebook, then
   Run Training(IEMOCAP).ipynb in Jupyter Notebook, then
   Run Analysis(IEMOCAP).ipynb in Jupyter Notebook.
   OR
   Run main_IEMOCAP.py in the terminal/command-line using the following command:

python main_IEMOCAP.py --epoch=100

2. For MSP-IMPROV Dataset:
   Run Data_Preprocess(IMPROV).ipynb in Jupyter Notebook, then
   Run Training(IMPROV).ipynb in Jupyter Notebook, then
   Run Analysis(IMPROV).ipynb in Jupyter Notebook.
   OR
   Run main_IMPROV.py in the terminal/command-line using the following command:

python main_IMPROV.py --epoch=100

3. For RAVDESS Dataset:
   Run Data_Preprocess(RAVDESS).ipynb in Jupyter Notebook, then
   Run Training(RAVDESS).ipynb in Jupyter Notebook, then
   Run Analysis(RAVDESS).ipynb in Jupyter Notebook.
   OR
   Run main_RAVDESS.py in the terminal/command-line using the following command:

python main_RAVDESS.py --epoch=100

Saving Model Checkpoints

By default, the code saves the model checkpoints in the log-1 folder.

Tensorboard Logging

The tensorboard log files are saved in the log-1 folder. These files can be accessed using the following command:

tensorboard --logdir "/log-1"