[Bug] Sound not immediately playing

[letsfindaway]

Describe the bug

MP3 sounds not starting or very lately or after multiple clicks. only on Debian, both with 1.7.1 (debian qt5) and 1.7.2 (debian qt6.4).

To Reproduce

1. Open the provided example document with MP3 sound files
2. Select page 1, 2 or 7, which contain audio files
3. Try to play the sound files

Expected behavior

Sound playback should start on first click

Actual behavior

Sound playback does not start on first click, bus probably on the second or third.

Context

• reproducibility : systematic
• OpenBoard version :
  • bug happens with : 1.7.2-dev
  • didn't check : others
• OS :
  • bug happens with : Debian 12
  • bug doesn't happen with : openSUSE Leap 15.6
  • didn't check : others
• Installation source : my own experimental builds of the latest release candidate

[letsfindaway]

First investigations to reproduce the bug:

• The bug does not happen with openSUSE Leap 15.6
• I could reproduce the problem however in a Debian 12 distrobox:

distrobox create --image docker.io/library/debian:bookworm-backports --name debian12
distrobox enter

# and then in the distrobox

sudo apt install /home/family/Downloads/openboard-qt6_1.7.2-rc.241009-1_amd64.deb
sudo apt install gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-plugins-bad
openboard

I also observed that switching to page 7 takes quite some time, while this is immediate on openSUSE. So it seems that already during creation of the media player widgets some problems occur.

I can also confirm that setting QT_MEDIA_BACKEND=ffmpeg solves the problem. So it is probably related to the gstreamer version 1.22.0 used on Debian 12. openSUSE Leap 15.6 is using gstreamer 1.24.0.

I then also installed the gstreamer apps

sudo apt install gstreamer1.0-plugins-base-apps

and then played the audio files from the command line like

gst-play-1.0 .local/share/OpenBoard/document/OpenBoard\ Document\ 2024-11-14\ 13-52-38.740/audios/\{5ee65b28-fd2e-43ed-b482-2fbd25040411\}.mp3

The file was playing immediately and without problems.

My next try is to do the same with Debian testing (trixie), which uses gstreamer 1.24.0.

[letsfindaway]

The problem also occurs with Debian testing and gstreamer 1.24. However it seems that Debian has switched the default Qt6 media backend from gstreamer to ffmpeg. So without setting an environment variable, ffmpeg is used as media backend and I have to set QT_MEDIA_BACKEND=gstreamer to produce the error.

This brought me to another idea: what backend is openSUSE Leap 15.6 using by default? And indeed: it is ffmpeg! I have the same error when I set the gstreamer backend using the environment variable.

So the solution is probably to set ffmpeg as backend on Linux. I assume we can even do this within the application itself, but I will have to check this.

[letsfindaway]

Yes, this works. Just add

#ifdef Q_OS_LINUX
    qputenv("QT_MEDIA_BACKEND", "ffmpeg");
#endif

to the main method and the problem is solved for any Linux distribution which is using gstreamer by default, but has ffmpeg available. I also tested this on Qt5, which does not have that choice and it causes no harm.

[letsfindaway]

Actually the switch of the default backend was performed by Qt, not by Debian. In Qt 6.4.2 as we have it in Debian 12 the default was gstreamer. With Qt 6.5, the default was changed to ffmpeg.

I found https://www.reddit.com/r/QtFramework/comments/10f79f8/qt_65_will_switch_to_ffmpeg_as_the_default_qt/ and they point to this commit: https://code.qt.io/cgit/qt/qtmultimedia.git/commit/?id=5a5fc56e06500c427f4db8666aa103151748588e

+    In Qt 6.5, we're setting \l {http://ffmpeg.org}{FFmpeg framework} as the default backend on
+    Windows, Linux, macOS/iOS, and Android. Native backends, namely gstreamer on Linux,
+    AVFoundation on macOS/iOS, WMF on Windows, and the MediaCodec framework on Android,
+    were the default in 6.4. These are still available but with limited support.
+    This means that we will strive to fix critical issues with the native backends, but don't
+    guarantee fixing minor issues and won't implement new features with native backends.
+    Furthermore, even some major issues with the gstreamer backend (on Linux) won't be fixed
+    since gstreamer is the hardest for debugging and is dependent on linux distributives issues.
+
+    We aim to align the behavior on all the platforms, especially, with the ffmpeg backend.
+    Despite this fact we still have platform-dependent issues with formats, codecs, advanced camera
+    features, and screen capturing. This happens since we use some platform API anyway,
+    even with ffmpeg, and there are specific problems with ffmpeg hardware acceleration.
+    Known backend-dependent limitations will be documented in the respective classes and methods.
+
+    In the case of issues with the default ffmpeg backend, we suggest testing with a native backend.
+    You can switch to native backends by setting the \c{QT_MEDIA_BACKEND} environment variable
+    to \c windows, \c gstreamer (on Linux), \c darwin (on macOS/iOS), or \c android:

[kaamui]

For 1.7.2, changing the backend to ffmpeg helps as it starts immediately, but then after multiple times playing the same sound the issue appears again

The reporter of the issue also described this behavior. What was your version of Debian and ffmpeg ?

[letsfindaway]

For 1.7.2, changing the backend to ffmpeg helps as it starts immediately, but then after multiple times playing the same sound the issue appears again

The reporter of the issue also described this behavior. What was your version of Debian and ffmpeg ?

Debian 12 and ffmpeg 7:5.1.6-0+deb12u1. This should be just part of an up-to-date Debian 12.

And I played the files dozens of times. No problem. I only found that if you do not stop an audio explictly but let it run to the end, then the play button changes again from "Pause" to "Play", but then you either have to press "Stop" in-between or press "Play" exactly twice to run it again.

Probably the reporter had this behavior in mind when reporting.

[letsfindaway]

And I played the files dozens of times. No problem. I only found that if you do not stop an audio explictly but let it run to the end, then the play button changes again from "Pause" to "Play", but then you either have to press "Stop" in-between or press "Play" exactly twice to run it again.

I could fix this by putting the player in the pause state after the audio was stopped or ended. Here:

OpenBoard/src/domain/UBGraphicsMediaItemDelegate.cpp

Lines 279 to 286 in 3b38085

	// updatePlayPauseState handles this functionality
	if (state == QMediaPlayer::StoppedState)
	{
	delegated()->setMediaPos(0);
	#ifdef Q_OS_OSX //media positionChanged signal is not always called in osx
	mMediaControl->updateTicker(0);
	#endif
	}

I added a line:

    // updatePlayPauseState handles this functionality
    if (state == QMediaPlayer::StoppedState)
    {
        delegated()->setMediaPos(0);
        delegated()->pause(); // set to pause after stopping 
#ifdef Q_OS_OSX //media positionChanged signal is not always called in osx
        mMediaControl->updateTicker(0);
#endif
    }

[kaamui]

The problem also occurs with Debian testing and gstreamer 1.24. However it seems that Debian has switched the default Qt6 media backend from gstreamer to ffmpeg. So without setting an environment variable, ffmpeg is used as media backend and I have to set QT_MEDIA_BACKEND=gstreamer to produce the error.

This brought me to another idea: what backend is openSUSE Leap 15.6 using by default? And indeed: it is ffmpeg! I have the same error when I set the gstreamer backend using the environment variable.

So the solution is probably to set ffmpeg as backend on Linux. I assume we can even do this within the application itself, but I will have to check this.

On my VM, with ffmpeg, the sound is played immediately on OpenBoard but the quality is really poor, like if the audio was not buffered fast enough. For you everything is OK when playing the sound ?

[kaamui]

Probably the reporter had this behavior in mind when reporting.

If you go on page 7 and play the short audios, the first play is OK but then the sound is not playing correctly, and end in not being played at all after a few attempts, even when adding the pause you suggested.

Reporter explained me that after the first play, the next play, the sound seems to not be played from start, and will randomly begin at a position until it cannot be played at all.

[kaamui]

If I don't play this part of the code on Linux, for sounds, it seems a bit better :

OpenBoard/src/domain/UBGraphicsMediaItemDelegate.cpp

Lines 245 to 256 in 80fb7dd

	#ifndef Q_OS_OSX
	if ((status == QMediaPlayer::LoadedMedia || status == QMediaPlayer::BufferedMedia)
	&& delegated()->mediaPosition() == delegated()->initialPos()
	&& !delegated()->isStopped()
	&& delegated()->firstLoad()
	)
	{
	delegated()->play();
	delegated()->pause();
	delegated()->setFirstLoad(false);
	}
	#endif

[kaamui]

On my VM, with ffmpeg, the sound is played immediately on OpenBoard but the quality is really poor, like if the audio was not buffered fast enough. For you everything is OK when playing the sound ?

I don't have hardware acceleration on my VM so it might explain this particular issue I'm the only one to encounter

[letsfindaway]

Test results

• All tests are using the 1.7.2 release version and the default package versions for Qt and ffmpeg.

Leap 15.6

• Qt 6.6.3
• ffmpeg 4.4
• ffmpeg is default backend
• all sounds play nicely, even repeatedly
• for the second time I have to press the play button twice
• message [mp3 @ 0x559cf9324500] Estimating duration from bitrate, this may be inaccurate appears multiple times
• message [mp3float @ 0x559cf994c4c0] overread, skip -7 enddists: -1 -1 appears multiple times

Debian 12

• Qt 6.4.2
• ffmpeg 5.1
• gstreamer is default backend
• start with QT_MEDIA_BACKEND=ffmpeg openboard
• all sounds play nicely, even repeatedly
• for the second time I have to press the play button twice
• message [mp3 @ 0x559cf9324500] Estimating duration from bitrate, this may be inaccurate appears multiple times
• message [mp3float @ 0x559cf994c4c0] overread, skip -7 enddists: -1 -1 appears multiple times

Debian Unstable (13)

• Qt 6.7.2
• ffmpeg 7.1
• ffmpeg is default backend
• all sounds play nicely, even repeatedly
• for the second time I have to press the play button twice
• message [mp3 @ 0x559cf9324500] Estimating duration from bitrate, this may be inaccurate appears multiple times

[letsfindaway]

If you want you may try the Debian Unstable version on your computer using distrobox:

# install distrobox and podman
sudo apt install distrobox podman

# download the package for Debian Unstable
wget https://download.opensuse.org/repositories/home:/letsfindaway:/release/Debian_Unstable/amd64/openboard_1.7.2-1_amd64.deb

# create and enter distrobox
distrobox create --image docker.io/library/debian:unstable --name debian13
distrobox enter debian13

# now you are in the distrobox - install openboard from the previously downloaded file
sudo apt install ./openboard_1.7.2-1_amd64.deb

# start openboard
openboard

# press Ctrl+D to leave distrobox
# if all your tests are done, you might delete all the stuff
distrobox stop debian13
distrobox rm debian13