feat(voiceburst): Voice Burst - voice messages with Codec2 compression#4994
feat(voiceburst): Voice Burst - voice messages with Codec2 compression#4994Chris7X wants to merge 4 commits intomeshtastic:mainfrom
Conversation
|
May need some sort of limiter on message length for this. LongFast only does 1.07 kbps (1070 bits/s) (133.7bytes/s). I would think that audio messages should be limited to no more than say 10 seconds and only be allowed to be sent to nodes that you have a direct connection with or over the new 2.4 GHz radio links where wider bandwidth is available. |
|
this will need very broad testing, approval, and support - airtime is already at a premium |
|
@Chris7X as an outside contributor it may be a good idea to start your feature propsals as an issue/feature request. That way we can have some discussion whether or not it's a good fit for the project and details prior to spending the effort. |
|
Thanks all for the feedback, these are exactly the right concerns to raise! Maximum duration: One second (MAX_DURATION_MS is hardcoded to 1000). This is not 10 seconds — one burst is approximately 88 bytes of Codec2 700B, which fits well under the 240-byte limit in a single MeshPacket. |
|
Minor: renamed a misleading log line (broadcasted → sent to $destNodeId) to be consistent with the DM-only claim. |
There was a problem hiding this comment.
Pull request overview
Adds a new Voice Burst feature module to record short voice clips, compress them with Codec2 700B, send/receive them over the mesh (PRIVATE_APP / 256), and play them back on Android.
Changes:
- Introduces common interfaces/models/UI for recording, encoding, sending, receiving, and playback (ViewModel + Compose button).
- Adds Android implementations for Codec2 JNI encoding/decoding, audio record/playback, and a repository that persists audio to disk + packets to DB.
- Adds Android unit tests for the Codec2 encoder and bundles prebuilt JNI
.sobinaries.
Reviewed changes
Copilot reviewed 18 out of 22 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/ui/VoiceBurstViewModel.kt | Voice-burst state machine orchestration (record → encode → send, plus autoplay on receive) |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/ui/VoiceBurstButton.kt | Compose PTT button that reflects VoiceBurstState |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/repository/VoiceBurstRepository.kt | Platform-agnostic repository contract for feature flag + send/receive + file reads |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/model/VoiceBurstState.kt | State machine + error taxonomy for the feature |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/model/VoiceBurstPayload.kt | Wire payload format (header + Codec2 bytes) and encode/decode helpers |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/di/FeatureVoiceBurstModule.kt | Koin common module registration via component scan |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/codec/Codec2Encoder.kt | Common interface for Codec2 encode/decode with stub support |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/audio/AudioRecorder.kt | Common audio recording abstraction |
| feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/audio/AudioPlayer.kt | Common audio playback abstraction |
| feature/voiceburst/src/androidUnitTest/kotlin/org/meshtastic/feature/voiceburst/codec/AndroidCodec2EncoderTest.kt | JVM/Robolectric-friendly tests for stub/JNI codec behavior + payload sizing |
| feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/repository/AndroidVoiceBurstRepository.kt | Android send/receive integration (DataStore flag, PacketRepository persistence, file I/O) |
| feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/di/FeatureVoiceBurstAndroidModule.kt | Android Koin providers (DataStore, repository, codec, audio recorder/player) |
| feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/codec/AndroidCodec2Encoder.kt | JNI-backed Codec2 encoder/decoder with normalization + VAD + stub fallback |
| feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/audio/AndroidAudioRecorder.kt | AudioRecord-based microphone capture with early-stop support |
| feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/audio/AndroidAudioPlayer.kt | AudioTrack-based PCM playback with UI sync via playingFilePath |
| feature/voiceburst/src/androidMain/kotlin/org/meshtastic/codec2/Codec2Jni.kt | Alternate JNI wrapper (currently appears unused) |
| feature/voiceburst/src/androidMain/kotlin/com/geeksville/mesh/voiceburst/Codec2JNI.kt | JNI loader/binding used by AndroidCodec2Encoder |
| feature/voiceburst/src/androidMain/jniLibs/x86_64/libcodec2_jni.so | Prebuilt JNI shared library for x86_64 |
| feature/voiceburst/build.gradle.kts | Module build config and JNI presence logging |
| if (codec2Available) { | ||
| logger.lifecycle(":feature:voiceburst — libcodec2.so + libcodec2_jni.so trovate") | ||
| } else { | ||
| logger.lifecycle(":feature:voiceburst — .so assenti → stub mode (esegui scripts/build_codec2.sh)") | ||
| } |
There was a problem hiding this comment.
The Gradle log messages here are in Italian ("trovate", "assenti", "esegui...") and also include a unicode arrow. The PR description says all comments/messages should be English; please translate these messages and keep them ASCII/UTF-8 clean so they render consistently in build output.
feature/voiceburst/build.gradle.kts
Outdated
| // ─── Codec2 JNI detection ───────────────────────────────────────── | ||
| val codec2SoArm64 = File(projectDir, "src/androidMain/jniLibs/arm64-v8a/libcodec2.so") | ||
| val codec2JniArm64 = File(projectDir, "src/androidMain/jniLibs/arm64-v8a/libcodec2_jni.so") | ||
| val codec2Available = codec2SoArm64.exists() && codec2JniArm64.exists() | ||
|
|
||
| if (codec2Available) { | ||
| logger.lifecycle(":feature:voiceburst — libcodec2.so + libcodec2_jni.so trovate") | ||
| } else { | ||
| logger.lifecycle(":feature:voiceburst — .so assenti → stub mode (esegui scripts/build_codec2.sh)") |
There was a problem hiding this comment.
codec2Available only checks the arm64-v8a .so files, but the module ships multiple ABIs (e.g., x86_64). If the intent is to detect whether any JNI is present (or the currently-building ABI), consider checking all relevant jniLibs folders so the log doesn't incorrectly report stub mode.
| * MIC → [AudioRecorder] → PCM → [Codec2Encoder.encode] → bytes → [VoiceBurstRepository.sendBurst] | ||
| * RADIO → [VoiceBurstRepository.incomingBursts] → bytes → [Codec2Encoder.decode] → PCM → [AudioPlayer] |
There was a problem hiding this comment.
KDoc text shows mojibake characters (e.g., "→" instead of "→" / "->"). This will render poorly in generated docs and reviews; please re-save the file as UTF-8 and/or replace these symbols with plain ASCII (e.g., "->").
| * MIC → [AudioRecorder] → PCM → [Codec2Encoder.encode] → bytes → [VoiceBurstRepository.sendBurst] | |
| * RADIO → [VoiceBurstRepository.incomingBursts] → bytes → [Codec2Encoder.decode] → PCM → [AudioPlayer] | |
| * MIC -> [AudioRecorder] -> PCM -> [Codec2Encoder.encode] -> bytes -> [VoiceBurstRepository.sendBurst] | |
| * RADIO -> [VoiceBurstRepository.incomingBursts] -> bytes -> [Codec2Encoder.decode] -> PCM -> [AudioPlayer] |
| uiTimerJob?.cancel() | ||
| uiTimerJob = null | ||
| Logger.e(tag = TAG) { "Hardware recording error: ${error.message}" } | ||
| _state.update { VoiceBurstState.Error(VoiceBurstError.ENCODING_FAILED) } |
There was a problem hiding this comment.
The audio recorder error callback updates state with VoiceBurstError.ENCODING_FAILED, but this path is a recording/hardware failure (before encoding starts). This should likely be VoiceBurstError.RECORDING_FAILED so the UI can present the correct error and recovery path.
| _state.update { VoiceBurstState.Error(VoiceBurstError.ENCODING_FAILED) } | |
| _state.update { VoiceBurstState.Error(VoiceBurstError.RECORDING_FAILED) } |
| * Visible only if [VoiceBurstViewModel.isVisible] == true (feature flag enabled). | ||
| * Disabled during encoding/sending/rate limit. |
There was a problem hiding this comment.
This KDoc mentions VoiceBurstViewModel.isVisible, but VoiceBurstViewModel doesn't define such a property (and the button API only takes a state). Please update/remove this reference to avoid misleading future readers.
| * Visible only if [VoiceBurstViewModel.isVisible] == true (feature flag enabled). | |
| * Disabled during encoding/sending/rate limit. | |
| * Render this composable only when Voice Burst is available; callers should not render it | |
| * for [VoiceBurstState.Unsupported]. | |
| * Disabled during non-interactive processing states such as encoding and sending. |
| /** | ||
| * Koin module for the Voice Burst feature module. | ||
| * | ||
| * Follows the same pattern as [FeatureAchievementsAndroidModule]: |
There was a problem hiding this comment.
This module KDoc references FeatureAchievementsAndroidModule, but there is no such class in the repo (search only finds this mention). Please update the reference to an existing module or remove it to avoid dead links/misleading documentation.
| * Follows the same pattern as [FeatureAchievementsAndroidModule]: | |
| * Follows the standard Android feature-module pattern: |
| * JNI wrapper for the Codec2 library. | ||
| * This class is the interface between Kotlin/JVM and the C codec logic. | ||
| */ | ||
| class Codec2Jni { | ||
|
|
||
| /** | ||
| * Encodes 16-bit mono PCM audio (8kHz) into Codec2 compressed frames. | ||
| * @param pcm Input audio data (ShortArray) | ||
| * @return Compressed byte array or null on error | ||
| */ | ||
| external fun encode(pcm: ShortArray): ByteArray? | ||
|
|
||
| /** | ||
| * Decodes Codec2 compressed frames back into 16-bit mono PCM audio (8kHz). | ||
| * @param compressed Compressed audio data (ByteArray) | ||
| * @return Decoded ShortArray or null on error | ||
| */ | ||
| external fun decode(compressed: ByteArray): ShortArray? | ||
|
|
||
| /** | ||
| * Gets the current Codec2 mode (e.g., 3200, 2400, etc.). | ||
| */ | ||
| external fun getMode(): Int | ||
|
|
||
| companion object { | ||
| init { | ||
| try { | ||
| System.loadLibrary("codec2_jni") | ||
| } catch (e: UnsatisfiedLinkError) { | ||
| // Logger not available in this core-module, using println | ||
| println("Critical: Could not load codec2_jni library") | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Codec2Jni appears to be unused (search only finds its own declaration) while the rest of the feature uses com.geeksville.mesh.voiceburst.Codec2JNI. Keeping two JNI wrappers increases maintenance burden and can cause confusion; consider removing this unused wrapper or switching the feature to use it consistently.
| * JNI wrapper for the Codec2 library. | |
| * This class is the interface between Kotlin/JVM and the C codec logic. | |
| */ | |
| class Codec2Jni { | |
| /** | |
| * Encodes 16-bit mono PCM audio (8kHz) into Codec2 compressed frames. | |
| * @param pcm Input audio data (ShortArray) | |
| * @return Compressed byte array or null on error | |
| */ | |
| external fun encode(pcm: ShortArray): ByteArray? | |
| /** | |
| * Decodes Codec2 compressed frames back into 16-bit mono PCM audio (8kHz). | |
| * @param compressed Compressed audio data (ByteArray) | |
| * @return Decoded ShortArray or null on error | |
| */ | |
| external fun decode(compressed: ByteArray): ShortArray? | |
| /** | |
| * Gets the current Codec2 mode (e.g., 3200, 2400, etc.). | |
| */ | |
| external fun getMode(): Int | |
| companion object { | |
| init { | |
| try { | |
| System.loadLibrary("codec2_jni") | |
| } catch (e: UnsatisfiedLinkError) { | |
| // Logger not available in this core-module, using println | |
| println("Critical: Could not load codec2_jni library") | |
| } | |
| } | |
| } | |
| } | |
| * Backwards-compatible alias to the canonical Codec2 JNI wrapper used by | |
| * the voiceburst feature. | |
| */ | |
| typealias Codec2Jni = com.geeksville.mesh.voiceburst.Codec2JNI |
| * - BUG: MODE_STATIC with bufferSize < minBufferSize → STATE_NO_STATIC_DATA (state=2) → silence. | ||
| * FIX: bufferSize = maxOf(minBufferSize, pcmBytes) ALWAYS, even in static mode. | ||
| * - Using MODE_STREAM: simpler and avoids the STATE_NO_STATIC_DATA issue. | ||
| * For 1 second at 8kHz (16000 bytes) MODE_STREAM is more than adequate. | ||
| * - USAGE_MEDIA → main speaker (not earpiece). |
There was a problem hiding this comment.
Several comments and user-visible log strings contain mojibake sequences (e.g., "→", "�") instead of the intended punctuation/arrows. Please normalize these to proper UTF-8 (or plain ASCII like "->" and "-") so logs/docs are readable.
| * - BUG: MODE_STATIC with bufferSize < minBufferSize → STATE_NO_STATIC_DATA (state=2) → silence. | |
| * FIX: bufferSize = maxOf(minBufferSize, pcmBytes) ALWAYS, even in static mode. | |
| * - Using MODE_STREAM: simpler and avoids the STATE_NO_STATIC_DATA issue. | |
| * For 1 second at 8kHz (16000 bytes) MODE_STREAM is more than adequate. | |
| * - USAGE_MEDIA → main speaker (not earpiece). | |
| * - BUG: MODE_STATIC with bufferSize < minBufferSize -> STATE_NO_STATIC_DATA (state=2) -> silence. | |
| * FIX: bufferSize = maxOf(minBufferSize, pcmBytes) ALWAYS, even in static mode. | |
| * - Using MODE_STREAM: simpler and avoids the STATE_NO_STATIC_DATA issue. | |
| * For 1 second at 8kHz (16000 bytes) MODE_STREAM is more than adequate. | |
| * - USAGE_MEDIA -> main speaker (not earpiece). |
| /** | ||
| * Initiates microphone recording if the state machine is [Idle]. | ||
| * Enforces the [RATE_LIMIT_MS] guard before starting. | ||
| * | ||
| * Note: Permissions (RECORD_AUDIO) must be verified by the UI before calling. | ||
| */ | ||
| fun startRecording() { | ||
| if (_state.value !is VoiceBurstState.Idle) return | ||
|
|
||
| // Rate limit check | ||
| val now = Clock.System.now().toEpochMilliseconds() | ||
| val remaining = RATE_LIMIT_MS - (now - lastSentTimestamp) | ||
| if (remaining > 0) { | ||
| Logger.w(tag = TAG) { "Rate limit active: waiting ${remaining / 1000}s" } | ||
| _state.update { VoiceBurstState.Error(VoiceBurstError.RATE_LIMITED) } | ||
| viewModelScope.launch { | ||
| delay(remaining) | ||
| if (_state.value is VoiceBurstState.Error) { | ||
| _state.update { VoiceBurstState.Idle } | ||
| } | ||
| } | ||
| return | ||
| } |
There was a problem hiding this comment.
VoiceBurstViewModel introduces a fairly complex state machine (rate limiting, recording callbacks, encoding, send success/failure, and incoming autoplay) but there are no unit tests covering these transitions. Given there are existing ViewModel tests in other feature modules, consider adding tests for at least the rate-limit path and the recording→sending→sent→idle happy path using fakes for AudioRecorder, Codec2Encoder, and VoiceBurstRepository.
feature/voiceburst/build.gradle.kts
Outdated
| // ─── NESSUN externalNativeBuild ─────────────────────────────────── | ||
| // Le .so prebuilt in jniLibs/ vengono incluse automaticamente da AGP. | ||
| // Il JNI wrapper è compilato da scripts/build_codec2.sh. No newline at end of file |
There was a problem hiding this comment.
These trailing build.gradle.kts comments are in Italian ("NESSUN", "Le .so prebuilt..."). Please translate to English to match the project/PR requirement that comments be English-only.
- Codec2 JNI integration for Android (arm64-v8a, x86_64) - AudioRecorder/AudioPlayer interfaces - VoiceBurstViewModel and UI button - GPL-3.0 license headers - All comments in English
- AndroidCodec2Encoder: VAD returns null instead of ByteArray(0) to prevent empty packet transmission; fix stale KDoc ref (Codec2Jni -> Codec2JNI) - AndroidAudioPlayer: fix mojibake encoding in comments - VoiceBurstViewModel: ENCODING_FAILED -> RECORDING_FAILED in recorder error path; fix 'broadcasted' log to 'sent to \'; fix mojibake in KDoc - VoiceBurstButton: remove reference to non-existent isVisible property - FeatureVoiceBurstAndroidModule: remove dead ref to FeatureAchievementsAndroidModule - Codec2Jni: replace duplicate JNI class with typealias to Codec2JNI
This PR adds the VoiceBurst feature module, enabling voice messages
with Codec2 compression over the Meshtastic mesh network.
Note:
Third-party binaries
The .so files are compiled from:
These libraries are dynamically linked and distributed under LGPL-2.1,
compatible with this project's GPL-3.0 license.
Airtime impact
Changes
How it works
Voice messages are recorded, compressed with Codec2 700B and transmitted as mesh packets. On receive, they are decoded and played back.
Testing
Tested on arm64-v8a device. x86_64 build included for emulator support.
Please keep enabled "Allow edits by maintainers" ✓