Skip to content

feat(voiceburst): Voice Burst - voice messages with Codec2 compression#4994

Open
Chris7X wants to merge 4 commits intomeshtastic:mainfrom
Chris7X:pr/voice-burst
Open

feat(voiceburst): Voice Burst - voice messages with Codec2 compression#4994
Chris7X wants to merge 4 commits intomeshtastic:mainfrom
Chris7X:pr/voice-burst

Conversation

@Chris7X
Copy link
Copy Markdown

@Chris7X Chris7X commented Apr 4, 2026

  • Codec2 JNI integration for Android (arm64-v8a, x86_64)
  • AudioRecorder/AudioPlayer interfaces
  • VoiceBurstViewModel and UI button
  • GPL-3.0 license headers
  • All comments in English

This PR adds the VoiceBurst feature module, enabling voice messages
with Codec2 compression over the Meshtastic mesh network.

Note:

Third-party binaries
The .so files are compiled from:

These libraries are dynamically linked and distributed under LGPL-2.1,
compatible with this project's GPL-3.0 license.

Airtime impact

  • Max burst duration: 1 second (~88 bytes Codec2 700B, single MeshPacket)
  • Minimum 30 seconds between sends (enforced in ViewModel)
  • DM-only: no mesh broadcast path implemented
  • Port: PRIVATE_APP (256), provisional

Changes

  • Codec2 JNI integration for Android (arm64-v8a, x86_64)
  • AudioRecorder/AudioPlayer interfaces (common + Android implementations)
  • VoiceBurstViewModel and UI button
  • GPL-3.0 license headers
  • All comments in English

How it works

Voice messages are recorded, compressed with Codec2 700B and transmitted as mesh packets. On receive, they are decoded and played back.

Testing

Tested on arm64-v8a device. x86_64 build included for emulator support.

Please keep enabled "Allow edits by maintainers" ✓

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 4, 2026

CLA assistant check
All committers have signed the CLA.

@Chris7X Chris7X marked this pull request as draft April 4, 2026 20:55
@Chris7X Chris7X marked this pull request as ready for review April 4, 2026 21:08
@shortwavesurfer2009
Copy link
Copy Markdown

shortwavesurfer2009 commented Apr 4, 2026

May need some sort of limiter on message length for this. LongFast only does 1.07 kbps (1070 bits/s) (133.7bytes/s). I would think that audio messages should be limited to no more than say 10 seconds and only be allowed to be sent to nodes that you have a direct connection with or over the new 2.4 GHz radio links where wider bandwidth is available.

@jamesarich
Copy link
Copy Markdown
Collaborator

this will need very broad testing, approval, and support - airtime is already at a premium

@jamesarich
Copy link
Copy Markdown
Collaborator

@Chris7X as an outside contributor it may be a good idea to start your feature propsals as an issue/feature request. That way we can have some discussion whether or not it's a good fit for the project and details prior to spending the effort.

@Chris7X
Copy link
Copy Markdown
Author

Chris7X commented Apr 5, 2026

Thanks all for the feedback, these are exactly the right concerns to raise!
Just to clarify what has actually been implemented, since the PR description wasn't explicit enough:

Maximum duration: One second (MAX_DURATION_MS is hardcoded to 1000). This is not 10 seconds — one burst is approximately 88 bytes of Codec2 700B, which fits well under the 240-byte limit in a single MeshPacket.
The rate limit is 30 seconds between consecutive sends (RATE_LIMIT_MS = 30000), which is enforced in the ViewModel before touching the radio.
DM-only: routing goes through the existing contactKey mechanism. There is no broadcast path in the current implementation.
Port: PRIVATE_APP (256): provisional and not using any registered mesh port. The code contains a TODO to address this issue upstream before any potential merge.

@Chris7X
Copy link
Copy Markdown
Author

Chris7X commented Apr 5, 2026

#4998

@Chris7X
Copy link
Copy Markdown
Author

Chris7X commented Apr 5, 2026

Minor: renamed a misleading log line (broadcasted → sent to $destNodeId) to be consistent with the DM-only claim.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Voice Burst feature module to record short voice clips, compress them with Codec2 700B, send/receive them over the mesh (PRIVATE_APP / 256), and play them back on Android.

Changes:

  • Introduces common interfaces/models/UI for recording, encoding, sending, receiving, and playback (ViewModel + Compose button).
  • Adds Android implementations for Codec2 JNI encoding/decoding, audio record/playback, and a repository that persists audio to disk + packets to DB.
  • Adds Android unit tests for the Codec2 encoder and bundles prebuilt JNI .so binaries.

Reviewed changes

Copilot reviewed 18 out of 22 changed files in this pull request and generated 14 comments.

Show a summary per file
File Description
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/ui/VoiceBurstViewModel.kt Voice-burst state machine orchestration (record → encode → send, plus autoplay on receive)
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/ui/VoiceBurstButton.kt Compose PTT button that reflects VoiceBurstState
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/repository/VoiceBurstRepository.kt Platform-agnostic repository contract for feature flag + send/receive + file reads
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/model/VoiceBurstState.kt State machine + error taxonomy for the feature
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/model/VoiceBurstPayload.kt Wire payload format (header + Codec2 bytes) and encode/decode helpers
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/di/FeatureVoiceBurstModule.kt Koin common module registration via component scan
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/codec/Codec2Encoder.kt Common interface for Codec2 encode/decode with stub support
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/audio/AudioRecorder.kt Common audio recording abstraction
feature/voiceburst/src/commonMain/kotlin/org/meshtastic/feature/voiceburst/audio/AudioPlayer.kt Common audio playback abstraction
feature/voiceburst/src/androidUnitTest/kotlin/org/meshtastic/feature/voiceburst/codec/AndroidCodec2EncoderTest.kt JVM/Robolectric-friendly tests for stub/JNI codec behavior + payload sizing
feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/repository/AndroidVoiceBurstRepository.kt Android send/receive integration (DataStore flag, PacketRepository persistence, file I/O)
feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/di/FeatureVoiceBurstAndroidModule.kt Android Koin providers (DataStore, repository, codec, audio recorder/player)
feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/codec/AndroidCodec2Encoder.kt JNI-backed Codec2 encoder/decoder with normalization + VAD + stub fallback
feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/audio/AndroidAudioRecorder.kt AudioRecord-based microphone capture with early-stop support
feature/voiceburst/src/androidMain/kotlin/org/meshtastic/feature/voiceburst/audio/AndroidAudioPlayer.kt AudioTrack-based PCM playback with UI sync via playingFilePath
feature/voiceburst/src/androidMain/kotlin/org/meshtastic/codec2/Codec2Jni.kt Alternate JNI wrapper (currently appears unused)
feature/voiceburst/src/androidMain/kotlin/com/geeksville/mesh/voiceburst/Codec2JNI.kt JNI loader/binding used by AndroidCodec2Encoder
feature/voiceburst/src/androidMain/jniLibs/x86_64/libcodec2_jni.so Prebuilt JNI shared library for x86_64
feature/voiceburst/build.gradle.kts Module build config and JNI presence logging

Comment on lines +30 to +34
if (codec2Available) {
logger.lifecycle(":feature:voiceburst — libcodec2.so + libcodec2_jni.so trovate")
} else {
logger.lifecycle(":feature:voiceburst — .so assenti → stub mode (esegui scripts/build_codec2.sh)")
}
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Gradle log messages here are in Italian ("trovate", "assenti", "esegui...") and also include a unicode arrow. The PR description says all comments/messages should be English; please translate these messages and keep them ASCII/UTF-8 clean so they render consistently in build output.

Copilot uses AI. Check for mistakes.
Comment on lines +25 to +33
// ─── Codec2 JNI detection ─────────────────────────────────────────
val codec2SoArm64 = File(projectDir, "src/androidMain/jniLibs/arm64-v8a/libcodec2.so")
val codec2JniArm64 = File(projectDir, "src/androidMain/jniLibs/arm64-v8a/libcodec2_jni.so")
val codec2Available = codec2SoArm64.exists() && codec2JniArm64.exists()

if (codec2Available) {
logger.lifecycle(":feature:voiceburst — libcodec2.so + libcodec2_jni.so trovate")
} else {
logger.lifecycle(":feature:voiceburst — .so assenti → stub mode (esegui scripts/build_codec2.sh)")
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

codec2Available only checks the arm64-v8a .so files, but the module ships multiple ABIs (e.g., x86_64). If the intent is to detect whether any JNI is present (or the currently-building ABI), consider checking all relevant jniLibs folders so the log doesn't incorrectly report stub mode.

Copilot uses AI. Check for mistakes.
Comment on lines +47 to +48
* MIC → [AudioRecorder] → PCM → [Codec2Encoder.encode] → bytes → [VoiceBurstRepository.sendBurst]
* RADIO → [VoiceBurstRepository.incomingBursts] → bytes → [Codec2Encoder.decode] → PCM → [AudioPlayer]
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KDoc text shows mojibake characters (e.g., "→" instead of "→" / "->"). This will render poorly in generated docs and reviews; please re-save the file as UTF-8 and/or replace these symbols with plain ASCII (e.g., "->").

Suggested change
* MIC → [AudioRecorder] → PCM → [Codec2Encoder.encode] → bytes → [VoiceBurstRepository.sendBurst]
* RADIO → [VoiceBurstRepository.incomingBursts] → bytes → [Codec2Encoder.decode] → PCM → [AudioPlayer]
* MIC -> [AudioRecorder] -> PCM -> [Codec2Encoder.encode] -> bytes -> [VoiceBurstRepository.sendBurst]
* RADIO -> [VoiceBurstRepository.incomingBursts] -> bytes -> [Codec2Encoder.decode] -> PCM -> [AudioPlayer]

Copilot uses AI. Check for mistakes.
uiTimerJob?.cancel()
uiTimerJob = null
Logger.e(tag = TAG) { "Hardware recording error: ${error.message}" }
_state.update { VoiceBurstState.Error(VoiceBurstError.ENCODING_FAILED) }
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The audio recorder error callback updates state with VoiceBurstError.ENCODING_FAILED, but this path is a recording/hardware failure (before encoding starts). This should likely be VoiceBurstError.RECORDING_FAILED so the UI can present the correct error and recovery path.

Suggested change
_state.update { VoiceBurstState.Error(VoiceBurstError.ENCODING_FAILED) }
_state.update { VoiceBurstState.Error(VoiceBurstError.RECORDING_FAILED) }

Copilot uses AI. Check for mistakes.
Comment on lines +57 to +58
* Visible only if [VoiceBurstViewModel.isVisible] == true (feature flag enabled).
* Disabled during encoding/sending/rate limit.
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This KDoc mentions VoiceBurstViewModel.isVisible, but VoiceBurstViewModel doesn't define such a property (and the button API only takes a state). Please update/remove this reference to avoid misleading future readers.

Suggested change
* Visible only if [VoiceBurstViewModel.isVisible] == true (feature flag enabled).
* Disabled during encoding/sending/rate limit.
* Render this composable only when Voice Burst is available; callers should not render it
* for [VoiceBurstState.Unsupported].
* Disabled during non-interactive processing states such as encoding and sending.

Copilot uses AI. Check for mistakes.
/**
* Koin module for the Voice Burst feature module.
*
* Follows the same pattern as [FeatureAchievementsAndroidModule]:
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module KDoc references FeatureAchievementsAndroidModule, but there is no such class in the repo (search only finds this mention). Please update the reference to an existing module or remove it to avoid dead links/misleading documentation.

Suggested change
* Follows the same pattern as [FeatureAchievementsAndroidModule]:
* Follows the standard Android feature-module pattern:

Copilot uses AI. Check for mistakes.
Comment on lines +20 to +54
* JNI wrapper for the Codec2 library.
* This class is the interface between Kotlin/JVM and the C codec logic.
*/
class Codec2Jni {

/**
* Encodes 16-bit mono PCM audio (8kHz) into Codec2 compressed frames.
* @param pcm Input audio data (ShortArray)
* @return Compressed byte array or null on error
*/
external fun encode(pcm: ShortArray): ByteArray?

/**
* Decodes Codec2 compressed frames back into 16-bit mono PCM audio (8kHz).
* @param compressed Compressed audio data (ByteArray)
* @return Decoded ShortArray or null on error
*/
external fun decode(compressed: ByteArray): ShortArray?

/**
* Gets the current Codec2 mode (e.g., 3200, 2400, etc.).
*/
external fun getMode(): Int

companion object {
init {
try {
System.loadLibrary("codec2_jni")
} catch (e: UnsatisfiedLinkError) {
// Logger not available in this core-module, using println
println("Critical: Could not load codec2_jni library")
}
}
}
}
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codec2Jni appears to be unused (search only finds its own declaration) while the rest of the feature uses com.geeksville.mesh.voiceburst.Codec2JNI. Keeping two JNI wrappers increases maintenance burden and can cause confusion; consider removing this unused wrapper or switching the feature to use it consistently.

Suggested change
* JNI wrapper for the Codec2 library.
* This class is the interface between Kotlin/JVM and the C codec logic.
*/
class Codec2Jni {
/**
* Encodes 16-bit mono PCM audio (8kHz) into Codec2 compressed frames.
* @param pcm Input audio data (ShortArray)
* @return Compressed byte array or null on error
*/
external fun encode(pcm: ShortArray): ByteArray?
/**
* Decodes Codec2 compressed frames back into 16-bit mono PCM audio (8kHz).
* @param compressed Compressed audio data (ByteArray)
* @return Decoded ShortArray or null on error
*/
external fun decode(compressed: ByteArray): ShortArray?
/**
* Gets the current Codec2 mode (e.g., 3200, 2400, etc.).
*/
external fun getMode(): Int
companion object {
init {
try {
System.loadLibrary("codec2_jni")
} catch (e: UnsatisfiedLinkError) {
// Logger not available in this core-module, using println
println("Critical: Could not load codec2_jni library")
}
}
}
}
* Backwards-compatible alias to the canonical Codec2 JNI wrapper used by
* the voiceburst feature.
*/
typealias Codec2Jni = com.geeksville.mesh.voiceburst.Codec2JNI

Copilot uses AI. Check for mistakes.
Comment on lines +38 to +42
* - BUG: MODE_STATIC with bufferSize < minBufferSize → STATE_NO_STATIC_DATA (state=2) → silence.
* FIX: bufferSize = maxOf(minBufferSize, pcmBytes) ALWAYS, even in static mode.
* - Using MODE_STREAM: simpler and avoids the STATE_NO_STATIC_DATA issue.
* For 1 second at 8kHz (16000 bytes) MODE_STREAM is more than adequate.
* - USAGE_MEDIA → main speaker (not earpiece).
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several comments and user-visible log strings contain mojibake sequences (e.g., "→", "�") instead of the intended punctuation/arrows. Please normalize these to proper UTF-8 (or plain ASCII like "->" and "-") so logs/docs are readable.

Suggested change
* - BUG: MODE_STATIC with bufferSize < minBufferSize → STATE_NO_STATIC_DATA (state=2) → silence.
* FIX: bufferSize = maxOf(minBufferSize, pcmBytes) ALWAYS, even in static mode.
* - Using MODE_STREAM: simpler and avoids the STATE_NO_STATIC_DATA issue.
* For 1 second at 8kHz (16000 bytes) MODE_STREAM is more than adequate.
* - USAGE_MEDIA → main speaker (not earpiece).
* - BUG: MODE_STATIC with bufferSize < minBufferSize -> STATE_NO_STATIC_DATA (state=2) -> silence.
* FIX: bufferSize = maxOf(minBufferSize, pcmBytes) ALWAYS, even in static mode.
* - Using MODE_STREAM: simpler and avoids the STATE_NO_STATIC_DATA issue.
* For 1 second at 8kHz (16000 bytes) MODE_STREAM is more than adequate.
* - USAGE_MEDIA -> main speaker (not earpiece).

Copilot uses AI. Check for mistakes.
Comment on lines +133 to +155
/**
* Initiates microphone recording if the state machine is [Idle].
* Enforces the [RATE_LIMIT_MS] guard before starting.
*
* Note: Permissions (RECORD_AUDIO) must be verified by the UI before calling.
*/
fun startRecording() {
if (_state.value !is VoiceBurstState.Idle) return

// Rate limit check
val now = Clock.System.now().toEpochMilliseconds()
val remaining = RATE_LIMIT_MS - (now - lastSentTimestamp)
if (remaining > 0) {
Logger.w(tag = TAG) { "Rate limit active: waiting ${remaining / 1000}s" }
_state.update { VoiceBurstState.Error(VoiceBurstError.RATE_LIMITED) }
viewModelScope.launch {
delay(remaining)
if (_state.value is VoiceBurstState.Error) {
_state.update { VoiceBurstState.Idle }
}
}
return
}
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VoiceBurstViewModel introduces a fairly complex state machine (rate limiting, recording callbacks, encoding, send success/failure, and incoming autoplay) but there are no unit tests covering these transitions. Given there are existing ViewModel tests in other feature modules, consider adding tests for at least the rate-limit path and the recording→sending→sent→idle happy path using fakes for AudioRecorder, Codec2Encoder, and VoiceBurstRepository.

Copilot uses AI. Check for mistakes.
Comment on lines +76 to +78
// ─── NESSUN externalNativeBuild ───────────────────────────────────
// Le .so prebuilt in jniLibs/ vengono incluse automaticamente da AGP.
// Il JNI wrapper è compilato da scripts/build_codec2.sh. No newline at end of file
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These trailing build.gradle.kts comments are in Italian ("NESSUN", "Le .so prebuilt..."). Please translate to English to match the project/PR requirement that comments be English-only.

Copilot uses AI. Check for mistakes.
Chris7X added 4 commits April 6, 2026 11:28
- Codec2 JNI integration for Android (arm64-v8a, x86_64)
- AudioRecorder/AudioPlayer interfaces
- VoiceBurstViewModel and UI button
- GPL-3.0 license headers
- All comments in English
- AndroidCodec2Encoder: VAD returns null instead of ByteArray(0) to prevent
  empty packet transmission; fix stale KDoc ref (Codec2Jni -> Codec2JNI)
- AndroidAudioPlayer: fix mojibake encoding in comments
- VoiceBurstViewModel: ENCODING_FAILED -> RECORDING_FAILED in recorder error
  path; fix 'broadcasted' log to 'sent to \'; fix mojibake in KDoc
- VoiceBurstButton: remove reference to non-existent isVisible property
- FeatureVoiceBurstAndroidModule: remove dead ref to FeatureAchievementsAndroidModule
- Codec2Jni: replace duplicate JNI class with typealias to Codec2JNI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants