[ocakciomursel-pixel]

What changed and why

Fixes #25148. Keep Omi's speaker identification and owner/People voice profiles enabled when Custom STT (custom_stt_mode=enabled) is active so live speaker matching continues to work whenever raw audio is forwarded (sendRawAudioToOmi=true), skip embedding extraction cleanly when raw audio is not streamed (sendRawAudioToOmi=false), and preserve upstream is_user and person_id metadata supplied directly by a Custom STT provider across segment merging, live transcript status projection, and post-conversation speaker resolution.

Line-Count-Exception: backend/routers/listen/transcripts.py | 1816 -> 1834 | Guard custom STT speaker-embedding queueing without raw audio and preserve upstream speaker metadata
Line-Count-Exception: backend/utils/conversations/speaker_resolution.py | 1567 -> 1571 | Preserve upstream custom STT speaker metadata during post-conversation speaker resolution

Product invariants affected

none

How it was verified

• Exercised the live session decision, profile loading, segment merging, live _speaker_detection / _apply_speaker_identity_statuses, and post-conversation apply_speaker_resolution paths across Custom STT sessions both with and without streamed raw audio (sendRawAudioToOmi=true and sendRawAudioToOmi=false), as well as upstream is_user and person_id payloads:
  • pytest tests/unit/test_user_speaker_embedding.py tests/unit/test_transcribe_decisions.py tests/unit/test_listen_custom_stt_without_raw_audio.py tests/unit/test_live_speaker_carry.py tests/unit/test_transcript_segment.py tests/unit/test_speaker_label_provenance.py tests/unit/test_listen_parity_runtime.py tests/unit/test_listen_runtime_regressions.py -q -> 223 passed
  • Backend ratchet guard suite (test_*ratchet*.py + test_reason_label_enums.py) -> 103 passed
  • Full speaker, Custom STT, and listen runtime unit regression sweep (91 files) -> 2630 passed
  • black --check --line-length 120 --skip-string-normalization -> 8 files would be left unchanged

Tests

• backend/tests/unit/test_user_speaker_embedding.py:
  • TestTranscribeFirestoreLoading::test_custom_stt_loads_owner_and_people_voice_profiles and test_excluded_profiles_are_not_read_or_loaded verify owner and People voice profiles load under custom_stt_mode='enabled' while include_speech_profile=False (freemium:on-device), onboarding_mode=True, and multi-channel sessions remain excluded.
  • TestCustomSttSpeakerIdentification verifies unlabeled Custom STT segments queue for speaker embedding matching when PCM audio is present in audio_ring_buffer, skip queueing cleanly when raw audio is absent, and preserve upstream is_user/person_id across TranscriptSegment.combine_segments, TranscriptProcessor._apply_speaker_identity_statuses, and apply_speaker_resolution.
• backend/tests/unit/test_transcribe_decisions.py:
  • test_speech_profile_and_speaker_id_gates and test_upstream_speaker_identity_predicate_and_mutator_are_separated verify decision gates without use_custom_stt plus separated has_upstream_speaker_identity / mark_upstream_speaker_identity helpers.

UI evidence

UI-Evidence: none -- backend Custom STT speaker identification and voice profile pipeline fix with no UI changes

Failure class (fixes)

Failure-Class: none

[Git-on-my-level]

Verdict: the core fix is correct and nicely scoped — speaker identification and voice profiles stay live under Custom STT, degrade cleanly when raw audio is not forwarded, and upstream is_user/person_id labels survive merging and resolution. Nothing blocking from my side; one scope confirmation and two nits below.

Verified on head 0369e4e:

• backend/utils/transcribe_decisions.py — should_load_speech_profile / should_enable_speaker_identification no longer special-case Custom STT; the skip moves to queue time via the new custom_stt_without_raw_audio() (ring buffer present but empty => no embedding work), which handles sendRawAudioToOmi=false correctly. preserve_upstream_speaker_identity() defers to matcher provenance (segment_assignments, speaker_label_source, speaker_match_source), so Omi's own assignments still win.
• backend/routers/listen/speakers.py — dropping use_custom_stt from the _load_profiles eligibility predicate mirrors the decision-function change; owner/People profile loads stay bounded by the existing failed-load caps.
• backend/routers/listen/transcripts.py — _apply_speaker_identity_statuses skips re-assignment for upstream-identified segments; since should_queue_speaker_embedding already refuses segments with person_id/is_user set, upstream-labeled turns never enqueue embedding work either.
• backend/models/transcript_segment.py — both merge gates now require is_user and person_id equality. This is the widest-blast-radius line of the PR: it applies to every transcription session, not just Custom STT (adjacent same-speaker segments carrying different identities no longer merge). The tightening looks right and is pinned by test_combine_segments_preserves_distinct_upstream_identities_on_same_speaker_label — flagging for an explicit maintainer nod.
• backend/utils/conversations/speaker_resolution.py — the post-conversation preserve path is gated on conversation.uses_custom_stt, so regular conversations keep today's resolution semantics.
• backend/docs/transcription.md — documents speaker_id / is_user / person_id and the raw-audio note; matches the implemented behavior.
• Tests — test_user_speaker_embedding.py adds ~290 lines covering raw-audio on/off and identity preservation including matcher retraction; test_transcribe_decisions.py pins the new decision table. Good coverage.

Notes:

1. preserve_upstream_speaker_identity in _apply_speaker_identity_statuses is not gated on Custom STT — any segment arriving with is_user/person_id and no matcher provenance is preserved in every flow. This seems intended (upstream = any provider/client), just confirming it is deliberate.
2. Readability nit: the predicate mutates speaker_identity_status (unknown -> user/not_user), and the two process_loop / _process_v2_batches call sites use it purely for that side effect with the return value discarded; a mark_upstream_identity-style name would read better in a follow-up.
3. The use_custom_stt parameters on the two decision functions are now ignored — consider dropping them in a follow-up cleanup.

CI: the Backend unit suite failure on this head is a CPU-duration guard flake in test_batch_upload_coalescing.py and test_conversation_erasure_queue_deploy.py (1.00s / 1.03s vs the 1.00s limit) — files this PR does not touch, and every test in them passed. Not related to this change.

Performance: embedding extraction only runs when raw audio actually exists in the ring buffer; no extra queries per segment. Cost: no new external calls. Security/privacy: raw audio is only processed for speaker ID when the user already forwards it; posture unchanged otherwise. Blast radius: non-Custom-STT users are only affected by the transcript_segment.py merge-gate tightening noted above.

USER IMPACT: with Custom STT plus 'Send raw audio to Omi' enabled, speaker names and owner attribution keep working; without raw audio, is_user / person_id labels supplied by your STT provider are preserved instead of being dropped.

Still wanted as a reviewer: maintainer sign-off on the merge-gate identity-equality tightening for non-Custom-STT sessions, and ideally one replay/e2e pass with a real Custom STT payload carrying is_user/person_id (unit coverage is thorough but synthetic). Leaving for human maintainer review before merge.

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

[ocakciomursel-pixel]

Thanks for the thorough review @Git-on-my-level!

1. Scope confirmation (_apply_speaker_identity_statuses & merge gate): Confirmed — keeping upstream is_user/person_id preservation ungated in _apply_speaker_identity_statuses is deliberate. Any segment arriving with is_user or person_id already populated and no Omi matcher/manual provenance (segment.id not in speaker.segment_assignments, speaker_label_source is None, speaker_match_source is None) came from an upstream provider or client payload rather than Omi's live embedding matcher, so _apply_speaker_identity_statuses should not overwrite it via speaker_to_person or stamp speaker_match_source = 'live_embedding'. Likewise, requiring a.is_user == b.is_user and a.person_id == b.person_id in TranscriptSegment.combine_segments aligns backend merging with the app's mergeTranscriptSegmentsBySpeaker (app/lib/models/stt_result.dart:213-214).
2. Predicate vs mutator split (Nit colors to the LED board #2): Addressed in 4a3c98346e — split preserve_upstream_speaker_identity into a pure side-effect-free predicate has_upstream_speaker_identity(...) -> bool and an explicit mutator mark_upstream_speaker_identity(segment) -> None used at the process_loop / _process_v2_batches construction sites and before continuing in _apply_speaker_identity_statuses / apply_speaker_resolution.
3. Unused use_custom_stt parameter cleanup (Nit Add printing BLE device address to serial monitor #3): Addressed in 4a3c98346e — removed the ignored use_custom_stt keyword parameter from should_load_speech_profile and should_enable_speaker_identification along with their runtime.py callers and test_transcribe_decisions.py cases.