Skip to content

feat(ios): unify voice states in Dynamic Island and Live Activity - #608

Open
JosssphZhou wants to merge 1 commit into
test/047-candidatefrom
feat/esp-1151-island-voice-state
Open

JosssphZhou wants to merge 1 commit into
test/047-candidatefrom
feat/esp-1151-island-voice-state

Conversation

@JosssphZhou

Copy link
Copy Markdown
Owner

Summary of changes

Target release: 0.4.8 (not intended for 0.4.7).

Unifies voice session status presentation across the Dynamic Island (compact, minimal, narrow, and expanded) and Lock Screen Live Activity using right-side visual differentiation with aligned bear poses and standard status words:

  • Expands PokiActivityAttributes.ContentState.Voice from 3 states to 8 states (connecting, listening, hearing, thinking, working, speaking, failed, ended), plus optional voiceLevel (3 waveform amplitude tiers) and voiceStep (e.g. 3/4) fields.
  • Adds aligned, badge-free ActivityBear-listening.png and ActivityBear-speaking.png assets (408×408, head-aligned to ready) for compact, minimal, and expanded views.
  • Consolidates duplicate voice-while-working Live Activity UI into a single unified voiceFace(_:) presentation table (ActivityFace.swift):
    • Connecting: needs bear + 5 quiet grey dots (ellipsis minimal badge) + expanded Connecting / 正在连接 + End button.
    • Listening (ready): ready bear + low sky-blue 5-bar waveform (mic.fill minimal badge) + expanded Listening / 正在聆听 + End button.
    • Listening (user speaking / hearing): listening bear + sky-blue 5-bar waveform scaled across 3 loudness tiers sampled from WebRTC microphone stats (mic.fill minimal badge) + expanded Listening / 正在聆听 + End button.
    • Thinking: thinking bear + 3 white dots (ellipsis minimal badge) + expanded Thinking / 正在思考 + End button.
    • Working: phone bear + sky-blue step progress (e.g. 3/4, iphone minimal badge) + expanded Working / 正在操作 + End button.
    • Responding: speaking bear + Poki brand-blue left-high 5-bar waveform (speaker.wave.2.fill minimal badge) + expanded Responding / 正在回复 + End button.
    • Connection failed: tripped bear + sky-blue keyboard icon (no text in compact/minimal) + expanded sky-blue Connection failed / 连接失败 + Type button linking to poki://chat?type=1 (dismisses voice error and focuses the message input), held for 3 seconds before dismissing.
    • Ended: done bear + light grey checkmark (no text in compact/minimal) + expanded Ended / 已结束 (no action button), held for 3 seconds before dismissing.
  • Adds 0.45 s cross-fade transitions (ActivityMotion.voice) respecting reduced motion and reduced luminance (Always-On Display).
  • Updates ActivityController.isCritical so voice state changes dispatch immediately without coalescing delay, and adds simulator voice sequence demo support in ActivityPreview.swift (#if DEBUG && targetEnvironment(simulator)).

Motivation

During live voice calls in the background, the Dynamic Island previously only exposed three coarse states (listening, thinking, speaking) with a static waveform icon, making it hard to distinguish connecting, waiting for speech, active user speech, thinking, phone tool operation, Poki responding, connection failure, and call completion at a glance. This 0.4.8 change implements the finalized right-side differentiation design across all Live Activity presentations.

Test plan

Built and tested on a disposable iPhone 17 Simulator (iOS 27.2, 402×874 pt) using xcodebuild build-for-testing and XCTest bundle injection:

  • ActivityBearAssetsTests: 1 test executed, 0 failures (verifies 136 pt, 27 pt compact, and 22 pt minimal assets for all 11 bear poses including .listening and .speaking).
  • LiveActivityTests: 18 tests executed, 0 failures (verifies all 8 voice states in English and Simplified Chinese, loudness tiers, merged working-during-voice presentation, and zero clipping inside the 12.33 pt bottom Dynamic Island clip zone).
  • VoiceModelTests: 46 tests executed, 1 skipped (pre-existing recorded-events test), 0 failures (verifies voice phase mapping, EndTalkIntent ending and 3 s hold dismissal, connection loss failed hold and dismissal, microphone loudness polling, and active turn step counts).

Screenshots or screen recording

Verified visually on the real system Dynamic Island (SpringBoard compact, long-press expanded, minimal preview, and Lock Screen card) across all 8 voice states in Simplified Chinese, plus continuous simulator screen recordings of both the full call lifecycle (connecting → listening → hearing with loudness tiers → thinking → working 1/4–3/4 → responding → interruption → ended → collapse) and the connection-failed lifecycle (connecting → failed keyboard icon → collapse after 3 s), compared side-by-side against the approved design specification and motion prototype.

AI assistance: Other (Gemini). I have reviewed and understand these changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant