Skip to content

ipc4: handler: bound get_large_config reply size to the reply buffer - #11239

Draft
lgirdwood wants to merge 1 commit into
thesofproject:mainfrom
lgirdwood:fix/ipc4-large-config-get-reply-size
Draft

lgirdwood wants to merge 1 commit into
thesofproject:mainfrom
lgirdwood:fix/ipc4-large-config-get-reply-size

Conversation

@lgirdwood

Copy link
Copy Markdown
Member

What this fixes

The simple-IPC-fuzz_sh job has been failing intermittently on unrelated PRs
(#11193, #11208, #11214, #11227 among them). The CI log only ever shows this:

==6279==ERROR: AddressSanitizer: stack-overflow on address 0xddedfffc (T8)
    #3 vfprintf
    #5 nsi_vprint_trace  zephyr/scripts/native_simulator/native/src/nsi_trace.c:94
    #6 posix_print_trace zephyr/boards/native/native_sim/posix_arch_if.c:61
Thread T8 created by T3 here:
    #7 k_work_queue_start zephyr/kernel/work.c:875
    #8 ipc_init           sof/src/ipc/ipc-common.c:799

That report is a red herring. The real failure is a Zephyr assert; the
assert -> k_panic -> z_spin_lock_valid assert loop that follows it is what
eventually exhausts the 8MB pthread stack native_sim gives the ipc_send_wq
thread. The actual assert text goes to stdout, which fuzz.sh closes via
-close_fd_mask=1 whenever -jobs > 1, so only the secondary ASan report on
stderr reaches the log.

Replaying the crash artifact locally shows the real failure immediately:

ASSERTION FAIL [!dsp_write_err] @ sof/src/include/sof/lib/mailbox.h:52

#1 mailbox_dspbox_write (offset=0, bytes=918016) sof/src/include/sof/lib/mailbox.h:52
#2 ipc_platform_send_msg (msg=0x7504c0 <msg_reply>) sof/src/platform/posix/ipc.c:435
#3 ipc_send_queued_msg ()                          sof/src/ipc/ipc-common.c:218
#4 ipc_work_handler (work=...)                     sof/src/ipc/ipc-common.c:386

A reply claiming ~900KB of payload for a 4KB mailbox.

Root cause

ipc4_get_large_config_module_instance() seeds data_offset from
config->extension.r.data_off_size — a 20-bit host-controlled field — and
passes it to drv->ops.get_large_config() as an in/out parameter. A module
whose .get_configuration produces no data returns success without writing
*data_offset_size, so the host-supplied value survives unchanged and is
published as msg_reply->tx_size. The payload itself lives in
ipc->comp_data, which is only SOF_IPC_MSG_MAX_SIZE bytes.

template_get_config() is the stub the fuzzer happened to find — it is
explicitly documented as "Not used in IPC4 systems" and just returns 0 — but
any IPC3-oriented .get_configuration behaves the same way.

Only the VENDOR_CONFIG_PARAM branch bounds data_off_size today (added in
d94306c), and it does so for the inbound hostbox read, not for the reply.

With asserts compiled out this is not merely cosmetic: memcpy_s() rejects
the copy but its return value is discarded, dcache_writeback_region() is
still asked to write back the out-of-range length, and the caller goes on to
send a reply header advertising a size that was never copied.

The change

Bound the reply size against the space actually left in the reply buffer
before it is published, and fail the command with
IPC4_INVALID_CONFIG_DATA_LEN otherwise. The bound is computed from data
rather than from SOF_IPC_MSG_MAX_SIZE directly because the non-vendor branch
advances data by sizeof(reply) under CONFIG_LIBRARY.

The same fix is applied to ipc4_process_large_config_get(), the
CONFIG_SOF_USERSPACE_LL variant, which has the identical flaw (and also
lacks the vendor-path hostbox bound its sibling has — not addressed here).

Verification

Built for native_sim/native/64 and replayed the crash artifacts from two
independent failing CI runs:

artifact before after
crash-cd99ca2d... (PR #11193, job simple-IPC-fuzz_sh (4)) ASSERTION FAIL [!dsp_write_err], rc=1 rc=0
crash-15eb8d66... (PR #11208, job simple-IPC-fuzz_sh (4)) ASSERTION FAIL [!dsp_write_err], rc=1 rc=0

Two independently-found inputs hitting the identical assert is what makes me
fairly confident this is the single root cause behind the recent intermittent
failures, rather than one of several.

Under gdb the new branch is confirmed to be the one taken:
data_offset = 918016, data_max = 4096, ret = 121
(IPC4_INVALID_CONFIG_DATA_LEN).

Also:

  • 420s x 8-job fuzz run over a ~10k-input corpus: no new crash artifacts.
  • intel_adsp/ace30/ptl cross-build clean.
  • checkpatch clean at SOF's 100-column limit.

Left alone deliberately

Two things make any fuzz assert much harder to diagnose than it should be.
Both look worth separate patches, and I would rather not fold them into a
fix:

  1. The assert recursion. ipc_send_queued_msg() holds ipc->lock when
    the assert fires, and the fatal path then trips z_spin_lock_valid()
    repeatedly, so any fuzz assert degenerates into an 8MB stack overflow
    instead of stopping at the assert.
  2. -close_fd_mask=1. With -jobs > 1 this discards the one line that
    identifies the bug.

Marked draft pending CI.

🤖 Generated with Claude Code

The IPC4 fuzzer (simple-IPC-fuzz_sh) aborts with an AddressSanitizer
stack-overflow report in the ipc_send_wq thread, inside vfprintf() under
posix_print_trace(). That report is a red herring: the real failure is an
assert, and the recursive assert -> k_panic -> assert loop that follows it
is what eventually exhausts the 8MB pthread stack native_sim gives the
thread. With -jobs>1 the harness passes -close_fd_mask=1, so printk (which
goes to stdout) is discarded and only the secondary ASan report on stderr
survives into the CI log.

The assert is:

  ASSERTION FAIL [!dsp_write_err] @ sof/src/include/sof/lib/mailbox.h:52

reached from ipc_send_queued_msg() -> ipc_platform_send_msg() ->
mailbox_dspbox_write(0, msg->tx_data, 918016), i.e. a reply claiming
~900KB of payload for a 4KB mailbox.

ipc4_get_large_config_module_instance() seeds data_offset from
config->extension.r.data_off_size, a 20-bit host-controlled field, and then
passes it to drv->ops.get_large_config() as an in/out parameter. A module
whose .get_configuration produces no data (template_get_config() is the one
the fuzzer found, but any IPC3-oriented stub behaves the same) returns
success without writing *data_offset_size, so the host-supplied value
survives unchanged and is published as msg_reply->tx_size. The payload
itself lives in ipc->comp_data, which is only SOF_IPC_MSG_MAX_SIZE bytes.

Only the VENDOR_CONFIG_PARAM branch bounds data_off_size today, and it does
so for the inbound hostbox read, not for the reply. With asserts compiled
out, memcpy_s() rejects the copy but the return value is discarded and
dcache_writeback_region() is still asked to write back the out-of-range
length, so the caller sends a reply header advertising a size that was
never copied.

Bound the reply size against the space actually left in the reply buffer
before it is published, and fail the command with
IPC4_INVALID_CONFIG_DATA_LEN otherwise. The bound is computed from data
rather than from SOF_IPC_MSG_MAX_SIZE directly because the non-vendor
branch advances data by sizeof(reply) under CONFIG_LIBRARY. The same fix is
applied to ipc4_process_large_config_get(), the CONFIG_SOF_USERSPACE_LL
variant, which has the identical flaw.

Reproduced and verified on native_sim/native/64 with the crash artifact
from the failing CI run (thesofproject/sof PR 11193, job simple-IPC-fuzz_sh
(4)): it aborts before the change and exits 0 after it. A subsequent
420s/8-job fuzz run over a ~10k-input corpus produced no new artifacts, and
an intel_adsp/ace30/ptl cross-build is clean.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants