Skip to content

CA-432805: Avoid fuse exit hanging in flush_all - #7364

Open
BengangY wants to merge 2 commits into
xapi-project:masterfrom
BengangY:private/bengangy/CA-432805-fuse-exit
Open

BengangY wants to merge 2 commits into
xapi-project:masterfrom
BengangY:private/bengangy/CA-432805-fuse-exit

Conversation

@BengangY

Copy link
Copy Markdown
Contributor

When the fuse exits xapi, Stdlib.exit runs flush_all, which can block forever writing a stale channel whose fd has been reused by an unread socket. The process then never exits.

  1. Exit with Unix._exit instead of Stdlib.exit, which skips flush_all.
  2. Start a timer when the fuse is lit that forces the process to exit if it is still running fuse_exit_grace seconds (default 60) after the fuse delay.

Stdlib.exit calls flush_all which does a blocking write() on every
unflushed out_channel. One stale channel whose fd was reused by an
unread socket blocks it forever. Use Unix._exit instead which doesn't
call flush_all.

Signed-off-by: Bengang Yuan <bengang.yuan@citrix.com>
@BengangY
BengangY marked this pull request as ready for review October 30, 2026 01:03
Start a timer in the fuse process to force exiting the process if it
is still running fuse_exit_grace seconds (default 60) after the fuse
delay.

Signed-off-by: Bengang Yuan <bengang.yuan@citrix.com>
Comment thread ocaml/xapi/xapi_fuse.ml
let force_exit_after_grace ~fuse_length ret_code =
let wait = fuse_length +. !Constants.fuse_exit_grace in
try
ignore (Thread.create (fun () -> Thread.delay wait ; Unix._exit ret_code) ()) ;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could there be a log that xapi exited in an unintended way here? Or do we want to avoid hanging on the log as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logging on exit may also block in syslog, so I didn't add a log.

Comment thread ocaml/xapi/xapi_fuse.ml Resolved
@BengangY
BengangY force-pushed the private/bengangy/CA-432805-fuse-exit branch from cc40ce8 to eb672a2 Compare November 2, 2026 06:11
@BengangY
BengangY requested a review from last-genius November 2, 2026 06:31
let fuse_time = ref 10.

(* the time the fuse allows for flush and exit before forcing the process out *)
let fuse_exit_grace = ref 60.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the reason you select 60 for fuse_exit_grace?
The thread is created before stats stop and rrdd backup, is it intentional?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked for the thread start to be moved above: #7364 (comment)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants