Repository navigation
Minor storage fixes - #7369
Open
last-genius wants to merge 2 commits into
Open
Minor storage fixes#7369last-genius wants to merge 2 commits into
last-genius wants to merge 2 commits into
Conversation
We've seen a case where multiple VDI.deactivate calls running in parallel get into a race between get_other_config and add_other_config, failing with the Duplicate_key exception. If one of the calls succeeded in adding other_config=content_id:, then simply continue. Signed-off-by: Andrii Sultanov <andriy.sultanov@vates.tech>
XCP-ng uses VBDs in dom0 for some use-cases, and failures can result in
leftover datapaths breaking unrelated VBDs.
To help avoid this, if attach succeeded but activate failed, detach
the dom0 VBD. This requires reordering the functions so that
attach_and_activate can call deactivate_and_detach.
Details of this issue:
This script can be used to check for leftover datapaths:
$ cat check_datapath.py
import json
d = json.load(open('/var/run/nonpersistent/xapi/storage.db'))
claims = {}
for sr, srv in d['host']['srs'].items():
for vdi, vv in (srv.get('vdis') or {}).items():
for dp, state in (vv.get('dps') or {}).items():
claims.setdefault(dp, []).append((sr, vdi, state))
for dp, owners in sorted(claims.items()):
print(dp, ' <-- DUPLICATE' if len(owners) > 1 else '')
for sr, vdi, state in owners:
print(' ', sr, vdi, state)
The following sequence simulates a failure to activate a VBD and shows how two
VBDs using the same datapath results in further errors down the line:
$ touch /tmp/fist_blktap_activate_inject_failure
$ xe vbd-plug uuid=$FIRST_VBD_UUID
Error code: SR_BACKEND_FAILURE_1200
Error parameters: , injected failure,
$ rm -f /tmp/fist_blktap_activate_inject_failure
$ python3 check_datapath.py
vbd/0/xvda
1d4a39c0-5029-a134-77c1-b189d2e60dfc 6016c6ad-1101-4d0a-bc32-bc9604d930ab ['Attached', 'RW']
$ xe vbd-destroy uuid=$FIRST_VBD_UUID
$ xe vbd-plug uuid=$SECOND_VBD_UUID
$ python3 check_datapath.py
vbd/0/xvda <-- DUPLICATE
1d4a39c0-5029-a134-77c1-b189d2e60dfc 7ceaa7c6-4c51-4536-9c84-7d25d11e3698 ['Activated', 'RW']
1d4a39c0-5029-a134-77c1-b189d2e60dfc 6016c6ad-1101-4d0a-bc32-bc9604d930ab ['Attached', 'RW']
$ xe vbd-unplug uuid=$SECOND_VBD_UUID
message: Expected 0 or 1 VDI with datapath, had 2
Only reboots will fix this, otherwise the storage.db needs to be manually
modified to drop the stale entry.
Signed-off-by: Andrii Sultanov <andriy.sultanov@vates.tech>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First commit fixes multiple
VDI.deactivaterunning in parallel.Second commit cleans up on
VBD.plugfailure fordom0VBDs.