Repository navigation
[BUG] cuvs-java com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues test fails intermittently #631
Description
Describe the bug
cuvs-java test fails intermittently, e.g.
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.031 s <<< FAILURE! -- in com.nvidia.cuvs.CagraRandomizedTest
[ERROR] com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues -- Time elapsed: 2.031 s <<< FAILURE!
java.lang.AssertionError: Not found in expected list: 1583
at __randomizedtesting.SeedInfo.seed([93DF604C314D221A:BAF3BCD46F1EDC5A]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at com.nvidia.cuvs/com.nvidia.cuvs.CuVSTestCase.compareResults(CuVSTestCase.java:83)
at com.nvidia.cuvs/com.nvidia.cuvs.CagraRandomizedTest.tmpResultsTopKWithRandomValues(CagraRandomizedTest.java:98)
at com.nvidia.cuvs/com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues(CagraRandomizedTest.java:29)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
...
Steps/Code to reproduce bug
It's not clear if setting the random seed (-Dtests.seed) actually improves reproducibility.
./build java
cd java/cuvs-java
mvn -Dtests.seed=93DF604C314D221A test
Expected behavior
The test passes successfully every time.
Environment details (please complete the following information):
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
- AWS g4dn.2xlarge, Ubuntu 22.04 + cuda 12.6
- Method of RAFT install: [conda, Docker, or from source]
- If method of install is [Docker], provide
docker pull&docker runcommands used
- If method of install is [Docker], provide
set the JAVA_HOME env variable to JDK 22
JAVA_HOME=/home/ubuntu/binaries/jdk-22.0.2/
Additional context
This fails in branch-25.02, with the commit that added the java-api:
commit 4ca47c92b03856e7d4ae328aabf6ee3e853a137e (...
Initial cut for a cuVS Java API (#450)
A Java API for cuVS for easy integration into Apache Lucene or other Java based projects.
Activity
ChrisHegarty commentedon Jan 31, 2025
/cc @chatman
bdice commentedon Oct 11, 2026
Root cause analysis from the flaky-test sweep (#2783):
CagraRandomizedIT.testResultsTopKWithRandomValues#1048, [BUG] [Java]CagraRandomizedIT.testResultsTopKWithRandomValuesfails randomly #1468). The Java bindings sized the prefilter host buffer withBitSet.toLongArray(), which drops trailing all-zero words, so the copy to the GPU read past the end of the buffer. Stray bits then let filtered-out rows in the first dropped 64-bit word through. The ids in the reports (192 and 448) are both multiples of 64. Fix prefilter bitmap host buffer under-allocation in Java bindings #2375 fixed this on 2026-07-29. Restoring the old buffer sizing in CI reproduces the same assertion,Not found in expected list: 192at n=195, and every leaked row is a filtered-out row at the start of a dropped word: https://github.com/NVIDIA/cuvs/actions/runs/381147592552k + 10that the test required. The returned ids and distances are correct (true rank 20 against a window of 20, for example). This still happens rarely on current main and in releases back to 25.02, so the test's expectation was wrong.#2830 re-enables
CagraRandomizedIT. For every query it checks distinct, valid ids, no rows excluded by the prefilter, sorted exact distances, and that a copy of a row is returned first. It checks recall only as a mean per memory kind, with bounds set from about 300 test runs measured in CI. While re-enabling the test it also found and fixed a third bindings bug: a prefilter that excludes every row was treated as "no filter". 400 CI runs of the new test (about 120,000 random cases) had no failures: https://github.com/NVIDIA/cuvs/actions/runs/38138513521