Clarify date-time fallback docs and simplify parsing (#1337)

CHANGELOG.md

@@ -10,8 +10,8 @@ This project does not adhere to [Semantic Versioning](https://semver.org/) and m
 - Opt-in `ExecutionConfig.maxEvaluationSteps` and `maxEvaluationDepth` limits for validation and schema walking, including walking without assertions. Exhaustion throws `ValidationLimitExceededException` instead of returning a partial validity result. Both limits default to unlimited; they bound admitted evaluation steps and schema frames, not work inside a keyword. (#1276)
 
 ### Changed
-- Without Ethlo on the classpath, the `date-time` format now checks the date with the rules of the `date` format and the time with the rules of the `time` format instead of `OffsetDateTime.parse(..)`. It rejects a time without seconds (`2020-01-01T12:00Z`), an offset without minutes (`+01`) and an empty fraction (`12:00:00.Z`), which Ethlo already rejected, as well as an offset with seconds (`+01:00:00`). It accepts an offset up to `23:59` and a fraction longer than nine digits, which RFC 3339 allows, and a leap second at the end of a June 30 or a December 31 in UTC, which it used to reject. Unlike Ethlo, it doesn't check that a leap second was actually inserted on that date. (#1316)
-- The `time` format now rejects a time without seconds (`12:00Z`) and a leap second with an hour or minute out of range (`24:59:60+01:00`), which it used to accept. (#1316)
+- Without Ethlo on the classpath, the `date-time` format now checks the date with the rules of the `date` format and the time with the rules of the `time` format instead of `OffsetDateTime.parse(..)`. It rejects a time without seconds (`2020-01-01T12:00Z`), an offset without minutes (`+01`) and an empty fraction (`12:00:00.Z`), which Ethlo already rejected, as well as an offset with seconds (`+01:00:00`). It accepts an offset up to `23:59` and a fraction longer than nine digits, which RFC 3339 allows, and a leap second at the end of a June 30 or a December 31 in UTC, which it used to reject. Compared with Ethlo, the fallback provides less accurate leap-second validation because it does not check that a leap second was actually inserted on that date. It also accepts `-00:00`, offsets beyond ±18:00, and fractions longer than nine digits, which Ethlo rejects. The fallback requires `T` or `t` between the date and time, rejecting a space separator such as `2020-01-01 12:00:00Z` that Ethlo accepts. (#1316, #1337)
+- The `time` format now rejects a leap second with an hour or minute out of range (`24:59:60+01:00`, `23:89:60+00:30`, `45:59:60+22:00`), which it used to accept. A time without seconds (`12:00Z`) was already rejected. (#1316, #1337)
 - Ignore zero `multipleOf` divisors returned by custom validators, regardless of scale, matching the default conversion. (#1328)
 - Display `multipleOf` divisors in error messages using plain notation when it fits within 32 characters or is no longer than `BigDecimal.toString()`, including values returned by custom validators. Check the length before expanding to plain notation to avoid excessive padding for extreme scales. (#1328)
 - `MultipleOfValidator.getDivisor` and `getDividend` now preserve exact integral and BigDecimal values. Custom validators, including overrides calling `super`, receive these exact values; validation uses hook results without further rounding. Integral dividend scales can change (for example, `2.0` becomes `2`), which affects scale-sensitive comparisons in overrides. Mixed integral/Double or Float inputs above 2^53 can produce different results. Precision already lost when parsing floating-point values is tracked separately in #1293. (#1286)

README.md

@@ -192,7 +192,9 @@ The YAML dependency can be excluded if this is not required. Attempting to proce
 </dependency>
 ```
 
-The Ethlo Time dependency can be excluded if accurate validation of the `date-time` format is not required. The `date-time` format will then use `java.time.OffsetDateTime` to determine if the `date-time` is valid .
+The Ethlo Time dependency can be excluded if less accurate leap-second validation is acceptable. The `date-time` format then uses an RFC 3339 fallback that validates `full-date "T" full-time` (also accepting lowercase `t` and `z`), requires seconds and an offset, and accepts numeric offsets up to ±23:59. Leap seconds are accepted only at the end of June 30 or December 31 in UTC; the fallback does not check whether a leap second was actually inserted on that date.
+
+The fallback also accepts `-00:00`, offsets beyond ±18:00, and fractions longer than nine digits, which Ethlo rejects. Ethlo accepts a space between the date and time, while the fallback requires `T` or `t`.
 
 ```xml
 <dependency>

src/main/java/com/networknt/schema/format/DateFormat.java

@@ -22,12 +22,19 @@ public boolean matches(ExecutionContext executionContext, String value) {
      * @return true if the value is a valid full-date
      */
     static boolean isValid(String value) {
+        return parse(value) != null;
+    }
+
+    /**
+     * @return the full-date, or null if the value is not a valid full-date
+     */
+    static LocalDate parse(String value) {
         try {
             LocalDate date = LocalDate.parse(value);
             int year = date.getYear();
-            return 0 <= year && year <= 9999;
+            return 0 <= year && year <= 9999 ? date : null;
         } catch (DateTimeParseException e) {
-            return false;
+            return null;
         }
     }
 

src/main/java/com/networknt/schema/format/DateTimeFormat.java

@@ -60,17 +60,17 @@ public static boolean isValid(String value) {
             if (separator != 'T' && separator != 't') {
                 return false;
             }
-            String date = value.substring(0, DATE_LENGTH);
+            LocalDate date = DateFormat.parse(value.substring(0, DATE_LENGTH));
             TimeFormat.FullTime time = TimeFormat.parse(value.substring(DATE_LENGTH + 1));
-            if (time == null || !DateFormat.isValid(date)) {
+            if (time == null || date == null) {
                 return false;
             }
             // TimeFormat.parse has checked that a leap second is at 23:59:60 UTC, so only the date is left
-            return time.second != 60 || isLastDayOfJuneOrDecemberInUtc(LocalDate.parse(date), time);
+            return time.second != 60 || isLastDayOfJuneOrDecemberInUtc(date, time);
         }
 
         private static boolean isLastDayOfJuneOrDecemberInUtc(LocalDate date, TimeFormat.FullTime time) {
-            LocalDate utc = date.atTime(time.hour, time.minute).minusMinutes(time.offsetMinutes).toLocalDate();
+            LocalDate utc = date.plusDays(Math.floorDiv(time.minutesInUtc(), 24 * 60));
             return utc.getMonth() == Month.JUNE && utc.getDayOfMonth() == 30
                     || utc.getMonth() == Month.DECEMBER && utc.getDayOfMonth() == 31;
         }

src/main/java/com/networknt/schema/format/TimeFormat.java

@@ -21,7 +21,6 @@
 
 import com.networknt.schema.ExecutionContext;
 
-import static java.time.format.DateTimeFormatter.ISO_LOCAL_TIME;
 import static java.time.temporal.ChronoField.*;
 
 /**
@@ -37,7 +36,12 @@ public class TimeFormat implements Format {
 
     private static final DateTimeFormatter formatter = new DateTimeFormatterBuilder()
             .parseCaseInsensitive()
-            .append(ISO_LOCAL_TIME)
+            .appendValue(HOUR_OF_DAY, 2)
+            .appendLiteral(':')
+            .appendValue(MINUTE_OF_HOUR, 2)
+            .appendLiteral(':')
+            .appendValue(SECOND_OF_MINUTE, 2)
+            .appendFraction(NANO_OF_SECOND, 0, 9, true)
             .appendOffset("+HH:MM", "Z")
             .parseLenient()
             .toFormatter();
@@ -74,6 +78,14 @@ static final class FullTime {
             this.second = second;
             this.offsetMinutes = offsetMinutes;
         }
+
+        /**
+         * Minutes from the start of the written date to this time in UTC. The result
+         * can fall outside the written date when the offset crosses midnight.
+         */
+        int minutesInUtc() {
+            return hour * 60 + minute - offsetMinutes;
+        }
     }
 
     /**
@@ -85,9 +97,6 @@ static final class FullTime {
      */
     static FullTime parse(String value) {
         try {
-            // partial-time is HH:MM:SS. ISO_LOCAL_TIME alone would also take HH:MM
-            if (value.length() < 8 || value.charAt(2) != ':' || value.charAt(5) != ':') return null;
-
             int pos = value.indexOf('Z');
             if (-1 != pos && pos != value.length() - 1) return null;
 
@@ -102,7 +111,7 @@ static FullTime parse(String value) {
                     fractionEnd++;
                 }
                 if (fractionEnd == fractionStart) return null;
-                // ISO_LOCAL_TIME parses at most nine digits, but RFC 3339 permits more.
+                // The formatter parses at most nine digits, but RFC 3339 permits more.
                 if (fractionEnd - fractionStart > 9) {
                     parsedValue = value.substring(0, fractionStart + 9) + value.substring(fractionEnd);
                 }
@@ -132,38 +141,14 @@ static FullTime parse(String value) {
             if (sec <= 59) {
                 return time;
             }
-            // Leap second check normalize to UTC to check if 23:59:60Z
-            hr = hr - offset / 60;
-            min = min - offset % 60;
-
-            if (min < 0) {
-                --hr;
-                min += 60;
-            }
-            if (hr < 0) {
-                hr += 24;
-            }
-            return isSpecialCaseLeapSecond(sec, min, hr) ? time : null;
+            // The remaining second is 60; it must be at 23:59 in UTC.
+            return Math.floorMod(time.minutesInUtc(), 24 * 60) == 23 * 60 + 59 ? time : null;
 
         } catch (DateTimeException e) {
             return null;
         }
     }
 
-    /**
-     * Determines if it is a valid leap second.
-     *
-     * See https://datatracker.ietf.org/doc/html/rfc3339#appendix-D
-     *
-     * @param sec second
-     * @param min minute
-     * @param hr  hour
-     * @return true if it is a valid leap second
-     */
-    private static boolean isSpecialCaseLeapSecond(long sec, long min, long hr) {
-        return (sec == 60 && min == 59 && hr == 23);
-    }
-
     @Override
     public String getName() {
         return "time";

src/test/java/com/networknt/schema/format/DateTimeFormatTest.java

@@ -69,7 +69,7 @@ void acceptsValidOffsetsAndLeapSecond(String value) {
     }
 
     /**
-     * OffsetDateTime.parse(..) accepts these, Ethlo doesn't. The two paths have to agree.
+     * OffsetDateTime.parse(..) accepts these three non-RFC 3339 values; both validation paths reject them.
      */
     @ParameterizedTest
     @ValueSource(strings = {
@@ -85,6 +85,13 @@ void bothPathsRejectValuesThatAreNotRfc3339(String value) {
         assertFalse(DateTimeFormat.JavaTimeOffsetDateTime.isValid(value), value);
     }
 
+    @Test
+    void ethloAcceptsSpaceSeparatorButFallbackRequiresT() {
+        String value = "2020-01-01 12:00:00Z";
+        assertTrue(DateTimeFormat.Ethlo.isValid(value));
+        assertFalse(DateTimeFormat.JavaTimeOffsetDateTime.isValid(value));
+    }
+
     @ParameterizedTest
     @ValueSource(strings = {
             "2020-01-01T12:00:00.+01:00",
@@ -106,10 +113,15 @@ void bothPathsRejectValuesThatAreNotRfc3339(String value) {
             "2020-01-01T45:59:60+22:00",
             "2021-03-15T23:59:60Z",
             "2016-12-30T23:59:60Z",
+            "2017-01-02T23:58:60+23:59",
+            "2016-12-30T00:00:60-23:59",
             "2016-12-31T23:59:60+01:00",
             "1998-12-31T15:59:60.123-07:00",
             "-001-01-01T00:00:00Z",
             "10000-01-01T00:00:00Z",
+            "-0001-12-31T23:59:60Z",
+            "0000-01-01T00:59:60+00:59",
+            "9999-12-31T23:59:60-00:01",
             "2020-01-01T12:00:00+24:00",
             "2020-01-01T12:00:00+0100",
             "2020-01-01T12:00:00+01:0",
@@ -133,10 +145,15 @@ void fallbackRejectsValuesThatAreNotRfc3339(String value) {
             "2015-06-30T23:59:60Z",
             "2017-01-01T00:59:60+01:00",
             "2017-01-01T01:59:60+02:00",
+            "2017-01-01T23:58:60+23:59",
+            "2016-12-31T00:00:60-23:59",
             "2016-12-31T18:59:60-05:00",
             "1998-12-31T15:59:60.123-08:00",
             "2015-07-01T08:59:60+09:00",
             "0000-01-01T00:00:00Z",
+            // The four-digit year applies to the written date, even when UTC is in year -1.
+            "0000-01-01T00:59:60+01:00",
+            "9999-12-31T23:59:60Z",
             "9999-12-31T23:59:59Z",
             "2020-01-01T12:00:00.1234567890Z",
             "2020-01-01T12:00:00+23:59",
@@ -202,7 +219,9 @@ private byte[] read(String name) throws ClassNotFoundException {
 
         // An offset beyond 18:00 is where the fallback and Ethlo differ: Ethlo rejects it
         assertTrue(((List<?>) validate.invoke(schema, "\"2020-01-01T12:00:00+23:59\"", json)).isEmpty());
+        assertTrue(((List<?>) validate.invoke(schema, "\"2017-01-01T00:59:60+01:00\"", json)).isEmpty());
         assertFalse(((List<?>) validate.invoke(schema, "\"2020-01-01T12:00Z\"", json)).isEmpty());
         assertFalse(((List<?>) validate.invoke(schema, "\"2020-01-01T24:59:60+01:00\"", json)).isEmpty());
+        assertFalse(((List<?>) validate.invoke(schema, "\"2016-12-30T23:59:60Z\"", json)).isEmpty());
     }
 }

src/test/java/com/networknt/schema/format/TimeFormatTest.java

@@ -34,6 +34,8 @@ enum ValidTimeFormatInput {
         Z_OFFSET_LEAP_SECOND("23:59:60Z"),
         POSITIVE_OFFSET_LEAP_SECOND("07:59:60+08:00"),
         NEGATIVE_OFFSET_LEAP_SECOND("15:59:60-08:00"),
+        POSITIVE_MAX_OFFSET_LEAP_SECOND("23:58:60+23:59"),
+        NEGATIVE_MAX_OFFSET_LEAP_SECOND("00:00:60-23:59"),
         Z_OFFSET_MIN_TIME("00:00:00Z"),
         Z_OFFSET("23:59:59Z"),
         Z_OFFSET_ONE_FRACTION_DIGIT("12:00:00.1Z"),
@@ -86,6 +88,8 @@ enum InvalidTimeFormatInput {
         TRAILING_NEWLINE_AFTER_OFFSET("12:00:00+01:00\\n"),
         NO_SECONDS("12:00Z"),
         ONE_DIGIT_HOUR("1:00:00Z"),
+        ONE_DIGIT_MINUTE("12:0:00Z"),
+        ONE_DIGIT_SECOND("12:00:0Z"),
         HOUR_OVERFLOW_LEAP_SECOND("24:59:60+01:00"),
         MINUTE_OVERFLOW_LEAP_SECOND("23:89:60+00:30"),
         HOUR_AND_OFFSET_OVERFLOW_LEAP_SECOND("45:59:60+22:00");