Skip to content

JpegEncoderTests are not reliable #1550

Description

@antonfirsov

Prerequisites

  • I have written a descriptive issue title
  • I have verified that I am running the latest version of ImageSharp
  • I have verified if the problem exist in both DEBUG and RELEASE mode
  • I have searched open and closed issues to ensure it has not already been reported

Description

Our current validation logic in JpegEncoderTests is too tolerant, which caused #1549 to go through the quality gate completely unnoticed. The method for calculating the tolerance percentage is result of trial-and error experiments (dealing with platform differences if memory serves well):

/// <summary>
/// Anton's SUPER-SCIENTIFIC tolerance threshold calculation
/// </summary>
private static ImageComparer GetComparer(int quality, JpegSubsample subsample)
{
float tolerance = 0.015f; // ~1.5%
if (quality < 50)
{
tolerance *= 10f;
}
else if (quality < 75 || subsample == JpegSubsample.Ratio420)
{
tolerance *= 5f;
if (subsample == JpegSubsample.Ratio420)
{
tolerance *= 2f;
}
}
return ImageComparer.Tolerant(tolerance);
}

The method above returns a comparer of 15% tolerance, for subsample=420, quality=100, which doesn't really make sense. The image in #1549 (comment) has ~4% difference compared to the image before encoding.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions