Now that Xcode 10 is GM, I want to try 8bit quantized models.
The issue is that my models have already been quantized to 16bit, and when I run the quantization script on them again, their size doesn't change.
It does work if I take the original model and quantize it straight to 8bit though, so I'm scratching my head on this.
Original =>6.7 MB
Original -> 16bit => 3.4 MB
Original -> 8bit => 1.7 MB
Original -> 16bit -> 8bit : 3.4 MB
Now that Xcode 10 is GM, I want to try 8bit quantized models.
The issue is that my models have already been quantized to 16bit, and when I run the quantization script on them again, their size doesn't change.
It does work if I take the original model and quantize it straight to 8bit though, so I'm scratching my head on this.
Original =>6.7 MB
Original -> 16bit => 3.4 MB
Original -> 8bit => 1.7 MB
Original -> 16bit -> 8bit : 3.4 MB