• Marc Ferland's avatar
    iio: dac: ad5592r: fix temperature channel scaling value · 279428df
    Marc Ferland authored
    The scale value for the temperature channel is (assuming Vref=2.5 and
    the datasheet):
    
        376.7897513
    
    When calculating both val and val2 for the temperature scale we
    use (3767897513/25) and multiply it by Vref (here I assume 2500mV) to
    obtain:
    
      2500 * (3767897513/25) ==> 376789751300
    
    Finally we divide with remainder by 10^9 to get:
    
        val = 376
        val2 = 789751300
    
    However, we return IIO_VAL_INT_PLUS_MICRO (should have been NANO) as
    the scale type. So when converting the raw temperature value to the
    'processed' temperature value we will get (assuming raw=810,
    offset=-753):
    
        processed = (raw + offset) * scale_val
                  = (810 + -753) * 376
    	      = 21432
    
        processed += div((raw + offset) * scale_val2, 10^6)
                  += div((810 + -753) * 789751300, 10^6)
    	      += 45015
        ==> 66447
        ==> 66.4 Celcius
    
    instead of the expected 21.5 Celsius.
    
    Fix this issue by changing IIO_VAL_INT_PLUS_MICRO to
    IIO_VAL_INT_PLUS_NANO.
    
    Fixes: 56ca9db8 ("iio: dac: Add support for the AD5592R/AD5593R ADCs/DACs")
    Signed-off-by: default avatarMarc Ferland <marc.ferland@sonatest.com>
    Link: https://lore.kernel.org/r/20240501150554.1871390-1-marc.ferland@sonatest.com
    Cc: <Stable@vger.kernel.org>
    Signed-off-by: default avatarJonathan Cameron <Jonathan.Cameron@huawei.com>
    279428df
ad5592r-base.c 14.5 KB