Engineering Reference
Sensor Calibration, Accuracy and Error Terms Reference
Lookup for sensor sensitivity, offset, span, gain error, linearity, hysteresis, repeatability, resolution, accuracy, drift, uncertainty, and calibration methods.
- Reading Time
- 13 min
- Format
- Calibration term lookup
- Updated
- September 29, 2026
Accuracy and Error Terms
| Term | Definition | Common expression | Boundary |
|---|---|---|---|
| Sensitivity | Output change divided by measurand change | V/unit, mV/V, counts/unit | Can vary with operating point and temperature |
| Offset / zero error | Output at the defined zero input minus ideal output | Output units or %FS | Zero balance and bias are context-specific |
| Span | Full-scale output minus zero output | Output units | Not identical to full-scale reading when zero is nonzero |
| Gain / span error | Slope deviation from the ideal transfer | % reading or %FS | State calibration endpoints and reference |
| Linearity error | Deviation from a defined straight reference line | %FS or units | Endpoint, best-fit and zero-based methods differ |
| Hysteresis | Difference for increasing versus decreasing input at the same point | %FS or units | Requires a defined cycle and direction |
| Repeatability | Spread under repeated same-condition measurements | standard deviation, range or %FS | Not the same as accuracy |
| Resolution | Smallest discernible input or output increment | input units, output units or bits | Noise and ENOB can dominate nominal ADC LSB |
| Accuracy | Closeness to the accepted reference under stated conditions | %FS, % reading or units | Must include conditions and error model |
| Precision | Closeness among repeated results | statistical metric | A precise sensor may still be biased |
| Drift | Change with time, temperature, supply or another influence | units/condition | Requires interval and reference condition |
| Uncertainty | Quantified doubt associated with a measurement result | standard or expanded uncertainty | Not simply the sum of every datasheet maximum unless defining worst case |
Calibration Methods
| Method | What it corrects | Primary boundary |
|---|---|---|
| One-point zero | Correct offset at one reference | Does not correct gain or nonlinearity |
| Two-point linear | Fit slope and intercept from two distinct references | Assumes linear behavior between/around points |
| Multi-point linear regression | Least-squares slope and intercept | Residuals must still be inspected |
| Piecewise linear | Separate local slopes between calibration points | Interpolation only; discontinuity handling matters |
| Polynomial | Fit curvature with higher-order terms | Overfitting and extrapolation risk |
| Lookup table | Store measured calibration values | Interpolation, memory and monotonicity rules required |
| Temperature compensation | Apply calibration versus temperature | Temperature sensor and thermal lag add uncertainty |
| System calibration | Calibrate sensor plus analog chain and ADC | May not isolate component-level error sources |
Linear Calibration Relationships
| Need | Relationship | Boundary |
|---|---|---|
| Slope | m = (y2 - y1) / (x2 - x1) | x1 and x2 must differ |
| Intercept | b = y1 - mx1 | Preserve unit consistency |
| Forward conversion | y = mx + b | Model valid over stated calibration range |
| Inverse conversion | x = (y - b) / m | Ill-conditioned when slope approaches zero |
| Residual | eᵢ = yᵢ - ŷᵢ | Inspect pattern as well as maximum magnitude |
| Full-scale error | error / span × 100% | Define whether span is input or output full scale |
Canonical Calculation Anchors
| Case | Result | Interpretation |
|---|---|---|
| Two points (1,10), (5,30) | m = 5, b = 5 | Exact two-point linear fit |
| Three-point regression | m = 1.900, b = 0.133 | Maximum residual 0.067 |
| 0.5-4.5 V maps to 0-100 | m = 25.000 units/V | 2.5 V maps to 50 |
| ±10 mV electrical uncertainty | -0.250 / +0.250 engineering units | Linear first-order propagation |
| Load cell 2 mV/V at 5 V | 10.000 mV full scale | Ratiometric sensitivity times excitation |
| 350 Ω gauge, GF 2, 1000 µε | 0.700 Ω change | Ideal gauge-factor relation |
| Accelerometer +1g 1.65 V, -1g 0.65 V | 0.500 V/g; zero 1.150 V | Two-reference calibration |
| Vector (0, 0.7071, 0.7071)g | 1.000 g | Roll 45°, standard g 9.80665 m/s² |
| Pressure 0-100 maps 0.5-4.5 V | slope 0.040 pressure/V | Pressure type and units remain explicit |
Uncertainty and Error Combination
| Method | Use | Boundary |
|---|---|---|
| Worst-case sum | Guaranteed independent bounds that could align | Conservative; preserve sign only when known |
| Root-sum-square | Independent random standard-uncertainty components | Requires statistical justification |
| Sensitivity coefficients | Convert each input uncertainty into output units | Include covariance when inputs are correlated |
| Expanded uncertainty | Coverage interval U = kuc | State coverage factor and confidence interpretation |
| Monte Carlo | Nonlinear or non-Gaussian propagation | Input distributions and correlations must be credible |
Common Errors
- Calling resolution accuracy.
- Calling repeatability accuracy.
- Omitting the reference-line definition for linearity.
- Ignoring hysteresis direction.
- Using two-point calibration as proof of linearity.
- Extrapolating beyond calibrated range.
- Mixing % reading and % full-scale errors.
- Adding statistical uncertainty and hard limits without a model.
- Ignoring covariance between shared references.
- Using nominal ADC bits instead of ENOB under noise.
- Calibrating the sensor but not the analog chain.
- Ignoring temperature, mounting and aging drift.
Support reference
FAQ
What is the difference between accuracy and precision?
Accuracy describes closeness to an accepted reference. Precision describes agreement among repeated measurements. A system can be precise but biased.
What is sensor sensitivity?
Sensitivity is the change in output divided by the corresponding change in measurand, locally or over a defined range.
What is the difference between offset and span error?
Offset shifts the transfer function at zero or another reference point. Span error changes the slope or full-scale difference.
How does two-point calibration work?
Two distinct reference points define slope and intercept for a linear calibration. It does not prove the sensor is linear between or beyond those points.
What is linearity error?
It is deviation from a specified straight reference line. Endpoint, best-fit and other definitions can produce different numbers.
What is hysteresis?
It is the output difference at the same input when approached from increasing and decreasing directions under a specified cycle.
Is ADC resolution the same as sensor accuracy?
No. ADC LSB size is a quantization increment. Reference error, noise, ENOB, front-end error and sensor uncertainty also affect measurement accuracy.
Should independent errors be added by RSS?
Only when a statistical model and independence assumptions are justified. Worst-case bounded limits are combined differently.
What is calibration residual?
It is measured reference output minus the value predicted by the fitted calibration model at each calibration point.
Why is extrapolation risky?
The fitted model is supported only over its calibration range. Nonlinearity, saturation and temperature effects may grow outside it.
Does calibration remove drift?
A calibration corrects conditions represented during calibration. Time, temperature, mounting, supply and mechanical changes can introduce later drift.
Connected Engineering Content
Related Resources
Related Calculators
Strain Gauge Calculator
Calculate strain gauge resistance change, gauge factor, microstrain, quarter-bridge, half-bridge, full-bridge output, mV/V sensitivity, and tolerance references.
Load Cell Output Calculator
Calculate load-cell output voltage, mV/V sensitivity, excitation voltage, full-scale output, applied load, zero balance, tare offset, overload reference, and tolerance range.
Pressure Sensor Scaling Calculator
Scale pressure sensor voltage and 4-20 mA signals, convert pressure to signal, signal to pressure, analyze live zero, calibration, zero/span error, and loop receiver resistors.
Sensor Calibration Calculator
Fit sensor calibration equations from raw signal and reference data, calculate slope, offset, R², RMSE, residuals, inverse calibration, and calibration tables.
Accelerometer Voltage & g Calculator
Convert analog accelerometer voltage to signed acceleration using zero-g bias and mV/g sensitivity, with calibration, range, ADC resolution, tolerance, and drift references.
Accelerometer Tilt & Vector Calculator
Calculate three-axis accelerometer vector magnitude, roll, pitch, tilt from vertical, horizontal elevation, normalized XYZ projection, gravity consistency, and offset-corrected static tilt references.
Analog Sensor Linear Scaling Calculator
Scale analog sensor voltage or current signals into engineering units, invert engineering targets back to signal values, calculate slope, intercept, span percentage, reverse ranges, 4-20 mA, 0-10 V, 0-5 V, and electrical uncertainty references.