Skip to main content
Numerical methods
Lesson 56 of 75
Introduction

Floating-point error

Objective. Track absolute, relative and rounding error.

  1. 1. Learn
  2. 2. Worked example
  3. 3. Practice
  4. 4. Feedback
  5. 5. Continue

Learn: the key idea

Floating-point error studies how storing real numbers in finite binary precision introduces rounding error, measured as absolute error |x̂ − x| or relative error |x̂ − x|/|x|. Every arithmetic operation can amplify these small errors, especially subtraction of nearly equal numbers, called catastrophic cancellation. For example computing 1 − cos(x) for small x directly loses precision because both terms are close to 1; rewriting as 2sin²(x/2) avoids the cancellation. Relative error is usually more meaningful than absolute error because it scales with the size of the quantity being measured.

Worked example

A floating-point computation gives x̂ = 2.003 as an approximation to the true value x = 2. Find the relative error.

2.003 … 2
Illustrated card showing 2.003 … 2
  1. 1Absolute error: |2.003 − 2| = 0.003
  2. 2Relative error formula: |x̂−x|/|x|
  3. 3Compute: 0.003/2 = 0.0015
  4. 4Express as a percentage: 0.15%

Answer: 0.0015 (0.15%)

Concept mastery

Based on your past attempts at this lesson — the question types to practise again come first.

Complete a practice set to see your concept breakdown here.

Practice

Practice happens on its own screen, one question at a time. Answers stay hidden until you submit yours, and you can stop and pick up at the same question later.

Start practising

Common mistake. Reporting only absolute error when comparing numbers of very different sizes; an absolute error of 0.01 is negligible for a value near 10000 but huge for a value near 0.001. Use relative error instead.

Your progress

Tick each step as you finish it. Your exact reading position is saved automatically, so Continue drops you back on the same line.

0 of 5 steps complete

Your place saves automatically as you read.

My dashboard