Introduction
Filtering data is one of the most common things you do with it: the exam scores above 60, the days hotter than 30 degrees, the readings that are not missing. With lists you write a loop with an if. With NumPy you write the condition once, as an expression on the whole array, and get a boolean mask that selects, counts and changes the matching elements in one step.
How it works
Comparison operators on an array work element-wise and give an array of booleans:
a = np.array([4, 9, 1, 7])
print(a > 5) # [False True False True]
print(a[a > 5]) # [9 7]
print(np.count_nonzero(a > 5)) # 2
Indexing with a boolean array of the same shape keeps the elements where it is True. The result is always 1-D and always a copy. Because True counts as 1, np.count_nonzero(mask) or mask.sum() counts the matches.
To combine conditions, use the element-wise operators & (and), | (or) and ~ (not), with parentheses around each comparison:
print((a > 2) & (a < 8)) # [ True False False True]
The parentheses are required: & binds more tightly than >, so a > 2 & a < 8 means something else and fails. Python's own and and or do not work on arrays at all; they raise "The truth value of an array with more than one element is ambiguous".
np.where(condition, x, y) builds a new array taking x where the condition holds and y elsewhere; x and y may be arrays or single values:
print(np.where(a > 5, a, 0)) # [0 9 0 7]
Finally, masked assignment changes the original array in place, only where the mask is true: a[a > 8] = 8 caps every value at 8.
Example
# A model of boolean masks: a comparison gives one True or False per element.
scores = [55, 82, 47, 91, 68]
mask = [s >= 60 for s in scores] # scores >= 60
print("mask:", mask)
print("passed:", [s for s, m in zip(scores, mask) if m]) # scores[mask]
print("count:", sum(mask)) # np.count_nonzero(mask)
middle = [(s >= 60) and (s < 90) for s in scores] # (scores >= 60) & (scores < 90)
print("60 to 89:", [s for s, m in zip(scores, middle) if m])
print("where:", [s if m else 0 for s, m in zip(scores, mask)]) # np.where(mask, scores, 0)
capped = [85 if s > 85 else s for s in scores] # scores[scores > 85] = 85
print("capped:", capped)Line by line
- The mask holds one
TrueorFalseper score, likescores >= 60. - Selecting with the mask keeps the passing scores; summing the mask counts them.
- Two conditions combine element by element, like
&in NumPy. - The
whereline keeps passing scores and puts 0 elsewhere; the last line caps high scores, like a masked assignment.
Expected result
mask: [False, True, False, True, True]
passed: [82, 91, 68]
count: 3
60 to 89: [82, 68]
where: [0, 82, 0, 91, 68]
capped: [55, 82, 47, 85, 68]
In the real world
Masks clean data (data[~np.isnan(data)] drops missing values), select pixels by colour in images, and pick out events above a threshold in signals. pandas uses exactly the same syntax for filtering rows: df[(df.age > 30) & (df.city == "Dhaka")].
Common mistakes
- Using and / or instead of
&and|, which raises the "ambiguous" error. - Leaving out parentheses around each comparison.
- Expecting a[mask] to keep the shape: it is always 1-D.
- Changing the result of a[mask] and expecting the original to change; assign with
a[mask] = ...instead.
Best practices
- Name masks that you use more than once:
passed = scores >= 60. - Use
np.whereto build a new array and masked assignment to change one in place. - Count with
np.count_nonzeroormask.sum(), never a loop.
Summary
Comparisons on arrays give boolean masks. Indexing with a mask selects the matching elements as a new 1-D array, &, | and ~ combine conditions (each in parentheses), np.where chooses between two values per element, and a[mask] = value changes the matching elements in place.
Quick check
Key takeaways
- a > 5 gives a boolean array; a[a > 5] selects the elements where it is True.
- Combine conditions with &, | and ~, each condition in parentheses.
- np.where picks between two values per element; a[mask] = v changes elements in place.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.