PYTHON / NUMPY
Why NumPy: arrays versus lists
Explain and demonstrate why a NumPy array uses less memory and computes faster than an equivalent Python list, and when a list is still the right choice.
What you will learn
- Estimate memory for a list of ints versus an array using getsizeof and nbytes
- Explain boxing, pointer indirection, and contiguity as the real source of the speedup
- Predict how + and * differ between lists and arrays and why
- Recognize when array homogeneity and fixed size make a list the better container
Understanding Why NumPy: arrays versus lists
A Python list of 1000 integers is not 1000 integers in a row. It is a block of 1000 pointers, each 8 bytes, and each pointer leads to a separate PyObject somewhere else on the heap that carries a reference count, a type pointer, a digit count, and only then the actual value. That is 28 bytes per small int plus 8 bytes for the slot, so roughly 36 bytes to store a number that needs 8. A NumPy array of the same 1000 integers is one contiguous buffer of 8000 bytes plus a small header describing shape, strides, and dtype.
The speed difference follows directly from that layout. Adding two lists element by element makes the interpreter fetch a pointer, check the object's type, unbox the C integer, allocate a brand new int object for the result, and store its pointer. NumPy already knows every element is an int64 sitting 8 bytes after the previous one, so a compiled loop can read, add, and write raw machine values with no per-element type dispatch and no allocation. Contiguity also means the CPU prefetches useful data into cache instead of chasing pointers to scattered objects.
The cost of that layout is flexibility. An array has one dtype for all elements, so mixing a string into numbers forces everything to a common type, and writing 9.7 into an int64 array truncates it silently. An array also has a fixed size: np.append does not extend anything, it allocates a new buffer and copies, which is why growing an array in a loop is quadratic. For a handful of values, or for heterogeneous data you keep appending to, a list is still the correct tool; arrays pay off when the same operation applies to thousands of same-typed numbers at once.
import sys
import numpy as np
nums = list(range(1, 1001))
arr = np.arange(1, 1001)
int_object_bytes = sum(sys.getsizeof(n) for n in nums)
pointer_bytes = 8 * len(nums)
print("one int object :", sys.getsizeof(nums[0]), "bytes")
print("1000 int objects :", int_object_bytes, "bytes")
print("list pointer slots :", pointer_bytes, "bytes")
print("array buffer :", arr.nbytes, "bytes")
print("array itemsize :", arr.itemsize, "bytes")
print("array dtype :", arr.dtype)An array is one contiguous block of fixed-width typed values, while a list is a block of pointers to independent Python objects, and every speed and memory difference between them comes from that.
Worked examples
The same operators mean different things
Shows that + and * treat a list as a container to join and an array as a vector of numbers.
import numpy as np
a = [1, 2, 3]
b = [4, 5, 6]
x = np.array(a)
y = np.array(b)
print(a + b)
print(x + y)
print(a * 2)
print(x * 2)Example explained
Line 1a + b builds a new 6-element list because list.__add__ is concatenation of containers.
Line 2x + y produces 3 sums because ndarray.__add__ works on the numeric buffer, not on the container.
Line 3a * 2 repeats the pointers, so the list holds the same three int objects twice.
Line 4x * 2 scales every stored value, which is why the array result still has length 3.
One dtype for the whole buffer
Shows how a fixed element width forces truncation and type promotion that a list never performs.
import numpy as np
arr = np.array([1, 2, 3])
arr[0] = 9.7
print(arr, arr.dtype)
promoted = np.array([1, 2, 3.5])
print(promoted, promoted.dtype)
mixed = [1, 2, 3.5, "four"]
print(mixed)Example explained
Line 1arr[0] = 9.7 must fit into 8 bytes of int64, so the fractional part is dropped, giving 9 rather than 10.
Line 2np.array([1, 2, 3.5]) cannot store an int and a float side by side, so all three become float64.
Line 3The list keeps an int, a float, and a str together because each slot is just a pointer to any object.
Fixed size and boxed scalars
Shows that np.append copies into a new buffer and that indexing one element still hands back a Python-level object.
import numpy as np
a = np.arange(4)
b = np.append(a, 4)
print(a)
print(b)
print(b.base is None)
print(type(a[0]).__name__, type([0, 1][0]).__name__)Example explained
Line 1a is unchanged: an array's buffer has a fixed length, so nothing can be appended in place.
Line 2b.base is None shows b owns a fresh buffer, meaning np.append allocated and copied all elements.
Line 3type(a[0]) is numpy.int64, a wrapper object created on the spot, which is why per-element Python loops over arrays are not faster than over lists.
Important notes
The 28-byte int and 8-byte pointer figures are CPython on 64-bit; np.arange defaults to int32 on Windows, so nbytes and dtype there will read 4000 and int32.
For arrays of a few elements the per-call NumPy overhead dominates, so a plain list can genuinely be faster; the advantage grows with element count.
Common mistakes
Growing results with arr = np.append(arr, value) inside a loop: every call copies the entire buffer, turning an O(n) job into O(n^2) and often running slower than the list version.
Keeping the for loop and only swapping the list for an array: each arr[i] boxes a numpy scalar, so you add overhead instead of removing it and see no speedup.
Expecting x + y to concatenate as lists do: it adds elementwise instead, and with different lengths it raises a ValueError about incompatible shapes rather than joining the data.
Try it yourself
Change, predict, then run
Build nums = list(range(1, 100001)) and arr = np.arange(1, 100001), print the estimated list cost (28 * len + 8 * len) next to arr.nbytes, then confirm sum(nums) and arr.sum() give the same number.
Open the Python workspaceCheck your understanding
You sum 10 million integers. Why is arr.sum() much faster than sum(py_list) on the same values?
- The values sit in one contiguous block of fixed-width bytes, so a compiled loop adds raw machine integers without creating a Python object per element
- NumPy automatically spreads the addition across all available CPU cores
- NumPy caches the result of earlier aggregations on the array and reuses it
- Lists re-verify the type of each element on access while arrays skip that check, though both still allocate the same objects
Show answer
The win comes from layout: known dtype plus contiguous storage lets NumPy run one C loop over raw int64 values, skipping pointer chasing, type dispatch, and result allocation. Multithreading is tempting but wrong: sum on an array runs on a single core, and the multithreading NumPy does use comes from BLAS in linear algebra routines, not from reductions.