================================================================================ NUMPY - USER GUIDE (Android Python STB) Generated by RIMI ================================================================================ Covers: what numpy is, install/verify, arrays, dtypes, math, indexing, reshaping, linear algebra, random, file I/O, numpy + Pillow, printing in the terminal, and common pitfalls. Written for: Python 3.12.2 (RIMI build) on Android Version: numpy 2.5.2 Scripts dir: /storage/emulated/0/PythonSTB/Scripts/ Installed: /data/user/0/com.pythonstb.rimi/files/python/lib/python3.12/site-packages/ ================================================================================ -------------------------------------------------------------------------------- 1) WHAT IS NUMPY? -------------------------------------------------------------------------------- NumPy is the fundamental library for fast numerical computing in Python. It adds: - n-dimensional arrays (ndarray) - faster and more compact than Python lists - element-wise math without writing loops - linear algebra (matrix multiply, inverse, solve, eig, ...) - random numbers, FFT, sorting, statistics - a bridge to C/Python extensions (OpenCV, Pillow, pandas, scipy, ...) Typical speedup vs plain Python loops: 10x - 100x or more. This Android build of numpy is compiled with OpenBLAS, so matrix operations are optimized (BLAS/LAPACK) on both arm64 and x86_64. import numpy as np print(np.__version__) # 2.5.2 print(np.show_config()) # shows BLAS/LAPACK backend + build info -------------------------------------------------------------------------------- 2) INSTALL / VERIFY -------------------------------------------------------------------------------- Install (already done, but if you ever reinstall): pip install numpy # or install the wheel file directly Quick smoke test - run in your app: import numpy as np a = np.arange(12).reshape(3, 4) print(a) print("sum:", a.sum(), "max:", a.max(), "shape:", a.shape, "dtype:", a.dtype) print("blas:", np.__config__.show("build") if hasattr(np.__config__, "show") else "n/a") Expected output pattern: [[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] sum: 66 max: 11 shape: (3, 4) dtype: int64 -------------------------------------------------------------------------------- 3) ARRAYS - CREATION BASICS -------------------------------------------------------------------------------- import numpy as np # from a list a = np.array([1, 2, 3]) # 1-D, dtype int64 b = np.array([[1, 2, 3], [4, 5, 6]]) # 2-D shape (2, 3) c = np.array([1.0, 2.0, 3.0]) # float64 # zeros / ones / full / identity np.zeros((3, 4)) np.ones((2, 2)) np.full((2, 3), 7.5) np.eye(4) # 4x4 identity np.identity(3) # ranges np.arange(10) # 0..9 np.arange(0, 1, 0.1) # 0.0, 0.1, ... 0.9 np.linspace(0, 1, 5) # 5 evenly spaced points 0..1 np.logspace(1, 3, 3) # 10, 100, 1000 # random rng = np.random.default_rng(seed=42) # reproducible rng.random((3, 3)) # uniform [0,1) rng.integers(0, 10, size=(2, 5)) # integers 0..9 rng.normal(0, 1, size=(4,)) # normal dist rng.permutation(10) # shuffled 0..9 # important attributes a = np.array([[1, 2, 3], [4, 5, 6]]) a.shape # (2, 3) a.ndim # 2 a.size # 6 a.dtype # dtype('int64') a.itemsize # bytes per element a.nbytes # total bytes -------------------------------------------------------------------------------- 4) DTYPES (data types) -------------------------------------------------------------------------------- np.int8 np.int16 np.int32 np.int64 # signed integers np.uint8 np.uint16 np.uint32 np.uint64 # unsigned integers np.float32 np.float64 # floats (f32 is half memory) np.complex64 np.complex128 # complex np.bool_ # boolean np.str_ np.bytes_ # strings (avoid for math) 'f4','f8','i1','i2','i4','i8','u1','u2','u4','u8' # short aliases a = np.array([1, 2, 3], dtype=np.uint8) b = a.astype(np.float32) # convert c = a.astype('f4') Rules of thumb: - use np.float64 (default) for general math - use np.float32 or np.uint8 for big arrays / images (half the memory) - beware overflow: np.array([200], np.uint8) + 100 wraps to 44 - beware int division: np.array([5]) // 2 == 2 (floor), np.array([5]) / 2 == 2.5 -------------------------------------------------------------------------------- 5) INDEXING AND SLICING -------------------------------------------------------------------------------- a = np.arange(12).reshape(3, 4) # [[ 0 1 2 3] # [ 4 5 6 7] # [ 8 9 10 11]] a[0] # row 0: [0 1 2 3] a[0, 2] # scalar 2 a[:, 1] # column 1: [1 5 9] a[1:, :2] # rows 1..2, cols 0..1 a[-1] # last row a[::2] # every other row # boolean masking a[a > 5] # 1-D array of values > 5 a[(a > 2) & (a < 8)] # combine masks with & | a[a % 2 == 0] # even values # fancy indexing with arrays a[[0, 2]] # rows 0 and 2 idx = np.array([3, 1]) a[:, idx] # columns 3 and 1 # assignment with masks a[a < 5] = 0 # zero out everything below 5 a[:, 0] = -1 # set first column -------------------------------------------------------------------------------- 6) SHAPES - RESHAPE / FLATTEN / TRANSPOSE / BROADCAST -------------------------------------------------------------------------------- a = np.arange(24) a.reshape(4, 6) # same data, new shape a.reshape(2, 3, 4) # 3-D a.reshape(-1, 6) # -1 = auto: (4, 6) a.ravel() # flatten to 1-D (may copy) a.flatten() # always a copy, 1-D a.T # transpose a.reshape(4, 6).T.shape # (6, 4) # add a new axis v = np.array([1, 2, 3]) v[np.newaxis, :].shape # (1, 3) v[:, np.newaxis].shape # (3, 1) # BROADCASTING: shapes line up from the right m = np.ones((3, 4)) m + 1 # scalar broadcast m * np.array([10, 20, 30, 40]) # row vector broadcast over rows m + np.array([[1], [2], [3]]) # column vector broadcast over cols # rule: dimensions must be equal or one of them must be 1 -------------------------------------------------------------------------------- 7) MATH - ELEMENT-WISE AND REDUCTIONS -------------------------------------------------------------------------------- a = np.array([1., 2., 3., 4.]) a + 1, a - 1, a * 2, a / 2, a ** 2, -a # element-wise np.sqrt(a), np.abs(a), np.exp(a), np.log(a) np.sin(a), np.cos(a), np.tan(a), np.arctan(a) np.round(a), np.floor(a), np.ceil(a), np.clip(a, 1.5, 3.5) np.sign(a), np.mod(a, 2), np.power(a, 3) np.maximum(a, 2), np.minimum(a, 3) # reductions (default: over all elements) a.sum() a.mean() a.min() a.max() a.std() a.var() a.prod() a.argmax() a.argmin() a.cumsum() a.cumprod() np.median(a) np.percentile(a, 50) np.ptp(a) # peak-to-peak # along an axis m = np.arange(6).reshape(2, 3) m.sum(axis=0) # per column: [3 5 7] m.sum(axis=1) # per row: [3 12] m.max(axis=0), m.min(axis=1) # comparisons return boolean arrays (a > 2) # array([False, False, True, True]) np.any(a > 2) # True np.all(a > 2) # False np.count_nonzero(a > 2) # 2 -------------------------------------------------------------------------------- 8) LINEAR ALGEBRA (OpenBLAS accelerated) -------------------------------------------------------------------------------- a = np.array([[1., 2.], [3., 4.]]) b = np.array([[5., 6.], [7., 8.]]) a @ b # matrix multiply (preferred) np.matmul(a, b) # same a.dot(b) # same (older style) a * b # ELEMENT-WISE, NOT matrix multiply np.linalg.inv(a) # inverse np.linalg.det(a) # determinant np.linalg.solve(a, np.array([1., 2.])) # solve a x = b np.linalg.eig(a) # eigenvalues + eigenvectors np.linalg.norm(a) # Frobenius norm np.linalg.qr(a), np.linalg.svd(a) np.linalg.pinv(a) # pseudo-inverse np.linalg.matrix_power(a, 3) # vector ops v = np.array([1., 2., 3.]) w = np.array([4., 5., 6.]) np.dot(v, w) # dot product 32.0 np.cross(v, w) # cross product np.inner(v, w) # inner product # useful on the device: transform a 3D point / homography M = np.array([[1., 0., 10.], [0., 1., 20.], [0., 0., 1.]]) p = np.array([5., 6., 1.]) out = M @ p -------------------------------------------------------------------------------- 9) STACKING, SPLITTING, CONCATENATING -------------------------------------------------------------------------------- a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) np.concatenate((a, b)) # [1 2 3 4 5 6] np.stack((a, b)) # shape (2, 3) np.vstack((a, b)) # vertical: shape (2, 3) np.hstack((a, b)) # horizontal: [1 2 3 4 5 6] np.dstack((a, b)) # depth: shape (1, 3, 2) m1 = np.ones((2, 2)) m2 = np.zeros((2, 2)) np.vstack((m1, m2)) # (4, 2) np.hstack((m1, m2)) # (2, 4) # split x = np.arange(10) np.split(x, 2) # two arrays of 5 np.array_split(x, 3) # uneven split np.hsplit(m1, 2), np.vsplit(m1, 2) -------------------------------------------------------------------------------- 10) RANDOM NUMBERS -------------------------------------------------------------------------------- rng = np.random.default_rng(2026) # always seed for reproducibility rng.random((2, 3)) # [0,1) floats rng.integers(1, 7, size=10) # die rolls 1..6 rng.normal(loc=0, scale=1, size=(3, 3)) rng.uniform(0, 10, size=5) rng.choice(np.arange(5), size=10, replace=True) rng.shuffle(np.arange(10)) # in place rng.standard_normal((4,)) # old style (np.random.rand etc.) also works, but default_rng is preferred. -------------------------------------------------------------------------------- 11) SAVE / LOAD DATA (files) -------------------------------------------------------------------------------- a = np.arange(12).reshape(3, 4) # numpy binary format (fast, compact, one array per file) np.save("/storage/emulated/0/Download/a.npy", a) b = np.load("/storage/emulated/0/Download/a.npy") # compressed multi-array archive np.savez("/storage/emulated/0/Download/data.npz", x=a, y=a * 2) d = np.load("/storage/emulated/0/Download/data.npz") d["x"], d["y"] # or np.savez_compressed(...) for smaller files # text (human readable) np.savetxt("/storage/emulated/0/Download/a.csv", a, delimiter=",") c = np.loadtxt("/storage/emulated/0/Download/a.csv", delimiter=",") # note: loadtxt returns float64; use dtype= to control np.savetxt("/storage/emulated/0/Download/a.tsv", a, delimiter="\t", fmt="%.2f") # plain binary (raw, no header) a.tofile("/storage/emulated/0/Download/a.bin") np.fromfile("/storage/emulated/0/Download/a.bin", dtype=np.int64) -------------------------------------------------------------------------------- 12) NUMPY + PILLOW (images are just arrays) -------------------------------------------------------------------------------- from PIL import Image import numpy as np # PIL image -> numpy array (H, W, C) im = Image.open("/storage/emulated/0/Download/photo.jpg").convert("RGB") arr = np.asarray(im) print(arr.shape) # (height, width, 3) print(arr.dtype) # uint8 # numpy array -> PIL image img2 = Image.fromarray(arr) img2.save("/storage/emulated/0/Download/out.jpg", quality=95) # grayscale -> (H, W) gray = np.asarray(im.convert("L")) # alpha -> (H, W, 4) rgba = np.asarray(im.convert("RGBA")) # process with numpy, then convert back arr2 = arr[:, ::-1] # mirror arr3 = np.clip(arr.astype(np.int16) + 50, 0, 255).astype(np.uint8) # brighten arr4 = 255 - arr # invert arr5 = arr.copy(); arr5[..., 0] = 255 # force red channel to max Image.fromarray(arr2).save("/storage/emulated/0/Download/mirror.png") Image.fromarray(arr4).save("/storage/emulated/0/Download/invert.png") # crop = slice crop = arr[100:200, 50:150] Image.fromarray(crop).save("/storage/emulated/0/Download/crop.png") # resize with numpy (nearest) - better to use PIL resize normally small = arr[::4, ::4] # nearest-neighbour downsample # build a gradient image h, w = 200, 200 yy, xx = np.mgrid[0:h, 0:w] grad = np.stack([ (xx * 255 // max(1, w - 1)).astype(np.uint8), (yy * 255 // max(1, h - 1)).astype(np.uint8), ((xx + yy) * 255 // max(1, (w + h) - 2)).astype(np.uint8), ], axis=2) Image.fromarray(grad).save("/storage/emulated/0/Download/gradient.png") IMPORTANT: np.asarray(im) may share memory with the PIL image. If you modify arr in place (arr[...] = ...), the image changes too. Use arr.copy() when you need an independent buffer. uint8 arithmetic overflows (255+1 -> 0). Cast to int16 first for math: arr.astype(np.int16). -------------------------------------------------------------------------------- 13) PRINTING ARRAYS / MATRICES IN THE TERMINAL -------------------------------------------------------------------------------- a = np.arange(12).reshape(3, 4) print(a) # default pretty print print(a.tolist()) # as nested Python lists # control the output import numpy as np np.set_printoptions( precision=2, # decimals for floats threshold=20, # max elements before "..." edgeitems=3, linewidth=120, suppress=True, # avoid scientific notation for small numbers formatter={"float_kind": lambda v: f"{v:7.2f}"}, ) print(a) # a small helper to show an array as a grid of numbers def print_matrix(m): m = np.asarray(m) for row in m: print(" ".join(f"{v:8.3f}" for v in row)) print_matrix(np.random.default_rng(1).random((3, 5))) # print a 2D array as colored blocks (with ANSI) def print_heatmap(m, cols=40): m = np.asarray(m, dtype=np.float64) if m.ndim != 2: m = m.reshape(m.shape[0], -1) h, w = m.shape # resample columns to terminal width (nearest) if w > cols: m = m[:, ::max(1, w // cols)][:, :cols] h, w = m.shape lo, hi = m.min(), m.max() rng = (hi - lo) or 1.0 out = [] for row in m: line = "" for v in row: t = (v - lo) / rng # 0..1 r = int(255 * t) g = int(255 * (1 - t)) line += f"\x1b[48;2;{r};{g};0m " out.append(line + "\x1b[0m") print("\n".join(out)) # example: heatmap of a sinc function import numpy as np x = np.linspace(-6, 6, 80) y = np.linspace(-6, 6, 40) yy, xx = np.meshgrid(y, x, indexing="ij") z = np.sinc(np.sqrt(xx ** 2 + yy ** 2)) print_heatmap(z, cols=60) # ASCII density plot from a 2D array RAMP = " .:-=+*#%@" def print_ascii_grid(m, cols=60): m = np.asarray(m, dtype=np.float64) if w := m.shape[1] > cols: m = m[:, ::w // cols + 1] lo, hi = m.min(), m.max() rng = (hi - lo) or 1.0 for row in m: print("".join(RAMP[int((v - lo) / rng * (len(RAMP) - 1))] for v in row)) print_ascii_grid(z, cols=70) -------------------------------------------------------------------------------- 14) USEFUL EVERYDAY SNIPPETS -------------------------------------------------------------------------------- Statistics of a numeric column: data = np.array([1., 2., 3., 4., 100.]) print("mean %.2f std %.2f median %.2f min %.0f max %.0f" % ( data.mean(), data.std(), np.median(data), data.min(), data.max())) Normalize to [0, 1]: x = np.array([3., 1., 2., 0.]) n = (x - x.min()) / (x.max() - x.min()) Standard score (z-score): z = (x - x.mean()) / x.std() One-hot encode categories: cats = np.array([0, 2, 1, 2, 0]) onehot = np.eye(3)[cats] Extract the diagonal of a matrix: np.diag(np.arange(9).reshape(3, 3)) # [0 4 8] Histogram: values, edges = np.histogram(rng.normal(size=1000), bins=20) Find the index of the maximum in each row: m = rng.random((5, 8)) m.argmax(axis=1) Clip and cast for image math: arr.astype(np.float32) * 1.2 + 10 -> clip -> uint8 Simple FIR smoothing: kernel = np.ones(5) / 5 smooth = np.convolve(signal, kernel, mode="same") Measure elapsed time: import time t0 = time.perf_counter() ... work ... print("elapsed %.3f s" % (time.perf_counter() - t0)) -------------------------------------------------------------------------------- 15) NUMPY + PANDAS / OTHER PACKAGES -------------------------------------------------------------------------------- numpy is the foundation for many packages already on the device: import numpy as np import pandas as pd df = pd.DataFrame({"a": [1, 2, 3], "b": [4.0, 5.0, 6.0]}) print(df) arr = df.to_numpy() # DataFrame -> numpy array # pandas is built on numpy; everything in this guide applies. # OpenCV (if installed) also exchanges buffers directly: # cv2.cvtColor(img_np, cv2.COLOR_BGR2RGB) -------------------------------------------------------------------------------- 16) PITFALLS & NOTES ON THIS BUILD -------------------------------------------------------------------------------- - Version is 2.5.2. numpy 2.x changed some 1.x behaviors: * np.array(None) no longer allowed * np.find_common_type removed * copy keyword defaults changed (np.array(..., copy=None) is common) Code written for numpy 1.x may need small fixes. - uint8 overflow: cast to a wider dtype before math on images. - Integer division: use // for floor, / for true (float) division. - Broadcasting: shapes must be equal or one must be 1; (3,1) * (1,4) -> (3,4). - np.asarray may share memory with the source (PIL image); use .copy() to detach. - OpenBLAS is compiled in - @ and np.linalg.* are fast; no action needed. - Wheels here are tagged cp312-cp312-linux_aarch64 / linux_x86_64 (this Android build uses the "linux" platform tag for numpy). Make sure you install the wheel that matches the device ABI (arm64 phone vs x86_64 emulator). - Temp files are not needed: save directly to /storage/emulated/0/Download/ or any folder the app can write. - For very large arrays, watch memory: a float64 array of 10M elements uses 80 MB. Use float32 or appropriate dtypes when possible. - Reinstall safety: keep a copy of the wheel file (numpy-2.5.2-cp312-cp312-linux_aarch64.whl or _x86_64) in /storage/emulated/0/Download/ so you can reinstall if needed. ================================================================================ END OF GUIDE Generated by RIMI ================================================================================