Written lessons | Md. Mahbubur Rahman

Tensors and shapes

Mon, 01 Jan 0001 00:00:00 +0000

A tensor stores values in a multidimensional array. In image learning, dimensions often describe the batch, channels, height, and width.

For a batch of eight RGB images resized to 224 by 224 pixels, the usual PyTorch shape is (8, 3, 224, 224). A single-channel depth input would normally have shape (8, 1, 224, 224).

import torch
rgb = torch.zeros(8, 3, 224, 224)
depth = torch.zeros(8, 1, 224, 224)
rgbd = torch.cat([rgb, depth], dim=1)
print(rgbd.shape) # torch.Size([8, 4, 224, 224])

Concatenation only aligns array dimensions. It does not register the cameras, convert depth units, or remove invalid depth values. Those steps must be handled before treating the channels as corresponding observations.

Try it

Change the batch size, then inspect every tensor shape. Consider which preprocessing steps are shared between RGB and depth and which require separate handling.

Training and evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Training adjusts model parameters using examples and a loss function. Validation informs choices such as hyperparameters and stopping time. The test set evaluates the final procedure after those choices are fixed.

A reliable sequence

Define the prediction target and the unit of independence.
Split the data before fitting preprocessing or selecting a model.
Fit using training data and choose settings using validation data.
Evaluate the frozen procedure on the held-out test data.

For repeated observations of plants, an image-level random split can place the same plant in both training and test sets. A plant-level split is more appropriate when the goal is performance on unseen plants.

Evaluation mode

model.eval()
with torch.inference_mode():
 predictions = model(test_inputs)

Evaluation mode changes the behavior of layers such as dropout and batch normalization. Disabling gradient recording reduces the work required during inference. These operations serve different purposes.

Try it

Write down the grouping variable for your own dataset. Check whether any group appears in more than one partition.

Working with RGB-D observations

Mon, 01 Jan 0001 00:00:00 +0000

RGB describes appearance. Depth describes distance according to the sensor’s coordinate system and unit convention. Combining them requires attention to calibration, alignment, missing measurements, and scale.

Before modeling

Record the sensor, depth units, and calibration used for each collection.
Verify alignment between the color image, depth image, and any segmentation masks.
Keep track of invalid depth values instead of treating them as valid zero-distance observations.
Apply spatial transformations consistently across corresponding channels and masks.

Evaluating a benefit from depth

Compare RGB and RGB-D models using the same data partitions and training budget. Check results by occlusion severity or other relevant conditions. An improvement in aggregate performance alone does not explain when depth contributes.

Try it

Inspect a color image, a depth image, and a validity mask side by side. Look for boundary misalignment and regions where the sensor did not return a usable measurement.