Deep_Learning_Memo

Deep Learning Memo

Introduction

When traning a neural network, there is lot’s of details that require attention, like how to set the learning rate, how to plot the fig, how to manage your project, etc. These details have no relations with the fundamental basis or the first principle of machine learning, they are just a sort of words of experience.

For green hands, there exists a huge gap between simply comprehending and copying other peoples neural networks from coding one’s own neural network. It’s a tough but effective process. Thus, I want to record my road of practicing deep learning by updating my Blog. This passage won’t discuss the basic principle and only involve my own experiences and insights during my practical process.

If you want to update this blog with your own experiences, just contact with the author.

20250310 Using matplotlib in Linux

When using matplotlib in my WSL subsystem, I encountered errors as follows:

1
2
3
4
5
6
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, minimal, minimalegl, offscreen, vnc, webgl, xcb.

[1]    5870 IOT instruction (core dumped)  python try_plotting.py

It seems that Linux cannot load the Qt platform plugin, which as a result failed to load the images.

Solutions

We cannot let Linux to directly show the plotted figure in the terminal for its lack of GUI interface. But we can realize it by changing Matplotlib’s backend: 

1
2
3
4
5
6
7
8
9
10
11
12
13
import matplotlib as mpl
mpl.use('Agg')
#...
#...
import matplotlib.plt as plt
# forbid scientific notation
np.set_printoptions(suppress=True)


# Then showing the figure by saving the figures.

plt.savefig("save/img.png")

Of course, you can use juypter notebook instead. Remember! Use plt.close() if you want to save more than one image.

20250310 Arranging my file-structure

The contents below is generated by deepseek-v3.

Simplified Project Structure

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
your_project/
│
├── data/                    # Raw and processed data
│   ├── raw/                 
│   └── processed/           
│
├── models/                  # Trained models
│   ├── checkpoints/         # Model checkpoints
│   └── final/               # Final models
│
├── outputs/                 # Training outputs
│   ├── logs/                # Logs (e.g., TensorBoard)
│   ├── images/              # Generated images
│   └── plots/               # Plots
│
├── scripts/                 # Scripts
│   ├── train.py             # Training script
│   └── utils.py             # Utility functions
│
└── README.md                # Project description

Code to Auto-Create Directories

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import os
from datetime import datetime

# Base directory
base_dir = "your_project"

# Subdirectories
dirs = {
    "data": ["raw", "processed"],
    "models": ["checkpoints", "final"],
    "outputs": ["logs", "images", "plots"],
    "scripts": []
}

# Create directories
for parent_dir, sub_dirs in dirs.items():
    parent_path = os.path.join(base_dir, parent_dir)
    os.makedirs(parent_path, exist_ok=True)
    for sub_dir in sub_dirs:
        sub_path = os.path.join(parent_path, sub_dir)
        os.makedirs(sub_path, exist_ok=True)

# Timestamped output directory
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
output_dir = os.path.join(base_dir, "outputs", timestamp)
os.makedirs(output_dir, exist_ok=True)

print(f"Directories created. Output directory: {output_dir}")

Usage in Training Script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import os
import matplotlib.pyplot as plt

# Example output directory
output_dir = "your_project/outputs/20231025_123456"

# Save model checkpoint
model_checkpoint_path = os.path.join(output_dir, "model_checkpoint.pth")
# torch.save(model.state_dict(), model_checkpoint_path)  # PyTorch example

# Save training log
log_file_path = os.path.join(output_dir, "training_log.txt")
with open(log_file_path, "w") as f:
    f.write("Epoch 1, Loss: 0.123\n")

# Save generated image
image_path = os.path.join(output_dir, "training_plot.png")
plt.plot([1, 2, 3], [4, 5, 6])
plt.savefig(image_path)
plt.close()

print(f"Training outputs saved to: {output_dir}")

Additional Tips

  1. Use a Config File:

    1
    2
    3
    4
    5
    # config.py
    BASE_DIR = "your_project"
    DATA_DIR = os.path.join(BASE_DIR, "data")
    MODELS_DIR = os.path.join(BASE_DIR, "models")
    OUTPUTS_DIR = os.path.join(BASE_DIR, "outputs")

  2. Use Logging:

    1
    2
    3
    import logging
    logging.basicConfig(filename=os.path.join(output_dir, "training.log"), level=logging.INFO)
    logging.info("Epoch 1, Loss: 0.123")

  3. Version Control:
    Add to .gitignore:

    1
    2
    3
    your_project/data/
    your_project/models/
    your_project/outputs/

Slices in Torch

Pytorch的切片操作和C++等语法风格存在巨大的差异。

在C++中，假设matrix是一个二维数组，那么matrix[i][j]就会返回这个数组在$(i, j)$位置处的元素的值。（从本质上讲就是对一个二级指针取两次引用）

在Python中，[]为认为是切片操作，或者形式化地说就是取原先数组的一个部分，从实现上将就是实现原数组的浅拷贝过程。对于Torch的张量也是同样如此。并且Python中切片是可以叠加的，换句话说，在Python中matrix[i][j]在本质上是首先切片一个数组$i$行（因为行是第一个维度），接着这个数组就从$(m \times n)$的二维数组变成了$n$的一维数组，接下来第二次切片在原先切片的基础之上取$j$，等效地取出$A[i][j]$中的元素。

但是在很多时候这会产生误会！例如我希望取这个矩阵的第一列，很显然的想法是matrix[:][0]，但是很遗憾，这会取出第一行。因为切片操作是按顺序进行的，matrix[:]相当于返回这个原来的数组，nothing changes.

正确的做法：matrix[:, 0]

例如这个代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def slice_indexing_practice(x: Tensor) -> Tuple[Tensor, Tensor, Tensor, Tensor]:
    """
    Given a two-dimensional tensor x, extract and return several subtensors to
    practice with slice indexing. Each tensor should be created using a single
    slice indexing operation.

    The input tensor should not be modified.

    Args:
        x: Tensor of shape (M, N) -- M rows, N columns with M >= 3 and N >= 5.

    Returns:
        A tuple of:
        - last_row: Tensor of shape (N,) giving the last row of x. It should be
          a one-dimensional tensor.
        - third_col: Tensor of shape (M, 1) giving the third column of x. It
          should be a two-dimensional tensor.
        - first_two_rows_three_cols: Tensor of shape (2, 3) giving the data in
          the first two rows and first three columns of x.
        - even_rows_odd_cols: Two-dimensional tensor containing the elements in
          the even-valued rows and odd-valued columns of x.
    """
    assert x.shape[0] >= 3
    assert x.shape[1] >= 5
    last_row = x[-1]
    third_col = x[:, 2]
    first_two_rows_three_cols = x[:2, :3]
    even_rows_odd_cols = x[::2, 1::2] 
    out = (
        last_row,
        third_col,
        first_two_rows_three_cols,
        even_rows_odd_cols,
    )
    return out

节选自 umich CV assignment 1

Artificial Intelligence
#Artificial Intelligence #Deep Learning #Convolutional Neural Networks #AlexNet #Image Clssification
Deep_Learning_Memo
https://xiyuanyang-code.github.io/posts/Deep-Learning-Memo/
Author
Xiyuan Yang
Posted on
March 10, 2025
Updated on
May 18, 2025
Licensed under