# tutorial for using tensor board import torch import torchvision from torch.utils.tensorboard import SummaryWriter from torchvision import datasets, transforms
print(torch.__version__) print(f"Numbers of GPU available: {torch.cuda.device_count()}")
# Writer will output to ./runs/ directory by default writer = SummaryWriter()
transform = transforms.Compose( [transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))] ) trainset = datasets.MNIST( "data/mnist_train", train=True, download=True, transform=transform ) trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True) model = torchvision.models.resnet50(weights=None) # Have ResNet model take in grayscale rather than RGB model.conv1 = torch.nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False) images, labels = next(iter(trainloader))
The SummaryWriter class provides a high-level API to create an event file in a given directory and add summaries and events to it. The class updates the file contents asynchronously. This allows a training program to call methods to add data to the file directly from the training loop, without slowing down training.
Scalars
You can write Scalars into Tensor Board.
This is one of the most common things to log. You can use add_scalar() to record a single numerical value, such as a loss or accuracy, at each training step. This is great for tracking your model’s performance over time.
Adding scalars for single data
Parameters:
tag (str) – Data identifier
scalar_value (float or string/blobname) – Value to save
global_step (int) – Global step value to record
walltime (float) – Optional override default walltime (time.time()) with seconds after epoch of event
new_style (boolean) – Whether to use new style (tensor field) or old style (simple_value field). New style could lead to faster data loading.
# demo 2: generate a random tensor for simulation for step inrange(10): img_random = np.zeros((3, 100, 100)) for i inrange(3): img_random[i] = np.random.random((100, 100)) writer.add_image("random_image", img_random, step)
# If you have non-default dimension setting, set the dataformats argument. writer.add_image("my_image_HWC", img_HWC, 0, dataformats="HWC") writer.close()
Add multiple images
Add batched image data to summary.
Note that this requires the pillow package.
Parameters
tag (str) – Data identifier
img_tensor (torch.Tensor, numpy.ndarray, or string/blobname) – Image data
global_step (int) – Global step value to record
walltime (float) – Optional override default walltime (time.time()) seconds after epoch of event
dataformats (str) – Image data format specification of the form NCHW, NHWC, CHW, HWC, HW, WH, etc.
Compared to add_image, it will add one dimension for batches, where the image_tensor is (N,3,H,W).
1 2 3 4 5 6 7 8 9 10 11 12
from torch.utils.tensorboard import SummaryWriter import numpy as np
img_batches = np.zeros((16, 3, 100, 100))
for step inrange(16): for channel inrange(3): img_batches[step, channel] = ( np.arange(0, 10000).reshape(100, 100) / 10000 / 16 * step )
writer.add_images("test for batches", img_batches, 1, dataformats="NCHW")
add_figure(tag, figure, global_step=None, close=True, walltime=None): Render matplotlib figure into an image and add it to summary.
Parameters
tag (str) – Data identifier
figure (Union[Figure, list[‘Figure’]]) – Figure or a list of figures
global_step (Optional[int]) – Global step value to record
close (bool) – Flag to automatically close the figure
walltime (Optional[float]) – Optional override default walltime (time.time()) seconds after epoch of event
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
import matplotlib.pyplot as plt
writer = SummaryWriter("test_dir/figure")
# generate a matplotlib image fig, ax = plt.subplots() x = np.linspace(0, 2 * np.pi, 100) y = np.cos(x) ax.plot(x, y) ax.set_title("Cos(x)")
writer.add_figure("cos_plot", fig, global_step=0)
plt.close(fig) writer.close()
Text
Parameters
tag (str) – Data identifier
text_string (str) – String to save
global_step (int) – Global step value to record
walltime (float) – Optional override default walltime (time.time()) seconds after epoch of event
1 2 3
writer_3 = SummaryWriter("test_dir/test_for_text") writer_3.add_text("tag-1", "Hello world") writer_3.add_text("tag-2", "wow, this is fanastic!")
Graph
add_graph(model, input_to_model=None, verbose=False, use_strict_trace=True)[source][source] Add graph data to summary.
Parameters
model (torch.nn.Module) – Model to draw.
input_to_model (torch.Tensor or list of torch.Tensor) – A variable or a tuple of variables to be fed.
verbose (bool) – Whether to print graph structure in console.
use_strict_trace (bool) – Whether to pass keyword argument strict to torch.jit.trace. Pass False when you want the tracer to record your mutable container types (list, dict)
mat (torch.Tensor or numpy.ndarray) – A matrix which each row is the feature vector of the data point, its size is (N,D), where N is the number of data and D is the feature dimension.
metadata (list) – A list of labels, each element will be converted to string
label_img (torch.Tensor) – Images correspond to each data point, (N,C,H,W).
global_step (int) – Global step value to record
tag (str) – Name for the embedding
metadata_header (list) – A list of headers for multi-column metadata. If given, each metadata must be a list with values corresponding to headers.
Plotting a precision-recall curve lets you understand your model’s performance under different threshold settings. With this function, you provide the ground truth labeling (T/F) and prediction confidence (usually the output of your model) for each target. The TensorBoard UI will let you choose the threshold interactively.
The visualization is based on Three.js, so it allows users to interact with the rendered object. Besides the basic definitions such as vertices, faces, users can further provide camera parameter, lighting condition, etc. Please check https://threejs.org/docs/index.html#manual/en/introduction/Creating-a-scene for advanced usage.
Parameters
tag (str) – Data identifier
vertices (torch.Tensor) – List of the 3D coordinates of vertices. (顶点坐标)
colors (torch.Tensor) – Colors for each vertex. (顶点颜色)
faces (torch.Tensor) – Indices of vertices within each triangle. (Optional) (面片索引)
config_dict – Dictionary with ThreeJS classes names and configuration.
global_step (int) – Global step value to record
walltime (float) – Optional override default walltime (time.time()) seconds after epoch of event
import torch from torch.utils.tensorboard import SummaryWriter
vertices_tensor = torch.as_tensor( [ [1, 1, 1], [-1, -1, 1], [1, -1, -1], [-1, 1, -1], ], dtype=torch.float, ).unsqueeze(0) # print(vertices_tensor.shape): (1,4,3), where 1 is the batch size and 4 is the number of vertices
colors_tensor = torch.as_tensor( [ [255, 0, 0], [0, 255, 0], [0, 0, 255], [255, 0, 255], ], dtype=torch.int, ).unsqueeze(0) # print(colors_tensor.shape): (1,4,3), the same as above
Add a set of hyperparameters to be compared in TensorBoard.
hparam_dict (dict) – Each key-value pair in the dictionary is the name of the hyper parameter and it’s corresponding value. The type of the value can be one of bool, string, float, int, or None.
metric_dict (dict) – Each key-value pair in the dictionary is the name of the metric and it’s corresponding value. Note that the key used here should be unique in the tensorboard record. Otherwise the value you added by add_scalar will be displayed in hparam plugin. In most cases, this is unwanted.
hparam_domain_discrete – (Optional[Dict[str, List[Any]]]) A dictionary that contains names of the hyperparameters and all discrete values they can hold
run_name (str) – Name of the run, to be included as part of the logdir. If unspecified, will use current timestamp.
global_step (int) – Global step value to record
1 2 3 4 5 6 7 8
from torch.utils.tensorboard import SummaryWriter
with SummaryWriter("test_dir/hparams") as w: for i inrange(5): w.add_hparams( {"lr": 0.1 * i, "bsize": i}, {"hparam/accuracy": 10 * i, "hparam/loss": 10 * i}, )
Demo
In this section, we will train a complex model (CNN + LSTM + Transformer), and use SummaryWriter to log loss curve and other parameters.
import torch import torch.nn as nn import torch.optim as optim from torch.utils.tensorboard import SummaryWriter from torch.utils.data import DataLoader, TensorDataset import numpy as np