Utilizing torch.index_select, torch.collect and torch.take
In some conditions, you’ll have to do some superior indexing / choice with Pytorch, e.g. reply the query: “how can I choose parts from Tensor A following the indices laid out in Tensor B?”
On this submit we’ll current the three commonest strategies for such duties, particularly torch.index_select, torch.collect and torch.take. We’ll clarify all of them intimately and distinction them with each other.
Admittedly, one motivation for this submit was me forgetting how and when to make use of which perform, ending up googling, shopping Stack Overflow and the, in my view, comparatively temporary and never too useful official documentation. Thus, as talked about, we right here do a deep dive into these capabilities: we encourage when to make use of which, give examples in 2- and 3D, and present the ensuing choice graphically.
I hope this submit will carry readability about mentioned capabilities and take away the necessity for additional exploration — thanks for studying!
And now, with out additional ado, let’s dive into the capabilities one after the other. For all, we first begin with a 2D instance and visualize the ensuing choice, after which transfer to considerably extra complicated instance in 3D. Additional, we re-implement the executed operation in easy Python — s.t. you may have a look at pseudocode as one other supply of data what these capabilities do. In the long run, we summarize the capabilities and their variations in a desk.
torch.index_select selects parts alongside one dimension, whereas maintaining the opposite ones unchanged. That’s: maintain all parts from all different dimensions, however decide parts within the goal dimensions following the index tensor. Let’s reveal this with a 2D instance, wherein we choose alongside dimension 1:
num_picks = 2values = torch.rand((len_dim_0, len_dim_1))
indices = torch.randint(0, len_dim_1, measurement=(num_picks,))
# [len_dim_0, num_picks]
picked = torch.index_select(values, 1, indices)
The ensuing tensor has form [len_dim_0, num_picks]: for each component alongside dimension 0, we’ve picked the identical component from dimension 1. Let’s visualize this:
