PyTorch - adjacent ()

Question

I was looking at this example of the LSTM language model on github (link) . What this does at all is pretty clear to me. But I'm still trying to figure out what the call is doing contiguous(), what happens in the code several times.

For example, on line 74/75 of the code, the target LSTM sequences are also entered. The data (stored in ids) is two-dimensional, where the first dimension is the packet size.

for i in range(0, ids.size(1) - seq_length, seq_length):
    # Get batch inputs and targets
    inputs = Variable(ids[:, i:i+seq_length])
    targets = Variable(ids[:, (i+1):(i+1)+seq_length].contiguous())

So, as a simple example, when using batch size 1 and seq_length10 inputsit targetslooks like this:

inputs Variable containing:
0     1     2     3     4     5     6     7     8     9
[torch.LongTensor of size 1x10]

targets Variable containing:
1     2     3     4     5     6     7     8     9    10
[torch.LongTensor of size 1x10]

So, in general, my question is: what does it mean contiguous()and why do I need it?

, , , , .

targets , inputs ?

EDIT: contiguous(), .

RuntimeError: invalid argument 1: input is not contiguous at .../src/torch/lib/TH/generic/THTensor.c:231

, , contiguous() .

( , GitHub.)

!

Answer 1

PyTorch Tensor, , . :

narrow(), view(), expand() transpose()

: transpose(), PyTorch , Tensor, . !

x = torch.randn(3,2)
y = torch.transpose(x, 0, 1)
x[0, 0] = 42
print(y[0,0])
# prints 42

. x , y , , . , "" , . , !

contiguous(), , , .

. PyTorch , , RuntimeError: input is not contiguous, contiguous().

Answer 2

Pytorch:

contiguous() → Tensor

Returns a contiguous tensor containing the same data as self 

. , self .

contiguous . , contiguous , , .

Answer 3

contigous() , , c c++,

Answer 4

tenor.contiguous() , . contiguous() , () , (). -, :

aaa = torch.Tensor( [[1,2,3],[4,5,6]] )
print(aaa.stride())
print(aaa.is_contiguous())
#(3,1)
#True

stride() return (3,1) , : ( ) 3 . ( ) 1 . , .

come :

bbb = aaa.transpose(0,1)
print(bbb.stride())
print(bbb.is_contiguous())

ccc = aaa.narrow(1,1,2)   ## equivalent to matrix slicing aaa[:,1:3]
print(ccc.stride())
print(ccc.is_contiguous())


ddd = aaa.repeat(2,1 )   # The first dimension repeat once, the second dimension repeat twice
print(ddd.stride())
print(ddd.is_contiguous())

## expand is different from repeat  if a tensor has a shape [d1,d2,1], it can only be expanded using "expand(d1,d2,d3)", which
## means the singleton dimension is repeated d3 times
eee = aaa.unsqueeze(2).expand(2,3,3)
print(eee.stride())
print(eee.is_contiguous())

fff = aaa.unsqueeze(2).repeat(1,1,8).view(2,-1,2)
print(fff.stride())
print(fff.is_contiguous())

#(1, 3)
#False
#(3, 1)
#False
#(3, 1)
#True
#(3, 1, 0)
#False
#(24, 2, 1)
#True

, , transpose(), small() , expand() . , repeat() view() . , : , ?

The answer is that the view () function cannot be applied to a continuous tensor. This is probably due to the fact that view () requires the tensor to be stored continuously so that it can quickly change shape in memory. eg:

bbb.view(-1,3)

we get the error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-63-eec5319b0ac5> in <module>()
----> 1 bbb.view(-1,3)

RuntimeError: invalid argument 2: view size is not compatible with input tensor size and stride (at least one dimension spans across two contiguous subspaces). Call .contiguous() before .view(). at /pytorch/aten/src/TH/generic/THTensor.cpp:203

To solve this problem, simply add contiguous () to the continuous tensor, create a continuous copy and then apply view ()

bbb.contiguous().view(-1,3)
#tensor([[1., 4., 2.],
        [5., 3., 6.]])