Soft DTW with ignore_padding_token

[anupsingh15]

Hello,

I have a batch of pairs of sequences. Each pair contains sequences of different lengths, which are padded to equal lengths. Is there a way to ignore these padded elements to compute the soft-dtw alignment? For example, https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html provides a feature to ignore a particular class index to compute cross-entropy loss.

Do you suggest any workaround to compute the dtw loss efficiently in such a case? I can only think of processing (removing paddings) each pair sample individually, but this will be too slow.

[YannCabanes]

Hello @anupsingh15,

For now I do not know how to help you to deal with the padding to compute the DTW loss efficiently.

In tslearn, the padding of a dataset can be done using the function to_time_series_dataset from the file:
https://github.com/tslearn-team/tslearn/blob/main/tslearn/utils/utils.py
The time series will be padded with nan values at the end of the time series.
For single time series, the nan values can be removed using the function to_time_series from the same file by setting the option remove_nans to True.

If you want, I can try to look for a better solution if you send in this conversation your codes with a small dataset detailing what you would like to do and your current solution.

[anupsingh15]

Hi @YannCabanes,

I have two sets, each containing N sequences, X = {x_i} and Y={y_i}, i=1,.., N. Each x_i and y_i are of different lengths lx_i and ly_i, respectively, but they are padded (left and right for my task) to the same lengths. Since tslearn.metrics.SoftDTWLossPyTorch apparently requires both x_i and y_i to be of the same lengths so I cannot simply use loss = soft_dtw_loss(X, Y).mean(). For now, I am computing the soft dtw loss for each sample in a batch using a for loop as:

for i in range(len(X)): 
    mask_x = # create mask to extract original x_i (no padding and length lx_i) 
    mask_y = # create mask to extract original y_i (no paddings and length ly_i)
    xi_unpadded = x[i][mask_x]
    yi_unpadded = y[i][mask_y]
    loss_i = self.soft_dtw_loss(xi_unpadded.unsqueeze(0), yi_unpadded.unsqueeze(0)).mean()/(xi_unpadded.shape[0]+yi_unpadded.shape[0]) # normalize the dtw loss with sum of lengths of sequences

This way of computing the loss over the batch is slow. I wonder if you can create a functionality which accepts the X and Y along with masks M_X and M_Y that indicates padding indices for each sequence x_i and y_i so that tslearn.metrics.SoftDTWLossPyTorch ignores the padded sequence elements when computing the loss.

[charavelg]

Maybe you could use a dedicated dist_func within tslearn.metrics.SoftDTWLossPyTorch . This parameter could be use to wrap masking somethink like:

def dist_fun(x, y):
     return SoftDTWLossPyTorch._euclidean_squared_dist(
         x[~torch.isnan(x)].reshape(x.shape[0], -1, x.shape[2]),
         y[~torch.isnan(y)].reshape(y.shape[0], -1, y.shape[2])
     )
SoftDTWLossPyTorch(dist_func=dist_fun)(X, Y)