Question: import torch from torch.distributions import constraints from torch.distributions.categorical import Categorical <del> from torch.distributions.utils import clamp_probs, broadcast_all, log_sum_exp </del> <ins> from torch.distributions.utils import clamp_probs, broadcast_all, _log_sum_exp </ins> from torch.distributions.distribution import Distribution from torch.distributions.transformed_distribution import TransformedDistribution from torch.distributions.transforms import ExpTransform

Leaving this issue here for higher visibility. Originally reported by here: Also mentioned by Piggybacking off of this issue, has strange behavior (reported here: )
I have fixed this locally. Could I send a PR?
Yes, go ahead!


Question: begin{array}{ll} i_t = sigma(W_{ii} x_t + b_{ii} + W_{hi} h_{(t-1)} + b_{hi})  f_t = sigma(W_{if} x_t + b_{if} + W_{hf} h_{(t-1)} + b_{hf})  <del> g_t = tanh(W_{ig} x_t + b_{ig} + W_{hc} h_{(t-1)} + b_{hg})  </del> <ins> g_t = tanh(W_{ig} x_t + b_{ig} + W_{hg} h_{(t-1)} + b_{hg})  </ins> o_t = sigma(W_{io} x_t + b_{io} + W_{ho} h_{(t-1)} + b_{ho})  c_t = f_t c_{(t-1)} + i_t g_t  h_t = o_t tanh(c_t)

Hi, Looking at the formulas of LSTM in , in the third equation (gt), the suffix of the second W should be changed from W{hc} to W{hg}. The correct formula: gt=tanh(W{ig}xt+b{ig}+W{hg}h{(t−1)}+b_{hg}) Cheers, Navid


Question: tmp3 = torch.cuda.FloatTensor(t.size()) self.assertEqual(tmp3.data_ptr(), ptr[0], 'allocation not re-used') <ins> def test_noncontiguous_pinned_memory(self): # See issue #3266 x = torch.arange(0, 10).view((2, 5)) self.assertEqual(x.t(), x.t().pin_memory()) </ins> def test_caching_pinned_memory(self): cycles_per_ms = get_cycles_per_ms()

Hello, I was experimenting with a custom which receives a batch of different sized integer sequences and pads them to form 2D tensors. I noticed a weird issue for which I created the following MWE: It's probably because of the different memory ordering on Python's side and CUDA's allocator. Tested on K40 and GTX1080 with driver 375.39, CUDA8.
Pin memory is missing a call inside. Should be an easy fix
fixed via


Question: - [Releases and Contributing](#releases-and-contributing) - [The Team](#the-team) <del> | Python |  **`Linux CPU`**   |  **`Linux GPU`** | |--------|--------------------|------------------| | 2.7.8  | [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | | | 2.7    | [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py2)](https://build.pytorch.org/job/pytorch-master-py2)  | | 3.5    | [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py3)](https://build.pytorch.org/job/pytorch-master-py3)  | | Nightly| [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | | </del> <ins> | System | Python | Status | | --- | --- | --- | | Linux CPU | 2.7.8, 2.7, 3.5, nightly | [![Build Status](https://travis-ci.org/pytorch/pytorch.svg?branch=master)](https://travis-ci.org/pytorch/pytorch) | | Linux GPU | 2.7 | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py2)](https://build.pytorch.org/job/pytorch-master-py2) | | Linux GPU | 3.5 | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py3)](https://build.pytorch.org/job/pytorch-master-py3) | </ins> ## More about PyTorch

We should explore using TorchDynamo to capture/optimize optimizer code. I don't anticipate any major issues here, but we should make sure it plays nice with AOT Autograd. cc
linking issue related to performance of this:


Question: ]] [[ <del> name: free return: self arguments: - THTensor* self ]] [[ name: retain return: self arguments: - THTensor* self ]] [[ </del> python_name: index name: THPTensor_(getValue)<true> only_register: True

Dataloader call getitem as many times as indices in the current batch. In case datasets support a list of indices in one call, or a native python slice object, add a getbatch (optional) to pytorch dataset class. If dataloader sees the dataset has such a method implemented, make it fetch the batch corresping to the list of indices with one getbatch call rather than many getitem calls. This saves the trouble of calling getitem many times in cases where for example a disk file has to be opened then closed for every call and the underlying data structure supports a slicing or lists of indices for example hdf5 files.
will support something similar
is the latest version of this.


Question: class InstanceNorm2d(_InstanceNorm): r"""Applies Instance Normalization over a 4d input that is seen as a mini-batch of 3d inputs .. math:: y = frac{x - mean[x]}{ sqrt{Var[x]} + epsilon} * gamma + beta The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. Gamma and beta are learnable parameter vectors of size C (where C is the input size).

why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:


Question: The upper triangular part of the matrix is defined as the elements on and above the diagonal. <del> The argument :attr:`k` controls which diagonal to consider. </del> <ins> The argument :attr:`diagonal` controls which diagonal to consider. </ins> <del> - :attr:`k` = 0, is the main diagonal. - :attr:`k` > 0, is above the main diagonal. - :attr:`k` < 0, is below the main diagonal. </del> <ins> - :attr:`diagonal` = 0, is the main diagonal. - :attr:`diagonal` > 0, is above the main diagonal. - :attr:`diagonal` < 0, is below the main diagonal. </ins> Args: input (Tensor): the input `Tensor` <del> k (int, optional): the diagonal to consider </del> <ins> diagonal (int, optional): the diagonal to consider </ins> out (Tensor, optional): The result `Tensor` Example::

The states that the argument controls which diagonal to consider. However, this argument is actually named in pytorch 0.2.0.


Question: If :attr:`upper` is ``False``, then lower triangular portion is used. <ins> .. note:: The eigenvalues are returned in ascending order. If :attr:`input` is a batch of matrices, then the eigenvalues of each matrix in the batch is returned in ascending order. </ins> .. note:: Irrespective of the original strides, the returned matrix `V` will be transposed, i.e. with strides `V.contiguous().transpose(-1, -2).stride()`.

Documentation for does not indicate whether or not singular values are returned in descending order. My suspicion is that they are since both LAPACK and MAGMA's ?gesdd return them in descending order. It seems like this should be clarified in the docs to avoid unnecessary sorting, or mistaken assumptions. <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --


Question: embedding_dim (int): the size of each embedding vector max_norm (float, optional): If given, will renormalize the embeddings to always have a norm lesser than this norm_type (float, optional): The p of the p-norm to compute for the max_norm option <del> scale_grad_by_freq (boolean, optional): if given, this will scale gradients by the frequency of </del> <ins> scale_grad_by_freq (bool, optional): if given, this will scale gradients by the frequency of </ins> the words in the dictionary. mode (string, optional): 'sum' | 'mean'. Specifies the way to reduce the bag. Default: 'mean' <del> sparse (boolean, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </del> <ins> sparse (bool, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </ins> more details regarding sparse gradients. Attributes:

I have a straightforward use case. Given a token (or word), lookup the embedding of the word, pass that through a linear layer and perform L2 regression. If the token corresponds to the of the embedding layer, throws an error saying OS: macOS High Sierra PyTorch version: 0.3.1 How you installed PyTorch (conda, pip, source): Tried both conda and pip Python version: 3.6.4 CUDA/cuDNN version: CPU GPU models and configuration: NA GCC version (if compiling from source): NA A script to reproduce the bug:
I got the same trouble, and it throw this error after some epochs of training. Does anyone have some solutions?


Question: g = tanh(W_{ig} x + b_{ig} + W_{hc} h + b_{hg})  o = mathrm{sigmoid}(W_{io} x + b_{io} + W_{ho} h + b_{ho})  c' = f * c + i * g  <del> h' = o * tanh(c_t)  </del> <ins> h' = o * tanh(c')  </ins> end{array} Args:

When converting a tensor to dlpack, the strides of single-value tensors () should be normalized to . This is the fallout conclusion from . Here is a demo of what is wrong: Collecting environment information... PyTorch version: 1.10.0a0+git2aa19f3 Is debug build: False CUDA used to build PyTorch: Could not collect ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.4 LTS (x8664) GCC version: (GCC) 9.4.0 Clang version: 12.0.1 ( ) CMake version: version 3.21.1 Libc version: glibc-2.31 Python version: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] (64-bit runtime) Python platform: Linux-5.4.0-122-generic-x8664-with-glibc2.31 Is CUDA available: False CUDA runtime version: 11.2.152 GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1660 SUPER Nvidia driver version: 470.141.03 cuDNN version: /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: False Versions of relevant libraries: [pip3] mypy==0.812 [pip3] mypy-extensions==0.4.3 [pip3] numpy==1.21.2 [pip3] torch==1.10.0a0+git2aa19f3 [pip3] torchaudio==0.10.0a0+ [conda] magma-cuda112 2.5.2 1 pytorch [conda] mkl 2021.3.0 h726a3e6557 conda-forge [conda] mkl-include 2021.3.0 h726a3e6557 conda-forge [conda] numpy 1.21.2 py39hdbf815f0 conda-forge [conda] torch 1.12.0a0+git1f29b31 dev0 <develop[conda] torchaudio 0.10.0a0+ pypi0 pypi
does this capture the problem?


Question: def calculate_gain(nonlinearity, param=None): """Return the recommended gain value for the given nonlinearity function. The values are as follows: ============ ========================================== nonlinearity gain ============ ==========================================

why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:


Question: def index_select(self, dim, index): return IndexSelect(dim)(self, index) <ins> def gather(self, dim, index): return Gather(dim)(self, index) def scatter(self, dim, index, source): return Scatter(dim)(self, index, source) def scatter_(self, dim, index, source): return Scatter(dim, True)(self, index, source) </ins> def masked_copy(self, mask, variable): return MaskedCopy()(self, mask, variable)

<!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --In the getting started section section the following quote appears: I guess this is from previous versions where CUDA support was not built-in and this can be removed, correct? cc
This seems to be easy fixable docs issue
If gets merged this is fixed.
Thanks for the PR Just merged it.


Question: # Example of using Container class Net(nn.Container): def __init__(self): <del> super(Net, self).__init__( conv1 = nn.Conv2d(1, 20, 5), relu  = nn.ReLU() ) </del> <ins> super(Net, self).__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.relu  = nn.ReLU() </ins> def forward(self, input): output = self.relu(self.conv1(x)) return output

When I enable the MPS backend on Mac in libtorch, it now crashes with this error when exiting: If I remove: the crash disappears. It only affects exiting/destructing the app. Most likely it is the MPS memory allocator closing down. nightly 1st August 2022. Backtrace (basically it is torch::jit::Module destructor, leading to C10::StorageImpl destructor): cc
, can you please provide a small test-case to repro the issue ?
I'me using next C++ code, but can't reproduce this issue void toTest() { torch::Tensor inputTensor = torch::randn({1, 3, 256, 256}); (torch::kMPS); } Tested with the latest mps_graph branch build. Could you please provide a minimal reproducible test?
The crash is in torch::jit::Module so you'd likely need to be running a forward. I'll try to get a reproduce. It's not happening in Python which makes it a bit harder.
I've tried to reproduce with a Module, for that i've created test module using code for resnet18 from this sample And next created next C++ code std::vector<torch::jit::IValueinputs; inputs.pushback(torch::ones({1, 3, 224, 224}).to(torch::kMPS)); at::Tensor output = module.forward(inputs).toTensor().to(torch::kCPU); std::cout << (1,0, 5) << 'n'; With this code I see no issues, it runs forward and prints expected output: -1.2350 -0.0649 0.2181 0.0655 -0.2242 [ CPUFloatType{1,5} ]
, removing the "High priority" label, as we are not able to reproduce the issue. Please provide us a failing case and we will take a look.
We weren't able to reproduce this issue, so I'll close it. Please reopen the issue if you see again. Thanks.


Question: # (3) initialize mean square values and square gradient storage if not 'm' in state: <del> state['m'] = x.new().resize_as_(dfdx).fill_(1) </del> <ins> state['m'] = x.new().resize_as_(dfdx).zero_() </ins> state['tmp'] = x.new().resize_as_(dfdx)

We should unify this with our normal list creating code for the reasonable cases cc
Similar issue to this, I believe:

The hub tests are broken on macOS. I'm seeing SSL certificate related errors when trying to hit GitHub. Run the macOS tests on CI with TestHub enabled. I tried installing the latest version of from conda-forge (which pulled in ) but this didn't fix the issue. cc
Looks like the same error I'm getting trying to use Ubuntu 18.04 with fresh install of conda and Pytorch 1.2
I think this will all be solved with .

Porting TH operators is essential for code simplicity and performance reasons. Porting guides and Q&A are available in umbrella issue: Feel free to add as a reviewer to get a prioritized review.

This is of particular use in list comprehensions: This should work:
I'll need to adapt list comprehensions to use Iterables. I'll take a look!

I haven't looked into this in depth but this looks fishy: More here: cc
is this still an issue in 1.3.1?
I can repro this on master:
The problem is: removes the parameters from the module's parameter list and makes them normal class attributes: applies only to parameters, not to the weight attributes.  (which is called from acts on which includes the value of the weight attributes and the bias parameters. So, it contains a mix of CUDA and CPU tensors.
This is my workaround for right now: This applies weight norm to an rnn module and overrides the function to deal with the error.
This appears to be fixed. do you want to verify?
I tested this on nightly and it worked for me, so closing.

As mentioned in title, I want to make sure if pytorch can deal with the situation that each input in a batch has different shapes and thus different computation graphs.
Please don't open duplicated issue for discussion. Context will be missed. Try to discuss about it here:

Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting I can work on this if this can be to pytorch. Please let me know. Thanks!
This should be fairly straightforward to add in user code using . Eventually it could be to core, but its utility should probably be validated in external repos first.


Question: Example:: >>> tensor = torch.tensor((), dtype=torch.float64) <del> >>> tensor.new_ones((2, 3)) tensor([[ 1.,  1.,  1.], [ 1.,  1.,  1.]], dtype=torch.float64) </del> <ins> >>> tensor.new_zeros((2, 3)) tensor([[ 0.,  0.,  0.], [ 0.,  0.,  0.]], dtype=torch.float64) </ins> """.format(**new_common_args))

The example given in the docs for newzeros() is actually an example for newones(). Should be a trivial fix. (Sorry, not quite sure what the etiquette/process is for fixing small errors like these). Thanks. The example should read:
Thanks for the catch, If you'd like to take a stab at fixing it, you can fork the pytorch repo and contribute a pull request. Otherwise, I can make the replacement.
fixed via


Question: return obj; } <ins> PyObject * THSPTensor_(NewEmpty)() { THSTensorPtr tensor(THSTensor_(new)(LIBRARY_STATE_NOARGS)); THSTensor_(initStorage)(tensor.get()); return THSPTensor_(New)(tensor.release()); } </ins> static void THSPTensor_(dealloc)(THSPTensor* self) { if (self->cdata)

This causes to not work properly on newly created empty sparse tensors. In the example below, both should print 1 instead.


Question: <ins> import os import sys import glob import platform import warnings from itertools import chain from .env import check_env_flag from .cuda import WITH_CUDA, CUDA_HOME def gather_paths(env_vars): return list(chain(*(os.getenv(v, '').split(':') for v in env_vars))) is_conda = 'conda' in sys.version or 'Continuum' in sys.version conda_dir = os.path.join(os.path.dirname(sys.executable), '..') WITH_NCCL = WITH_CUDA and platform.system() != 'Darwin' WITH_SYSTEM_NCCL = False NCCL_LIB_DIR = None NCCL_INCLUDE_DIR = None NCCL_ROOT_DIR = None if WITH_CUDA and not check_env_flag('NO_SYSTEM_NCCL'): ENV_ROOT = os.getenv('NCCL_ROOT_DIR', None) # NCCL_ROOT_DIR takes precedence over NCCL_LIB_DIR lib_paths = list(filter(bool, [ os.path.join(ENV_ROOT, 'lib') if ENV_ROOT is not None else None, os.path.join(ENV_ROOT, 'lib64') if ENV_ROOT is not None else None, os.getenv('NCCL_LIB_DIR'), os.path.join(CUDA_HOME, 'lib'), os.path.join(CUDA_HOME, 'lib64'), '/usr/lib/x86_64-linux-gnu/', '/usr/lib/powerpc64le-linux-gnu/', '/usr/lib/aarch64-linux-gnu/', ] + gather_paths([ 'LIBRARY_PATH', ]) + gather_paths([ 'LD_LIBRARY_PATH', ]))) if os.getenv('NCCL_INCLUDE_DIR') is not None: warnings.warn("Ignoring environment variable NCCL_INCLUDE_DIR because " "NCCL_INCLUDE_DIR is implicitly assumed as " "$NCCL_ROOT_DIR/include or $NCCL_LIB_DIR/../include") if is_conda: lib_paths.append(os.path.join(conda_dir, 'lib')) for path in lib_paths: if path is None or not os.path.exists(path): continue if glob.glob(os.path.join(path, 'libnccl*')): if os.path.exists((os.path.join(path, '../include/nccl.h'))): NCCL_LIB_DIR = path NCCL_INCLUDE_DIR = path break if NCCL_LIB_DIR is not None: WITH_SYSTEM_NCCL = True NCCL_ROOT_DIR = os.path.abspath(os.path.join(NCCL_LIB_DIR, "../")) </ins>

Currently, you can have a libnccl on your system, and pytorch setup favors that instead of compiling one itself from source. The code that takes care of this is: However, this breaks gloo build because gloo gets a path to NCCL via:
I got the error during the installation: Is it related to this problem?
that's a dupe of
no, that's a consequence of , now you have to do before compiling. I think you don't want to do recursive submodule update, because that will pull in gtest and another copy of nccl.
Alternatively we can pass new flags to gloo to prevent it from using its own NCCL
Thanks works for me.
You can prevent it from using its own nccl, but its cmake still has to be able to find some nccl.
I think should be sufficient. I'm testing.
fixes it when we dont have an external NCCL and pytorch compiles it's own NCCL?
I don't see how NCCLEXTERNAL fixes anything, looks like in gloo NCCLEXTERNAL is used just to set dependency if gloo is compiling its own nccl (which we don't want in any case).
The only thing I tested when I was that the vendored nccl was not being built (even if its submodule was checked out); I tested this by adding an to one of the headers. I'll test with the other configurations.
Actually, there are a few bugs here. First, even though GLOOFLAGS looks like it's setting NCCLROOTDIR: it actually isn't because the gloo invocation is which expands to but only is passed to cmake. Suppose you fix this. nccl is still not going to be picked up, because INSTALLDIR is PyTorch's local install dir, but if you disabled nccl builds, there will never actually be a nccl here. So cmake is just going to do it's standard search to find a nccl somewhere. Inversely, if you ever built nccl, and then turn it off, we will keep picking up that locally built version until you delete it. So we shouldn't do that. Suppose you fix this.

Then I think gloo will (continue to) correctly pick up system nccl and be built with it, and the final thing to fix is that build of PyTorch csrc doesn't know how to search for nccl headers somewhere. This is probably related to . I wonder, maybe we shouldn't do any autodetection at all, and force the user to pass us the include and library paths of nccl if they want to build with system.
FYI, I committed a change to gloo to stop bundling NCCL itself. It will now happily pick up whatever NCCL you give it through NCCLROOTDIR or by trying to find it in the standard systems paths. See facebookincubator/gloo You do have to set to make Gloo try and find it and compile in support.
So to fix this, you should be able to not specify anything aside from if you want to use system NCCL, and add if you have some custom location (either a PyTorch bundled version or a custom install altogether).


Question: | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides <del> for :attr:`padding` number of points | :attr:`dilation` controls the spacing between the kernel points. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. </del> <ins> for :attr:`padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. </ins> |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. <ins> At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> .. note::

Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

When building from source, I got this error. My system is OSX 10.12.4, conda, CUDA 8.0. Any thoughts? Thanks!
can you try adding to the file and see if that fixes things? if so i'll commit it.
actually, that file already has memory included. Are you following instructions from Especially: This part is important:
works nicely! Thanks a lot!

The suggest that takes a as a parameter, but it actually must be wrapped in a Variable first.
you are looking at the Tensor docs. When operating on Variable, it is implied from the tutorials that all functions operating on Variables have to use Variables. We didn't write separate docs for Variable itself.

something wrong with import torchvision Traceback (most recent call last): File "", line 3, in <moduleimport torchvision File "", line 4, in <moduleimport torchvision.transforms as transforms ImportError: No module named transforms
you have a file called for some reason. delete it or change your pythonpath to ignore it.
Hey, I have the same problem persisting when I import torchvision. I have no other file by the same name. ImportError Traceback (most recent call last) <ipython-input-6-3baf6837d8f1in <module() 8 from import Dataset 9 from import DataLoader ---10 from torchvision import transforms ImportError: No module named torchvision Any idea why this is happening?
Do you have torchvision installed (it is a separate package)?
pip Install torchvision try this
from data import BaseTransform,VOC_CLASSES as labelmap this is not working
pip install torchvision

Hi, I need to pass input through one nn.Module, then argmax the output, and then pass that output to a second nn.Module. I want to backprop through the argmax back to the weights of the first module. That's impossible right now, though, it seems. When I loop through the gradient of the weights of my first module (with the following code): firstmodel.zerograd() secondmodel.zerograd() loss.backward() # retainvariables=True for param in secondmodel.parameters(): print( 'data: ', ) print( 'grad data: ', ) for param in firstmodel.parameters(): print( 'data: ', ) print( 'grad data: ', ) I get the following error, stating that those gradients can no longer be viewed/exist: -main()
argmax is not differentiable (and even if you try to think about it as a function differentiable almost everywhere, its derivative is always 0, so you won't get anything meaningful).

def inithidden(self): h0 = Variable(FloatTensor(self.numlayers, self.batchlen, self.hiddendim).zero(),  requiresgrad=True) c0 = Variable(FloatTensor(self.numlayers, self.batchlen, self.hiddendim).zero(),  requiresgrad=True) return (h0, c0) def forward(self, sequences): lstmout, = ((1, self.batchlen, self.inputdim), ) ... = nn.LSTM(inputdim, hiddendim, numlayers, dropout = dropout, batchfirst=True) = self.inithidden() output = model(traininput) produces the following error although I defined weight and input dimensions as in the documentation: File "/usr/local/lib/python2.7/dist-", line 202, in call result = self.forward(input, kwargs) File "/usr/local/lib/python2.7/dist-", line 91, in forward output, hidden = func(input, self.allweights, hx) File "/usr/local/lib/python2.7/dist-", line 327, in forward return func(input, fargs, fkwargs) File "/usr/local/lib/python2.7/dist-", line 227, in forward nexth, output = func(input, hidden, weight) File "/usr/local/lib/python2.7/dist-", line 67, in forward hy, output = inner(input, hidden[l], weight[l]) File "/usr/local/lib/python2.7/dist-", line 96, in forward hidden = inner(input[i], hidden, *weight) File "/usr/local/lib/python2.7/dist-", line 22, in LSTMCell gates = F.linear(input, wih, bih) + F.linear(hx, whh, bhh) File "/usr/local/lib/python2.7/dist-", line 744, in add return (other) File "/usr/local/lib/python2.7/dist-", line 293, in add return (other, False) File "/usr/local/lib/python2.7/dist-", line 287, in add return Add(inplace)(self, other) File "/usr/local/lib/python2.7/dist-", line 20, in forward return a.add(b) RuntimeError: inconsistent tensor size at /data/users/soumith/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensorMath.c:827
I can't reproduce your problem. Could you include a short runnable example that exhibits this behavior?

You also don't need to manually init hidden anymore, the following would work as well...
Thanks! After removing problem solved.

Hi, I would like to implement a sharing weight 'alexnet' with two input batches of images, and the fully connected layers are concatenated together like this: However, when I run it, I come across the following error: Does anyone know about it? Thank you very much.
You forgot to call on the output of . In the future please remember that we're using the issues for bug reports and feature requests only. If you have questions or need help, please post .


Question: for module in (nn.RNN, nn.LSTM, nn.GRU): for bias in (True, False): for bidirectional in (False, True): <del> for dropout in (0, 1):  # Because of dropout randomness, can only compare 0 and 1 for batch_first in (False, True): num_directions = 2 if bidirectional else 1 if batch_first: input_val = torch.randn(batch, seq_length, input_size) else: input_val = torch.randn(seq_length, batch, input_size) hx_val = torch.randn(num_layers * num_directions, batch, hidden_size) rnn = module(input_size, </del> <ins> for batch_first in (False, True): num_directions = 2 if bidirectional else 1 if batch_first: input_val = torch.randn(batch, seq_length, input_size) else: input_val = torch.randn(seq_length, batch, input_size) hx_val = torch.randn(num_layers * num_directions, batch, hidden_size) rnn = module(input_size, hidden_size, num_layers, bias=bias, dropout=dropout, bidirectional=bidirectional, batch_first=batch_first) outputs_cpu = forward_backward( False, rnn, input_val, hx_val, rnn.all_weights) rnn_gpu = module(input_size, </ins> hidden_size, num_layers, bias=bias,

cuDNN backend for RNN stabilizes at large mem usage. The following code stabilizes for me at 14+GiB reported by nvidia-smi. The same RNN without cuDNN uses <2.5GiB.
Oops... any idea what's going on with the garbage collector in this case?
are you running this on a P100 GPU?
Yes, I observe similar behavior on Titan X Pascal
hmmm, do cudnn RNNs have weird memory-oversubscribing properties? If it wasn't for CuDNN RNN, then it's not unreasonable. we use a custom CUDA allocator that will oversubscribe if needed, but will free the memory blocks it's holding onto if it hits OOM / failed allocations.
Tried the same RNN in torch and am getting ~2.6GiB on nvidia-smi.
With cuDNN I'm getting a constant usage of 2241MB on my machine using PyTorch (M40).
are you using cudnnv5, or v6?
v5
no, cudnn does not do memory oversubscribing, and already tried the same network in lua torch with cudnn, where it gets expected memory usage. Also, he is getting OOMs on slightly bigger networks, where apparently caching allocator should have tried to free previous blocks. Interesting that can not repro this.
Just to make sure did you set when you ran? I left it as False in the repro.
I can repro in the docker image built from the current version on P100, and I go OOM after 8 batches with the same image on 1080, which generally should not happen - if there is enough memory to go through 1 batch, memory usage should not increase afterwards.
No, I just copied your script and ran it. Weird thing is that it goes OOM after 18 batches once I enable . Without it memory remains stable.
A lot of cudaMalloc allocations go not through THCudaMalloc, and never get freed:
So not going through THCudaMalloc is actually expected for storages allocated with THCStorage_(newWithSize), but what's not expected is to every time cudaMalloc bytes of rnn workspace, instead of reusing earlier allocations. (~784 MB allocations are for workspace, ~156MB allocations happen somewhere in criterion).
My guess is that the bug is at the Python level.

Either: Something is holding on to a Tensor longer than it should Something is creating a circular reference which prevents quick clean-up
The issue is that in creates cyclic references. It's made worse by the fact that it holds on to more memory than it should (like the workspace). You can see this by adding a as the first line in your for loop. With the explicit call to the GC every iteration, the memory usage doesn't explode.
how can these cyclic references be fixed?
Not sure yet, but I will fix it tomorrow with


Question: return type(self), (self.tolist(),) def __repr__(self): <del> return repr(str(self)) </del> <ins> return str(self) </ins> def __str__(self): # All strings are unicode in Python 3, while we have to encode unicode

In torch, printing a tensor will print the whole tensor, even if that one is huge. This is sometimes annoying, especially when using a notebook, as it crashes it. Numpy has a very nice way to display big tensors by truncating them. Would it be possible so have something similar for pytorch?
Yeah, we haven't implemented it yet, but that's definitely on our roadmap
Implemented in .
I should say that I have some artefacts that appear with while printing a tensor. Here is an example output: gives Curiously, if I simply do It works fine and print nicely, with the truncation for large tensors. I'm using Python 2.7.6 (probably the one that came with my system and not from anaconda). Maybe it's just a bad setup in my machine?
hmmm, it works fine on python 2.7.12. wonder what changed between .6 and .12
ok i got a repro. I opened my ipython and did this:
my example was flawed, sorry. Using works for me, but when only entering it prints the s, as in your example.
Is it possible to tell the truncation point?
yes, use the function


Question: ] if WITH_CUDNN: main_libraries += ['cudnn'] <del> library_dirs.append(CUDNN_LIB_DIR) </del> <ins> library_dirs.insert(0, CUDNN_LIB_DIR) </ins> # NOTE: these are at the front, in case there's another cuDNN in CUDA path include_dirs.insert(0, CUDNN_INCLUDE_DIR) if not IS_WINDOWS:

If there was two libcudnn files ( and ) in the folder wouldn't it result in taking the older version ()? Maybe adding a warning or something to insure its the right lib?
I think and are quite possibly related. EDIT: also

Compilation goes without a problem but importing torch gives ImportError: /lib/python3.6/site-packages/torch/C.cpython-36m-x8664-linux- undefined symbol: cudnnSetConvolutionGroupCount Compiling from source v0.2.0 works fine. Set up is Python 3.6.2 :: Anaconda custom (64-bit) Cuda compilation tools, release 8.0, V8.0.61 gcc (GCC) 4.8.5 centos-release-scl-rh-2-
It's likely a configuration issue of your system. You built with cuDNN7, but cuDNN6 is loaded at runtime. Can you please run and paste the output in a gist?
OK, here's the .
That's weird. It doesn't even seem to be linked. Can you post the output of ? Did you build from source? Can you clean the repo ( ASSUMING YOU DID NO CHANGES YOURSELF - THIS MIGHT DESTROY THEM), build form scratch, and post the build log in a gist as well?
Hey Adam, I made a mistake. This is the correct . And the ldd just in case.
Ok. cuDNN isn't linked. How did you install PyTorch?
Compiled from source, following the instructions.
Please rebuild and post the build log
Done, here's the .
The build log looks correct. I think the problem is that you have multiple Python environments with different PyTorch packages. In your run time log, torch is loaded from , but in your build log it's installed in . Cleaning up the global install should fix it for you.
Ok, so I made sure pytorch wasn't installed in the global python. Checked & removed from the Anaconda . Now the only shows Still, get Again, runtime and the of the from the error
Eh, I'm running out of tricks. Maybe there's sth wrong with the compiler? you can see that there's a flag when building . Try prepending before the build command
The sysadmin installed both cuDNN 6.0 and 7.0 and that's causing the problem. I followed but still found it necessary to insert CUDNNLIBDIR at the beginning of the list with .

Now, after compiling the import doesn't throw any errors, so I'm guessing it's ok.
hmm yeah, it should also go in the front of . Can you please submit a PR with the fix?
Will do
i believe this patch in the second last comment has to be adapted for your refactor.


Question: return output @staticmethod <del> @once_differentiable </del> def backward(ctx, grad_output, grad_indices=None): <del> grad_input = grad_output.new(ctx.input_size).zero_() </del> <ins> grad_input = Variable(grad_output.data.new(ctx.input_size).zero_(), requires_grad=True) </ins> if ctx.return_indices: indices, = ctx.saved_tensors else: indices = ctx.indices dim = ctx.dim if ctx.dim is not None else grad_output.dim() - 1 <del> return (grad_input.scatter_(dim, indices, grad_output),) + (None,) * ctx.num_flags </del> <ins> return (grad_input.scatter(dim, Variable(indices), grad_output),) + (None,) * ctx.num_flags </ins> class Sort(_MultiSelectionFunction):

Just a very small grammatical issue. ! Suggested fix: "mutates" to "mutate" <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --
Thanks for pointing this out. Please feel free to open a pull request for this.
Thanks for adding labels. Forgot to do that. I will make a pull request


Question: #  binary_linux_libtorch_3.6m_cpu_test: #    environment: #      BUILD_ENVIRONMENT: "libtorch 3.6m cpu" <del> #    resource_class: gpu.medium </del> <ins> #    resource_class: gpu.nvidia.small </ins> #    <<: *binary_linux_test # #  binary_linux_libtorch_3.6m_cu90_test: #    environment: #      BUILD_ENVIRONMENT: "libtorch 3.6m cu90" <del> #    resource_class: gpu.medium </del> <ins> #    resource_class: gpu.nvidia.small </ins> #    <<: *binary_linux_test #

As it still using deprecated resource class, see See example of the failure in
Followup: we need to plan better --we knew it was gonna happen and none of us fixed it early enough. Action item: go through tasks about deprecation in sprint planning


Question: for weight in self.parameters(): weight.data.uniform_(-stdv, stdv) <ins> def check_forward_args(self, input, hidden, batch_sizes): is_input_packed = batch_sizes is not None expected_input_dim = 2 if is_input_packed else 3 if input.dim() != expected_input_dim: raise RuntimeError( 'input must have {} dimensions, got {}'.format( expected_input_dim, input.dim())) if self.input_size != input.size(-1): raise RuntimeError( 'input.size(-1) must be equal to input_size. Expected {}, got {}'.format( fn.input_size, input.size(-1))) if is_input_packed: mini_batch = batch_sizes[0] else: mini_batch = input.size(0) if self.batch_first else input.size(1) num_directions = 2 if self.bidirectional else 1 expected_hidden_size = (self.num_layers * num_directions, mini_batch, self.hidden_size) def check_hidden_size(hx, expected_hidden_size, msg='Expected hidden size {}, got {}'): if tuple(hx.size()) != expected_hidden_size: raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size()))) if self.mode == 'LSTM': check_hidden_size(hidden[0], expected_hidden_size, 'Expected hidden[0] size {}, got {}') check_hidden_size(hidden[1], expected_hidden_size, 'Expected hidden[1] size {}, got {}') else: check_hidden_size(hidden, expected_hidden_size) </ins> def forward(self, input, hx=None): is_packed = isinstance(input, PackedSequence) if is_packed:

I run RNN sucessfully with the code. and I run embedding example. Then, got RuntimeError. I set to use nothing seems wrong. And the error info seems not the dim problem? The error is so strange.
What exactly do you mean by "embedding example"? Could you provide a code snippet that causes the RuntimeError?
To better reproduce the error, I use the official code example of The origin runs well: Due to use , I have to make batch first. And then , it raises error.
Thank you for the example! The say that setting means that the first dim of input is the batch, ie, input has size . however is unchanged, so the following should work: I do agree the error message could be better, though


Question: } __device__ __forceinline__ void operator()(T* out, T* in) { <del> *out = mask(in) ? *in : ScalarConvert<int, T>::to(0); </del> <ins> *out = mask(out) ? *in : ScalarConvert<int, T>::to(0); </ins> } __device__ __forceinline__ void operator()(T* v) {

On recent PyTorch master (built from source; 0.4.0a0+) on a P100, CUDA 9:
I think the results on CPU meets the defination of triu (upper triangular part of the matrix). It is really hard to handle with a tensor from expand operation with multi-thread not only on GPU but also on CPU. The unexpected result will also happen on CPU if we apply openmp to it. I think it is necessary to eleminate the difference between the implementation and defination.
It doesn’t matter if it’s cpu or not. triu doesn’t have an underscore which means it doesn’t operate in-place and should always give the same result (in new memory)


Question: } template <typename Dtype, typename Acctype> <ins> __launch_bounds__(CUDA_NUM_THREADS) </ins> __global__ void col2im_kernel(const int n, const Dtype* data_col, const int height, const int width, const int channels, const int kernel_h, const int kernel_w,

I'm trying to run a variant of ERFNet on an NVIDIA TX-2 running Jetpack 3.2 (CUDA 9.0 and CuDNN 7). I get the following error: RuntimeError: cuda runtime error (7) : too many resources requested for launch at ../../../pytorch/torch/lib/THCUNN/im2col.h:120 This error indicates the size of the model + overhead is too large for the GPU? But this is roughly a 700mb model trying to perform inference on a single 640x512 grayscale image on a GPU that has roughly 6.5Gb of free space. I even tried training a new model on images at half that resolution and get the same error. Any tips/feedback is appreciated. PyTorch or Caffe2: PyTorch How you installed PyTorch (conda, pip, source): Source Build command you used (if compiling from source): Followed jetson-reinforcement github OS: Ubuntu 16.04 on Nvidia TX2 PyTorch version: 0.3.0 Python version: 3.6 CUDA/cuDNN version: CUDA 9.0 , CUDNN 7 GPU models and configuration: Tegra X2 GCC version (if compiling from source): 5.4 CMake version: 3.5.1 Versions of any other relevant libraries:
we should fix the launch parameters. I presume we cant use as many threads per block on the TX2 as we use on desktop GPUs. can you tell us what the limits of TX2 GPUs are for the fix to im2col
Threads per block and maximum blocks in the grid are actually the same for TX2 as they are for desktop GPUs , only number of registers is smaller. do you know what might be causing this?
as said, the number of threads per block is constant across different NVIDIA GPUs, the Tegra series included. Is it possible that it was compiled to require more registers than is available to the TX2, and maybe the kernel invocation of im2col requires some sort of launch_bounds() command?
TX2 is arm64 platform. I presume pytorch is compiled from source. In this case, there isn't a chance that it was compiled to requre more registers than available on TX2 I think. The build log will be helpful to see.
thanks for the quick response.

I will try dig up the build log tomorrow and post it here.
if there are no launch bounds, it is in fact possible that a kernel is compiled to request more registers than available. At the compile time, compiler does not know how many threads you'll want to launch with, so, potentially, it can use too many registers per thread to later satisfy runtime requirements, and, e.g. trying to launch with 512 or 1024 threads could fail (not even a single block can be put on an SM), whereas launching with 256 would succeed.
is there a way where we can audit that GPU constraints on our server setting (i.e. not actually sitting and compiling TX2 PyTorch)
Adding launch_bounds with the max number of threads the kernel is going to be launched with will cause compiler not to overuse registers. We had to do it e.g. for interp kernels when cuda 9 started using more registers
Ahh, that was as I suspected Thank you for clearing this up. do you guys think a fix will be released in the near future (week?). Thanks for all the help
Works now, thanks!
hello, I met the same error on tx2, complied pytorch 0.3.0.. But the error is this: 'cuda runtime error(7): too many resources requested for launch at '. The file is different with '', is that I need add this line 'launch_bounds(1024)' to every function in file ''? Thank you!
Hello I am also facing a similar issue: Here's my setup: How you installed PyTorch (conda, pip, source): Source Build command you used (if compiling from source): Followed jetson-reinforcement github OS: Ubuntu 16.04 on Nvidia TX2 PyTorch version: 0.5.0a0+ (torchvision 0.2.1) Python version: 3.5 CUDA/cuDNN version: CUDA 9.0 , CUDNN 7 GPU models and configuration: Tegra X2 GCC version (if compiling from source): 5.4 CMake version: 3.12.2 Source code: Full log: Do you know what it could be?

I know that '' without launch_bounds(1024) leads to this error, but I don't know how to fix it... Tried this solution , but still not working (after compilation)
My issue was solved by following
Hi, I have the same problem. Can this problem solved by gitting clone the new pytorch, than changing CUDANUMTHREADS =256 in the below two file and compiling it? !
that appears to fix some of the functions, but not perhaps all - not sure. For more info, see:


Question: std::make_shared<autograd::Eval>()->replaceSubgraph(inputs, outputs); } <del> Node* recordTraceHelper(std::string op, // TODO: make this a Symbol at::ArrayRef<Variable> inputs, at::ArrayRef<Variable> outputs) { </del> <ins> Node* recordTrace(std::string op, // TODO: make this a Symbol at::ArrayRef<Variable> inputs, at::ArrayRef<Variable> outputs) { </ins> auto state = getTracingState(inputs); auto& graph = state->graph; // TODO: Technically, we could reduce the scope of the lock, but since we

when executing:
I'm seeing this as well as of last night. Its preceded by many warnings about : But the first error is the one about . I went back, and I get a good install as late as commit


Question: add_docstr(torch._C.triu, """ <del> triu(input, k=0, out=None) -> Tensor </del> <ins> triu(input, diagonal=0, out=None) -> Tensor </ins> Returns the upper triangular part of the matrix (2D Tensor) :attr:`input`, the other elements of the result Tensor :attr:`out` are set to 0.

The states that the argument controls which diagonal to consider. However, this argument is actually named in pytorch 0.2.0.


Question: tensor_view = tensor.view(5, 3) tensor_view.fill_(random.uniform(0, 1)) self.assertEqual((tensor_view-tensor).abs().max(), 0) <ins> self.assertEqual(empty.view_as(empty), empty) self.assertEqual(empty.view(0), empty) </ins> def test_expand(self): result = torch.Tensor()

Please check and fix all the PyTorch web links format from to , like in the official site.
closed via


Question: // most fields THPFunction* self = (THPFunction*)obj; new (&self->cdata) torch::autograd::PyFunction(obj); <ins> new (&self->cdata_ptr) std::weak_ptr<torch::autograd::PyFunction>(); </ins> self->cdata.num_inputs = -1; self->cdata.is_stochastic = PyObject_IsInstance(obj, THPStochasticFunctionClass); return obj;

there is a bug. the code is from The code for installing cuda10.2 should be corrected to The code for installing cuda10.2 should be corrected to cc
Thank you for reporting the typo

This is how I implement the decoder of a sequence to sequence model I'm not sure about the new autograd mechanics but this worked in the previous version. If I didn't make the unrolling codes a function it will work. It will also work with CPU. I compiled from source with Python 2.7, CUDA 8.0 and Cudnn 6
Thanks for the minimal reproduction! It broke OpenNMT and a big internal model but we weren't looking forward to isolating what parts of those had the issue.
That's great, I'll take a look today. Thanks!
I've pushed a fix to .
How does the inference work in this case? I mean, once you train it how do you test it in the test phase?


Question: self.assertEqual(input_var.requires_grad, broadcasted_var.requires_grad) @unittest.skipIf(not TEST_MULTIGPU, "multi-GPU not supported") <ins> def test_broadcast_no_grad(self): x = torch.randn(1, 2, dtype=torch.cuda.float32, requires_grad=True) with torch.no_grad(): broadcasted = Broadcast.apply((0, 1), x) self.assertTrue(x.requires_grad) for output in broadcasted: self.assertFalse(output.requires_grad) @unittest.skipIf(not TEST_MULTIGPU, "multi-GPU not supported") </ins> def test_replicate(self): module = nn.Linear(10, 5).float().cuda() input = Variable(torch.randn(2, 10).float().cuda())

OS: ubuntu 16.04 PyTorch version: 0.4.0a0+ How you installed PyTorch (conda, pip, source): source Python version: 3.6.3 CUDA/cuDNN version: v9.1, cudnn-9.1-linux-x64-v7 GPU models and configuration: 4x Tesla M60 GCC version (if compiling from source): 5.4.0 When using context () on a model with dataparallel operation (same with ) on 1 GPU the flag requires_grad does not come back to True. If only one GPU is used then everything works fine. Script: On 2 GPUs, this outputs which is wrong in the last line. On 1 GPU (set ) the output is correct.
is looking into it


Question: int batch_size = THTensor_(size)(input, 0); THTensor_(resize1d)(output, batch_size); <ins> int invalid_target = -1;  // We cannot throw an exception inside omp parallel </ins> int i; #pragma omp parallel for private(i) for (i = 0; i < batch_size; i++) { int cur_target = THTensor_fastGet1d(target, i) - TH_INDEX_BASE; <del> if (cur_target == ignore_index) { THTensor_fastSet1d(output, i, 0.0f); continue; </del> <ins> if (cur_target >= 0 && cur_target < n_classes) { if (cur_target == ignore_index) { THTensor_fastSet1d(output, i, 0.0f); continue; } real cur_weight = weights ? THTensor_fastGet1d(weights, cur_target) : 1.0f; THTensor_fastSet1d(output, i, -THTensor_fastGet2d(input, i, cur_target) * cur_weight); } else { THAtomicCompareAndSwap(&invalid_target, -1, cur_target); </ins> } <del> real cur_weight = weights ? THTensor_fastGet1d(weights, cur_target) : 1.0f; THTensor_fastSet1d(output, i, -THTensor_fastGet2d(input, i, cur_target) * cur_weight); </del> <ins> } if (invalid_target >= 0) { THError("Target %d out of bounds", invalid_target); </ins> } return;

Claim: When I use an invalid label with F.crossentropy with reduce=False, I don't receive an error message. I do receive an error message when reduce=True. Code: Result: No error message received. Expected error message: Context: I was using , accidentally used an invalid label, and got extremely large losses and extremely small losses, and inconsistently so. Environment: OS: Ubuntu 16.04 on Docker and host PyTorch version: 0.3.0.post4 How you installed PyTorch: conda Python version: 3.6.3


Question: limits = new_limits # Find the first and the last bin defining the support of the histogram: <del> cum_counts = np.cumsum(np.greater(counts, 0, dtype=np.int32)) </del> <ins> cum_counts = np.cumsum(np.greater(counts, 0)) </ins> start, end = np.searchsorted(cum_counts, [0, cum_counts[-1] - 1], side="right") start = int(start) end = int(end) + 1

The use of in is deprecated a while ago, and the deprecation expired in Numpy 1.24+: Source: Minimal example: Error message with Numpy 1.24+: This error does not occur in Numpy 1.23.5. Numpy should have print a deprecation warning about this: but I think PyTorch has suppressed the warning.
(related: there are a few more numpy1.24 compatibility issues at )
so how can use add_histogram function in numpy 1.24.1?
Since this issue has been resolved, I believe you can use  as before (i.e., as in NumPy < 1.24).
I was using Python 3.9, torch 1.9.1 (with CUDA 11.1) and numpy 1.24.2 to run something today and still ran into this exact problem, don't really know about pytorch 2.0+ though. Thank you for the post though, otherwise might have cost me several more hours to figure this out.
I have this issue during yolo v7 training - cum_counts = (np.greater(counts, 0, dtype=np.int32)) TypeError: No loop matching the specified signature and casting was found for ufunc greater.
Hi, I'm still getting this error with numpy 1.24 and tensorboard 2.15.
same problem with numpy==1.24.3 torch==1.13.1+cu117 tensorboard==2.13.0
Same problem with numpy == 1.24.0 tensorboard == 2.14.0 torch == 1.13.1
Same problem with numpy == 1.26.4 tensorboard == 2.16.2 torch == 1.13.1+cu116
Using can solve this issue
same problem when I used with numpy==1.26.4 torch==1.13.0 tensorboard==2.16.2
go to the path of the in your or your main env like this -C:Userslegionanaconda3envsganLibsite-packagestorchutilstensorboard open and change following line to


Question: if (threadIdx.x == 0) { // Make sure the sum of our distribution didn't overflow assert(!isinf(sum)); <ins> assert(THCNumerics<AccT>::gt(sum, accZero)); </ins> asmem[0] = sum; smem[0] = sampled[curDist];

Looking at the values returned from tn.multinomial(1) and counting them, it seems like it returns the values of the multinomial distribution taken from t, i.e. that it has ignored the signs of the values. However, it seem like always returns a tensor filled with zeros.
looking into this


Question: <del> FROM nvidia/cuda:9.1-cudnn7-devel-ubuntu16.04 RUN echo "deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list </del> <ins> FROM nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 </ins> RUN apt-get update && apt-get install -y --no-install-recommends  build-essential

OS: MacOS Sierra PyTorch version: How you installed PyTorch (conda, pip, source): Just pulled the dockerfile I pulled the dockerfile just now and tried to build it using a tool that we plan on open-sourcing at some point. The command translates to a simple .
CC
It seems to work now.


Question: # for-loop bodies don't define scopes, so we have to save the variables # we want to close over in some way def do_test(self, name=name, self_size=self_size, args=new_args, test_name=test_name, <del> test_output_indices=test_output_indices): </del> <ins> output_process_fn=output_process_fn): </ins> def check(name): is_magic_method = name[:2] == '__' and name[-2:] == '__' is_inplace = name[-1] == "_" and not is_magic_method

I am having problem in running the backward propagation with svd. In particular, it seems working when the first dimension of the matrix x (below) is greater than the second. But I get the following error if the first is smaller than the second. PyTorch version: 2018.04.20 Is debug build: No CUDA used to build PyTorch: 8.0.61 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: Could not collect Python version: 3.5 Is CUDA available: No CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA Versions of relevant libraries: [pip3] numpy (1.13.3) [pip3] torch (2018.4.20) [pip3] torchvision (0.2.0) [conda] pytorch 0.3.1 py35cuda8.0.61cudnn7.0.52 pytorch [conda] pytorch-nightly 2018.04.20 py35cuda8.0.61cudnn7.1.21 pytorch [conda] torchvision 0.2.0 py35heaa392f_1 pytorch
Repro'ed on 0.4 branch. works btw.
What does "Repro'ed" means? Should I install from that branch? I need to compute the kernel of a matrix, therefore I need the last columns of the matrix V (which is why I have to pass ).
It's a shorthand for "reproduced", which means that the bug still hasn't been fixed. Sorry, we'll try to get to it soon.


Question: but they must be :ref:`broadcastable <broadcasting-semantics>`. .. math:: <del> text{out}_i = min(text{tensor}_i, text{other}_i) """ + r""" </del> <ins> text{{out}}_i = min(text{{tensor}}_i, text{{other}}_i) </ins> .. note:: When the shapes do not match, the shape of the returned output tensor follows the :ref:`broadcasting rules <broadcasting-semantics>`.

with , both max and min are broken <img width="880" alt="Screen Shot 2020-03-23 at 1 10 07 PM" src="https://user-"on , min is broken <img width="875" alt="Screen Shot 2020-03-23 at 1 13 23 PM" src="https://user-"
Fixed by and


Question: endforeach(flag_var) endif() <del> set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fno-omit-frame-pointer") set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_STATIC_LINKER_FLAGS_DEBUG} -fno-omit-frame-pointer") </del> <ins> set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fno-omit-frame-pointer -O0") set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_STATIC_LINKER_FLAGS_DEBUG} -fno-omit-frame-pointer -O0") </ins> if (USE_ASAN) set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fsanitize=address") set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_STATIC_LINKER_FLAGS_DEBUG} -fsanitize=address")

I think this may have been intentional, so I want to float this first before I apply the change. Historically, PyTorch did DEBUG builds with , because it prevents inlining, which means that you can inspect variables in all stack frames. However, in master, DEBUG mode appears to be compiled with optimizations, which means that any inlined functions are non-inspectable. The following patch will turn off optimizations when compiled in DEBUG mode. Is there any reason not to apply this?


Question: // See Note [Numpy memory management] &THNumpyArrayAllocator, new NumpyArrayAllocator(numpy_array))); <ins> THStorage_(clearFlag)(storage.get(), TH_STORAGE_RESIZABLE); </ins> result = THTensor_(newWithStorage)(LIBRARY_STATE storage, 0, sizes, strides); } else

When both PyTorch and Pandas are imported, calling results in a segfault. The issue was originally reported by It looks like calling returns a null pointer here: Examining in GDB: I suspect it's both PyTorch and Pandas doing threading magic and somehow getting in each other's way. I haven't yet been able to provide a minimal example that triggers this behaviour and does not use my package; it's likely that I'm doing something odd that causes this. More documentation here: Possibly related issues: and Python version: 3.6 PyTorch version: v0.2.0 Pandas version: 0.20.1


Question: def parameters(self, memo=None): if memo is None: memo = set() <del> super(Container, self).parameters(memo) </del> <ins> for p in super(Container, self).parameters(memo): yield p </ins> for module in self.children(): for p in module.parameters(memo): yield p

While using torch on a Macbook pro with M1 ship, using python 3.8.10 and torch 1.12.1, I came across a weird bug that happened because I forgot to put a tensor on MPS. The reproducible code is the following: As you can see, if is on MPS and on CPU, the line wrongly replaces the index in place of the index . I would expect to have an error saying both tensors should be on the same device instead. I get an error: cc
The error seems to happen somewhere within Also I'm getting a warning "The operator 'aten::nonzero' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications" when printing tensors on MPS.
Hi - thanks for reporting this issue. This is actually an allowed usecase (copying CPU tensors into MPS tensors / MPS -CPU / MPS -MPS): The issue is on MPS side when copying a CPU tensor into a strided MPS tensor. I'll take a look.


Question: HingeEmbeddingLoss, MarginRankingLoss backend.register_function('Linear', Linear) <del> backend.register_function('BatchNorm', BatchNorm) </del> backend.register_function('ConvNd', ConvNd) backend.register_function('RNN', RNN) backend.register_function('RNNTanhCell', RNNTanhCell)

As Arthur Szlam reports, fb-internal cudnn is still lagging behind, and giving batch-size 1024 with batchnorm is raising an error from cudnn. Need to check for compile-time version and disable this codepath
The 1024 limitation was removed in 5.1.10


Question: import torch from torch.distributions import constraints from torch.distributions.distribution import Distribution <del> from torch.distributions.utils import probs_to_logits, logits_to_probs, log_sum_exp, lazy_property, broadcast_all </del> <ins> from torch.distributions.utils import probs_to_logits, logits_to_probs, _log_sum_exp, lazy_property, broadcast_all </ins> class Categorical(Distribution):

Leaving this issue here for higher visibility. Originally reported by here: Also mentioned by Piggybacking off of this issue, has strange behavior (reported here: )
I have fixed this locally. Could I send a PR?
Yes, go ahead!


Question: def fn(*inputs): output = getattr(inputs[0], name)(*inputs[1:]) <del> if test_output_indices is None: return output else: return tuple(output[i] for i in test_output_indices) </del> <ins> return output_process_fn(output) </ins> if not is_inplace and name not in EXCLUDE_GRADCHECK: run_grad_and_gradgrad_checks(self, name, test_name, fn,

I am having problem in running the backward propagation with svd. In particular, it seems working when the first dimension of the matrix x (below) is greater than the second. But I get the following error if the first is smaller than the second. PyTorch version: 2018.04.20 Is debug build: No CUDA used to build PyTorch: 8.0.61 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: Could not collect Python version: 3.5 Is CUDA available: No CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA Versions of relevant libraries: [pip3] numpy (1.13.3) [pip3] torch (2018.4.20) [pip3] torchvision (0.2.0) [conda] pytorch 0.3.1 py35cuda8.0.61cudnn7.0.52 pytorch [conda] pytorch-nightly 2018.04.20 py35cuda8.0.61cudnn7.1.21 pytorch [conda] torchvision 0.2.0 py35heaa392f_1 pytorch
Repro'ed on 0.4 branch. works btw.
What does "Repro'ed" means? Should I install from that branch? I need to compute the kernel of a matrix, therefore I need the last columns of the matrix V (which is why I have to pass ).
It's a shorthand for "reproduced", which means that the bug still hasn't been fixed. Sorry, we'll try to get to it soon.


Question: -DCMAKE_SHARED_LINKER_FLAGS="$LDFLAGS"  -DCMAKE_INSTALL_LIBDIR="$INSTALL_DIR/lib"  -DCUDA_NVCC_FLAGS="$C_FLAGS"  <del> -DCWRAP_FILES_BASE="$BASE_DIR/torch"  </del> <ins> -Dcwrap_files="$CWRAP_FILES"  </ins> -DTH_INCLUDE_PATH="$INSTALL_DIR/include"  -DTH_LIB_PATH="$INSTALL_DIR/lib"  -DTH_LIBRARIES="$INSTALL_DIR/lib/libTH$LD_POSTFIX"

There are previous issues addressing this, but I want to write this up with a plan in a new issue. We should make differentiable instead of having a wrapper class. The Variable class adds conceptual overhead without sufficient benefit. It makes it harder for new users to adopt PyTorch by adding a second API which is similar but not exactly the same as Tensor. There are few prerequisites: on s need to be at least as fast as operations on () need scalar support at the Python level () The constructor will be deprecated and become a no-op. The accessor will be deprecated and return a view that has requiresgrad=False. The constructors will take an optional keyword argument There will be a number of internal changes, but we shouldn't break most existing code: performance reasons, we will no longer build up the graph of operations by default when unless the operation uses stochastic variables. We'll use a context manager or global variable to enable graph creation. new differentiable Tensor will need support all the operations currently on Tensor, including in-place and operations. -place operations on leaf Variables which requiregrad are not currently supported. In the new API, the tensor's value will be updated and it will remain a leaf node. (This is important for optimizers, which update leaf Variables in-place). There are probably a few other subtle differences between Tensor and Variable APIs. I'll update this issue as we address them. See also: 23/738 # Greg will be fixing the tests, Tensor vs. Variable requiresgrad 2/1837 # Just testbroadcast from TestTorch OK OK # error at top level OK OK OK OK OK OK OK OK 10/120 # missing sspaddmm, dsmm/mm, addmm 1/257 TestTorch.testbroadcast 2OK
Properly handling in-place might not be as easy as it seems. Sometimes you actually need to move the Variable in the graph. Consider this case:
One of the cases of differences between variable and tensor is computation of var/std:
IMO Variable gives user a clear difference between graph node and data. .data attribute and Variable constructor provide convenient and explicit ways to take control of the graph (or graphs).
This is done. (Still some clean-up to do)
Is the conda PyTorch out of date? I'm getting the following error: .
Yes, we haven't released 0.4, which has the Variable Tensor merge, yet


Question: z3-solver==4.12.2.0 tensorboard==2.13.0 optree==0.9.1 <ins> # NB: test_hparams_* from test_tensorboard is failing with protobuf 5.26.0 in # which the stringify metadata is wrong when escaping double quote protobuf==3.20.2 </ins>

Platforms: macos, mac This test was disabled because it is failing on main branch (). cc
<!-- validation-comment-start --<bodyHello there! From the DISABLED prefix in this issue title, it looks like you are attempting to disable a test in PyTorch CI. The information I have parsed is below: Test name: Platforms for which to skip the test: mac, macos Disabled by Within ~15 minutes, will be disabled in PyTorch CI for these platforms: mac, macos. Please verify that your test name looks correct, e.g., . To modify the platforms list, please include a line in the issue body, like below. The default action will disable the test for all platforms if no platforms list is specified. We currently support the following platforms: asan, dynamo, inductor, linux, mac, macos, rocm, slow, win, windows.</body<!-- validation-comment-end --


Question: return PyLong_FromLong((int64_t) driverVersion); } <ins> PyObject * THCPModule_getCompiledVersion(PyObject *self) { return PyLong_FromLong((long) CUDA_VERSION); } </ins> PyObject * THCPModule_getRNGState(PyObject *_unused) { HANDLE_TH_ERRORS

There's quite a few user-reports of installing the CUDA 7.5 binary of pytorch and then running on GTX 1080 / 1070, and there will be reports of installing 7.5 / 8.0 binaries and running on V100. Raising a user-warning in such situations and letting them know will fix the problem.
The version of CUDA used to compile the binary should be available thanks to this PR:
I think is a more robust way of getting the CUDA used to compile PyTorch, as currently if there are multiple CUDA installed in the machine, the heuristics used in might not match those used by CMake FindCUDA. I think that we should modify the to return the cuda version used during compilation. I have the impression that it will not require changing much because doesn't require cuda to execute. What do you think?


Question: If :attr:`compute_uv` is ``False``, the returned `U` and `V` matrices will be zero matrices of shape :math:`(m times m)` and :math:`(n times n)` respectively. :attr:`some` will be ignored here. <ins> .. note:: The singular values are returned in descending order. If :attr:`input` is a batch of matrices, then the singular values of each matrix in the batch is returned in descending order. </ins> .. note:: The implementation of SVD on CPU uses the LAPACK routine `?gesdd` (a divide-and-conquer algorithm) instead of `?gesvd` for speed. Analogously, the SVD on GPU uses the MAGMA routine `gesdd` as well.

Documentation for does not indicate whether or not singular values are returned in descending order. My suspicion is that they are since both LAPACK and MAGMA's ?gesdd return them in descending order. It seems like this should be clarified in the docs to avoid unnecessary sorting, or mistaken assumptions. <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --


Question: auto set_history = [&](Variable& var, uint32_t output_nr, bool is_input, bool is_modified, bool is_differentiable) { if (!is_differentiable) { <del> if (!var.requires_grad()) return; </del> <ins> if (!var.requires_grad()) { return; } </ins> // NB: we don't support returning non-differentiable views that could require grad <del> // (this could happen if someone were to return an input to the function). </del> if (var.is_view()) { throw std::runtime_error("Returning Variables sharing storage with other Variables " "that require grad is not supported in Python functions. " "Please submit a feature request if you hit this error."); } <del> var.detach_(); </del> <ins> // Return detached aliases of inputs, instead of changing their requires_grad // property. if (is_input) { var = var.detach(); } else { var.detach_(); } </ins> } else if (is_modified) { if (var.is_leaf() && var.requires_grad()) { throw std::runtime_error("a leaf Variable that requires grad has been used in an in-place operation.");

OS: ubuntu 16.04 PyTorch version: 0.4.0a0+ How you installed PyTorch (conda, pip, source): source Python version: 3.6.3 CUDA/cuDNN version: v9.1, cudnn-9.1-linux-x64-v7 GPU models and configuration: 4x Tesla M60 GCC version (if compiling from source): 5.4.0 When using context () on a model with dataparallel operation (same with ) on 1 GPU the flag requires_grad does not come back to True. If only one GPU is used then everything works fine. Script: On 2 GPUs, this outputs which is wrong in the last line. On 1 GPU (set ) the output is correct.
is looking into it


Question: Otherwise, :attr:`dim` is squeezed (see :func:`torch.squeeze`), resulting in the output tensor having 1 fewer dimension than :attr:`input`."""}) factory_common_args = merge_dicts(common_args, parse_kwargs(""" dtype (:class:`torch.dtype`, optional): the desired data type of returned tensor. Default: if ``None``, uses a global default (see :func:`torch.set_default_tensor_type`).

with , both max and min are broken <img width="880" alt="Screen Shot 2020-03-23 at 1 10 07 PM" src="https://user-"on , min is broken <img width="875" alt="Screen Shot 2020-03-23 at 1 13 23 PM" src="https://user-"
Fixed by and


Question: """Fills the input Tensor or Variable with values according to the method described in "Understanding the difficulty of training deep feedforward neural networks" - Glorot, X. & Bengio, Y. (2010), using a normal distribution. The resulting tensor will have values sampled from :math:`N(0, std)` where <del> :math:`std = gain times sqrt{2 / (fan_in + fan_out)}`. Also known as Glorot initialisation. </del> <ins> :math:`std = gain times sqrt{2 / (fan_in + fan_out)}`. Also known as Glorot initialisation. </ins> Args: tensor: an n-dimensional torch.Tensor or autograd.Variable

why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:


Question: #include "torch/csrc/autograd/engine.h" #include <set> <ins> #include <string> </ins> #include <THPP/THPP.h> using thpp::Tensor;

Not sure what is the reason for these errors, any suggestions? I suspect the clang version is not supported? Here is the
clang have more strict checking , pull request will fixed it.
fixed, thanks to


Question: """ return self._apply(lambda t: t.cuda(device_id)) <del> def cpu(self, device_id=None): </del> <ins> def cpu(self): </ins> """Moves all model parameters and buffers to the CPU.""" return self._apply(lambda t: t.cpu())

as shown in source code, this parameters is not used. Any reason that we keep it?
this can be cleaned up.


Question: uniforms = clamp_probs(self.logits.new(self._extended_shape(sample_shape)).uniform_()) gumbels = -((-(uniforms.log())).log()) scores = (self.logits + gumbels) / self.temperature <del> return scores - log_sum_exp(scores) </del> <ins> return scores - _log_sum_exp(scores) </ins> def log_prob(self, value): K = self._categorical._num_events

Leaving this issue here for higher visibility. Originally reported by here: Also mentioned by Piggybacking off of this issue, has strange behavior (reported here: )
I have fixed this locally. Could I send a PR?
Yes, go ahead!


Question: def make_grad_input(fn): def call_grad_input(self, bufs, input, weight, grad_output): backend = type2backend[type(input)] <del> grad_input = input.new().resize_as_(input).zero_() </del> <ins> grad_input = input.new().resize_as_(input) </ins> kernel_size = weight.size()[2:] args = parse_arguments(self, fn.arguments[5:], bufs, kernel_size) getattr(backend, fn.name)(backend.library_state, input, grad_output,

THNN convolutions are slower due to unnecessary .zero() calls on gradients: is not needed same for bias, prob worth adding a constant in C-side that controls whether to override grad or not
Same for batch_norm and auto thnn functions, not sure if all of them though. Prob the speedups will be quite significant.
I think they are needed. THNN functions accumulate the gradient instead of overwriting the buffer, don't they?
only grad wrt weight, inputs are overriden
We should reopen this, still many functions have unnecessary .zero() calls on gradinput and gradparam, e.g.: : : most of the functions don't need it
I think the ones are currently necessary.
for nn yes, but cudnn has: and on cudnnConvolutionBackwardFilter and on cudnnBatchNormalizationBackward which control addition of gradients and should be set to 1 and 0 in pytorch cudnn (it is and 1 in )
Yeah, we can remove it on BatchNorm. If you look at Conv, it already only zeros out grad_weight for cunn.
this is now fixed in master:


Question: } auto SavedVariable::unpack() -> std::shared_ptr<Variable> { <del> if (!data) return nullptr; </del> <ins> if (!data) { if (version) { throw std::runtime_error(ERR_BACKWARD_TWICE); } return nullptr; } </ins> int current_version = **version; if (expected_version != current_version) {

Usually calling backwards twice gives this message: But conv2d just segfaults. $ pt A B Segmentation fault (core dumped)


Question: ('trace', (M, M), ()), ('cross', (S, 3), ((S, 3),)), ('cross', (S, 3, S), ((S, 3, S), 1), 'dim'), <ins> ('inverse', (S, S), (), '', (), [skipIfNoLapack]), </ins> ('clone', (S, M, S), ()), ('eq', (S, S, S), ((S, S, S),)), ('ne', (S, S, S), ((S, S, S),)),

Command to install the latest nightly binary: installs : However, PyTorch itself reports : I guess this version tag might be defined during the build process from e.g. and then forwarded from , but also source builds should use based on the tag. Unsure, if this is just a reporting issue or if we are building the nightly binaries with and executing them with , so creating this issue to dig a bit further into it. CC cc
This is really weird, I've put up measure in place to validate against it... Will investigate today (and moving to pytorch repo)
can you please check that 2.2.0 reports (and is build) against the correct version of nccl library?
Confirmed for nightlies : Same issue for RC:
Looking at the build log Looks like we are compiling with: USESYSTEMNCCL
On Windows I see following exception: We compiling with should we return None rather then exception ?
In this case we are indeed using 2.18.1: Do you know why we are using the system-wide NCCL installation instead of the one? Should we delete the system NCCL and use by default?
Looks like its set here: Looks like it was by: Let me propose the PR to unset this flag : USESYSTEM_NCCL
I guess we are using the system NCCL installation to be able to during the runtime. If we would use the module I assume we would link NCCL into the PyTorch binaries. If so, we should make sure to update the NCCL version whenever is updated. This could break our dependency on the NCCL pip package.
Yes. Also when we update this nccl here:
We used following path  What would be good path for nccl 2.19.3 and cuda 12.1 ? As per this issue: Will be using 12.0 ncck 2.19.3
Installing system NCCL into the nightly build docker container feels great, but we need to figure out, how to keep it in sync with PyTorch. I wonder if one can, for example, just check nccl version from this requirements file and build from source?
yes probably we should build and upload nccl for our supported cuda versions, same way as we build docker containers, by following details here:
Verified with final rc


Question: case NPY_DOUBLE: return kDouble; case NPY_FLOAT: return kFloat; case NPY_HALF: return kHalf; <ins> case NPY_LONGLONG: </ins> case NPY_INT64: return kLong; case NPY_INT32: return kInt; case NPY_INT16: return kShort;

Minimal example: Torch initializes the tensor with values which are different from the numpy array. I've seen issues for similar kinds of problems (see ). Maybe the easiest solution to all these initialization problems might be, to check for the same values of the original and initialized array/tensor instead of initializing tensors silently with wrong numbers. What do you think? Furthermore, I've opened an issue which is also related this one in terms of initializing a numpy array with a number of type and then initializing a torch tensor from that array, see Feel free to merge them if you think that both issues can be solved together.
Could you please provide some information about what version of pytorch and numpy you're using? ! It works as expected for me on master
I'm using torch 0.3.0.post4 (latest one on website of PyTorch) and numpy 1.13.3.
I'm running the same thing, this is strange. In the first line , I assumed that came from . Is this the case?
Nope, it's indeed the python built-in type. Actually this issue came up when I tried to get the shape of (i.e. , , returns ).
Thanks for the clarification, I'm now seeing your results. Looks like the tensor constructor doesn't work well with python2.7 long types in this case
Sure, no problem. Thanks for looking into it!
The problem seems to be that np.longlong and np.int64 are the same thing (at least on a 64bit system, I think), but the NPYLONGLONG and NPYINT64 macros are different... I'll put up a pr for this soon.

A minimal example: Although has and this type is listed as supported, the array can't be converted. Consider that is initialized with an upper bound of type . I'm not completely sure, whether this issue is related to torch or numpy, but numpy initializes the array with values as expected, whereas torch throws an inconsistent error message.
Edit: nevermind, I didn't run the same exact thing ! Works for me on master and 0.3. Seems like it could be related to


Question: #endif /* !IS_CUDA */ #undef BUILD_REAL_FMT

On Python 3.5.2 errors out with . map and map2 seem to be work just fine.