Question: } }; <ins> // Similar to shared_from_this. There's a problem that the Python object // and its cdata depend on each other being alive, so we can't keep // shared_ptrs as members, but we'd like to be able to manage the lifetime of // the objects using shared_ptrs in the C++ graph. The only way to get a new // shared_ptr that references them is through THPFunction_asFunction. When // called for the first time it will allocate a new shared_ptr and save a // weak_ptr in cdata_ptr attr. Later, when we try to take another reference, // we'll try to lock cdata_ptr and return its value if successful. Otherwise it // means that all shared_ptrs returned previously have been freed, so we can // create a new one. This ensures that this object is managed by at most one // shared_ptr control block at any time - a guarantee we depend on in other places // (e.g. we use weak_ptrs in SavedVariable because we know it won't go out of scope). </ins> std::shared_ptr<PyFunction> THPFunction_asFunction(THPFunction* self) { if (!self) { return std::shared_ptr<PyFunction>(); } Py_INCREF((PyObject*)self); <del> return std::shared_ptr<PyFunction>(&self->cdata, Decref()); </del> <ins> auto ptr = self->cdata_ptr.lock(); if (ptr) return ptr; ptr = std::shared_ptr<PyFunction>(&self->cdata, Decref()); self->cdata_ptr = ptr; return ptr; </ins> }

there is a bug. the code is from The code for installing cuda10.2 should be corrected to The code for installing cuda10.2 should be corrected to cc
Thank you for reporting the typo

This is how I implement the decoder of a sequence to sequence model I'm not sure about the new autograd mechanics but this worked in the previous version. If I didn't make the unrolling codes a function it will work. It will also work with CPU. I compiled from source with Python 2.7, CUDA 8.0 and Cudnn 6
Thanks for the minimal reproduction! It broke OpenNMT and a big internal model but we weren't looking forward to isolating what parts of those had the issue.
That's great, I'll take a look today. Thanks!
I've pushed a fix to .
How does the inference work in this case? I mean, once you train it how do you test it in the test phase?


Question: x.__repr__() str(x) <ins> x = torch.DoubleTensor([1e-324, 1e-323, 1e-322, 1e307, 1e308, 1e309]) x.__repr__() str(x), </ins> def test_unsqueeze(self): x = torch.randn(2, 3, 4) y = x.unsqueeze(1)

on print(torch.DoubleTensor(1)) I got ZeroDivisionError: float division by zero but torch.DoubleTensor(0)) or torch.DoubleTensor(2)) work just fine. I work in Jupyter notebook pytorch 0.1.12
Note that returns undefined memory. Could you find the specific content of the tensor that gave this error, by printing ? I'm not sure if this bug has been fixed in the latest release already.
the error is pretty interesting. I restarted Jupyter notebook. # many freaking signatures: print(torch.FloatTensor(1)) print(torch.IntTensor(1)) print(torch.DoubleTensor(1)) # no error right after initial run. #print(torch.DoubleTensor(0)) print(torch.DoubleTensor(2)) print(torch.ByteTensor(1)) print(torch.CharTensor(1)) print(torch.ShortTensor(1)) print(torch.LongTensor(1)) steps to reproduce the error: 1). run, no errors 2) repeat, no error 3) uncomment ..DoubleTensor(0), run, got an error (see below): 4) comment ..DouleTensor(0), run still have this error 2.+24 [torch.FloatTensor of size 1] 7.+08 [torch.IntTensor of size 1] ZeroDivisionError Traceback (most recent call last) <ipython-input-5-3629c250c505in <module() 2 print(torch.FloatTensor(1)) 3 print(torch.IntTensor(1)) ----4 print(torch.DoubleTensor(1)) # ZeroDivisionError: float division by zero 5 #print(torch.DoubleTensor(0)) 6 print(torch.DoubleTensor(2)) C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in str(self) 126 # characters to replace unicode characters with.

127 if sys.versioninfo (3,): --128 return (self) 129 else: 130 if hasattr(, 'encoding'): C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in str(self) 284 return '[{} with no dimension]n'.format(torch.typename(self)) 285 elif self.ndimension() == 1: --286 strt = vectorstr(self) 287 elif self.ndimension() == 2: 288 strt = matrixstr(self) C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in vectorstr(self) 270 if () < PRINTOPTS.threshold: 271 return (strt + --272 'n'.join(ident + (val / scale) for val in self) + 273 'n') 274 else: C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in <genexpr(.0) 270 if () < PRINT_OPTS.threshold: 271 return (strt + --272 'n'.join(ident + (val / scale) for val in self) + 273 'n') 274 else: ZeroDivisionError: float division by zero
Ok, I got a repro. Here is the failing case: Fails with the error you mentioned. This seems to be an edge case though, as both and works. The issue is in , where becomes zero because is a too large negative number. Maybe we should clip to be positive?


Question: res2 += i * j self.assertEqual(res1, res2) <ins> # Test 0-strided for tname, _prec in types.items(): v1 = torch.randn(1).type(tname).expand(100) v2 = torch.randn(100).type(tname) res1 = torch.dot(v1, v2) res2 = 0 for i, j in zip(v1, v2): res2 += i * j self.assertEqual(res1, res2) def test_ger(self): types = { 'torch.DoubleTensor': 1e-8, 'torch.FloatTensor': 1e-4, } for tname, _prec in types.items(): v1 = torch.randn(100).type(tname) v2 = torch.randn(100).type(tname) res1 = torch.ger(v1, v2) res2 = torch.zeros(100, 100).type(tname) for i in range(100): for j in range(100): res2[i, j] = v1[i] * v2[j] self.assertEqual(res1, res2) # Test 0-strided for tname, _prec in types.items(): v1 = torch.randn(1).type(tname).expand(100) v2 = torch.randn(100).type(tname) res1 = torch.ger(v1, v2) res2 = torch.zeros(100, 100).type(tname) for i in range(100): for j in range(100): res2[i, j] = v1[i] * v2[j] self.assertEqual(res1, res2) def test_addmv(self): types = { 'torch.DoubleTensor': 1e-8, 'torch.FloatTensor': 1e-4, } for tname, _prec in types.items(): t = torch.randn(10).type(tname) m = torch.randn(10, 100).type(tname) v = torch.randn(100).type(tname) res1 = torch.addmv(t, m, v) res2 = torch.zeros(10).type(tname) res2 += t for i in range(10): for j in range(100): res2[i] += m[i, j] * v[j] self.assertEqual(res1, res2) # Test 0-strided for tname, _prec in types.items(): t = torch.randn(1).type(tname).expand(10) m = torch.randn(10, 1).type(tname).expand(10, 100) v = torch.randn(100).type(tname) res1 = torch.addmv(t, m, v) res2 = torch.zeros(10).type(tname) res2 += t for i in range(10): for j in range(100): res2[i] += m[i, j] * v[j] self.assertEqual(res1, res2) def test_addmm(self): types = { 'torch.DoubleTensor': 1e-8, 'torch.FloatTensor': 1e-4, } for tname, _prec in types.items(): M = torch.randn(10, 25).type(tname) m1 = torch.randn(10, 50).type(tname) m2 = torch.randn(50, 25).type(tname) res1 = torch.addmm(M, m1, m2) res2 = torch.zeros(10, 25).type(tname) res2 += M for i in range(10): for j in range(25): for k in range(50): res2[i, j] += m1[i, k] * m2[k, j] self.assertEqual(res1, res2) # Test 0-strided for tname, _prec in types.items(): M = torch.randn(10, 1).type(tname).expand(10, 25) m1 = torch.randn(10, 1).type(tname).expand(10, 50) m2 = torch.randn(50, 25).type(tname) res1 = torch.addmm(M, m1, m2) res2 = torch.zeros(10, 25).type(tname) res2 += M for i in range(10): for j in range(25): for k in range(50): res2[i, j] += m1[i, k] * m2[k, j] self.assertEqual(res1, res2) </ins> def _testMath(self, torchfn, mathfn): size = (10, 5) # contiguous

It seems like autodiff of matrix terms of .mv() doesn't behave as I'd expect. But if we use .mm(), things work fine: python A = Variable((3,2), requiresgrad=True) x = Variable((2), requiresgrad=True) (A.mm(x[:,None])).sum().backward() print(A.grad) print(x.grad)
Hi, As a temporary fix, you can use . The problem is the behavior between the backward and the backward. The sum returns a strided gradient which is not supported by MKL/BLAS implementations of (used in the backward). not sure where this was introduced and how to fix this properly. Should the blas wrapper clone the input if it is strided? Or should it just raise an error (meaning that we need to add some s in some places in the code like the sum backward)?
It should clone the input. Still, this problem should have been caught. We must be missing some return code checks from BLAS
Ah apparently these functions have no return codes and the docs clearly state that vector strides can't be 0... We have to fix those bindings


Question: from ._utils import _type, _cuda, _range from functools import reduce from itertools import chain <ins> import sys </ins> import math

It seems that the following link that can be reached from here (check the screenshot) is broken: Page: Broken link: Screenshot: ! cc
this should be transferred to , but I don't have permissions to do that. Fix is available at pytorch/hub.


Question: set(BLAS_SET_BY_USER ${BLAS_SET_BY_USER} CACHE STRING "Marks whether BLAS was manually set by user or auto-detected") endif() <ins> # These lines are an attempt to make find_package(cuda) pick up # libcuda.dylib, and not cuda.framework.  It doesn't work all # the time, but it seems to help for some users. # TODO: replace this with a more robust fix if(APPLE) set(CMAKE_FIND_FRAMEWORK LAST) set(CMAKE_FIND_APPBUNDLE LAST) endif() </ins> # ---[ Options. # Note to developers: if you add an option below, make sure you also add it to # cmake/Summary.cmake so that the summary prints out the option values.

error occurs when linking the cuda library ld: can't map file, errno=22 file '/usr/local/cuda/lib/stubs/cuda.framework' for architecture x8664 Provide a short description. I have tracked down in findlibrary(CUDACUDALIB cuda PATHS ${CUDATOOLKITROOTDIR} PATHSUFFIXES lib lib64 lib/stubs lib64/stubs lib/x64) finds the cuda.framework instead of Have confirm it was broken by this commit () Any idea why it is linking the cuda.framework? CUDA used to build PyTorch: 9.2 OS: Mac OSX 10.13.4 GCC version: Could not collect CMake version: version 3.11.1 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.2.64 GPU models and configuration: Could not collect Nvidia driver version: Could not collect cuDNN version: Probably one of the following: /usr/local/cuda/lib/libcudnn_static.a
cc
From cmake doc , CMake will use a -framework A, and a -F<fullPathto link the framework to the target, however it is not. I have tried separating ::cuda from Caffe2PUBLICCUDADEPENDENCY_LIBS, still not linking properly. Removing ::cuda passes the build though.
can you post your cmake output that contains the list of CUDA-related libraries that cmake found? Is this statically linking CUDA? What's the actual linking command that is failing (probably need to pass VERBOSE=1 to the make command to see this)? None of us will be able to repro on our end as we do not have macs with GPUs :( , but we can try to answer questions and give pointers as to where to look.
thanks for reporting this and helping reproduce. Can you try and add this to the root around line 162? You'll then want to run to clear the directory and try again.

[ 94%] Linking CXX shared library cd /Users/hongbinliu/pytorch/build/ && /usr/local/miniconda3/bin/cmake -E cmakelinkscript --verbose=1 /Library/Developer/CommandLineTools/usr/bin/clang++ -msse4.1 -msse4.2 --std=c++11 -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -DONNXNAMESPACE=onnxc2 -O2 -fPIC -Wno-narrowing -Wno-invalid-partial-specialization -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-unused-parameter -Wno-unknown-warning-option -Wno-unknown-pragmas -DHAVEAVXCPUDEFINITION -DHAVEAVX2CPUDEFINITION -O3 -isysroot -mmacosx-version-min=10.10 -dynamiclib -Wl,-headerpadmaxinstallnames -compatibilityversion 1.0.0 -currentversion 1.0.0 -o -installname -Wl,-rpath,/usr/local/miniconda3/lib -Wl,-rpath,/usr/local/cuda/lib -Wl,-rpath,/Users/hongbinliu/pytorch/build/lib /usr/local/cuda/lib/libcudartstatic.a -Wl,-rpath,/usr/local/cuda/lib /usr/local/cuda/lib/libcudartstatic.a -Wl,-rpath,/usr/local/cuda/lib /usr/local/cuda/lib/stubs/cuda.framework /usr/local/cuda/lib/libcublasdevice.a ld: can't map file, errno=22 file '/usr/local/cuda/lib/stubs/cuda.framework' for architecture x8664 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: [] Error 1 make[1]: [] Error 2 make: * [all] Error 2
Thx, adding that did work. However, when I run training with cuda, it throws an RuntimeError: cannot initialize CUDA without ATen_cuda library. I guess this is another issue anyway.
Same issue.

<img width="545" alt="downloads" src="https://user-"Also, I found a small problem when copying the dylibs to torch/lib, it is replacing the symbolic link with the actual file (screenshot above). Replacing cp -r to cp -R does the trick, please let me know if we want a pr.
Have tracked down the problem was in , always return null because in Registry.h line 77 the (key) is always 0; ObjectPtrType Create(const SrcType& key, Args... args) { ` Suspect CUDAHooksRegistry() is creating new instance, because of namespace? Is it working in Linux? Any help will be appreciated.
I think Mac OS X with CUDA is broken at the moment. We think the problem has something to do with not getting a dependency on (or, I guess, , now)
Hi ezyang, thanks for your hint. I just tried before . It gives me "Key already registered". I will leave it to you guys, and I can help with testing, please @ me.
Thanks for giving it a try. Sounds like is being loaded correctly, so my initial guess was wrong. Would it be possible to print the keys of the map to see what is in it when you get the key already registered error message? And maybe also, you noted that there was never anything at the key "CUDAHooks" when you didn't have the dlopen line. Is the map always empty in that case, or is there something registered, perhaps at a different name? (I made this bug when I developed the patch and figured it out by seeing that there was another CUDAHooks thing but named slightly differently). I'd like to debug this myself but I can't get PyTorch to build with Mac OS X and CUDA at the moment. (See also the bugs I've recently filed)
Yes, the map was always empty when calling CUDAHooksRegistry(). Loading the dylib works as expected that it is reusing the same static Registry and registers the key. However calling CUDAHooksRegistry() doesn't get the same static Registry instance, instead it creates a new one. Could it be relating to how mac handles dynamic loading? e.g different memory space?
Successfully reproed.

And yes, it looks like the dynamic dependencies are setup correctly:
BTW, fix for the cuda.framework error didn't work for me. Where exactly did you end up placing the block in the OTOH, this patch worked for me: (It's totally the wrong way to solve the problem but it might help someone)
and I diagnosed the problem, fix coming.
That is great!
Go try . You'll need to do a clean build from scratch. Also, you'll still need a cuda.framework fix of some sort or another. Working on a more robust version of this fix.
Thanks ezyang, it is working.


Question: if (tensor->isCuda() != isCuda) { throw invalid_tensor(isCuda ? "CUDA" : "CPU", tensor->isCuda() ? "CUDA" : "CPU"); } <ins> if (tensor->type() != type) { throw invalid_tensor(thpp::toString(type), thpp::toString(tensor->type())); } </ins> } }

the error only happens in very specific conditions. (change the view of input tensor and use weight converted from numpy array.) The output of functional conv2d is far off in scale (e12 or so.)
this happens because generates double precision tensor by default, whereas torch generates single precision. We need to add guards for tensor types in THNN and THCUNN.
It's a problem with function, that only . Should be a simple fix.
Whoops! Yeah it just ignores the "thpp::Type type" parameter
what type is expected here?
a is a FloatTensor but weights is a DoubleTensor. Change: to: (or change a to (...).double())
cool thanks!


Question: Py_INCREF(input_var); output_var = input_var; input_var_.creator = THPFunction_asFunction(self); <ins> input_var_.requires_grad = self->cdata.requires_grad; </ins> } else { // If the leaf Variable has been returned, we have to move it after the // current function to ensure the gradient is computed correctly.

import torch as th from torch.autograd import Variable x = Variable(th.Tensor([1 , 2 , 3]) , requiresgrad = False) y = Variable(th.Tensor([100]) , requiresgrad = True) x = x.cuda(0) y = y.cuda(0) x[0] = y print(x.requiresgrad) If ((0) (0)) is executed then the output is False, True otherwise. I think the result that x.requiresgrad = True is more natural.
I further checked some in-place operator in GPU mode(say, x = Variable(...).cuda(0) ), and find out that after the operation is done, the requiresgrad of the Variable doesn't change. For example: import torch as th from torch.autograd import Variable x = Variable(torch.Tensor([1]) , requiresgrad = False) y = Variable(torch.Tensor([2]) , requiresgrad = True) x = x.cuda(0) y = y.cuda(0) x.add(y) print(x.requires_grad) the output is False
It doesn't have to do with Variable holding a CUDA tensor, but rather with the fact that it's no longer a leaf. I've sent a fix in .
Thanks for the report!


Question: return torch._C._cuda_getDeviceName(device) <ins> def get_device_capability(device): """Gets the cuda capability of a device. Arguments: device (int): device for which to return the name. This function is a no-op if this argument is negative. Returns: tuple(int, int): the major and minor cuda capability of the device """ if device >= 0: return torch._C._cuda_getDeviceCapability(device) </ins> @contextlib.contextmanager def stream(stream): """Context-manager that selects a given stream.

There's quite a few user-reports of installing the CUDA 7.5 binary of pytorch and then running on GTX 1080 / 1070, and there will be reports of installing 7.5 / 8.0 binaries and running on V100. Raising a user-warning in such situations and letting them know will fix the problem.
The version of CUDA used to compile the binary should be available thanks to this PR:
I think is a more robust way of getting the CUDA used to compile PyTorch, as currently if there are multiple CUDA installed in the machine, the heuristics used in might not match those used by CMake FindCUDA. I think that we should modify the to return the cuda version used during compilation. I have the impression that it will not require changing much because doesn't require cuda to execute. What do you think?


Question: try: _loadlib() except Exception: <del> warnings.warn('cuDNN library not found. Check your LD_LIBRARY_PATH') </del> <ins> warnings.warn('cuDNN library not found. Check your {libpath}'.format( libpath={ 'darwin': 'DYLD_LIBRARY_PATH', 'win32': 'PATH' }.get(sys.platform, 'LD_LIBRARY_PATH'))) </ins> return False return True

The official website does not provide the torchvison version installation method. Here is the error message when torchvison 0.3.0 is used. torchvision/C.cpython-36m-x8664-linux- undefined symbol: ZN2at7getTypeERKNS6TensorE That should be caused by the mismatch version between pytorch-nightly and torchvision 0.3.0.
It should be that matches with , but they are not provided currently.
I tried installing it with pip as per , but now I get when importing . I did a similar thing on June 25th. It worked then. Any known fix for this?
do we document what to do here?
I finally solved this by building from source with Before running that make sure to upgrade and to use . Note: this will not work if you have on your system when building since you'd get version mismatch between and . Tested only on .
you build pytorch from source or just torchvision? or both of them.
I installed from conda with: I then cloned the repo and ran inside the repo. I guess this would also work if you do , but I haven't tried it.
Just a reminder (what I encountered just now): one should uninstall previous ` from source, or they can mix up and mess up everything.
There are nightlies for torchvision under Windows now.
I have problems using Pytorch with my Geforce RTX 3060, (sm86). I tried to install with help of : Create a new venv, based on python 3.7.10. Activate it. conda install pytorch-nightly cudatoolkit=10.0 -c pytorch (I also did it without the "=10.0") After this, Pytorch detects the sm86 with no errors, problem sm86 solved with pytorch-nightly Now, I have a problem with torchvision. tried installing torchvision with File "C:repovisiontorchvisionmodels", line 195, in Inception3 AttributeError: module '' has no attribute 'unused' tried installing torchvision version 0.3.0, from , but an error rose: tried installing torchvision at origin/master, from , but another error rose: Here is the list of packages installed:


Question: transposedIndices.emplace_back(); } } <del> return std::make_tuple<>(self.permute(dims), std::move(transposedIndices)); </del> <ins> return std::make_tuple(self.permute(dims), std::move(transposedIndices)); </ins> } static std::vector<int64_t> computeLinearStride(const Tensor & tensor) {

In PyTorch master 0.4 , variables cannot be indexed. This used to work in PyTorch 0.2 :
This is breaking some of Pyro's inference algorithms on PyTorch master:


Question: finally: torch.backends.cudnn.enabled = prev <ins> def test_rnn_args_check(self): input_size = 3 hidden_size = 5 num_layers = 2 batch_size = 4 seq_len = 6 num_directions = 1 def test(input_shape, hidden_shape, mode): for input, hidden in get_inputs(input_shape, hidden_shape, mode): model = getattr(nn, mode)(input_size, hidden_size, num_layers) self.assertRaises(RuntimeError, lambda: model(input, hidden)) correct_input_shape = (seq_len, batch_size, input_size) correct_hidden_shape = (num_layers * num_directions, batch_size, hidden_size) def update_tuple(tup, dim, delta): new_tup = list(tup) new_tup[dim] = delta return tuple(new_tup) def get_inputs(input_shape, hidden_shape, mode): '''returns list( tuple(input, hidden) ) where input, hidden are inputs to a model''' input = Variable(torch.randn(input_shape)) hidden = Variable(torch.randn(hidden_shape)) if mode is not 'LSTM': return [(input, hidden)] if hidden_shape == correct_hidden_shape: return [(input, (hidden, hidden))] good_hidden = Variable(torch.randn(correct_hidden_shape)) return [ (input, (hidden, good_hidden)), (input, (good_hidden, hidden)), ] rnn_modes = ['RNN', 'GRU', 'LSTM'] for mode in rnn_modes: # Incorrect input batch size input_shape = update_tuple(correct_input_shape, 1, -1) hidden_shape = correct_hidden_shape test(input_shape, hidden_shape, mode) # Incorrect hidden batch size input_shape = correct_input_shape hidden_shape = update_tuple(correct_hidden_shape, 1, -1) test(input_shape, hidden_shape, mode) # Incorrect input size input_shape = update_tuple(correct_input_shape, 2, -1) hidden_shape = correct_hidden_shape test(input_shape, hidden_shape, mode) # Incorrect hidden size input_shape = correct_input_shape hidden_shape = update_tuple(correct_hidden_shape, 2, -1) test(input_shape, hidden_shape, mode) # Incorrect hidden[0] input_shape = correct_input_shape hidden_shape = update_tuple(correct_hidden_shape, 0, -1) test(input_shape, hidden_shape, mode) </ins> def test_rnn_initial_hidden_state(self): rnn_modes = ['RNN', 'GRU', 'LSTM'] for mode in rnn_modes:

I run RNN sucessfully with the code. and I run embedding example. Then, got RuntimeError. I set to use nothing seems wrong. And the error info seems not the dim problem? The error is so strange.
What exactly do you mean by "embedding example"? Could you provide a code snippet that causes the RuntimeError?
To better reproduce the error, I use the official code example of The origin runs well: Due to use , I have to make batch first. And then , it raises error.
Thank you for the example! The say that setting means that the first dim of input is the batch, ie, input has size . however is unchanged, so the following should work: I do agree the error message could be better, though


Question: make_variable(std::move(std::get<3>(tensors)))); } <del> std::vector<Variable> VariableType::as_variable(TensorList tl) const { </del> <ins> static std::vector<Variable> as_variable(TensorList tl) { </ins> std::vector<Variable> variables; for (auto& t : tl) { variables.emplace_back(make_variable(std::move(t)));

See Docker run fails with , which is due to the fact that is specified in docker container: Which results in: CI cc
Check that adding to manywheels image results in a failure:
Tentatively marking hi-pri since nightly testing are failing
This is resolved, here is the successful run:


Question: <ins> #include <stdint.h> #include <stddef.h> enum THPByteOrder { THP_LITTLE_ENDIAN = 0, THP_BIG_ENDIAN = 1 }; THPByteOrder THP_nativeByteOrder(); void THP_decodeInt16Buffer(int16_t* dst, const uint8_t* src, THPByteOrder order, size_t len); void THP_decodeInt32Buffer(int32_t* dst, const uint8_t* src, THPByteOrder order, size_t len); void THP_decodeInt64Buffer(int64_t* dst, const uint8_t* src, THPByteOrder order, size_t len); void THP_decodeFloatBuffer(float* dst, const uint8_t* src, THPByteOrder order, size_t len); void THP_decodeDoubleBuffer(double* dst, const uint8_t* src, THPByteOrder order, size_t len); </ins>

<!-- A clear and concise description of what the bug is. --The target name is used by a test executable in mkldnn. Steps to reproduce the behavior: BUILDTEST=1 build <!-- If you have a code sample, error messages, stack traces, please provide it here as well --<!-- A clear and concise description of what you expected to happen. --CMake configure passes. Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): v1.5.0-rc2 OS (e.g., Linux): Windows How you installed PyTorch (, , source): source Build command you used (if compiling from source): python build Python version: 3.x CUDA/cuDNN version: 9.2 / 10.1 / 10.2 / NA GPU models and configuration: Tesla M60 Any other relevant information: <!-- Add any other context about the problem here. --We should not build tests for mkldnn.
I have encountered the same error, how do I resolve it?
It is actually mkldnn that is out of sync. So running the following script should help.
Thanks a lot for this I posted this on the PyTorch Discussions but to no avail, and now I've finally understood the reason of this error. Thanks a lot. Surprisingly, for me the build works out if I build from the master repo, but changing to a different version always fails.


Question: input=(torch.randn(2, 3, 5, 5, 5) * 1000)), OldModuleTest(nn.VolumetricMaxPooling, (2, 2, 2, 2, 2, 2), <del> input_size=(2, 3, 5, 5, 5), </del> <ins> input=(torch.randn(2, 3, 5, 5, 5) * 1000), </ins> desc='stride'), OldModuleTest(nn.VolumetricMaxPooling, (2, 2, 2, 2, 2, 2, 1, 1, 1), <del> input_size=(2, 3, 5, 5, 5), </del> <ins> input=(torch.randn(2, 3, 5, 5, 5) * 1000), </ins> desc='stride_padding'), OldModuleTest(nn.VolumetricReplicationPadding, (1, 2, 3, 4, 5, 6),

Repro: stack: says he's already investigating a similar problem in internal models. cc
we think may be root cause
I think this might be correct? I am seeing CPU devices explicitly created in the model, and in dynamo. I do not believe this is a bug, but we need much better tooling around detecting devices.
In that case we need to add float16 support to the cpu backend (ugh)
The node being copied CPU=CUDA is: so the only CPU op involved is -- which we are actually generating ourselves. This patch might help, by doing the conversion on GPU: Though after applying it we get a new error:
After applying some partial fixes in pytorch/torchdynamo, I tried the minifier -- but it didn't seem to work. cc I was running I get a lot of printouts of: But then it finishes I get:
If I change where the cpu device is set to "cuda" in the model source - and remove an assert on our end, I can get the same exact error. That might make for an easier minification and repro story. Let me see if that minimizes on my end.
Please rebase to this PR,
pytorch/torchdynamo gets the same error without needing to do dtype conversion, potentially easier for debugging?
Minified version (requires pytorch/torchdynamo)
ref pytorch/torchdynamo


Question: THCAssertSameGPU(THCTensor_(checkGPU)(state, 2, self_, src_)); THArgCheck(src_->nDimension == 2, 1, "expected a matrix"); <del> THCTensor *src = src_; if (self_ == src_) src = THCTensor_(newContiguous)(state, src_); </del> <ins> if (self_ != src_) THCTensor_(resizeAs)(state, self_, src_); </ins> <del> int64_t stride0 = src->stride[0]; int64_t stride1 = src->stride[1]; real *start = THCTensor_(data)(state, src); </del> <ins> int64_t stride0 = self_->stride[0]; int64_t stride1 = self_->stride[1]; real *start = THCTensor_(data)(state, self_); </ins> TensorTriOp<real, 0> op(start, stride0, stride1, k); if (self_ == src_) { <del> if (!THC_pointwiseApply1(state, src, op)) { </del> <ins> if (!THC_pointwiseApply1(state, src_, op)) { </ins> THArgCheck(false, 2, CUTORCH_DIM_WARNING); } } else { <del> THCTensor_(resizeAs)(state, self_, src); </del> <ins> THCTensor_(resizeAs)(state, self_, src_); </ins> <del> if (!THC_pointwiseApply2(state, self_, src, op)) { </del> <ins> if (!THC_pointwiseApply2(state, self_, src_, op)) { </ins> THArgCheck(false, 2, CUTORCH_DIM_WARNING); } } <del> if (self_ == src_) THCTensor_(freeCopyTo)(state, src, src_); </del> THCudaCheck(cudaGetLastError()); }

On recent PyTorch master (built from source; 0.4.0a0+) on a P100, CUDA 9:
I think the results on CPU meets the defination of triu (upper triangular part of the matrix). It is really hard to handle with a tensor from expand operation with multi-thread not only on GPU but also on CPU. The unexpected result will also happen on CPU if we apply openmp to it. I think it is necessary to eleminate the difference between the implementation and defination.
It doesn’t matter if it’s cpu or not. triu doesn’t have an underscore which means it doesn’t operate in-place and should always give the same result (in new memory)


Question: line_search_fn = group['line_search_fn'] history_size = group['history_size'] <del> state = self.state['global_state'] </del> <ins> # NOTE: LBFGS has only global state, but we register it as state for # the first param, because this helps with casting in load_state_dict state = self.state[self._params[0]] </ins> state.setdefault('func_evals', 0) state.setdefault('n_iter', 0)

Hi, I encountered this bug: The code skeleton is like: It seems the loaded are , and I've tried a workaround to move to , but it still has the same bug.
Could you provide a full script to reproduce the problem?
maybe you can try like this, () (beta1).add_(1 - beta1, ())
Sorry, I missed the reply email. I am afraid that I am unable to provide a reproducer now. It is a work I am doing for the OpenNMT-py , trying to use for doing lr update. And I encoutered this problem when testing the case. So I factored out the code skeleton about this problem above. I've tried several methods, including tricks like what suggests, but it still happens. Per my analysis, it is because the previous training was done on gpu, so when saving the , the stored states(tensors) are of version. During resuming, when we load the saved optimizer, loads this version to cpu(the model() can be moved to gpu easily, but seems lacking this ability?) , so this problem emerges.
Try moving optimizer state to the GPU memory manually after loading it from the checkpoint. I agree that having an method for this operation would be nice.
thanks. My work was suspended due to other problems, when resumed, I will try your method.
I'm afraid solution won't work. It will make the error go away, but your optimizer will no longer be training the model. You should recreate optimizers after casting modules to a different type or device, and you can use to restore the state from a previous copy. This currently doesn't work, but we should fix it (by copying from the data from the state dict, instead of using the tensors directly - this allows for cross-device or cross-type updates).
, yep, your method is what I currently use, it works. But I will wait for upstream to fix this problem though. Thanks for your great works!
Ah, my bad. I forgot to update the line where the optimizer is recreated. But otherwise, the following should do the job, right?
ah, right. That should work
Except that you should use instead of
I had a similar problem. When I save the optimizer state from a GPU other than GPU 0 and then load the state it still loads everything to GPU 0.

Specifying in didn't work either. 's solution solves this though.
Hi guys, I have a very similar problem as the one in this thread, here's my code: And then once I resume, I got KeyErrors on my optimizer: Do you guys know how to fix this issue? BTW, I have 8 GPUs used, I'm guessing if this issue was because of that?
were you able to solve your issue with multiple-GPUs?
Thanks for asking. Yeah, I'm able to, turns out the issue is because I used an early version of PyTorch, after I updated the version, it works like a charm!
Just a comment, this problem is caused by makes all state tensor are moved into GPU0 so by moving the state of an optimizer to specific GPU will fix this problem.
Hi how do you know the specific GPU location of different tensors in advance?
Would a feature where all () calls always makes use of an automatically generated CPU version be feasible ? And then at resume the () would make use of the "current" device being used (or any better strategy). At the moment it seems we need a lot of boilerplater code to ensure saving and loading are consistents across devices for models/optimizer/scheduler/etc.
I have met similar problem, I recreated Adam optimizer without () after reloading model, () and DataParallel(model) according to solution.
thanks, it work!
Hi, as you say that: every time moving model to other device, we should build optimizer, but, if we move the model to other device and move back, should we build the optimizer again? here is an example code: does optimizer run as expected? also, if doing , should we rebuild optimizer ?
Is there a problem if you switch the order to something like this? Meaning moving the model to 'cuda' but only loading it's state dict from checkpoint after loading the optimizer's state dict first?
The problem can be concluded that the optimizer's state will be loaded to the device as same as the model. You must load the model to GPU at first, and then load the optimizer's state. So that both the model and the optimizer's state are loaded in GPU.

Instead of moving optimizer to cuda after loading it in cpu, you could load the checkpoint directly in cuda:
I've independently rediscovered that this works :) Should read until the end of the thread next time
I find my codes still have the problem. I tried my best to range the modules as the examples shown in the above. can anyone give me some hints? I also tried this
For me, doesn't solve the issue, but the one by works. Any idea why? I create the optimizer, load the state (with map location to CUDA), pass to the train loop, where the mode is pushed to device (though I assume it's not needed if the loaded model is pushed to cuda by maplocation). Specifically, I would get stuck at call, throwing the device mismatch error.
Still it doesn't work for me. Could you please help me?


Question: inputs, module_kwargs = scatter_kwargs(inputs, module_kwargs, device_ids, dim) if len(device_ids) == 1: return module(*inputs[0], **module_kwargs[0]) <del> replicas = replicate(module, device_ids[:len(inputs)]) outputs = parallel_apply(replicas, inputs, module_kwargs, device_ids) </del> <ins> used_device_ids = device_ids[:len(inputs)] replicas = replicate(module, used_device_ids) outputs = parallel_apply(replicas, inputs, module_kwargs, used_device_ids) </ins> return gather(outputs, output_device, dim)

has broken DataParallel tests. Now that deviceids are explicitly sent to parallelapply, this assert gets triggered if inputs are not big enough to be on all devices (e.g. batch size of 20 on 8 GPUs gets chunked into 6*3+2, so that 8-th GPU is idle, and assert gets triggered).
your latest fix in doesn't fix this completely. Here's a failing stack-trace:
How is it possible that the tests didn't catch that in the PR?
it's because our build box which only has 2 GPUs doesn't hit this case. It's failing on 4-GPU setting (maybe batch size becomes too small then)


Question: grad = grad.index(c); } <del> int64_t num_features = grad.size(-1); </del> <ins> int64_t num_features = grad_.size(-1); </ins> auto weight_size = std::array<int64_t, 2>{{ num_weights, num_features }}; <ins> auto& dense_type = grad.type(); auto& sparse_type = dense_type.toBackend(grad.is_cuda() ? kSparseCUDA : kSparseCPU); // check if all our grad come from padding_idx if (grad.numel() == 0) { return sparse_type.sparse_coo_tensor(indices_.type().tensor(), dense_type.tensor(), weight_size); } </ins> auto index = indices.view({1, -1}); auto values = grad.contiguous().view({-1, num_features}); <del> auto& sparse_type = grad.type().toBackend(grad.is_cuda() ? kSparseCUDA : kSparseCPU); </del> return sparse_type.sparse_coo_tensor(index, values, weight_size); }

I have a straightforward use case. Given a token (or word), lookup the embedding of the word, pass that through a linear layer and perform L2 regression. If the token corresponds to the of the embedding layer, throws an error saying OS: macOS High Sierra PyTorch version: 0.3.1 How you installed PyTorch (conda, pip, source): Tried both conda and pip Python version: 3.6.4 CUDA/cuDNN version: CPU GPU models and configuration: NA GCC version (if compiling from source): NA A script to reproduce the bug:
I got the same trouble, and it throw this error after some epochs of training. Does anyone have some solutions?


Question: THCTensor *newValues = THCTensor_(newNarrow)(state, sorted, dimension, k, 1); THCudaLongTensor *newIndices = THCudaLongTensor_newNarrow(state, sorted_indices, dimension, k, 1); <ins> THCTensor_(free)(state, sorted); THCudaLongTensor_free(state, sorted_indices); </ins> if (!keepdim) { THCTensor_(squeeze1d)(state, newValues, newValues, dimension); THCudaLongTensor_squeeze1d(state, newIndices, newIndices, dimension);

has a memory leak on the GPU when using PyTorch version 0.3.1. The memory leak goes away if I downgrade to PyTorch 0.3.0. I was able to reproduce the memory leak on 3 separate machines using P100s, 1080TIs and Titan Xs.How you installed PyTorch (conda, pip, source): conda OS: Ubuntu 16.04 PyTorch version: 0.3.1.post3 Python version: 3.6.3 CUDA version: 8.0.61 GPU models and configuration: 4x P100s, 2x 1080 TIs, 4x Titan X (3 separate machines)
thanks for the report. I can reproduce the bug on master. We'll get it fixed.


Question: Best practices -------------- <ins> Device-agnostic code ^^^^^^^^^^^^^^^^^^^^ Due to the structure of PyTorch, you may need to explicitly write device-agnostic (CPU or GPU) code; an example may be creating a new tensor as the initial hidden state of a recurrent neural network. The first step is to determine whether the GPU should be used or not. A common pattern is to use Python's `argparse` module to read in user arguments, and have a flag that can be used to disable CUDA, in combination with `torch.cuda.is_available()`. In the following, `args.cuda` results in a flag that can be used to cast tensors and modules to CUDA if desired:: import argparse import torch parser = argparse.ArgumentParser(description='PyTorch Example') parser.add_argument('--disable-cuda', action='store_true', help='Disable CUDA') args = parser.parse_args() args.cuda = not args.disable_cuda and torch.cuda.is_available() If modules or tensors need to be sent to the GPU, `args.cuda` can be used as follows:: x = torch.Tensor(8, 42) net = Network() if args.cuda: x = x.cuda() net.cuda() When creating tensors, an alternative to the if statement is to have a default datatype defined, and cast all tensors using that. An example when using a dataloader would be as follows:: dtype = torch.cuda.FloatTensor for i, x in enumerate(train_loader): x = Variable(x.type(dtype)) When working with multiple GPUs on a system, you can use the `CUDA_VISIBLE_DEVICES` environment flag to manage which GPUs are available to PyTorch. To manually control which GPU a tensor is created on, the best practice is to use the `torch.cuda.device()` context manager:: print("Outside device is 0")  # On device 0 (default in most scenarios) with torch.cuda.device(1): print("Inside device is 1")  # On device 1 print("Outside device is still 0")  # On device 0 If you have a tensor and would like to create a new tensor of the same type on the same device, then you can use the `.new()` function, which acts the same as a normal tensor constructor. Whilst the previously mentioned methods depend on the current GPU context, `new()` preserves the device of the original tensor. This is the recommended practice when creating modules in which new tensors/variables need to be created internally during the forward pass:: x_cpu = torch.FloatTensor(1) x_gpu = torch.cuda.FloatTensor(1) x_cpu_long = torch.LongTensor(1) y_cpu = x_cpu.new(8, 10, 10).fill_(0.3) y_gpu = x_gpu.new(x_gpu.size()).fill_(-5) y_cpu_long = x_cpu_long.new([[1, 2, 3]]) If you want to create a tensor of the same type and size of another tensor, and fill it with either ones or zeros, `torch.ones_like()` or `torch.zeros_like()` are provided as more convenient functions (which also preserve device):: x_cpu = torch.FloatTensor(1) x_gpu = torch.cuda.FloatTensor(1) y_cpu = torch.ones_like(x_cpu) y_gpu = torch.zeros_like(x_gpu) </ins> Use pinned memory buffers ^^^^^^^^^^^^^^^^^^^^^^^^^

A common issue - for both new and more seasoned users, including myself - is how to write nice device-agnostic code. There seems to be several potential solutions (using a global CUDA flag, always calling , using etc.), used by various people in various bits of code. Ideally, we should have "best practices" documented somewhere, like ; or at least some place where we can link to.
Alternative proposal - we provide a method that deals with the most common use cases (tensors, variables and networks). We could have something similar to Chainer's , something like a , etc. We're not going to be "write once, run everywhere", but we can probably catch most use cases. Pros: It makes it much simpler for people writing CPU/1 GPU code (presumably a majority of users). Depending on the approach it'll probably be fine for multiple GPUs too. Cons: An abstraction of this form isn't foolproof, but I can't immediately think of any major issues. Setting everything to CUDA by default is probably going to break things, but a wrapper, being more manual, can be used selectively enough.
I agree we should figure out a way to make writing device-agnostic code easier. I don't think fakecuda is a good idea, it's too magical and has global side effects. Here's a list of options I see, along with some comments. In general I'm not 100% happy with any of those, but I don't have any better ideas right now. Example: I think that sooner or later we should add arg to functions anyway, so it seems like a good enough extension. I think I'm fine with this option too, but I'm afraid you would either need a single global module captured inside functions, or use it everywhere inside. In general it can be simulated using the approach using partials (of course this is an inefficient implementation, but you get the idea): This is fine, except that you have to call it a lot and it makes the code super verbose. A similar alternative that people sometimes use now is this: This is really the best option, as it will give you a tensor of exactly the same type and on exactly the same device as the source. This is usually the desired behaviour, which is why we use it all over the place in the core library.

On the other hand, it feels a bit unnatural, and is annoying when you do more complicated things, because you first construct the tensor and only then fill it with the data. and helps a lot with that
Thanks for more in-depth feedback on some options. For existing practices, highlighting , and are probably the way to go. In terms of what to do when there aren't existing tensors, what I see as the main target is the . Precisely, can we do and without using the if statement? The cast/dtype solution seems good for tensors, but anything we can do for modules?
I've put together some of this and other stuff I know into "best practices" documentation in I think we still want to think about future possibilities, but there's plenty that's worth noting down now for users.
It would indeed be amazing if PyTorch could make it simpler to write device agnostic code. I really like your first suggestion. Just a quick note from my (applied) perspective: maybe it would be possible to split up the and the into two attributes? First of all, I think this would make it easier to handle cases where tensors are on different gpus and second of all, it would allow to write code like this: ((5, 5), , ) # Returns on same device as x Don't know how practical this is though.
one thing I like about suggestion is that it makes it explicit and obvious when something is happening on cuda, rather than requiring users to know that non-negative device numbers refer to cuda. Maybe this is a bad idea, but what about also taking an optional kw in every function that takes a but to have both and non- dtypes? So passing would give you a cuda float tensor on the current GPU, passing would give you a CPU float, and passing would give you a cuda tensor on GPU 5. Obviously we'd need error checking to check things like non- devices don't have non-negative s specified (which is kind of silly because if the device is specified it tells you everything), but it does seem to make the common case of not specifying a device more clear.
Speaking only as an end-use with no particular insight into how difficult it would be to code: These suggestions make me feel like end users will end up having to think more about device semantics, not less.

I'd like to see the API move away from having the tensor type imply the storage location entirely. Have a context manager that sets cpu vs. cuda, default device, etc. Something like: Then once this exists, class wraps its calls to the with an appropriate context, as do Modules. I think that this results in everything Just Working(tm).
I'm not convinced that making it easy for end users to avoid thinking about device semantics is the correct goal. The different devices have different performance characteristics, amount of memory/cache, incur data transfer costs/synchronizations, etc. My worry is that by papering over these issues, we make it really easy to write non-performant code and really difficult to figure out why. This discussion is similar to a discussion in the distributed systems community: is a good discussion of why trying to unify all interfaces might not be the best idea (there are admittedly additional complexities in distributed computing like partial failures). Anyway, that's why I like making it obvious when there are cuda storage / device transfers going on, because those are obvious places to look at performance.
I'm not an expert on any of this, but about the only two things that bug me about using is when I have to cast are I don't have a prototypical tensor at hand (e.g. in a module it feels unnatural to use so it might be neat to have a "canonical tensor" in modules that reacts to .cuda() and whose is exposed as or so. I want a different type is just a bit much. Other than that, I think using it is cool, I think that there is not much advantage in over But this is only from my limited experience.
Regarding the "canonical tensor" it's hard, because the module can have a lot of different parameters that have different types and are on different devices. I agree that chains are awful. I think (and similar for all other types) would be a convenient helper.
Oh yes, something like would be cool (or ). For the "canonical tensor", I imagine that, for basic applications, having something that "does the right thing" whether called before or after would be great. For more advanced applications, the user could create their own set of prototype tensors and just do , or so. But that is just pure imagination, as I don't have multiple GPUs.

:slightlysmilingface:
there are several potential goals here, and they are somewhat conflicting. One is to make CPU/GPU code seamless for end users, with just one flag somewhere. On the other hand, that's likely to results in less-performant code and harder debugging. It seems like the conversation is going towards device-aware code, but making it a little less clunky in places. Sounds like is a good step forward, and was probably in the pipeline anyway?
a latecomer here: i agree that it's not a good idea to hide things behind the curtain from users. though, in many cases, it might be easier to to have a global behavior of ignoring, e.g., gpu (i.e., make .cuda() do nothing). could we at least give some kind of global switch allowing this behiavour?
Closing as 0.4 has plenty of options for this


Question: os.mkdir(os.path.join(tmp_dir, 'barrier')) os.mkdir(os.path.join(tmp_dir, 'test_dir')) if backend == 'mpi': <del> mpiexec = 'mpiexec -n 3 --noprefix {}'.format(python) </del> <ins> # test mpiexec for --noprefix option devnull = open(os.devnull, 'w') noprefix_opt = '--noprefix' if subprocess.call( 'mpiexec -n 1 --noprefix bash -c ""', shell=True, stdout=devnull, stderr=subprocess.STDOUT) == 0 else '' mpiexec = 'mpiexec -n 3 {} {}'.format(noprefix_opt, python) </ins> return_code = run_test(mpiexec, test_module, test_directory, options) else:

I installed to get available, but the test run failed this way: Log: So it sounds like we need a specific MPI backend for the tests, any old one won't do. CC and
Hmm can you try removing ?
Thank you, It's work!
I am sorry and , how to remove --noprefix,,? should I reinstall the mpi using --prefix,,? thank you
It needs to be removed from the script
Thank you so much adam, I have solved it Sent from Mail for Windows 10


Question: #include "function.h" <ins> #include <string> </ins> #include <THPP/THPP.h> #include "variable.h"

Version include <stringin some cpp files: define order in The pre-defined struct is not passed by clang in macOS, so we should reorder them.
Simple fix


Question: ('cross', (S, 3), ((S, 3),)), ('cross', (S, 3, S), ((S, 3, S), 1), 'dim'), ('inverse', (S, S), (), '', (), [skipIfNoLapack]), <ins> ('gesv', (S, S), ((S, S),), '', (), [skipIfNoLapack]), </ins> ('clone', (S, M, S), ()), ('eq', (S, S, S), ((S, S, S),)), ('ne', (S, S, S), ((S, S, S),)),

There seems to be regression in the dynamo export of the MONAI VNet model as regards . Export script: The versions I'm using is torch 2.5.0.dev20240625+cu121 Previous torch versions such as were able to export this model successfully. Perhaps dynamo was generating FX graph without the expm1 node previously? cc: PyTorch version: 2.5.0.dev20240625+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 22.04.4 LTS (x8664) GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Clang version: Could not collect CMake version: version 3.29.0 Libc version: glibc-2.35 Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime) Python platform: Linux-6.5.0-35-generic-x8664-with-glibc2.35 Is CUDA available: True CUDA runtime version: 12.4.131 CUDAMODULELOADING set to: LAZY GPU models and configuration: GPU 0: NVIDIA RTX A5000 Nvidia driver version: 535.171.04 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True CPU: Architecture: x8664 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 20 On-line CPU(s) list: 0-19 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i9-9820X CPU @ 3.

30GHz CPU family: 6 Model: 85 Thread(s) per core: 2 Core(s) per socket: 10 Socket(s): 1 Stepping: 4 CPU max MHz: 4200.0000 CPU min MHz: 1200.0000 BogoMIPS: 6599.98 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constanttsc art archperfmon pebs bts repgood nopl xtopology nonstoptsc cpuid aperfmperf pni pclmulqdq dtes64 monitor dscpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse41 sse42 x2apic movbe popcnt tscdeadlinetimer aes xsave avx f16c rdrand lahflm abm 3dnowprefetch cpuidfault epb catl3 cdpl3 invpcidsingle pti ssbd mba ibrs ibpb stibp tprshadow flexpriority ept vpid eptad fsgsbase tscadjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdta avx512f avx512dq rdseed adx smap clflushopt clwb intelpt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqmllc cqmoccupllc cqmmbmtotal cqmmbmlocal dtherm ida arat pln pts hwp hwpactwindow hwpepp hwppkgreq vnmi mdclear flushl1d archcapabilities Virtualization: VT-x L1d cache: 320 KiB (10 instances) L1i cache: 320 KiB (10 instances) L2 cache: 10 MiB (10 instances) L3 cache: 16.5 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-19 Vulnerability Gather data sampling: Mitigation;

Microcode Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable Vulnerability Meltdown: Mitigation; PTI Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable Vulnerability Retbleed: Mitigation; IBRS Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and _user pointer sanitization Vulnerability Spectre v2: Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable Versions of relevant libraries: [pip3] numpy==1.24.4 [pip3] onnx==1.16.0 [pip3] onnxscript==0.1.0.dev20240625 [pip3] optree==0.11.0 [pip3] pytorch-quantization==2.1.2 [pip3] pytorch-triton==3.0.0+ [pip3] torch==2.5.0.dev20240625+cu121 [pip3] torch-tensorrt==2.3.0a0 [pip3] torchaudio==2.4.0.dev20240625+cu121 [pip3] torchdata==0.7.1a0 [pip3] torchtext==0.17.0a0 [pip3] torchvision==0.20.0.dev20240625+cu121 [conda] Could not collect
I am not very sure, but this looks like the kind of ops that should get decomp'ed. do you know? could you help take a look as part of the op support work you are doing? Thanks!
Yes, it looks like there is a graph-changed in dynamo, which dynamo now decomposes elu for some reasons. Using can preserve elu and then successfully export though.

Yes can run but we would need to be resolved first since the weights are still not exported properly.
I can take a look at this.
should fix this.

Here, I can't export trivial net with sort() using dynamoexport(). Legacy () works fine, dynamoexport errors out : Unsupported FX nodes: {'callfunction': ['']}. Pytorch nightly 06/03 cc
a missing op, could you take this?
Not sure if related:
Hmmm. Ideally, ONNX should add a sort op. It doesn't have one. I wonder what the legacy exporter converts it to.
should fix this issue.

Are there any plans to implement something like tensorflow's and ? exists but it doesn't have the backward step implemented.
One workaround for the moment would be to write as , which should support backprop since . Adding support for should be trivial, the gradient for it can be found in
I'll add backward support for , which should be equivalent to . I'll try to do it later today.
How difficult would it be to add in batched versions (where only the final two dimensions are square) of GESV and inverse? Are there any plans for this?
If we have batched version of , then adapting the autograd part is very simple. There is an open issue already that discuss implementing batched gesv
Actually, there is an open PR for that, so it might be coming soon
I'll rebase the batched version of gesv. It'll come in soon
May I ask why gesv only support . As I saw from , it support that A is a MxN matrix, and B is a NxK matrix, then solve a matrix X (NxK). If I want to use a non-square matrix A in gesv, would you mind to let me know how to implement in PyTorch?
this doesn't exactly answer your question but you can add zero-rows to the A matrix until it becomes square.
thanks a lot, I find the solution, can solve the problem.
Any plan to add gradient calculation for gels?
I am also wondering if will have differentiation support. My current workaround is to use instead
I actually implemented overdetermined case following tensorflow implementation. For backward pass I only did basic grad test. On Monday, March 18, 2019, Johnson Zhong <:
Thanks, seems to work fine!


Question: lr=1e-1) ) <ins> def test_adagrad_sparse(self): self._test_rosenbrock_sparse( lambda params: optim.Adagrad(params, lr=1e-1) ) </ins> def test_adamax(self): self._test_rosenbrock( lambda params: optim.Adamax(params, lr=1e-1),

and I talked about the intended semantics of sparsemask when given a sparse tensor as an argument, and we came to the following conclusions: with an uncoalesced sparse tensor does not make sense, full stop. Unlike values, where your update might be linear and work correctly on an uncoalesced sparse tensor, there isn't any good reason why you want to pluck out values from the dense tensor multiple times, it is basically always a bug. the (eventual) documentation for sparsemask, we will recommend calling coalesce before doing any operations involving sparsemask (or nonlinear updates to indices/values). should add a test to sparsemask which checks if the flag of the input tensor is true, and raise an error if it is not. This is a cheap safety check which will prevent people from forgetting to call coalesce (as in (2)). There is a performance consequence to calling coalesce every iteration of training. If the gradient is producing coalesced vectors, we can alleviate this problem by adding a preprocessing step to coalesce() which tests if the input is already coalesced, and bugs out in that case. However, I think this is out of scope for this particular ticket for now.


Question: set /a NO_NNPACK=1 ) <ins> IF "%DEBUG%"=="1" ( set BUILD_TYPE=Debug ) ELSE ( set BUILD_TYPE=Release ) </ins> IF "%CMAKE_GENERATOR%"=="" ( set CMAKE_GENERATOR_COMMAND= set MAKE_COMMAND=msbuild INSTALL.vcxproj /p:Configuration=Release

The current master triggers the following error messages from Visual Studio 2015 (Debug build): 2------ Build started: Project: ATen, Configuration: Debug x64 ------ 2cl : Command line error D8016: '/Ox' and '/RTC1' command-line options are incompatible ... 14LINK : fatal error LNK1104: cannot open file '....srcATenDebug' It appears that they are triggered by *.c files in the ATen project, like "srcTHvectorAVX.c"
Why do you need Debug build?
Isn't ATen meant to support both Debug and Release builds? FYI I'm involved with the project at which does support both build configurations, and we would like to be able to link our project with ATen.
Well, you can remove the flags in as a workground.
We should patch this file to ignore this flag when building the library in debug mode. It shouldn't be too hard
I catched similar errors , when I Tryed to build Debug Version I try to build Release and catched next errors e:pytorchatensrcatencpuvec256vec256int.h(184): note: see reference to function template instantiation 'at::vec256::anonymous-namespace'::Vec256<int16t::blend<1(at::vec256::anonymous-namespace'::Vec256<int16t)' being compiled 28e:pytorchatensrcatencpuvec256vec256int.h(147): error C3861: 'mm256extract_epi16': identifier not found What I should do?
There're two issues in your description. The first question is easy. You'll also have to specify the flag to 1. The second question is about your compiler. I guess you are using VS2015 or earlier versions of VS2017. Make sure the version in the output is 19.11 after your enter in CMD.
Yes I am using VS2015. My Compiler is 19.0.2. Must I use VS 2017 in any case?
Sorry, but the answer is yes. The AVX2 functions was in VS2017 and they are used in PyTorch. So VS2017 19.11 is a must.
No problem:) Thank you.


Question: cudnnDestroyConvolutionDescriptor(desc); } void set(cudnnDataType_t dataType, int dim, int* pad, int* stride, int * upscale) { <ins> cudnnDataType_t mathType = dataType; if (dataType == CUDNN_DATA_HALF) mathType = CUDNN_DATA_FLOAT; </ins> CHECK(cudnnSetConvolutionNdDescriptor(desc, dim, pad, stride, upscale, <del> CUDNN_CROSS_CORRELATION, dataType)); </del> <ins> CUDNN_CROSS_CORRELATION, mathType)); </ins> } };

to reproduce Will result in:
What's your GPU and CUDA version?
TitanX Maxwell, cuda 8.0
In cudnn bindings convolution math type is inferred from input data type (and inferred to be equal to input data type): Math type for convolutions with half inputs and weights should be CUDNNDATAFLOAT because 1) computations with half math are not supported anywhere except Pascal and TX1 2) Even on Pascal, 6.1 cards have very low performance fp16 math 3) fp16 math comes with a set of accuracy problems
Thanks. Seems like batchnorm fp16 support is still missing. e.g: will result in
Ah, yes. fp16 support was removed from cudnn bn backend (because it still requires fp32 for parameters), and probably it is not properly routed to or not supported in nn backend. BTW, be careful with truncating bn params to 16 bits, in our experiments on wide resnets/cifar it cost ~ half a percent of accuracy, though there may be ways to overcome this.
Is there an update regarding this? Could you describe how to truncate the bn params?
did you update PyTorch to a recent version? The snippet no longer raises an error for me
Thanks for the fast response. I installed the newest version via conda. Just retried the above code snippet, it works indeed. Shouldn't it be possible to "half" a entire model like this: For this i get: CUDNNSTATUSBAD_PARAM
Your input size is wrong, models in torchvision expect specific input sizes. You can play around with the batch size, but number of channels and image size are fixed. When I change input size to (64,3,224,224) the snippet above does not return an error.


Question: fn.w_desc = rnn.init_weight_descriptor(fn, fn.weight_buf) # Slice off views into weight_buf <del> params = rnn.get_parameters(fn, handle, fn.weight_buf) </del> all_weights = [[p.data for p in l] for l in self.all_weights] <ins> params = rnn.get_parameters(fn, handle, fn.weight_buf) </ins> # Copy weights and update their storage rnn._copyParams(all_weights, params)

While the fast path for cuDNN RNNs introduced in includes a check intended to fall back to copying weights if weights are moved or aliased, this only works if the aliasing happens after the weights are first compacted. But the most natural place to tie weights is inside a module's method, which runs before , the typical time when is first invoked. Here's an MWE. The three runs (CPU, weight tying before , and weight tying after ) should all give the same results, and they do before , but they have not since:


Question: return !was_null && requires_grad == var.requires_grad() && is_volatile == var.is_volatile(); } <del> Node* recordTraceHelper(std::string op, at::ArrayRef<Variable> inputs, at::ArrayRef<Variable> outputs); // These overloads are intended to simplify code generation inline Node* recordTrace(std::string op, std::initializer_list<Variable> inputs, const Variable& output) { return recordTraceHelper(op, inputs, {output}); } inline Node* recordTrace(std::string op, std::initializer_list<Variable> inputs, const std::tuple<Variable, Variable>& outputs) { return recordTraceHelper(op, inputs, {std::get<0>(outputs), std::get<1>(outputs)}); } inline Node* recordTrace(std::string op, std::initializer_list<Variable> inputs, const std::tuple<Variable, Variable, Variable>& outputs) { return recordTraceHelper(op, inputs, {std::get<0>(outputs), std::get<1>(outputs), std::get<2>(outputs)}); } inline Node* recordTrace(std::string op, at::TensorList inputs, const Variable& output) { // TODO: Eliminate the intermediate vector allocation return recordTraceHelper(op, variable_list(inputs.begin(), inputs.end()), {output}); } </del> <ins> Node* recordTrace(std::string op, at::ArrayRef<Variable> inputs, at::ArrayRef<Variable> outputs); </ins> }}} // namespace torch::jit::tracer

when executing:
I'm seeing this as well as of last night. Its preceded by many warnings about : But the first error is the one about . I went back, and I get a good install as late as commit


Question: ('topk', small_3d_unique, lambda t: [2, 1, True, True], 'dim_desc_sort'), ('trace', medium_2d, lambda t: [],), ('tril', medium_2d, lambda t: [],), <ins> ('tril', medium_2d_expanded, lambda t: [], 'zero_stride', types, True), </ins> ('tril', medium_2d, lambda t: [2], 'positive'), ('tril', medium_2d, lambda t: [-2], 'negative'), ('triu', medium_2d, lambda t: [],), <ins> ('triu', medium_2d_expanded, lambda t: [], 'zero_stride', types, True), </ins> ('triu', medium_2d, lambda t: [2], 'positive'), ('triu', medium_2d, lambda t: [-2], 'negative'), ('unsqueeze', new_t(2, 3, 4), lambda t: [2],),

On recent PyTorch master (built from source; 0.4.0a0+) on a P100, CUDA 9:
I think the results on CPU meets the defination of triu (upper triangular part of the matrix). It is really hard to handle with a tensor from expand operation with multi-thread not only on GPU but also on CPU. The unexpected result will also happen on CPU if we apply openmp to it. I think it is necessary to eleminate the difference between the implementation and defination.
It doesn’t matter if it’s cpu or not. triu doesn’t have an underscore which means it doesn’t operate in-place and should always give the same result (in new memory)


Question: END_HANDLE_TH_ERRORS } <ins> #ifndef THC_GENERIC_FILE static PyObject * THPStorage_(fromBuffer)(PyObject *_unused, PyObject *args, PyObject *keywds) { HANDLE_TH_ERRORS PyObject *obj = NULL; const char* byte_order_str = NULL; Py_ssize_t count = -1, offset = 0; Py_buffer buffer; static char *kwlist[] = {"buffer", "byte_order", "count", "offset", NULL}; const char* argtypes; #if defined(TH_REAL_IS_BYTE) || defined(TH_REAL_IS_CHAR) argtypes = "O|snn"; #else argtypes = "Os|nn"; #endif if (!PyArg_ParseTupleAndKeywords(args, keywds, argtypes, kwlist, &obj, &byte_order_str, &count, &offset)) { return NULL; } #if !(defined(TH_REAL_IS_BYTE) || defined(TH_REAL_IS_CHAR)) THPByteOrder byte_order; if (strcmp(byte_order_str, "native") == 0) { byte_order = THP_nativeByteOrder(); } else if (strcmp(byte_order_str, "big") == 0) { byte_order = THP_BIG_ENDIAN; } else if (strcmp(byte_order_str, "little") == 0) { byte_order = THP_LITTLE_ENDIAN; } else { PyErr_Format(PyExc_ValueError, "invalid byte_order '%s' (expected 'big', 'little', or 'native')", byte_order_str); return NULL; } #endif if (PyObject_GetBuffer(obj, &buffer, PyBUF_SIMPLE) < 0) return NULL; if (offset < 0 || offset > buffer.len) { PyErr_Format(PyExc_ValueError, "offset must be non-negative and no greater than buffer length (%ld)", (long) buffer.len); return NULL; } if (count < 0) { if ((buffer.len - offset) % sizeof(real) != 0) { PyErr_Format(PyExc_ValueError, "buffer size must be a multiple of element size"); return NULL; } count = (buffer.len - offset) / sizeof(real); } if (offset + (count * (Py_ssize_t)sizeof(real)) > buffer.len) { PyErr_Format(PyExc_ValueError, "buffer is smaller than requested size"); return NULL; } uint8_t* src = (uint8_t*) buffer.buf; THStorage* storage = THStorage_(newWithSize)(count); #if defined(TH_REAL_IS_BYTE) || defined(TH_REAL_IS_CHAR) memcpy(storage->data, src + offset, count); #elif defined(TH_REAL_IS_SHORT) THP_decodeInt16Buffer(storage->data, src + offset, byte_order, count); #elif defined(TH_REAL_IS_INT) THP_decodeInt32Buffer(storage->data, src + offset, byte_order, count); #elif defined(TH_REAL_IS_LONG) THP_decodeInt64Buffer(storage->data, src + offset, byte_order, count); #elif defined(TH_REAL_IS_FLOAT) THP_decodeFloatBuffer(storage->data, src + offset, byte_order, count); #elif defined(TH_REAL_IS_DOUBLE) THP_decodeDoubleBuffer(storage->data, src + offset, byte_order, count); #else #error "Unknown type" #endif return (PyObject*)THPStorage_(newObject)(storage); END_HANDLE_TH_ERRORS } #endif </ins> PyObject * THPStorage_(writeFile)(THPStorage *self, PyObject *file) { HANDLE_TH_ERRORS

<!-- A clear and concise description of what the bug is. --The target name is used by a test executable in mkldnn. Steps to reproduce the behavior: BUILDTEST=1 build <!-- If you have a code sample, error messages, stack traces, please provide it here as well --<!-- A clear and concise description of what you expected to happen. --CMake configure passes. Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): v1.5.0-rc2 OS (e.g., Linux): Windows How you installed PyTorch (, , source): source Build command you used (if compiling from source): python build Python version: 3.x CUDA/cuDNN version: 9.2 / 10.1 / 10.2 / NA GPU models and configuration: Tesla M60 Any other relevant information: <!-- Add any other context about the problem here. --We should not build tests for mkldnn.
I have encountered the same error, how do I resolve it?
It is actually mkldnn that is out of sync. So running the following script should help.
Thanks a lot for this I posted this on the PyTorch Discussions but to no avail, and now I've finally understood the reason of this error. Thanks a lot. Surprisingly, for me the build works out if I build from the master repo, but changing to a different version always fails.


Question: The transposed convolution operator multiplies each input value element-wise by a learnable kernel, and sums over the outputs from all input feature planes. <del> **This module can be seen as the exact reverse of Conv3d**. It is sometimes (but incorrectly) refered to as a deconvolutional operation. </del> <ins> This module can be seen as the gradient of Conv3d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation). </ins> | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides <del> for :attr:`padding` number of points </del> <ins> for :attr:`padding` number of points. </ins> | If :attr:`output_padding` is non-zero, then the output is implicitly zero-padded on one side <del> for :attr:`output_padding` number of points | :attr:`groups` controls the connections between inputs and outputs. </del> <ins> for :attr:`output_padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. </ins> |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. <ins> At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`output_padding` can either be: <del> - a single ``int`` -- in which case the same value is used for the height and width dimension </del> <ins> - a single ``int`` -- in which case the same value is used for the depth, height and width dimensions </ins> - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the width dimension and the third `int` for the width dimension

Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

When building from source, I got this error. My system is OSX 10.12.4, conda, CUDA 8.0. Any thoughts? Thanks!
can you try adding to the file and see if that fixes things? if so i'll commit it.
actually, that file already has memory included. Are you following instructions from Especially: This part is important:
works nicely! Thanks a lot!

The suggest that takes a as a parameter, but it actually must be wrapped in a Variable first.
you are looking at the Tensor docs. When operating on Variable, it is implied from the tutorials that all functions operating on Variables have to use Variables. We didn't write separate docs for Variable itself.

something wrong with import torchvision Traceback (most recent call last): File "", line 3, in <moduleimport torchvision File "", line 4, in <moduleimport torchvision.transforms as transforms ImportError: No module named transforms
you have a file called for some reason. delete it or change your pythonpath to ignore it.
Hey, I have the same problem persisting when I import torchvision. I have no other file by the same name. ImportError Traceback (most recent call last) <ipython-input-6-3baf6837d8f1in <module() 8 from import Dataset 9 from import DataLoader ---10 from torchvision import transforms ImportError: No module named torchvision Any idea why this is happening?
Do you have torchvision installed (it is a separate package)?
pip Install torchvision try this
from data import BaseTransform,VOC_CLASSES as labelmap this is not working
pip install torchvision

Hi, I need to pass input through one nn.Module, then argmax the output, and then pass that output to a second nn.Module. I want to backprop through the argmax back to the weights of the first module. That's impossible right now, though, it seems. When I loop through the gradient of the weights of my first module (with the following code): firstmodel.zerograd() secondmodel.zerograd() loss.backward() # retainvariables=True for param in secondmodel.parameters(): print( 'data: ', ) print( 'grad data: ', ) for param in firstmodel.parameters(): print( 'data: ', ) print( 'grad data: ', ) I get the following error, stating that those gradients can no longer be viewed/exist: -main()
argmax is not differentiable (and even if you try to think about it as a function differentiable almost everywhere, its derivative is always 0, so you won't get anything meaningful).

def inithidden(self): h0 = Variable(FloatTensor(self.numlayers, self.batchlen, self.hiddendim).zero(),  requiresgrad=True) c0 = Variable(FloatTensor(self.numlayers, self.batchlen, self.hiddendim).zero(),  requiresgrad=True) return (h0, c0) def forward(self, sequences): lstmout, = ((1, self.batchlen, self.inputdim), ) ... = nn.LSTM(inputdim, hiddendim, numlayers, dropout = dropout, batchfirst=True) = self.inithidden() output = model(traininput) produces the following error although I defined weight and input dimensions as in the documentation: File "/usr/local/lib/python2.7/dist-", line 202, in call result = self.forward(input, kwargs) File "/usr/local/lib/python2.7/dist-", line 91, in forward output, hidden = func(input, self.allweights, hx) File "/usr/local/lib/python2.7/dist-", line 327, in forward return func(input, fargs, fkwargs) File "/usr/local/lib/python2.7/dist-", line 227, in forward nexth, output = func(input, hidden, weight) File "/usr/local/lib/python2.7/dist-", line 67, in forward hy, output = inner(input, hidden[l], weight[l]) File "/usr/local/lib/python2.7/dist-", line 96, in forward hidden = inner(input[i], hidden, *weight) File "/usr/local/lib/python2.7/dist-", line 22, in LSTMCell gates = F.linear(input, wih, bih) + F.linear(hx, whh, bhh) File "/usr/local/lib/python2.7/dist-", line 744, in add return (other) File "/usr/local/lib/python2.7/dist-", line 293, in add return (other, False) File "/usr/local/lib/python2.7/dist-", line 287, in add return Add(inplace)(self, other) File "/usr/local/lib/python2.7/dist-", line 20, in forward return a.add(b) RuntimeError: inconsistent tensor size at /data/users/soumith/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensorMath.c:827
I can't reproduce your problem. Could you include a short runnable example that exhibits this behavior?

You also don't need to manually init hidden anymore, the following would work as well...
Thanks! After removing problem solved.

Hi, I would like to implement a sharing weight 'alexnet' with two input batches of images, and the fully connected layers are concatenated together like this: However, when I run it, I come across the following error: Does anyone know about it? Thank you very much.
You forgot to call on the output of . In the future please remember that we're using the issues for bug reports and feature requests only. If you have questions or need help, please post .


Question: flat_input = tuple(_iter_variables(input)) flat_output = super(NestedIOFunction, self)._do_forward(*flat_input) nested_output = self._nested_output <del> nested_variables = _map_tensor_fromiter(iter(flat_output))(self._nested_output) </del> <ins> nested_variables = _unflatten(flat_output, self._nested_output) </ins> return nested_variables def backward(self, *gradients): <del> nested_gradients = _map_tensor_fromiter(iter(gradients))(self._nested_output) </del> <ins> nested_gradients = _unflatten(gradients, self._nested_output) </ins> del self._nested_output result = self.backward_extended(*nested_gradients) del self._to_save_nested

cuDNN backend for RNN stabilizes at large mem usage. The following code stabilizes for me at 14+GiB reported by nvidia-smi. The same RNN without cuDNN uses <2.5GiB.
Oops... any idea what's going on with the garbage collector in this case?
are you running this on a P100 GPU?
Yes, I observe similar behavior on Titan X Pascal
hmmm, do cudnn RNNs have weird memory-oversubscribing properties? If it wasn't for CuDNN RNN, then it's not unreasonable. we use a custom CUDA allocator that will oversubscribe if needed, but will free the memory blocks it's holding onto if it hits OOM / failed allocations.
Tried the same RNN in torch and am getting ~2.6GiB on nvidia-smi.
With cuDNN I'm getting a constant usage of 2241MB on my machine using PyTorch (M40).
are you using cudnnv5, or v6?
v5
no, cudnn does not do memory oversubscribing, and already tried the same network in lua torch with cudnn, where it gets expected memory usage. Also, he is getting OOMs on slightly bigger networks, where apparently caching allocator should have tried to free previous blocks. Interesting that can not repro this.
Just to make sure did you set when you ran? I left it as False in the repro.
I can repro in the docker image built from the current version on P100, and I go OOM after 8 batches with the same image on 1080, which generally should not happen - if there is enough memory to go through 1 batch, memory usage should not increase afterwards.
No, I just copied your script and ran it. Weird thing is that it goes OOM after 18 batches once I enable . Without it memory remains stable.
A lot of cudaMalloc allocations go not through THCudaMalloc, and never get freed:
So not going through THCudaMalloc is actually expected for storages allocated with THCStorage_(newWithSize), but what's not expected is to every time cudaMalloc bytes of rnn workspace, instead of reusing earlier allocations. (~784 MB allocations are for workspace, ~156MB allocations happen somewhere in criterion).
My guess is that the bug is at the Python level.

Either: Something is holding on to a Tensor longer than it should Something is creating a circular reference which prevents quick clean-up
The issue is that in creates cyclic references. It's made worse by the fact that it holds on to more memory than it should (like the workspace). You can see this by adding a as the first line in your for loop. With the explicit call to the GC every iteration, the memory usage doesn't explode.
how can these cyclic references be fixed?
Not sure yet, but I will fix it tomorrow with


Question: strt = _tensor_str(self) size_str = 'x'.join(str(size) for size in self.size()) <del> device_str = '' if not self.is_cuda else ' (GPU {})'.format(self.get_device()) </del> <ins> device_str = '' if not self.is_cuda else  ' (GPU {})'.format(self.get_device()) </ins> strt += '[{} of size {}{}]n'.format(torch.typename(self), <del> size_str, device_str) </del> <ins> size_str, device_str) </ins> return 'n' + strt

It seems that the following link that can be reached from here (check the screenshot) is broken: Page: Broken link: Screenshot: ! cc
this should be transferred to , but I don't have permissions to do that. Fix is available at pytorch/hub.


Question: <ins> #include "byte_order.h" static inline uint16_t decodeUInt16LE(const uint8_t *data) { return (data[0]<<0) | (data[1]<<8); } static inline uint16_t decodeUInt16BE(const uint8_t *data) { return (data[1]<<0) | (data[0]<<8); } static inline uint32_t decodeUInt32LE(const uint8_t *data) { return (data[0]<<0) | (data[1]<<8) | (data[2]<<16) | (data[3]<<24); } static inline uint32_t decodeUInt32BE(const uint8_t *data) { return (data[3]<<0) | (data[2]<<8) | (data[1]<<16) | (data[0]<<24); } static inline uint64_t decodeUInt64LE(const uint8_t *data) { return (((uint64_t)data[0])<< 0) | (((uint64_t)data[1])<< 8) | (((uint64_t)data[2])<<16) | (((uint64_t)data[3])<<24) | (((uint64_t)data[4])<<32) | (((uint64_t)data[5])<<40) | (((uint64_t)data[6])<<48) | (((uint64_t)data[7])<<56); } static inline uint64_t decodeUInt64BE(const uint8_t *data) { return (((uint64_t)data[7])<< 0) | (((uint64_t)data[7])<< 8) | (((uint64_t)data[5])<<16) | (((uint64_t)data[4])<<24) | (((uint64_t)data[3])<<32) | (((uint64_t)data[2])<<40) | (((uint64_t)data[1])<<48) | (((uint64_t)data[0])<<56); } THPByteOrder THP_nativeByteOrder() { uint32_t x = 1; return *(uint8_t*)&x ? THP_LITTLE_ENDIAN : THP_BIG_ENDIAN; } void THP_decodeInt16Buffer(int16_t* dst, const uint8_t* src, THPByteOrder order, size_t len) { for (size_t i = 0; i < len; i++) { dst[i] = (int16_t) (order == THP_BIG_ENDIAN ? decodeUInt16BE(src) : decodeUInt16LE(src)); src += sizeof(int16_t); } } void THP_decodeInt32Buffer(int32_t* dst, const uint8_t* src, THPByteOrder order, size_t len) { for (size_t i = 0; i < len; i++) { dst[i] = (int32_t) (order == THP_BIG_ENDIAN ? decodeUInt32BE(src) : decodeUInt32LE(src)); src += sizeof(int32_t); } } void THP_decodeInt64Buffer(int64_t* dst, const uint8_t* src, THPByteOrder order, size_t len) { for (size_t i = 0; i < len; i++) { dst[i] = (int64_t) (order == THP_BIG_ENDIAN ? decodeUInt64BE(src) : decodeUInt64LE(src)); src += sizeof(int64_t); } } void THP_decodeFloatBuffer(float* dst, const uint8_t* src, THPByteOrder order, size_t len) { for (size_t i = 0; i < len; i++) { union { uint32_t x; float f; }; x = (order == THP_BIG_ENDIAN ? decodeUInt32BE(src) : decodeUInt32LE(src)); dst[i] = f; src += sizeof(float); } } void THP_decodeDoubleBuffer(double* dst, const uint8_t* src, THPByteOrder order, size_t len) { for (size_t i = 0; i < len; i++) { union { uint64_t x; double d; }; x = (order == THP_BIG_ENDIAN ? decodeUInt64BE(src) : decodeUInt64LE(src)); dst[i] = d; src += sizeof(double); } } </ins>

<!-- A clear and concise description of what the bug is. --The target name is used by a test executable in mkldnn. Steps to reproduce the behavior: BUILDTEST=1 build <!-- If you have a code sample, error messages, stack traces, please provide it here as well --<!-- A clear and concise description of what you expected to happen. --CMake configure passes. Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): v1.5.0-rc2 OS (e.g., Linux): Windows How you installed PyTorch (, , source): source Build command you used (if compiling from source): python build Python version: 3.x CUDA/cuDNN version: 9.2 / 10.1 / 10.2 / NA GPU models and configuration: Tesla M60 Any other relevant information: <!-- Add any other context about the problem here. --We should not build tests for mkldnn.
I have encountered the same error, how do I resolve it?
It is actually mkldnn that is out of sync. So running the following script should help.
Thanks a lot for this I posted this on the PyTorch Discussions but to no avail, and now I've finally understood the reason of this error. Thanks a lot. Surprisingly, for me the build works out if I build from the master repo, but changing to a different version always fails.


Question: optimizer (Optimizer): Wrapped optimizer. milestones (list): List of epoch indices. Must be increasing. gamma (float): Multiplicative factor of learning rate decay. <del> Default: -0.1. </del> <ins> Default: 0.1. </ins> last_epoch (int): The index of last epoch. Default: -1. Example:

as shown in source code, this parameters is not used. Any reason that we keep it?
this can be cleaned up.


Question: """Fills the input Tensor or Variable with values according to the method described in "Understanding the difficulty of training deep feedforward neural networks" - Glorot, X. & Bengio, Y. (2010), using a uniform distribution. The resulting tensor will have values sampled from :math:`U(-a, a)` where <del> :math:`a = gain times sqrt{2 / (fan_in + fan_out)} times sqrt{3}`. Also known as Glorot initialisation. </del> <ins> :math:`a = gain times sqrt{2 / (fan_in + fan_out)} times sqrt{3}`. Also known as Glorot initialisation. </ins> Args: tensor: an n-dimensional torch.Tensor or autograd.Variable

why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:


Question: AT_PRIVATE_CASE_TYPE(at::ScalarType::Long, int64_t, __VA_ARGS__)         AT_PRIVATE_CASE_TYPE(at::ScalarType::Short, int16_t, __VA_ARGS__)        default:                                                                 <del> AT_ERROR("%s not implemented for '%s'", (NAME), the_type.toString());  </del> <ins> AT_ERROR(#NAME, " not implemented for '", the_type.toString(), "'");  </ins> }                                                                          }()

is not available in the docs page, but the doc string has been in . This was mentioned earlier in

Oh, bilinear, I missed that last time I looked at it. I'll add a docstring.

returns nothing. Might be useful to have some linter in the build process to find Python functions missing

Pytorch 0.4.0 gives a segmentation fault with dilated 2D convolutions over a specific size. import torch import as nn model = nn.Conv2d(inchannels=64, outchannels=64, kernelsize=5, dilation=2) model(([1, 64, 1166, 1166])) # This runs model(([1, 64, 1167, 1167])) # This crashes PyTorch version: 0.4.0 Is debug build: No CUDA used to build PyTorch: 9.1.85 OS: CentOS Linux 7 (Core) GCC version: (GCC) 4.8.5 (Red Hat 4.8.5-28) CMake version: version 2.8.12.2 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 8.0.61 GPU models and configuration: GPU 0: GeForce GTX 1080 Ti GPU 1: TITAN X (Pascal) GPU 2: Quadro K2200 Nvidia driver version: 390.59 cuDNN version: Probably one of the following: /usr/local/cuda- /usr/local/cuda- /usr/local/cuda-7.5/lib64/libcudnnstatic.a /usr/local/cuda- /usr/local/cuda-8.0/lib64/libcudnn_static.a
do you think this is the same as the segfault you showed me a week ago on convolution?
ah yes. int overflow. I only fixed conv3d because I didn’t want to hurt perf. I’ll fix all of them.

This should be an easy one.. The function is implemented, just no sign of it in the docs.


Question: char *bytes = (char *) data; int64_t remaining = sizeof(real) * size; while (remaining > 0) { <del> ssize_t result = write(fd, bytes, remaining); </del> <ins> // we write and read in 1GB blocks to avoid bugs on some OSes ssize_t result = write(fd, bytes, THMin(remaining, 1073741824)); </ins> if (result < 0) throw std::system_error(result, std::system_category()); bytes += result;

Running this on OSX: More details of failure in Ubuntu 14.04.1 in
Maybe it's caused by a ? I just test this code and it doesn't work (on Mac OS X 10.12.3 with Python 3.6):
That thread links to this one , which says it’s an OS bug and proposes a workaround patch that hasn’t been merged. There’s also a numpy issue thread, which links to this Torch7 commit that implements a workaround
We're doing a loop in case it writes less, but we always pass the full amount of remaining bytes. Putting a cap (std::min with INT_MAX) on the size of a single write might help.
I can't reproduce it on CentOS. I think it might have been a report for a PyTorch version prior to fixing .


Question: c[1].fill_(20) self.assertEqual(c[1], c[3], 0) <ins> def test_from_buffer(self): a = bytearray([1, 2, 3, 4]) self.assertEqual(torch.ByteStorage.from_buffer(a).tolist(), [1, 2, 3, 4]) shorts = torch.ShortStorage.from_buffer(a, 'big') self.assertEqual(shorts.size(), 2) self.assertEqual(shorts.tolist(), [258, 772]) ints = torch.IntStorage.from_buffer(a, 'little') self.assertEqual(ints.size(), 1) self.assertEqual(ints[0], 67305985) f = bytearray([0x40, 0x10, 0x00, 0x00]) floats = torch.FloatStorage.from_buffer(f, 'big') self.assertEqual(floats.size(), 1) self.assertEqual(floats[0], 2.25) </ins> if __name__ == '__main__': unittest.main()

<!-- A clear and concise description of what the bug is. --The target name is used by a test executable in mkldnn. Steps to reproduce the behavior: BUILDTEST=1 build <!-- If you have a code sample, error messages, stack traces, please provide it here as well --<!-- A clear and concise description of what you expected to happen. --CMake configure passes. Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): v1.5.0-rc2 OS (e.g., Linux): Windows How you installed PyTorch (, , source): source Build command you used (if compiling from source): python build Python version: 3.x CUDA/cuDNN version: 9.2 / 10.1 / 10.2 / NA GPU models and configuration: Tesla M60 Any other relevant information: <!-- Add any other context about the problem here. --We should not build tests for mkldnn.
I have encountered the same error, how do I resolve it?
It is actually mkldnn that is out of sync. So running the following script should help.
Thanks a lot for this I posted this on the PyTorch Discussions but to no avail, and now I've finally understood the reason of this error. Thanks a lot. Surprisingly, for me the build works out if I build from the master repo, but changing to a different version always fails.


Question: # use faster C loader if available from yaml import CLoader as YamlLoader except ImportError: <del> from yaml import YamlLoader </del> <ins> from yaml import Loader as YamlLoader </ins> GENERATED_COMMENT = CodeTemplate("""

Build error: According to , the should be modified as:
I the same problem when installing pytorch from source. You mean the file is wrong?
Although the PyYAML documentation site suggests that is possible, I could not perform it. That is a reason for the failing. This has been reported as well: I believe that the should have this instead: In fact when I did this: This indicates that the import didn't happen in PyYAML itself.
you guys are right, the case has to be modified to: . The original issue (my guess) is that your pyyaml had some compile error and the CLoader wasn't successfully installed.


Question: TEST_CUDA = torch.cuda.is_available() TEST_MULTIGPU = TEST_CUDA and torch.cuda.device_count() >= 2 TEST_CUDNN = TEST_CUDA and torch.backends.cudnn.is_acceptable(torch.cuda.FloatTensor(1)) <ins> TEST_CUDNN_VERSION = TEST_CUDNN and torch.backends.cudnn.version() </ins> PRECISION = 1e-5 module_tests = [

cuDNN backend for RNN stabilizes at large mem usage. The following code stabilizes for me at 14+GiB reported by nvidia-smi. The same RNN without cuDNN uses <2.5GiB.
Oops... any idea what's going on with the garbage collector in this case?
are you running this on a P100 GPU?
Yes, I observe similar behavior on Titan X Pascal
hmmm, do cudnn RNNs have weird memory-oversubscribing properties? If it wasn't for CuDNN RNN, then it's not unreasonable. we use a custom CUDA allocator that will oversubscribe if needed, but will free the memory blocks it's holding onto if it hits OOM / failed allocations.
Tried the same RNN in torch and am getting ~2.6GiB on nvidia-smi.
With cuDNN I'm getting a constant usage of 2241MB on my machine using PyTorch (M40).
are you using cudnnv5, or v6?
v5
no, cudnn does not do memory oversubscribing, and already tried the same network in lua torch with cudnn, where it gets expected memory usage. Also, he is getting OOMs on slightly bigger networks, where apparently caching allocator should have tried to free previous blocks. Interesting that can not repro this.
Just to make sure did you set when you ran? I left it as False in the repro.
I can repro in the docker image built from the current version on P100, and I go OOM after 8 batches with the same image on 1080, which generally should not happen - if there is enough memory to go through 1 batch, memory usage should not increase afterwards.
No, I just copied your script and ran it. Weird thing is that it goes OOM after 18 batches once I enable . Without it memory remains stable.
A lot of cudaMalloc allocations go not through THCudaMalloc, and never get freed:
So not going through THCudaMalloc is actually expected for storages allocated with THCStorage_(newWithSize), but what's not expected is to every time cudaMalloc bytes of rnn workspace, instead of reusing earlier allocations. (~784 MB allocations are for workspace, ~156MB allocations happen somewhere in criterion).
My guess is that the bug is at the Python level.

Either: Something is holding on to a Tensor longer than it should Something is creating a circular reference which prevents quick clean-up
The issue is that in creates cyclic references. It's made worse by the fact that it holds on to more memory than it should (like the workspace). You can see this by adding a as the first line in your for loop. With the explicit call to the GC every iteration, the memory usage doesn't explode.
how can these cyclic references be fixed?
Not sure yet, but I will fix it tomorrow with


Question: getattr(self, self.id().split(".")[2])() sys.exit(0) <del> def _join_and_reduce(self): </del> <ins> def _join_and_reduce(self, fn): skip_ok = getattr(fn, "skip_if_no_cuda_distributed", False) </ins> for p in self.processes: p.join(self.JOIN_TIMEOUT) <del> self.assertEqual(p.exitcode, 0) </del> <ins> if not skip_ok: self.assertEqual(p.exitcode, 0) if skip_ok: first_process = self.processes[0] # do this first so we don't give an error message about mismatched exit codes if the first isn't valid assert first_process.exitcode == 0 or first_process.exitcode == SKIP_IF_NO_CUDA_EXIT_CODE for p in self.processes: self.assertEqual(p.exitcode, first_process.exitcode) if first_process.exitcode == SKIP_IF_NO_CUDA_EXIT_CODE: raise unittest.SkipTest("cuda is not available") </ins> elif BACKEND == 'mpi': dist.init_process_group(init_method=INIT_METHOD, backend='mpi')

is causing a crash when we try to build pytorch. Crash looks like this: (all times in US/West) Our 7AM, 8AM builds and earlier were all fine but 9AM, 10AM, 11AM builds all crashed with that same traceback. The new import was in which I think went in between 8AM and 9AM today See command below: See here if needed for : <!-- A clear and concise description of what you expected to happen. --Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): building from master OS (e.g., Linux): linux How you installed PyTorch (, , source): source Build command you used (if compiling from source): see above under 'reproduce' Python version: 3.6
maybe easier to read for build command used:
please try in the CircleCi environment?
in our build script seems to have fixed this. Thanks !


Question: <del> d6015d42d9a1834bc7595c4bd6852562fb80b30b </del> <ins> 23512dbebd44a11eb84afbf53c3c071dd105297e </ins>

*Status could be: ongoing Torch Bench testing is broken with errors that look like Inductor CI has been broken since 5:41 PM PT on June 14, 2024. It was detected the next day and root caused the morning on June 17th. The affected inductor ci jobs (those related to torchbench perf) shall be marked as unstable until this issue is mitigated. We suspect it's due to the numpy 2.0 update. () Ongoing Current plan is for to pin numpy for torchbench workflows TBD
This can be fixed by the following PRs: PyTorch PR: TorchBench PR:
I'll close those unstable issue once trunk recovers