1This directory contains the low-level tensor libraries for PyTorch, 2as well as the new ATen C++ bindings. 3 4The low-level libraries trace their lineage from the original Torch. There are 5multiple variants of the library, summarized here: 6 7* TH = TorcH 8* THC = TorcH Cuda 9* THCS = TorcH Cuda Sparse (now defunct) 10* THNN = TorcH Neural Network (now defunct) 11* THS = TorcH Sparse (now defunct) 12 13(You'll also see these abbreviations show up in symbol names.) 14 15## Reference counting 16 17PyTorch employs reference counting in order to permit tensors to provide 18differing views on a common underlying storage. For example, when you call 19view() on a Tensor, a new THTensor is allocated with differing dimensions, 20but it shares the same c10::StorageImpl with the original tensor. 21 22Unfortunately, this means we are in the business of manually tracking reference 23counts inside our C library code. Fortunately, for most of our library code implementing 24tensor operations, there is only one rule you have to remember: 25 26> **Golden Rule of Reference Counting:** You must either FREE or RETURN 27> a pointer which was returned by a function whose name begins with 28> `new` or which you called `retain` on. 29> If you return this pointer, your function name must begin with `new`. 30 31In a long function, there may be many invocations of functions with `new` in 32their name. Your responsibility is to go through each of them and ensure 33that there is a matching `free` for it for EACH exit point of the function. 34 35### Examples 36 37Suppose you want to get a reference to the indices of a sparse tensor. This 38function is called `newIndices`. The `new` means you MUST free it when you're 39done (usually at the end of your function.) (It's worth noting that 40`newIndices` doesn't actually allocate a fresh indices tensor; it just gives 41you a pointer to the existing one.) DO NOT directly access the member 42variables of the struct. 43 44``` 45THIndexTensor *indices = THSTensor_(newIndices)(state, sparse); 46// ... do some stuff ... 47THIndexTensor_(free)(state, indices); 48``` 49 50Let's take a look at the implementation of `newIndices`. This doesn't free the 51return result of `newNarrow`, but returns it. This justifies the `new` in its 52name. 53 54``` 55THIndexTensor *THSTensor_(newIndices)(const THSTensor *self) { 56 // ... 57 return THIndexTensor_(newNarrow)(self->indices, 1, 0, self->nnz); 58} 59``` 60 61Passing an object to another function does NOT absolve you of responsibility 62of freeing it. If that function holds on to a pointer to the object, it 63will `retain` it itself. 64 65``` 66 THByteStorage *inferred_size = THByteStorage_newInferSize(size, numel); 67 THTensor_(setStorage)(self, tensor->storage, tensor->storageOffset, inferred_size, NULL); 68 c10::raw::intrusive_ptr::decref(inferred_size); 69``` 70 71Sometimes, you have a tensor in hand which you'd like to use directly, but 72under some conditions you have to call, e.g., `newContiguous`, to get it into 73the correct form: 74 75``` 76 if (!(k_->stride(3) == 1) || !(k_->stride[2] == k_->size(3))) { 77 kernel = THTensor_(newContiguous)(k_); 78 } else { 79 THTensor_(retain)(k_); 80 kernel = k_; 81 } 82 ... 83 c10::raw::intrusive_ptr::decref(kernel); 84``` 85 86In this case, we have (redundantly) called `retain` on `k_`, so that we can 87unconditionally free `kernel` at the end of the function; intuitively, you 88want it to be possible to replace the conditional expression with an equivalent 89function call, e.g., `kernel = THTensor_(newContiguous2D)(k_)`. 90 91### Tips 92 93* If you have an early exit in a function (via a `return`), don't forget to 94 `free` any pointers which you allocated up to this point. If at all possible, 95 move early exits prior to these allocations, so that you don't have to clean up. 96 97* Very occasionally, you may be able to implement an algorithm more efficiently 98 if you "destroy" its input. This is a `move`; after moving an object away, 99 you must NOT `free` it. This is the one exception to the rule, and at the 100 moment there is only one instance of `move` in the code base. 101 102* We use `THError` to signal error cases, and fortunately, 103 you do NOT need to make sure you've freed everything before calling `THError`, 104 because by default, it aborts the entire process. However, it's good style 105 to call `THError` before performing any allocations, since in some cases we 106 sketchily throw a C++ exception and try to recover (in particular, the test 107 suite does this.) 108