313 questions
0
votes
1
answer
65
views
Linking matplot++ and Libtorch together in a program causes memory corruption problem
I made a minimal example that crashes. For the crash it is required that the torch libraries are linked. I made sure that GLIBCXX_USE_CXX11_ABI=0 for all 3 components. Setup is Ubuntu, g++, C++17.
Any ...
- 11
0
votes
0
answers
78
views
cmake libtorch include issue. only <torch/XXX.h> can be found by cmake
My C++ code is like this
#include <torch/torch.h>
#include <torch/script.h>
#include <torch/autograd/profiler.h>
and my CMakeLists.txt is writen like this
...
0
votes
1
answer
13k
views
CUDA error: no kernel image is available for execution on the device
Im using the latest LibTorch with CUDA 12.6 installed on Windows 10
Libtorch is showing CUDA is available, however, I get this error when I try to use
device = torch::Device(torch::kCUDA, 0);
...
- 3,714
3
votes
0
answers
100
views
How can I view the value of a LibTorch tensor in Visual Studio debugger
In the image below I am running the same PPO minimalist project. On the right side is the original code in python with PyTorch in VSCode. On the left side is the converted code in C++ with LibTorch in ...
- 3,714
3
votes
0
answers
73
views
Efficiency of torch matrix operations on CPU
I am currently working on optimizing PyTorch code running on CPU. On CUDA, you can write fused multiple operators into specialized kernels to reduce memory transfers, avoid intermediate data ...
- 3,213
0
votes
0
answers
14
views
Resolving Bug Of `jit_type_base.h` Of libtorch
What is a solution or workaround to the following error logged when debugging a solution (commit cb8e64063c91ef9c9b78829f577d5e52816b0623) that uses the Debug version of libtorch, is written in C++, ...
- 447
1
vote
0
answers
109
views
Error loading model using Torch TensorRT in Libtorch on Windows
Environment
Libtorch 2.5.0.dev (latest nightly) (built with CUDA 12.4)
CUDA 12.4
TensorRT 10.1.0.27
PyTorch 2.4.0+cu124
Torch-TensorRT 2.4.0
Python 3.12.8
Windows 10
Compile Torch-TensorRT with ...
- 11
1
vote
0
answers
109
views
Is torch::jit::trace not available in libtorch?
I am using libtorch in c++.
In python, torch.jit.trace function exists and works fine.
However, torch::jit::trace does not work in libtorch.
It said there is no "trace" member in namespace ...
- 11
0
votes
0
answers
41
views
How do I initialize a model with known weights in torch / luz?
The problem I am faced with is to be able to fit a series of models starting with progressively different inputs.
I am using the R interface to torch for this, which is based on pytorch. I have to use ...
- 2,406
-2
votes
1
answer
142
views
Linking error when building libtorch on MacOS
I am going along the pytorch api tutorial and I get the following error
ld: unknown options: --no-as-needed --as-needed --no-as-needed --as-needed
c++: error: linker command failed with exit code 1 (...
- 11
1
vote
0
answers
43
views
Segmentation fault when trying to copy memory from Gst buffer to torch tensor using torch::from_blob::clone()
I'm getting a segmentation fault when I'm trying to copy memory from gst buffer to torch tensor. Here is code. I'll mark where the segfault originates from:
static GstPadProbeReturn on_probe(
...
- 26
1
vote
0
answers
98
views
Calculate accuracy in libtorch (C++) when using Softmax
The below code is able to learn the generic pattern and it is able to predict as expected:
torch::manual_seed(42);
torch::nn::Sequential net(
torch::nn::Linear(300, 301),
torch::nn::...
2
votes
1
answer
169
views
Can a PyTorch Model Built in Debug Mode Be Used in Release Mode? TorchScript Model Load Error in Release Mode but Works in Debug Mode
I'm encountering an issue when trying to load a TorchScript model in my C++ application using LibTorch. The model loads and works fine in debug mode, but I get an exception when switching to release ...
1
vote
1
answer
706
views
How to avoid find_package_handle_standard_args package name warning
In LibTorch 2.0.1 I was able to use the library without installing it (download library and set CMAKE_PREFIX_PATH to allow CMake to find the library). Using LibTorch 2.5.1 and CUDA 12.6 I have to ...
1
vote
0
answers
743
views
Building PyTorch 2.5.1 (libtorch + main PyTorch) with CUDA 12.6.3 and cuDNN 9.5.1
My goal is to build PyTorch from source for use in image generation AI workflows (e.g., ComfyUI).
I would like to request guidance on the correct method to build PyTorch 2.5.1 (both libtorch and the ...
- 19
1
vote
0
answers
168
views
std::bad_alloc when loading PyTorch model in C++ using LibTorch
I'm trying to load a traced PyTorch model in C++ using LibTorch but encountering a std::bad_alloc error. I've set up a minimal example to demonstrate the issue.
The Problem:
When running my C++ ...
- 127
1
vote
1
answer
75
views
Use Boost python with libtorch
I am trying to create a python package from C++ code with Boost python. However when including libtorchin the code, the resulting python package shows strange errors such as
Boost.Python....
- 91
1
vote
0
answers
60
views
How does batch inference work in pytorch c++
I am looking for ways to batch inference instead of inference one request at a time in c++
For example, assuming each request in requests is a single prediction containing n feature, each feat
for (...
- 73
0
votes
0
answers
276
views
How to set up CMakeLists.txt for Libtorch in a C++ Project?
I have a simple project with use of libtorch. The structure of the project is the following:
MyLibtorchProject/
├── CMakeLists.txt
├── main.cpp
├── libtorch/
└── build/
The content of CMakeLists.txt ...
1
vote
1
answer
166
views
How to bundle libtorch with my rust binary?
I am developing an AI chat desktop application targeting Apple M chips. The app utilizes embedding models and reranker models, for which I chose Rust-Bert due to its capability to handle such models ...
- 2,881
4
votes
1
answer
116
views
Why is a namespace-qualified name available as a candidate?
Using libtorch, the following snippet causes an error:
#include <iostream>
#include <torch/torch.h>
using namespace torch;
int main() {
std::cout << zeros({}) << std::endl;
...
- 156k
4
votes
0
answers
89
views
OpenCV and Libtorch conflict
Ubuntu 24 system; CLion IDE. When trying to run code using OpenCV and Libtorch, the following errors occur:
main.cpp:34:(.text+0xa28): undefined reference to `cv::putText(cv::_InputOutputArray const&...
- 163
0
votes
1
answer
566
views
Unable to load DLL 'LibTorchSharp' or one of its dependencies: The specified procedure could not be found
I'm attempting to use ML.NET to build a model for a simple text classification, but using the CPU rather than GPU due to lacking one with CUDA. However, trying to execute my current code results in ...
0
votes
1
answer
341
views
PyTorch in C++ - how to get part of tensor for all dimensions?
I have the following tensor operation in Python:
A = A[
:,
h_offset:h_offset + fineSize,
w_offset:w_offset + fineSize
]
where *_offset and fineSize are some integers. I am ...
- 9,862
3
votes
3
answers
972
views
libtorch can't detect CUDA, while CUDA runtime can
#include <cuda_runtime.h>
#include <torch/torch.h>
#include <iostream>
int main() {
std::cout << "CUDA device count: " << torch::cuda::device_count() <&...
- 51
-1
votes
1
answer
155
views
Cannot open file: extension.h, but IncludePath is set [duplicate]
Using VS Code launched from Visual Studio 2019 professional developer terminal (windows 11). I am trying to include torch/extension.h by including the path of the conda environment in which I have ...
- 1
0
votes
0
answers
55
views
Which api in `libtorch` corresponding the `ctx.needs_input_grad` in pytorch when to define custom `torch.autograd.Function`?
When I tanslate the following python code to cpp code with libtorch api
class _SphericalHarmonics(torch.autograd.Function):
"""Spherical Harmonics"""
@...
- 101
0
votes
0
answers
109
views
Error running code using Libtorch C++ Windows (VS2022). Intel MKL FATAL ERROR
After Installing MKL I can't find those dll files. And the error continues.
Intel MKL FATAL ERROR: cannot load mkl_vml_avx2.1.dll or mkl_vml_def.1.dll, at runtime.
Libtorch Distrib I'm using is: CUDA ...
2
votes
1
answer
129
views
Weird behavior from CUDA (Libtorch) and OpenGL interop [closed]
I am trying to write functions to transform an OpenGL texture to a PyTorch tensor and back in a C++ app. To test that it works I added 128 to the tensor to basically brighten the image, and then ...
- 31
1
vote
1
answer
1k
views
How I can load model and inference using Libtorch (C++)?
I trained simple model for MNIST using it https://github.com/pytorch/examples/blob/main/cpp/mnist/mnist.cpp
I added code for saving model like below
string model_path = "model.pt";
torch::...
- 25
0
votes
0
answers
59
views
Why can inconsistent function names be called correctly in the scenario of calling functions defined by c++in python code?
I saw a project on GitHub that involved Python code to call functions defined in C++.
Strangely, the function name defined in C++ is A, but it can be called through the name B in Python, where A and B ...
0
votes
0
answers
47
views
How to transfer data between Aramdillo matrix and LibTorch tensor?
I am trying to transfer data between Aramdillo matrix and LibTorch tensor.
I referenced this project, but the conversion failed in my Linux environment.
Then I tried to convert using the from_blob ...
- 21
1
vote
1
answer
413
views
C program using a wrapper libtorch c++ library not linking
I'm trying to make a libtorch c++ library to use it in a C program, exporting library functions with extern "C". The library seems to compile and link correctly. But the main executable ...
- 1,054
0
votes
0
answers
689
views
Issues setting up LibTorch on Windows 11
Hello and thanks in advance!
When following the official LibTorch installation guide, I ran across a four separate errors when building the project. I have not found solutions to the last two errors ...
1
vote
1
answer
68
views
Why might the single act of linking against a library drastically degrade the runtime of a C++ program?
I have a reasonably large C++ project, and I would like to add LibTorch as a dependency. I've downloaded the pre-built LibTorch libraries from the website, and updated my CMakeFile as follows:
...
- 401
0
votes
0
answers
51
views
compatibility between nvidia driver and cuda [duplicate]
I have a RTX 4070 and it said nvidia-driver-535 is recommended.
and I installed the driver. but when I type nvidia-smi, it said CUDA version: 12.2
but I want to get CUDA 11.8 or 12.1
Question :
...
0
votes
0
answers
65
views
Memory address are the same after creating new tensor
std::map<std::string, torch::Tensor*> tmpFloatActivations = model.getActivations();
std::map<std::string, torch::Tensor*> floatActivations;
floatActivations.insert(tmpFloatActivations....
- 753
0
votes
0
answers
85
views
libtorch forward result unexpected
I use a small net. The net converges and the accuracy is 1.0 after 400 iterations. So far, so good.
conv1(torch::nn::Conv2dOptions(1, 15, /*kernel_size=*/3)),
conv2(torch::nn::Conv2dOptions(15, 30,...
0
votes
0
answers
93
views
Solving LNK4022 (not a unique symbol "Initialize" found) without changing export.def
I am trying to solve an LNK4022 issue (not a unique symbol "Initialize" found) but I can't modify the export.def file. The release is a DLL which can only be read by the final software if ...
- 1
2
votes
1
answer
241
views
best way to create libtorch (tch-rs) tensor from polars dataframe
What would be the most efficient way to create libtorch (tch-rs) tensor from polars dataframe?
I can only make it working with as_slice->from_slice like
Tensor::from_slice(
series.f64()?
....
- 3,074
1
vote
0
answers
116
views
Callgrind for CUDA program, execution time in percentages do not add up to 100%
I have a CNN image classification C++ program that uses LibTorch(Pytorch)/CUDA to improve the inference performance. I try using Callgrind：
valgrind --tool=callgrind ./my-bin
to profile it and find ...
- 537
0
votes
1
answer
930
views
Issue building Libtorch Rust Projects
Apologies if this isn't the best place to post this.
I'm trying to learn how to do Pytorch in Rust. Have previous experience with Libtorch in C++ and Pytorch in Python. I'm running into issues setting ...
- 11
1
vote
1
answer
3k
views
Using TorchScript model and got issue: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu
I trained an ALEBF-based model on python, and I chose to reason it in c++ for overall efficiency reasons. I chose torch.jit.trace in python to save the model and loaded the corresponding .pt file in c+...
- 21
1
vote
0
answers
96
views
Torchlib shared library segfaults
I'm trying to run example of object detection using YOLO and libtorch. Here is the source code:
//main.cpp
#include <ATen/core/stack.h>
#include <opencv2/core.hpp>
#include <opencv2/...
- 23
1
vote
1
answer
1k
views
How to convert a cudaArray to a Torch tensor?
I am experimenting with Torch and CUDA. Using torch::from_blob() I was able to do the following so far:
#include <cuda_runtime.h>
#include <torch/torch.h>
#include <iostream>
#...
- 9,862
1
vote
0
answers
54
views
Is it possible to build pytorch to dlopen from a separate libtorch installation?
I'd like to reuse my Linux libtorch installation in Pytorch as well. The benefits would be:
reducing unpacked Python package size by ~600MB of shared objects
certainty that I'm using the same backend ...
- 1,096
1
vote
1
answer
751
views
How to convert pytorch model to half for inference using c++
I am writing C++ code to load model and inference, the saved model is float32, I need to convert it to half precision just like:
model = torch.jit.load(model_path)
model.half()
seems that torch C++ ...
- 61
1
vote
0
answers
496
views
How to control GPU memory in libtorch/CUDA on Windows
My problem is not related to the code, it is related to the "GPU memory" listed in the Windows task manager.
Briefly about the problem: I have an RTX4090 video card with 24GB of video memory....
- 1,485
0
votes
0
answers
603
views
Libtorch performance issue when using multiple GPUs in multiple threads
I have a TorchScript model(traced from a pytorch model) and want to make use of multiple GPUs to do an inference task. The idea is as simple as follows:
Create multiple independent models from the ...
- 1,028
2
votes
1
answer
595
views
Difference between model training in PyTorch and libTorch
I was trying to train a model in C++ API (libtorch) and got bad results. I then compared in a detailed way to pytorch. My network is very simple (linear with 4 inputs and outputs and only one hidden ...
- 21