I am developing a product with heavy 3D graphics computations, **to a large extent closest point and range searches**. Some hardware optimization would be useful. While I know little about this, my boss (who has no software experience) advocates FPGA (because it can be tailored), while our junior developer advocates GPGPU with CUDA, because its cheap, hot and open. While I feel I lack judgement in this question, I believe CUDA is the way to go also because I am worried about flexibility, our product is still under strong development.

So, rephrasing the question, are there any reasons to go for FPGA at all? Or is there a third option?

 

CUDA or FPGA for special purpose 3D graphics computations?

...or just the threads in the current warp or block?

Also, when the threads in a particular block encounter (in the kernel) the following line

    __shared__  float srdMem[128];

will they just declare this space once (per block)?

They all obviously operate asynchronously so if Thread 23 in Block 22 is the first thread to reach this line, and then Thread 69 in Block 22 is the last one to reach this line, Thread 69 will know that it already has been declared?

Does __syncthreads() synchronize all threads in the grid?

I get what `blockDim` is, but I have a problem with `gridDim. Blockdim` gives the size of the block, but what is `gridDim`? On the Internet it says `gridDim.x` gives the number of blocks in the x coordinate.

How can I know what `blockDim.x * gridDim.x` gives?

How can I know that how many `gridDim.x` values are there in the x line?

For example, consider the code below:

    int tid = threadIdx.x + blockIdx.x * blockDim.x;
    double temp = a[tid];
    tid += blockDim.x * gridDim.x;

    while (tid &lt; count)
    {
        if (a[tid] &gt; temp)
        {
           temp = a[tid];
        }
        tid += blockDim.x * gridDim.x;
    }

I know that `tid` starts with 0. The code then has `tid+=blockDim.x * gridDim.x`. What is `tid` now after this operation?

Cuda gridDim and blockDim

I&#39;m trying to install CUDA 7.5 in my ubuntu 14.04. I followed everything in this guide (installation through package): http://developer.download.nvidia.com/compute/cuda/7.5/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf until post build section. Mainly, by running the following commands: 

    sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
    sudo apt-get update
    sudo apt-get install cuda

The thing is the folder /usr/local/cuda* does not exist after successful CUDA installation. Further trying to install cuda says that it is already the newest version.

    sudo apt-get install cuda
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    cuda is already the newest version.

    ls /usr/local/cuda*
    ls: cannot access /usr/local/cuda*: No such file or directory

It also does not find nvcc.

    nvcc
    The program &#39;nvcc&#39; is currently not installed. You can install it by typing:
    sudo apt-get install nvidia-cuda-toolkit
    
    sudo find /usr/ -name nvcc
    &lt;no output&gt;

What is wrong?

Where did CUDA get installed in my computer?

I&#39;m trying to compile  a cuda test program on Windows 7 via Command Prompt, 
I&#39;m this command:

    nvcc test.cu

But all I get is this error:

    nvcc fatal : Cannot find compiler &#39;cl.exe&#39; in PATH

What may be causing this error?

Error compiling CUDA from Command Prompt

I am completely new to terms related to HPC computing, but I just saw that EC2 released its new type of instance on AWS that&#39;s powered by the new Nvidia Tesla V100, which has both kinds of &quot;cores&quot;: Cuda Cores (5,120) and Tensor Cores (640). 
What is the difference between both?


What is the difference between cuda vs tensor cores?

Does anyone know a project which implements standard compression methods (like Zip, GZip, BZip2, LZMA,...) using NVIDIA&#39;s [CUDA library][1]?

I was wondering if algorithms which can make use of a lot of parallel tasks (like compression) wouldn&#39;t run much faster on a graphics card than with a dual or quadcore CPU.

What do you think about the pros and cons of such an approach?

  [1]: http://www.nvidia.com/object/cuda_home.html

Compression library using Nvidia&#39;s CUDA

I am currently working on a project in python, and I would like to make use of the GPU for some calculations. 

At first glance it seems like there are many tools available; at second glance, I feel like im missing something.

Copperhead looks awesome but has not yet been released. It would appear that im limited to writing low-level CUDA or openCL kernels; no thrust, no cudpp. If id like to have something sorted, im going to have to do it myself.

That doesnt seem quite right to me. Am I indeed missing something? Or is this GPU-scripting not quite living up to the hype yet?

Edit: GPULIB seems like it might be what I need. Documentation is rudimentary, and the python bindings are mentioned only in passing, but im applying for a download link right now. Anyone has experience with that, or links to similar free-for-academic-use GPU libraries? ReEdit: ok, python bindings are infact nonexistant.

Edit2: So I guess my best bet is to write something in C/CUDA and call that from python?

Python GPU programming

From some comments that I have read in here, for some reason it is preferable to have `Structure of Arrays` (`SoA`) over `Array of Structures` (`AoS`) for parallel implementations like CUDA? If that is true, can anyone explain why?
Thanks in advance!

Structure of Arrays vs Array of Structures

I am currently going through the tutorial examples on http://code.google.com/p/stanford-cs193g-sp2010/ to learn CUDA. The code which demostrates `__global__` functions is given below. It simply creates two arrays, one on the CPU and one on the GPU, populates the GPU array with the number 7 and copies the GPU array data into the CPU array. 

    #include &lt;stdlib.h&gt;
    #include &lt;stdio.h&gt;
    
    __global__ void kernel(int *array)
    {
      int index = blockIdx.x * blockDim.x + threadIdx.x;
    
      array[index] = 7;
    }
    
    int main(void)
    {
      int num_elements = 256;
    
      int num_bytes = num_elements * sizeof(int);
    
      // pointers to host &amp; device arrays
      int *device_array = 0;
      int *host_array = 0;
    
      // malloc a host array
      host_array = (int*)malloc(num_bytes);
    
      // cudaMalloc a device array
      cudaMalloc((void**)&amp;device_array, num_bytes);
    
      int block_size = 128;
      int grid_size = num_elements / block_size;
    
      kernel&lt;&lt;&lt;grid_size,block_size&gt;&gt;&gt;(device_array);
    
      // download and inspect the result on the host:
      cudaMemcpy(host_array, device_array, num_bytes, cudaMemcpyDeviceToHost);
    
      // print out the result element by element
      for(int i=0; i &lt; num_elements; ++i)
      {
        printf(&quot;%d &quot;, host_array[i]);
      }
     
      // deallocate memory
      free(host_array);
      cudaFree(device_array);
    } 



My question is why have they worded the `cudaMalloc((void**)&amp;device_array, num_bytes);` statement with a double pointer? Even [here][1] definition of cudamalloc() on says the first argument is a double pointer.  

Why not simply return a pointer to the beginning of the allocated memory on the GPU, just like the `malloc` function does on the CPU?


  [1]: http://www.clear.rice.edu/comp422/resources/cuda/html/group__CUDART__MEMORY_gc63ffd93e344b939d6399199d8b12fef.html

Use of cudamalloc(). Why the double pointer?

I am looking for help getting started with a project involving CUDA. My goal is to have a project that I can compile in the native g++ compiler but uses CUDA code. I understand that I have to compile my CUDA code in nvcc compiler, but from my understanding I can somehow compile the CUDA code into a cubin file or a ptx file.

Here are my questions:

1. How do I use nvcc to compile into a cubin file or a ptx file? Don&#39;t I need a -c or something?
2. Which file type do I want to use?
3. What are the g++ commands to correctly compile and link the project together?

Assume the following:

1. I have a file called &quot;main.cpp&quot; that has a main function in it and includes cuda.h.
2. I have another file called &quot;cudaFunc.cu&quot; that has CUDA code in it. Let&#39;s say, for instance, that I want to add two integer arrays that exist in main.cpp.

How can I compile CUDA code then link it to a C++ project?

With the rapid development of C++ compilers,especially the intel ones, and the abilities of directly applying SIMD functions in your C/C++ codes, does Fortran still hold any real advantage in the world of numerical computations?

I am from an applied maths background, my job involves a lot of numerical analysis, computations, optimisations and such, with a strictly defined performance-requirement.

I hardly know anything about Fortran, I have some experience in C/CUDA/matlab(if you consider the latter as a computer language to begin with), and my daily task involves analysis of very large data (e.g. 10GB-large matrix), and it seems the program at least spend 2/3 of its time on memory-accessing (thats why I send some of its job to GPU), do you people think it may worth the effects for me to trying the fortran routine on at least some performance-critical part of my codes to improve the performance of my program?

Because the complexity and things need to be done involved there, I will only go that routine if only there is significant performance benefit there, thanks in advance.

Fortran vs C++, does Fortran still hold any advantage in numerical analysis these days?

I have a GeForce GTX 580, and I want to make a statement about the total number of threads that can (ideally) actually be run in parallel, to compare with 2 or 4 multi-core CPU&#39;s.

deviceQuery gives me the following possibly relevant information: 

    CUDA Capability Major/Minor version number:    2.0
    (16) Multiprocessors x (32) CUDA Cores/MP:     512 CUDA 
    Maximum number of threads per block:           1024

I think I heard that each CUDA core can run a warp in parallel, and that a warp is 32 threads. Would it be correct to say that the card can run 512*32 = 16384 threads in parallel then, or am I way off and the CUDA cores are somehow not really running in parallel?

CUDA: How many concurrent threads in total?

I am trying to build [this project][1], which has CUDA as a dependency. But the cmake script cannot find the CUDA installation on the system:

    cls ~/workspace/gpucluster/cluster/build $ cmake ..
    -- The C compiler identification is GNU 4.7.1
    -- The CXX compiler identification is GNU 4.7.1
    -- Check for working C compiler: /usr/bin/gcc
    -- Check for working C compiler: /usr/bin/gcc -- works
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working CXX compiler: /usr/bin/c++
    -- Check for working CXX compiler: /usr/bin/c++ -- works
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    CMake Error at /usr/share/cmake/Modules/FindCUDA.cmake:488 (message):
      Specify CUDA_TOOLKIT_ROOT_DIR
    Call Stack (most recent call first):
      CMakeLists.txt:20 (find_package)


-- Configuring incomplete, errors occurred!


I&#39;ve tried adding it as an environment variable to `.bashrc`, to no effect:

    export CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.5

How do I `Specify CUDA_TOOLKIT_ROOT_DIR` correctly?

  [1]: http://www.staff.science.uu.nl/~faggi101/gpucluster.tar.gz

How to let cmake find CUDA

Looking through the answers and comments on CUDA questions, and in the [CUDA tag wiki][1], I see it is often suggested that the return status of every API call should checked for errors. The API documentation contains functions like `cudaGetLastError`, `cudaPeekAtLastError`, and `cudaGetErrorString`, but what is the best way to put these together to reliably catch and report errors without requiring lots of extra code?


  [1]: https://stackoverflow.com/tags/cuda/info

What is the canonical way to check for errors using the CUDA runtime API?

I have noticed that some newer TensorFlow versions are incompatible with older CUDA and cuDNN versions. Does an overview of the compatible versions or even a list of officially tested combinations exist? I can&#39;t find it in the TensorFlow documentation. 

Which TensorFlow and CUDA version combinations are compatible?

I have searched many places but ALL I get is HOW to install it, not how to verify that it is installed. I can verify my NVIDIA driver is installed, and that CUDA is installed, but I don&#39;t know how to verify CuDNN is installed. Help will be much appreciated, thanks!

PS.  
This is for a caffe implementation. Currently everything is working without CuDNN enabled.

How to verify CuDNN installation?

I am very confused by the different CUDA versions shown by running `which nvcc` and `nvidia-smi`. I have both cuda9.2 and cuda10 installed on my ubuntu 16.04. Now I set the PATH to point to cuda9.2. So when I run

```bash
$ which nvcc
/usr/local/cuda-9.2/bin/nvcc
```
However, when I run 

    $ nvidia-smi
    Wed Nov 21 19:41:32 2018       
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 410.72       Driver Version: 410.72       CUDA Version: 10.0     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  GeForce GTX 106...  Off  | 00000000:01:00.0 Off |                  N/A |
    | N/A   53C    P0    26W /  N/A |    379MiB /  6078MiB |      2%      Default |
    +-------------------------------+----------------------+----------------------+
                                                                                   
    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID   Type   Process name                             Usage      |
    |=============================================================================|
    |    0      1324      G   /usr/lib/xorg/Xorg                           225MiB |
    |    0      2844      G   compiz                                       146MiB |
    |    0     15550      G   /usr/lib/firefox/firefox                       1MiB |
    |    0     19992      G   /usr/lib/firefox/firefox                       1MiB |
    |    0     23605      G   /usr/lib/firefox/firefox                       1MiB |

So am I using cuda9.2 as `which nvcc` suggests, or am I using cuda10 as `nvidia-smi` suggests? I [saw this answer][1] but it does not provide direct answer to the confusion, it just asks us to reinstall the CUDA Toolkit, which I already did.



  [1]: https://stackoverflow.com/questions/34319877/nvcc-has-different-version-than-cuda

Different CUDA versions shown by nvcc and NVIDIA-smi

I&#39;m searching for a way to use the GPU from inside a docker container.

The container will execute arbitrary code so i don&#39;t want to use the privileged mode.

Any tips? 

From previous research i understood that `run -v` and/or LXC `cgroup` was the way to go but i&#39;m not sure how to pull that off exactly

Using GPU from a docker container?

I&#39;m trying to monitor a process that uses CUDA and MPI, is there any way I could do this, something like the command &quot;top&quot; but that monitors the GPU too?



A top-like utility for monitoring CUDA activity on a GPU

How are threads organized to be executed by a GPU?

Understanding CUDA grid dimensions, block dimensions and threads organization (simple explanation)

I have been using CUDA for a few weeks, but I have some doubts about the allocation of blocks/warps/thread.
**I am studying the architecture from a didactic point of view** (university project), so reaching peak performance is not my concern.

First of all, I would like to understand if I got these facts straight:

1. The programmer writes a kernel, and organize its execution in a grid of thread blocks.

2. Each block is assigned to a Streaming Multiprocessor (SM). Once assigned it cannot migrate to another SM.

3. Each SM splits its own blocks into Warps (currently with a maximum size of 32 threads). All the threads in a warp executes concurrently on the resources of the SM.

4. The actual execution of a thread is performed by the CUDA Cores contained in the SM. There is no specific mapping between threads and cores.

5. If a warp contains 20 thread, but currently there are only 16 cores available, the warp will not run.

6. On the other hand if a block contains 48 threads, it will be split into 2 warps and they will execute in parallel provided that enough memory is available.

7. If a thread starts on a core, then it is stalled for memory access or for a long floating point operation, its execution could resume on a different core.

Are they correct?

Now, I have a GeForce 560 Ti so according to the specifications it is equipped with 8 SM, each containing 48 CUDA cores (384 cores in total).

My goal is to make sure that every core of the architecture executes the SAME instructions. Assuming that my code will not require more register than the ones available in each SM, I imagined different approaches:

1. I create 8 blocks of 48 threads each, so that each SM has 1 block to execute. In this case will the 48 threads execute in parallel in the SM (exploiting all the 48 cores available for them)?

2. Is there any difference if I launch 64 blocks of 6 threads? (Assuming that they will be mapped evenly among the SMs)

3. If I &quot;submerge&quot; the GPU in scheduled work (creating 1024 blocks of 1024 thread each, for example) is it reasonable to assume that all the cores will be used at a certain point, and will perform the same computations (assuming that the threads never stall)?

4. Is there any way to check these situations using the profiler?

5. Is there any reference for this stuff? I read the CUDA Programming guide and the chapters dedicated to hardware architecture in &quot;Programming Massively Parallel Processors&quot; and &quot;CUDA Application design and development&quot;; but I could not get a precise answer.

How do CUDA blocks/warps/threads map onto CUDA cores?

Is there any quick command or script to check for the version of CUDA installed?

I found the manual of 4.0 under the installation directory but I&#39;m not sure whether it is of the actual installed version or not.

How to get the CUDA version?

I&#39;m working on a business project that is done in Java, and it needs huge computation power to compute business markets. Simple math, but with huge amount of data.

We ordered some CUDA GPUs to try it with and since Java is not supported by CUDA, I&#39;m wondering where to start. Should I build a JNI interface? Should I use JCUDA or are there other ways?

I don’t have experience in this field and I would like if someone could direct me to something so I can start researching and learning.




Using Java with Nvidia GPUs (CUDA)

Can anyone describe the differences between `__global__` and `__device__` ?

When should I use `__device__`, and when to use `__global__`?.

Difference between global and device functions

This is a question about how to determine the CUDA grid, block and thread sizes. This is an additional question to the one posted [here][1].

Following this link, the answer from talonmies contains a code snippet (see below). I don&#39;t understand the comment &quot;value usually chosen by tuning and hardware constraints&quot;.

I haven&#39;t found a good explanation or clarification that explains this in the CUDA documentation. In summary, my question is how to determine the optimal `blocksize` (number of threads) given the following code:


    const int n = 128 * 1024;
    int blocksize = 512; // value usually chosen by tuning and hardware constraints
    int nblocks = n / nthreads; // value determine by block size and total work
    madd&lt;&lt;&lt;nblocks,blocksize&gt;&gt;&gt;mAdd(A,B,C,n);


  [1]: https://stackoverflow.com/a/5643838/1292251

How do I choose grid and block dimensions for CUDA kernels?

I have troubles compiling some of the examples shipped with CUDA SDK.
I have installed the developers driver (version 270.41.19) and the CUDA toolkit,
then finally the SDK (both the 4.0.17 version).

Initially it didn&#39;t compile at all giving:

    error -- unsupported GNU version! gcc 4.5 and up are not supported!
 
I found the line responsible in 81:/usr/local/cuda/include/host_config.h  and changed it to:

    //#if __GNUC__ &gt; 4 || (__GNUC__ == 4 &amp;&amp; __GNUC_MINOR__ &gt; 4)
    #if __GNUC__ &gt; 4 || (__GNUC__ == 4 &amp;&amp; __GNUC_MINOR__ &gt; 6)

from that point on I got only a few of the examples to compile, it stops with:

    In file included from /usr/include/c++/4.6/x86_64-linux-gnu/bits/gthr.h:162:0,
                 from /usr/include/c++/4.6/ext/atomicity.h:34,
                 from /usr/include/c++/4.6/bits/ios_base.h:41,
                 from /usr/include/c++/4.6/ios:43,
                 from /usr/include/c++/4.6/ostream:40,
                 from /usr/include/c++/4.6/iterator:64,
                 from /usr/local/cuda/include/thrust/iterator/iterator_categories.h:38,
                 from /usr/local/cuda/include/thrust/device_ptr.h:26,
                 from /usr/local/cuda/include/thrust/device_malloc_allocator.h:27,
                 from /usr/local/cuda/include/thrust/device_vector.h:26,
                 from lineOfSight.cu:37:
    /usr/include/c++/4.6/x86_64-linux-gnu/bits/gthr-default.h:251:1: error: pasting         &quot;__gthrw_&quot; and &quot;/* Android&#39;s C library does not provide pthread_cancel, check for
    `pthread_create&#39; instead.  */&quot; does not give a valid preprocessing token
    make[1]: *** [obj/x86_64/release/lineOfSight.cu.o] Error 1
    
As some of the examples compile I reckon this is not a driver problem, but rather must have something to do with an unsupported gcc version. Downgrading is not an option as gcc4.6 has a whole system as a dependency at this point...    

CUDA incompatible with my gcc version

Question: Is there an emulator for a Geforce card that would allow me to program and test CUDA without having the actual hardware?

___

Info:

I&#39;m looking to speed up a few simulations of mine in CUDA, but my problem is that I&#39;m not always around my desktop for doing this development. I would like to do some work on my netbook instead, but my netbook doesn&#39;t have a GPU. Now as far as I know, you need a CUDA capable GPU to run CUDA. Is there a way to get around this? It would seem like the only way is a GPU emulator (which obviously would be painfully slow, but would work). But whatever way there is to do this I would like to hear.

I&#39;m programming on Ubuntu 10.04 LTS.

GPU Emulator for CUDA programming without the hardware

I have been reading the programming guide for CUDA and OpenCL, and I cannot figure out what a bank conflict is. They just sort of dive into how to solve the problem without elaborating on the subject itself. Can anybody help me understand it? I have no preference if the help is in the context of CUDA/OpenCL or just bank conflicts in general in computer science.

What is a bank conflict? (Doing Cuda/OpenCL programming)

I&#39;d like to hear from people with experience of coding for both. Myself, I only have experience with NVIDIA.

NVIDIA CUDA seems to be a lot more popular than the competition. (Just counting question tags on this forum, &#39;cuda&#39; outperforms &#39;opencl&#39; 3:1, and &#39;nvidia&#39; outperforms &#39;ati&#39; 15:1, and there&#39;s no tag for &#39;ati-stream&#39; at all). 

On the other hand, according to Wikipedia, ATI/AMD cards should have a lot more potential, especially per dollar. The fastest NVIDIA card on the market as of today, GeForce 580 ($500), is rated at 1.6 single-precision TFlops. AMD Radeon 6970 can be had for $370 and it is rated at 2.7 TFlops. The 580 has 512 execution units at 772 MHz. The 6970 has 1536 execution units at 880 MHz. 

How realistic is that paper advantage of AMD over NVIDIA, and is it likely to be realized in most GPGPU tasks? What happens with integer tasks? 

NVIDIA vs AMD: GPGPU performance

For debugging CUDA code and checking compatibilities I need to find out what nvidia driver version for the GPU I have installed. I found https://stackoverflow.com/questions/9727688/how-to-get-the-cuda-version but that does not help me here.

How to get the nvidia driver version from the command line?

In a multi-GPU computer, how do I designate which GPU a CUDA job should run on?  

As an example, when installing CUDA, I opted to install the `NVIDIA_CUDA-&lt;#.#&gt;_Samples` then ran several instances of the `nbody` simulation, but they all ran on one GPU 0; GPU 1 was completely idle (monitored using `watch -n 1 nvidia-dmi`).  Checking `CUDA_VISIBLE_DEVICES` using

    echo $CUDA_VISIBLE_DEVICES

I found this was not set.  I tried setting it using 

    CUDA_VISIBLE_DEVICES=1

then running `nbody` again but it also went to GPU 0.  

I looked at the related question, [how to choose designated GPU to run CUDA program?](https://stackoverflow.com/questions/25564574/how-to-choose-designated-gpu-to-run-cuda-program), but `deviceQuery` command is not in the CUDA 8.0 bin directory.  In addition to `$CUDA_VISIBLE_DEVICES$`, I saw other posts refer to the environment variable `$CUDA_DEVICES` but these were not set and I did not find information on how to use it.

While not directly related to my question, using `nbody -device=1` I was able to get the application to run on GPU 1 but using `nbody -numdevices=2` did not run on both GPU 0 and 1.

I am testing this on a system running using the bash shell, on CentOS 6.8, with CUDA 8.0, 2 GTX 1080 GPUs, and NVIDIA driver 367.44.

I know when writing using CUDA you can manage and control which CUDA resources to use but how would I manage this from the command line when running a compiled CUDA executable?

How do I select which GPU to run a job on?

I have a very simple Toshiba Laptop with i3 processor. Also, I do not have any expensive graphics card. In the display settings, I see **Intel(HD) Graphics** as display adapter. I am planning to learn some cuda programming. But, I am not sure, if I can do that on my laptop as it does not have any nvidia&#39;s cuda enabled GPU.

In fact, I doubt, if I even have a GPU o_o

So, I would appreciate if someone can tell me if I can do CUDA programming with the current configuration and if possible also let me know what does **Intel(HD) Graphics** mean? 

Can I run CUDA on Intel&#39;s integrated graphics processor?

I&#39;d like to extend my skill set into GPU computing. I am familiar with raytracing and realtime graphics(OpenGL), but the next generation of graphics and high performance computing seems to be in GPU computing or something like it.

I currently use an AMD HD 7870 graphics card on my home computer. Could I write CUDA code for this? (my intuition is no, but since Nvidia released the compiler binaries I might be wrong).

A second more general question is, Where do I start with GPU computing? I&#39;m certain this is an often asked question, but the best I saw was from 08&#39; and I figure the field has changed quite a bit since then. 

Is it possible to run CUDA on AMD GPUs?

General-purpose computing on graphics processing units ([GPGPU][1]) is a very attractive concept to harness the power of the GPU for any kind of computing. 

I&#39;d love to use GPGPU for image processing, particles, and fast geometric operations.

Right now, it seems the two contenders in this space are CUDA and OpenCL. I&#39;d like to know:

 - Is OpenCL usable yet from Java on Windows/Mac?
 - What are the libraries ways to interface to OpenCL/CUDA?
 - Is using JNA directly an option?
 - Am I forgetting something?

Any real-world experience/examples/war stories are appreciated.

  [1]: http://en.wikipedia.org/wiki/GPGPU

Best approach for GPGPU/CUDA/OpenCL in Java?

What is the relationship between a CUDA core, a streaming multiprocessor and the CUDA model of blocks and threads?

What gets mapped to what and what is parallelized and how? and what is more efficient, maximize the number of blocks or the number of threads?

----------
My current understanding is that there are 8 cuda cores per multiprocessor. and that every cuda core will be able to execute one cuda block at a time. and all the threads in that block are executed serially in that particular core.

Is this correct?

Streaming multiprocessors, Blocks and Threads (CUDA)

I observe substantial speedups in data transfer when I use pinned memory for CUDA data transfers. On linux, the underlying system call for achieving this is mlock. From the man page of mlock, it states that locking the page prevents it from being swapped out:

&gt; mlock() locks pages in the address range starting at addr and continuing for len bytes.  All pages that contain a part of the specified address range are guaranteed to be resident in RAM when the call returns successfully; 

In my tests, I had a fews gigs of free memory on my system so there was never any risk that the memory pages could&#39;ve been swapped out yet I still observed the speedup. Can anyone explain what&#39;s really going on here?, any insight or info is much appreciated.

Why is CUDA pinned memory so fast?

I know that `nvidia-smi -l 1` will give the GPU usage every one second (similarly to the following). However, I would appreciate an explanation on what `Volatile GPU-Util` really means. Is that the number of used SMs over total SMs, or the occupancy, or something else?

    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 367.48                 Driver Version: 367.48                    |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  Tesla K20c          Off  | 0000:03:00.0     Off |                    0 |
    | 30%   41C    P0    53W / 225W |      0MiB /  4742MiB |     96%      Default |
    +-------------------------------+----------------------+----------------------+
    |   1  Tesla K20c          Off  | 0000:43:00.0     Off |                    0 |
    | 36%   49C    P0    95W / 225W |   4516MiB /  4742MiB |     63%      Default |
    +-------------------------------+----------------------+----------------------+
    
    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID  Type  Process name                               Usage      |
    |=============================================================================|
    |    1      5193    C   python                                        4514MiB |
    +-----------------------------------------------------------------------------+

nvidia-smi Volatile GPU-Utilization explanation?

What is &quot;coalesced&quot; in CUDA global memory transaction? I couldn&#39;t understand even after going through my CUDA guide. How to do it? In CUDA programming guide matrix example, accessing the matrix row by row is called &quot;coalesced&quot; or col.. by col.. is called coalesced?
Which is correct and why?


In CUDA, what is memory coalescing, and how is it achieved?

When I run `nvidia-smi`, I get the following message:

&gt; Failed to initialize NVML: Driver/library version mismatch

An hour ago I received the same message and uninstalled my CUDA library and I was able to run `nvidia-smi`, getting the following result:

[![nvidia-smi-result][1]][1]

After this I downloaded `cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb` from [the official NVIDIA page][2] and then simply:

```lang-none
sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
```

Now I have CUDA installed, but I get the mentioned mismatch error.
_______________________________________________________

Some potentially useful information:

Running `cat /proc/driver/nvidia/version` I get:

```lang-none
NVRM version: NVIDIA UNIX x86_64 Kernel Module  378.13  Tue Feb  7 20:10:06 PST 2017
GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
```

I&#39;m running Ubuntu 16.04.2 LTS (Xenial Xerus).

The kernel release is 4.4.0-66-generic.

  [1]: https://i.stack.imgur.com/RyAeQ.png
  [2]: https://developer.nvidia.com/cuda-downloads






Nvidia NVML Driver/library version mismatch

I am Integrating `Matlab`, `C` and `Cuda` together in a project. I used Matlab mix in order to connect matlab mx function written in c with the cuda runtime library, a linking error appear about conflict in static release and dynamic release between the c file and the library. Can anyone solve this? 

    error LNK2038: mismatch detected for &#39;RuntimeLibrary&#39;: value &#39;MT_StaticRelease&#39; doesn&#39;t match value &#39;MD_DynamicRelease&#39; in file.obj. 


LNK2038: mismatch detected for &#39;RuntimeLibrary&#39;: value &#39;MT_StaticRelease&#39; doesn&#39;t match value &#39;MD_DynamicRelease&#39; in file.obj

when is calling to the `cudaDeviceSynchronize` function  really needed?. 

As far as I understand from the CUDA documentation, CUDA kernels are asynchronous, so it seems that we should call `cudaDeviceSynchronize` after each kernel launch.   However, I have tried the same code (training neural networks) with and without any `cudaDeviceSynchronize`, except one before the time measurement. I have found that I get the same result but with a  speed up between 7-12x (depending on the matrix sizes).  

So, the question is if there are any reasons to use `cudaDeviceSynchronize` apart of time measurement. 

For example:

- Is it needed before copying data from the GPU back to the host with `cudaMemcpy`?

- If I do matrix multiplications like


        C = A * B
        D = C * F

should I put `cudaDeviceSynchronize` between both? 

From my experiment It seems that I don&#39;t.

Why does `cudaDeviceSynchronize` slow the program so much?

When to call cudaDeviceSynchronize?

At the moment, i&#39;m trying to create a Java-application which uses CUDA-functionality. The connection between CUDA and Java works fine, but i&#39;ve got another problem and wanted to ask, if my thoughts about it are correct.

When i call a native function from Java, i pass some data to it, the functions calculates something and returns a result. Is it possible, to let the first function return a reference (pointer) to this result which i can pass to JNI and call another function that does further calculations with the result?

My idea was to reduce the overhead that comes from copying data to and from the GPU by leaving the data in the GPU memory and just passing a reference to it so other functions can use it.

After trying some time, i thought for myself, this shouldn&#39;t be possible, because pointers get deleted after the application ends (in this case, when the C-function terminates). Is this correct? Or am i just to bad in C to see the solution? 


Edit: 
Well, to expand the question a little bit (or make it more clearly): Is memory allocated by JNI native functions deallocated when the function ends? Or may i still access it until either the JNI application ends or when i free it manually?

Thanks for your input :)

Passing pointers between C and Java through JNI

I am a newbie to GPU programming. I have a laptop with NVIDIA GeForce GT 640 card. I am faced with 2 dilemmas, suggestions are most welcome.

 1. **If I go for CUDA -- Ubuntu or Windows** Clearly CUDA is more suitable to windows while it can be a severe issue to install on Ubuntu. I have seen some blogposts which claim to have installed CUDA 5 on [Ubuntu 11.10][1] and [Ubuntu 12.04][2]. However, I have not been able to get them to work. Also, [standard CUDA textbooks][3] prefer to work in the windows domain and are more or less silent in concern with Unix/Ubuntu installation and working.

 2. **[CUDA or OpenCL][4]** -- Now this is probably more trickier than my first question ! I have mostly come across GPGPU projects using CUDA/Nvidia but OpenCL is probably is the next best option in open source and installing in Ubuntu probably will not be an issue, though some suggestions here will be most useful. Am I sacrificing any functionality if I go for OpenCL and NOT CUDA ? 

Any help or suggestions ? 

  [1]: http://www.r-tutor.com/gpu-computing/cuda-installation/cuda5.0-ubuntu
  [2]: http://sn0v.wordpress.com/2012/12/07/installing-cuda-5-on-ubuntu-12-04/
  [3]: http://www.amazon.com/CUDA-Example-Introduction-General-Purpose-Programming/dp/0131387685
  [4]: http://wiki.tiker.net/CudaVsOpenCL
  [5]: http://devblogs.nvidia.com/parallelforall/cudacasts-episode-5-install-cuda-55-linux-package-manager/
  [6]: https://gist.github.com/rmcgibbo/6314452
  [7]: http://mhr3.blogspot.in/2013/06/opencl-on-ubuntu-1304.html
  [8]: http://heather.cs.ucdavis.edu/parprocbook

GPU Programming, CUDA or OpenCL?

My CUDA program crashed during execution, before memory was flushed. As a result, device memory remained occupied.

I&#39;m running on a GTX 580, for which `nvidia-smi --gpu-reset` is not supported.

Placing `cudaDeviceReset()` in the beginning of the program is only affecting the current context created by the process and doesn&#39;t flush the memory allocated before it.

I&#39;m accessing a Fedora server with that GPU remotely, so physical reset is quite complicated. 

So, the question is - Is there any way to flush the device memory in this situation?

How can I flush GPU memory using CUDA (physical reset is unavailable)

I tried running sample programs provided at [NVIDIA&#39;s official site][1]. Most of the programs ran smoothly except few where I get similar error messages. How can I fix that? Here&#39;s a sample of error message I got after running a program named &quot;MatrixMul&quot;. 

Note: I have installed both x32 and x64 [NVIDIA CUDA Toolkit][2] v5.0 on my Window7x64 OS.

    &#39;matrixMul.exe&#39;: Loaded &#39;C:\ProgramData\NVIDIA Corporation\CUDA Samples\v5.0\bin\win32\Debug\matrixMul.exe&#39;, Symbols loaded.
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\ntdll.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\kernel32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\KernelBase.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Program Files (x86)\NVIDIA GPU Computing Toolkit\CUDA\v5.0\bin\cudart32_50_35.dll&#39;, Binary was not built with debug information.
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\apphelp.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\AppPatch\AcLayers.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\sspicli.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\msvcrt.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\rpcrt4.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\cryptbase.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\sechost.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\user32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\gdi32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\lpk.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\usp10.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\advapi32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\shell32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\shlwapi.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\ole32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\oleaut32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\userenv.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\profapi.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\winspool.drv&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\mpr.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\imm32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\msctf.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\nvinit.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\nvcuda.dll&#39;, Binary was not built with debug information.
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\setupapi.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\cfgmgr32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\devobj.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\dwmapi.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Unloaded &#39;C:\Windows\SysWOW64\dwmapi.dll&#39;
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\nvapi.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\version.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\wintrust.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\crypt32.dll&#39;, Cannot find or open the PDB file
    &#39;matrixMul.exe&#39;: Loaded &#39;C:\Windows\SysWOW64\msasn1.dll&#39;, Cannot find or open the PDB file
    The thread &#39;Win32 Thread&#39; (0x12fc) has exited with code 0 (0x0).
    The thread &#39;Win32 Thread&#39; (0x18a0) has exited with code 0 (0x0).
    The program &#39;[3104] matrixMul.exe: Native&#39; has exited with code 0 (0x0)


  [1]: http://docs.nvidia.com/cuda/cuda-samples/index.html
  [2]: https://developer.nvidia.com/cuda-downloads

Error Message : Cannot find or open the PDB file

I&#39;m new to the CUDA paradigm. My question is in determining the number of threads per block, and blocks per grid. Does a bit of art and trial play into this? What I&#39;ve found is that many examples have seemingly arbitrary number chosen for these things.

I&#39;m considering a problem where I would be able to pass matrices - of any size - to a method for multiplication. So that, each element of C (as in C = A * B) would be calculated by a single thread. How would you determine the threads/block, blocks/grid in this case?

CUDA determining threads per block, blocks per grid

Does CUDA support recursion?

I have quite a good understanding about how to allocate and copy linear memory with `cudaMalloc()` and `cudaMemcpy()`. However, when I want to use the CUDA functions to allocate and copy 2D or 3D matrices, I am often befuddled by the various arguments, especially concerning pitched pointers which are always present when dealing with 2D/3D arrays. The documentation is good for providing a couple examples on how to use them but it assumes that I am familiar with the notion of padding and pitch, which I am not.

I usually end up tweaking the various examples I find in the documentation or somewhere else on the web, but the blind debugging that follows is quite painful, so my question is:

What is a pitch? How do I use it? How do I allocate and copy 2D and 3D arrays in CUDA?

How and when should I use pitched pointer with the cuda API?

I&#39;ve been looking for some information on coding CUDA (the nvidia gpu language) with C#.  I have seen a few of the libraries, but it seems that they would add a bit of overhead (because of the p/invokes, etc).  

  - How should I go about using CUDA in my C# applications?  Would it be better to code it in say C++ and compile that into a dll?
  - Would this overhead of using a wrapper kill any advantages I would get from using CUDA?
  - And are there any good examples of using CUDA with C#?

All Cuda Solutions on Gang of Coders

Total of 50 Cuda Solutions