CUDA kernel registration supports

[jiannanWang]

This pull request introduces the following changes:

1. Adds load_inline functionality and enables support for CUDA kernel registration.
2. Introduces a script for simple CUDA kernel creation
3. Adds tests for CUDA kernel registration

[msaroufim]

check the no implicit headers mode, otherwise this function will take 90s per call

[jiannanWang]

I set no_implicit headers to Ture and added the header to CUDA files. As a result, the running time for TestDirectoryBackendCUDA decreased from approximately 50 seconds to around 10 seconds.

[jiannanWang]

Update:

• set no_implicit_headers to True and the running time for TestDirectoryBackendCUDA dropped from 50s to 10s.
• CI env doesn't have CUDA_HOME and thus cannot run TestDirectoryBackendCUDA. TestDirectoryBackendCUDA is skipped in CI.

[msaroufim]

You should be able to set CUDA_HOME on the CI runners though, it's a T4 machine with a GPU

[jiannanWang]

You should be able to set CUDA_HOME on the CI runners though, it's a T4 machine with a GPU

Yeah I found that I can install the CUDA toolkit directly on the CI runner. Now the tests are running successfully:

test/test_directory_backend.py::TestDirectoryBackendCUDA::test_add_operation PASSED [ 18%]
test/test_directory_backend.py::TestDirectoryBackendCUDA::test_backend_loading PASSED [ 19%]
test/test_directory_backend.py::TestDirectoryBackendCUDA::test_kernel_directories_exist PASSED [ 20%]

[msaroufim]

Is this ready for a review?

[jiannanWang]

Is this ready for a review?

Yes!

[msaroufim]

I'm wondering if we can simplify this a a bit and only make the LLM spit out the .cu file - the cpp file should typically be quite simple for us to provide. see this as an example https://github.com/gpu-mode/reference-kernels/blob/main/problems/pmpp/vectoradd_py/solutions/correct/submission_cuda_inline.py#L48

[jiannanWang]

Solved! Added a new parameter load_cpp_source which by default set to false. It controls whether to load cpp source from the .cpp file (load_cpp_source=true) or to generate the cpp source content from the cuda source (oad_cpp_source=false)

[msaroufim]

mostly looking good, some minor questions