Clarification of stance on AI use

[Rua]

Recently there have been a few uses of AI/LLMs creeping into the c2rust project. First the postprocess command using LLMs internally, and now #1614. I don't think AI use is really ever justifiable, it taints the whole project, and people will not want to use it or contribute anymore. AI usage reverses the burden of understanding, where even those who generate code with it don't know why stuff is there. And if the project's own contributors don't know what they are writing anymore, and reviewers have to start asking why some unknown algorithm has made decisions on their code, that is not a good sign.

It would be helpful to have an official clarification from the c2rust maintainers about the use of AI. Will the project decide to reject it or accept it, with all that entails?

c2rust is a very useful project that has already helped many others. I started contributing my work because I value the project and want it to succeed. I really hope that the use of AI is reconsidered, because I don't feel comfortable contributing in good conscience if my work is going to be associated with it.

[thedataking]

Immunant and Galois' work on C2Rust is funded by the DARPA TRACTOR program. A key research question is whether LLMs can help make code migration more scalable and efficient. (The off-by-default postprocess command is part of that effort.)

We will be using the best available tools for a given job, including AI/LLMs when it makes sense; we do not intend to ban AI.

The concerns around code quality and burden of understanding are real. We are not lowering our code standards. Code must pass human code review. Contributors should understand their contributions and remain accountable for all aspects of the code they write (technical merit, utility, security, license compliance, etc.) regardless of the tools that helped them.

We will do our best to highlight where tools may use LLMs so it does not come as a surprise and make sure all AI features can be turned off.

Rua: we're tremendously grateful for your contributions; thank you! We understand that this may be a deal breaker and respect your position even though we disagree.

Re. Draft PR #1614: This is an experiment to understand whether codex or similar tools are good enough to help us with error-prone mechanical tasks like nightly bumps. The c2rust-refactoring tool (which is less used than the transpiler) has bitrotted and is pinned to a nightly from 2022. I can see how the language in the PR can raise quality concerns. The PR is marked as a draft and won't land without the same level of code review we'd require of a "roll by hand".

[Rua]

Will all AI-assisted PRs be clearly marked in the commit history? Have any been merged into the transpiler?

[randomPoison]

We're not going to require that PRs/commits indicate that they're AI-assisted, though contributors are welcome to disclose that if they feel it's relevant. My rationale for this is that the person putting up the PR is fully responsible for the PR, regardless of how it was written. If the PR is low quality or makes questionable changes that the author cannot explain or justify, then we will reject that change regardless of how it was authored.

I'd also like to give some context into how I've personally been using AI on c2rust, since I'm probably the person who's experimented with it the most at Immunant. I've used AI on the following PRs so far:

• Rework relooper algorithm for fun and profit #1558 - This is almost entirely hand-written code, with the exception of the dominator tree analysis and the related tests, which I used AI to write. I also found AI useful for exploration of the code, as it could act as a second set of eyes to help me catch subtle details and get a grasp of what the relooper algorithm was doing.
• refactor: Add pass for converting libc math functions #1586 and refactor: Add pass for converting exit and abort #1587 - Initial implementation was AI-generated, and then cleaned up manually before putting up the PR. In both cases it was helpful to be able to quickly put together an implementation while focusing on tests and getting the behavior I wanted, and I found it easier to go through and cleanup the implementation afterwards.
• Restore fix_unused_unsafe transform and fix AST printing bug #1595 - I initially tried to use AI to identify the bug and it got stuck trying to figure out the issue, so I ended up having to dig in and fix it manually.
• refactor: Remove unnecessary blocks when cleaning up unused unsafe #1611 - Initial implementation was AI-generated and then cleaned up manually. Again, I found it helpful to quickly get something working along with tests, and then it was easier to do cleanup and iteration after the fact with tests in place.
• Roll toolchain to nightly-2022-11-03 #1614 - This is the only one where I just let the AI do all of the work with minimal guidance. As noted, this is explicitly experimental and is not reflective of how we generally intend to use AI, and I'm currently going through and evaluating the changes and cleaning it up manually before we consider merging it. It only went up in a WIP PR so that my coworkers could see what it looked like and so I could use the PR interface to help with my audit of the changes.

With the exception of #1614, all of my AI-generated changes have met my quality standards and no significant quality issues have been flagged in code review so far. As such, I have not felt the need to explicitly call out my usage of AI in those PRs.

That being said, I do also want to acknowledge that AI is not perfect, it can absolutely make mistakes, and careless usage of it can easily be detrimental to the project. I do NOT trust the AI to go off and make changes on its own (again, #1614 is an exception for explicitly experimental purposes, and is still being manually reviewed), and I personally prefer to use it in a pair programming style, where I'm directing the changes and the AI is just responsible for typing out the code, which it can do a lot faster than I can. So far this seems like a reasonable approach to me.

[margaretjoanmiller]

So we just...don't care about the environmental costs and the fact that it's trained on stolen code and any output of the LLM may be licensed to someone else? That it's taxing our already shaky grid and making the lives of people near data centers worse?

[Rua]

There are real legal issues with claiming copyright on something you didn't actually make. If code not written by people isn't explicitly marked, then that may be used to call the copyright of more aspects of the project into question.

[fish4terrisa-MSDSM]

It's weird that this project is using LLMs to touch the code I'm processing... Like I'm generally fine with the main project being worked with ai tools(there're potential copyright issues, but it's faced by this projects maintainers, and can generally be avoid by using true open source llms(like olmo) and trained with public domain code and code licensed with compatible licenses.)
The real problem is the postprocess command and other future potential features where the AI will change the code I'm converting with c2rust... This is really concerning not only because now users will have to face license issues the reliablity of the generated code is also facing challenges. Currently the code generated is consistant and it's possible to keep the generated code working as expected because it's converted through fixed logic in code, but if llms get introduced... Idk llms arent known for being able to generate code reliably.
If c2rust would start to convert the code using llm, then the whole purpose of this project is kinda defeated: users can just feed their code to the llm and tell them to convert it themselves, what c2rust stands out is that it's meant to be consistant, reliable, predictable and understandable.

Even if you dont care about the environment and the copyright issues,

(tbf the environment is already doomed after all and not really many people care about copyright especially when the code stolen is probably from another small developer instead of a big corpo and even them dont usually waste money on this)

it's still weird to think about the potential unpredictable and unreliable code c2rust will generated with the introduction of llms.

[Rua]

it's still weird to think about the potential unpredictable and unreliable code c2rust will generated with the introduction of llms.

Yes, I absolutely do not feel like I can trust the output of an LLM. I'm relieved that the postprocess command is optional, and that the main part of c2rust remains deterministic and transparent. Still doesn't alleviate the concern about using LLMs to write c2rust's own code, though.