Skip to content

chore: add anti-slop protection for AI-generated spam PRs#642

Merged
anandgupta42 merged 2 commits intomainfrom
chore/anti-slop-protection
Apr 4, 2026
Merged

chore: add anti-slop protection for AI-generated spam PRs#642
anandgupta42 merged 2 commits intomainfrom
chore/anti-slop-protection

Conversation

@anandgupta42
Copy link
Copy Markdown
Contributor

@anandgupta42 anandgupta42 commented Apr 4, 2026

What does this PR do?

Adds automated detection and a contribution policy to combat low-quality AI-generated spam PRs.

Three changes:

  1. Anti-Slop GitHub Action (.github/workflows/anti-slop.yml) — runs peakoss/anti-slop on every PR open/reopen. Checks account age, profile completeness, merge history, spam usernames, PR quality, and a honeypot trap. OWNER/MEMBER/COLLABORATOR are auto-exempt.
  2. AI Contributions Policy (CONTRIBUTING.md) — sets expectations for AI-assisted contributions without being restrictive.
  3. Honeypot in PR template — hidden HTML comment that AI agents follow but humans ignore, triggering the blocked-terms check.

Type of change

  • Infrastructure / CI improvement
  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Issue for this PR

Closes #641

How did you verify your code works?

  • YAML validated with yaml-lint
  • Marker check passed (bun run script/upstream/analyze.ts --markers)
  • Reviewed anti-slop action docs to confirm all config options are valid
  • Verified exemptions cover all team members and CI bots

Checklist

  • I have tested this locally
  • I have not included unrelated changes

Summary by CodeRabbit

  • Documentation

    • Added an AI contributions policy to contributor guidelines clarifying expectations for AI-assisted work and human review.
    • Updated the pull request template to require a specific marker at the start of PR descriptions.
  • Chores

    • Added an automated pull-request quality workflow enforcing description, content and spam/compliance checks; failing checks close PRs and apply a review-needed label.

- Install peakoss/anti-slop GitHub Action with user signal checks
  (account age, profile completeness, merge history), honeypot trap,
  file protection, and conventional title enforcement
- Add AI contributions policy to CONTRIBUTING.md
- Add honeypot instruction to PR template for catching AI agents

Closes #641

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 4, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 56162bb5-91c9-4bc8-94a5-12cda2ed88fc

📥 Commits

Reviewing files that changed from the base of the PR and between 2affbe4 and 85b0f4a.

📒 Files selected for processing (1)
  • .github/workflows/anti-slop.yml
✅ Files skipped from review due to trivial changes (1)
  • .github/workflows/anti-slop.yml

📝 Walkthrough

Walkthrough

Introduces anti-spam protections: a new GitHub Actions workflow using peakoss/anti-slop, adds a honeypot HTML comment to the PR template, and adds an "AI Contributions Policy" section to CONTRIBUTING.md to define acceptable AI-assisted contributions and prohibit autonomous agent PRs.

Changes

Cohort / File(s) Summary
Anti-spam Workflow Configuration
.github/workflows/anti-slop.yml
Adds a new anti-slop GitHub Actions workflow (runs on pull_request_target) that invokes peakoss/anti-slop@v0 with scoped permissions and many enforcement inputs (PR description/title rules, blocked terms, commit checks, file path blocks, user-signal thresholds). On failure it closes the PR and labels it needs-review:blocked.
Honeypot & Contribution Policy
.github/pull_request_template.md, CONTRIBUTING.md
Adds an HTML comment honeypot instructing AI-generated contributions to include the keyword "PINEAPPLE" at the start of PR descriptions, and adds an "AI Contributions Policy" section to CONTRIBUTING.md prohibiting autonomous/unsupervised agent PRs while allowing human-reviewed AI assistance.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Contributor as Contributor
participant GitHub as GitHub (pull_request_target)
participant AntiSlop as anti-slop Action
participant Repo as Repository (PR & Labels)
Contributor->>GitHub: Open/Update PR
GitHub->>AntiSlop: Run anti-slop checks with configured inputs
AntiSlop-->>GitHub: Result (pass / fail)
alt fail
GitHub->>Repo: Close PR, add needs-review:blocked label, post failure message
else pass
GitHub->>Repo: Leave PR open (no extra label)
end

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 I sniffed a PR with pineapple sweet,
A little honeypot to foil each bot's feat.
With actions and rules I guard the gate,
Humans welcome — spam abate! 🍍✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The PR description does not follow the required template structure with 'Summary' and 'Test Plan' sections, though it provides comprehensive information about changes, verification, and a partial checklist. Restructure the description to follow the template: add a clear 'Summary' section at the start, include a dedicated 'Test Plan' section detailing how changes were verified, and ensure standard checklist format is used.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'chore: add anti-slop protection for AI-generated spam PRs' directly summarizes the main changes: adding infrastructure/CI improvements to combat low-quality AI-generated PRs through the anti-slop action.
Linked Issues check ✅ Passed The PR fully implements all three objectives from issue #641: the anti-slop action with user signal checks and honeypot, the AI contributions policy in CONTRIBUTING.md, and the hidden honeypot comment in the PR template.
Out of Scope Changes check ✅ Passed All three changes (anti-slop workflow, AI contributions policy, and PR template honeypot) directly address the objectives in issue #641 with no extraneous modifications.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/anti-slop-protection

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/anti-slop.yml (1)

4-5: Consider re-running checks on PR updates, not just open/reopen.

With only opened/reopened, post-open changes to title/description/commits can bypass this workflow until a reopen.

♻️ Suggested trigger expansion
 on:
   pull_request_target:
-    types: [opened, reopened]
+    types: [opened, reopened, edited, synchronize]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/anti-slop.yml around lines 4 - 5, The pull_request_target
trigger currently only fires for types [opened, reopened], which misses
post-open updates; update the trigger for pull_request_target to include
additional event types such as "synchronize" (for new commits) and "edited" (for
title/description changes) alongside the existing "opened" and "reopened" so the
workflow runs on PR updates as well.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/anti-slop.yml:
- Line 16: Replace the mutable action reference "uses: peakoss/anti-slop@v0"
with a pin to a full commit SHA for peakoss/anti-slop (e.g., "uses:
peakoss/anti-slop@<full-commit-sha>") so the workflow run under
pull_request_target uses an immutable reference; locate the string "uses:
peakoss/anti-slop@v0" in the workflow and update it to the specific commit SHA
from the action's GitHub repo, then commit the change.

---

Nitpick comments:
In @.github/workflows/anti-slop.yml:
- Around line 4-5: The pull_request_target trigger currently only fires for
types [opened, reopened], which misses post-open updates; update the trigger for
pull_request_target to include additional event types such as "synchronize" (for
new commits) and "edited" (for title/description changes) alongside the existing
"opened" and "reopened" so the workflow runs on PR updates as well.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: db5989a3-9284-4c04-acf0-9bb23db09753

📥 Commits

Reviewing files that changed from the base of the PR and between 9806411 and 2affbe4.

📒 Files selected for processing (3)
  • .github/pull_request_template.md
  • .github/workflows/anti-slop.yml
  • CONTRIBUTING.md

- Pin `peakoss/anti-slop` to full commit SHA (85daca18) for supply-chain
  safety since workflow uses `pull_request_target` with write permissions
- Add `edited` and `synchronize` triggers so checks re-run on PR updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@anandgupta42 anandgupta42 merged commit 5b6b31f into main Apr 4, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add anti-slop protection for AI-generated spam PRs

1 participant