AI DevOps 작업: AI 네이티브 저장소의 CI/CD를 위한 9가지 GitHub 작업

🔴 Live example: Open PR #1 (canonical demo) or any recent PR to see the full suite running on itself — context summary, quality score, and root cause hints, all from GITHUB_TOKEN . The full CI/CD layer for AI-native development — 8 GitHub Actions covering PR quality, safety, cost, infra, and behavioral testing. AI-native repos have problems that standard CI/CD doesn't solve. PRs flooded with AI slop. Unchecked LLM spend. Sensitive data leaking through AI outputs. MCP servers shipped without validation. Action tags silently compromised. Agent skills published without schema checks. Behavioral regressions invisible until production. This suite covers the full stack — eight GitHub Actions that work independently or as a pipeline. Most CI tells you something broke. This system tells you why — and what to do next. Three things this loop does that standard CI/CD can't: | Stage | What it does | |---|---| | 🔍 Detect | Catch regressions in AI behavior — not just code | | 🧠 Explain | Identify root cause: prompt change, model drift, data shift, cost spike | | 🔧 Fix | Turn failures into actionable feedback, automatically | Example output when your AI pipeline breaks: ❌ Eval failed Root Cause Analysis: → Knowledge drift (HIGH confidence) → RAG corpus changed 2 commits ago → Eval suite: 3/12 assertions failed Suggested fix: → Re-run evals with updated embeddings → Check: ai-workflow-evals + llm-cost-tracker AI systems don't fail like code — they degrade silently, drift with data, and pass tests while producing worse outputs. This is the CI layer that catches it. - Copy this into .github/workflows/ai-hygiene.yml in any repo: name: AI PR Hygiene on: pull_request: types: [opened, synchronize, reopened] jobs: hygiene: runs-on: ubuntu-latest permissions: pull-requests: write contents: read steps: - uses: actions/checkout@v4 - uses: ollieb89/[email protected] with: github-token: ${{ secrets.GITHUB_TOKEN }} - uses: ollieb89/[email protected] with: github-token: ${{ secrets.GITHUB_TOKEN }} threshold: 60 - uses: ollieb89/[email protected] with: github-token: ${{ secrets.GITHUB_TOKEN }} - Open a PR. - You'll see: 🔍 PR Context Summary Author: @you | Base: main ← feature/my-change 5 files changed (+120/-30) | Risk: low | Complexity: 3/10 Related issues: #42 ✅ PR Quality Score: 78/100 🟢 No correlated failure patterns detected. No API keys. No external services. Just GITHUB_TOKEN . New to the suite? Pick your entry point: | You want to... | Start with | |---|---| | Gate AI-generated PR slop | ai-pr-guardian | | Give AI reviewers full PR context | pr-context-enricher | | Stop secrets leaking from AI outputs | ai-output-redacter | | Lock down your supply chain | actions-lockfile-generator | | Catch agent behavioral regressions | ai-workflow-evals | | Validate your MCP server in CI | mcp-server-tester | | Publish agent skills safely | agent-skill-validator | | Understand why your AI pipeline broke | ai-root-cause-hints | | Track LLM spend before it hits your card | llm-cost-tracker | Each action works standalone. The full pipeline shows how they compose. | Action | What it solves | |---|---| | ai-pr-guardian | Scores PR quality 0–100, detects AI-generated slop, gates merges | | pr-context-enricher | Auto-generates rich context summaries: files, risk level, commit history, ready-to-paste AI reviewer prompt | | Action | What it solves | |---|---| | ai-output-redacter | Scans and redacts API keys, tokens, PII, and secrets from AI-generated outputs before they leave CI | | actions-lockfile-generator | Pins all uses: to full commit SHAs — prevents supply chain attacks | | Action | What it solves | |---|---| | ai-workflow-evals | Runs eval suites for prompts, agents, and workflows — catches behavioral regressions before merge | | mcp-server-tester | Validates MCP servers: health, protocol compliance, tool/resource discovery | | agent-skill-validator | Lints and validates agent skill repos (OpenClaw, Claude Code, Codex, Gemini) | | Action | What it solves | |---|---| | llm-cost-tracker | Tracks OpenAI/Anthropic/Gemini spend in CI, alerts on budget overruns | Don't know which action to use? Pick your problem: | My problem | Recommended stack | |---|---| | AI outputs regressed, don't know why | AI Debugging Stack — evals + cost tracker + root cause hints | | PRs are noisy and AI-sloppy | PR Hygiene Stack — context enricher + guardian + lockfile | | Worried about secrets leaking | AI Safety Stack — output redacter + lockfile + root cause hints | | Shipping agent skills or MCP servers | agent-skill-validator + mcp-server-tester | | Want everything | Full pipeline | jobs: ai-devops: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 # Enrich PR with context for AI reviewers - id: context uses: ollieb89/[email protected] with: github-token: ${{ secrets.GITHUB_TOKEN }} # Gate AI-generated / low-quality PRs before review - uses: ollieb89/[email protected] with: threshold: 60 on-low-quality: comment # Scan AI ou