HN 표시: AC 추적 - 승인 기준을 코드 및 테스트에 매핑한 다음 코드를 변경합니다.
hackernews
|
|
🔬 연구
#ac 추적
#python
#review
#리뷰
#코드 변이
#테스트
#승인 기준
#코드 변경
원문 출처: hackernews · Genesis Park에서 요약 및 분석
요약
AI 보조 코딩의 발전으로 심화되고 있는 소프트웨어 신뢰도 격차를 해소하기 위해, 테스트 통과가 실제 기준을 충족하는지 확인하는 도구 'ac-trace'가 오픈 소스로 공개되었습니다. 이 도구는 파이썬과 pytest 환경에서 인수 기준을 코드와 테스트에 매핑한 뒤, 코드를 변조(Mutate)하여 테스트가 오류를 제대로 감지하는지 검증합니다. 제작자는 이 실험적인 프로젝트에 대해 테스트 및 QA 업무, AI 개발 워크플로우를 다루는 분들의 피드백을 요청했습니다.
본문
ac-trace maps acceptance criteria to code and tests, then mutates the code to prove the tests actually catch the breakage. It is designed to fit an existing Python project, not force the project into a new architecture or workflow. You add a small amount of traceability metadata on top of your current codebase and keep using your current code, tests, and delivery process. Current scope: Python code and pytest tests. When code and tests are generated or heavily assisted by AI, teams often lose three things: - clear ownership of which code implements which acceptance criterion - confidence that the mapped tests actually protect that behavior - an easy way to review requirement -> implementation -> test coverage ac-trace gives that back by making the relationship explicit: AC ->code code ->tests tests -> proven by mutation checks, not only by green test runs - A YAML traceability map from acceptance criteria to code and tests - An inference mode for Python projects that derives the map from existing pytest tests annotated with AC ids - A mutation-based check that changes mapped code and runs only the mapped tests - A clear result per AC: Killed (tests failed as expected) orUnkilled (tests did not fail) - HTML and YAML reports that are easy to review locally or attach to CI The repo contains a small self-contained demo project in demo/ : demo/ acceptance_criteria.yaml traceability.yaml demo_api/ tests/ test_result_report.html The demo models a tiny quote API with four ACs: AC-1 : VIP discount at thresholdAC-2 : Shipping fee by speedAC-3 : Quote API returns a cost breakdownAC-4 : Health endpoint reports service status python3 -m venv .venv source .venv/bin/activate python3 -m pip install . All examples below use python -m ac_trace . - AC catalog: demo/acceptance_criteria.yaml - Hand-authored manifest: demo/traceability.yaml - Demo app: demo/demo_api/ - Demo tests: demo/tests/ python3 -m ac_trace infer demo/acceptance_criteria.yaml --output demo/traceability.generated.yaml This reads the AC catalog, scans annotated pytest tests, runs coverage, and produces a normal traceability manifest. python3 -m ac_trace manifest demo/traceability.yaml This validates the manifest first. If it is valid, it prints the AC -> code -> tests overview. python3 -m ac_trace run demo/traceability.yaml By default this: - validates the manifest - mutates each mapped code: item - runs only the mapped tests: - writes an HTML report to demo/test_result_report.html The demo intentionally contains an Unkilled AC, so this command exits with code 1 . That is expected and useful: it shows how the tool surfaces weak tests. python3 -m ac_trace run demo/traceability.yaml --ac AC-1 --report yaml --output demo/ac1-report.yaml Use this when you want machine-readable output or you only want to inspect one AC at a time. The model is simple: - Define acceptance criteria. - Map each AC to the code that implements it. - Map each AC to the tests that should protect it. - Mutate the mapped code and run only the mapped tests. - Mark the AC as Unkilled if any mapped test never fails. This is the key difference from a normal green test run: the tool checks whether the tests are sensitive to change, not only whether they pass on the original code. A mutation is a small, deliberate change in the implementation, made only for checking the tests. Examples: >= becomes> + becomes- 15.0 becomes16.0 ac-trace applies one mutation at a time to code mapped to an AC and runs the mapped tests. If the tests fail, that mutation is Killed . If tests stay green when behavior was changed, the tests are too weak for that part of the code. The intended adoption path is incremental. You do not need to redesign your project, replace pytest, or restructure your codebase. You add only the traceability layer: - an AC catalog - optionally AC annotations on existing pytest tests - a traceability manifest - your application architecture - your package layout - your existing pytest suite - your development workflow Create a YAML file that describes the ACs and where source and tests live: project_root: . source_paths: - src test_paths: - tests acceptance_criteria: - id: AC-101 title: VIP discount at threshold description: VIP customers with subtotal >= 100 receive a 10% discount. project_root is resolved relative to the catalog file. If you want automatic inference, annotate existing pytest tests with the AC ids they validate: from ac_trace.annotations import ac @ac("AC-101") def test_vip_discount_applies_at_threshold(): ... You do not need to rewrite the tests. You only add the AC link. You have two options. Option A: infer the manifest python3 -m ac_trace infer path/to/acceptance_criteria.yaml --output path/to/traceability.yaml Option B: write the manifest manually project_root: . acceptance_criteria: - id: AC-101 title: VIP discount at threshold description: VIP customers with subtotal >= 100 receive a 10% discount. code: - path: src/pricing/service.py symbol: calculate_discount lines: 1
Genesis Park 편집팀이 AI를 활용하여 작성한 분석입니다. 원문은 출처 링크를 통해 확인할 수 있습니다.
공유