From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World
Paper • 2605.10834 • Published • 4
Ian Gardner
igardner
AI & ML interests
mischief
Recent Activity
updated a collection about 21 hours ago
Check out later repliedto salma-remyx's post 10 days ago
If you can’t explain why your AI-generated contribution belongs in the repo, don’t put it in a maintainer’s queue 🙅🏻♀️
Before asking for review, you should be able to answer:
1. What project need does it address?
2. Where does it fit, and does it duplicate existing work?
3. What evidence shows it works, and will you own it through review?
If you can’t answer those, you haven’t saved anyone time. You’ve passed the buck to the maintainer.
Outrider has made us better contributors by doing more of this work before upstream review by
* reading contribution rules, accepted PRs, and open issues
* finding needs and integration points
* drafting the code, tests, and context.
We still decide what deserves to go upstream, verify the claims, coordinate with maintainers and contributors, and stay involved through review.
On huggingface/peft, only 4 of 20 Outrider runs opened draft PRs. Two contributions have now merged:
✅ Riemannian-preconditioned LoRA: https://github.com/huggingface/peft/pull/3382
✅ Super-Tuning: https://github.com/huggingface/peft/pull/3518
We have more contributions in review and far more ideas were filtered out before they reached a maintainer.
Full case study: https://remyx.ai/case-study
Outrider: https://github.com/remyxai/outrider repliedto salma-remyx's post 10 days ago
If you can’t explain why your AI-generated contribution belongs in the repo, don’t put it in a maintainer’s queue 🙅🏻♀️
Before asking for review, you should be able to answer:
1. What project need does it address?
2. Where does it fit, and does it duplicate existing work?
3. What evidence shows it works, and will you own it through review?
If you can’t answer those, you haven’t saved anyone time. You’ve passed the buck to the maintainer.
Outrider has made us better contributors by doing more of this work before upstream review by
* reading contribution rules, accepted PRs, and open issues
* finding needs and integration points
* drafting the code, tests, and context.
We still decide what deserves to go upstream, verify the claims, coordinate with maintainers and contributors, and stay involved through review.
On huggingface/peft, only 4 of 20 Outrider runs opened draft PRs. Two contributions have now merged:
✅ Riemannian-preconditioned LoRA: https://github.com/huggingface/peft/pull/3382
✅ Super-Tuning: https://github.com/huggingface/peft/pull/3518
We have more contributions in review and far more ideas were filtered out before they reached a maintainer.
Full case study: https://remyx.ai/case-study
Outrider: https://github.com/remyxai/outriderOrganizations
None yet