0
/ 100
Solid foundation. Invest in docs and CI to grow from here.
Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.
Top fixes
Highest-impact changes first, ranked by point weight
- 1Tests18pt
Wire your tests to a documented command (e.g. a test script in your build config) so the suite is reproducible.
- 2CI/CD14pt
Add a lint step to catch style issues automatically.
- 3CI/CD14pt
Add `tsc --noEmit`, `mypy`, or `cargo check` to catch type errors before they merge.
- 4CI/CD14pt
Upload coverage to Codecov, Coveralls, or report it with `--coverage` flags.
Working through the fixes? Let every push regrade itself.
The free GitHub App rescans this repo on every push and posts the grade as a commit check, so the score climbs without coming back to rescan by hand.
Scorecard
Every check, grouped by category and sorted worst-first
Documentation
82
README is present.
CONTRIBUTING guide found.
README documents how to install the project.
Licensed under MIT.
Engineering
74
No issue or PR templates found (−100 pts).
→ Add .github/ISSUE_TEMPLATE/ with bug_report.md and feature_request.md to guide contributors. It dramatically improves issue quality.
Lockfile present (Gemfile.lock). Installs are reproducible.
Test files detected (tests).
CI is configured (.github/workflows/test.yml).
Linter or formatter configured ([tool.ruff] / [tool.black] in pyproject.toml).
Project health
98
Dependency manifest found (Gemfile).
Repository has a description.
Actively maintained (pushed within the last month).
.gitignore present.
Repository health signals
Activity, community, and responsiveness at scan time
Activity
- -Commits (30d / 90d)
- 42Forks
- 5Releaseslatest 2mo ago
Community
- -Community health
- -authors own >50% of commits
- 82Watchers
Responsiveness
- 17hMedian issue response
- 7d 5hMedian PR merge time
- 44Open issues
Repository files17 root entries
- .githubGood: CI is configured (.github/workflows/test.yml).
- docsGood: CONTRIBUTING guide found.Issue: CONTRIBUTING guide contents could not be read (−28 pts vs a readable file).Fix: Move the file to the repo root or docs/CONTRIBUTING.md so its setup, style, test, and PR sections can be graded.
- every_eval_ever
- testsGood: Test files detected (tests).
- tools
- utils
- _config.yml
- .gitignoreGood: .gitignore present.
- eval.schema.json
- GemfileGood: Dependency manifest found (Gemfile).
- Gemfile.lockGood: Lockfile present (Gemfile.lock). Installs are reproducible.
- instance_level_eval.schema.json
- LICENSEGood: Licensed under MIT.
- post_codegen.py
- pyproject.toml
- README.mdGood: README is present.Good: README is well structured with multiple sections.Issue: No screenshots or images in the README (−20 pts).Fix: Add a GIF, screenshot, or logo image. It is the fastest way to show what your project does.Good: README has code examples.Good: README links to a live demo or deployed app.Issue: No status badges in the README (−10 pts).Fix: Add CI/build status badges from shields.io or your CI provider to signal project health.Good: README documents how to install the project.Good: README documents how to run the project.
- uv.lock
Add this badge to your README
It updates automatically each time the repo is re-graded.
[](https://www.repo-grade.com/report/evaleval/every_eval_ever)