How we check Codex listings
A listing with ✓ Install-checked in Codex installed cleanly with the real Codex CLI and Codex loaded something usable from it. Right now 893 listings carry the badge (checked with Codex CLI 0.163.0), and 168 more have a check note instead of a badge.
Listings install and run in your own Codex. We don't list paid remote-control or agent-relay services.
What the check does
- We install the listing with the Codex CLI (0.163.0) in a brand-new, empty home folder that is deleted afterwards. Codex is never signed in: no ChatGPT account and no OpenAI API key.
- We run the same install shown on the listing's page. Official plugins:
codex plugin add <name>@openai-api-curated, from a copy of OpenAI's plugin catalog taken once per run. Community plugins:codex plugin marketplace add owner/repo, thencodex plugin add; when a marketplace holds several plugins, we install each one it lists. MCP servers: the listing'scodex mcp addline. For servers that open a browser sign-in, we stop as soon as Codex says the server was added and never complete the sign-in. - Skills: we run the skill-installer script that ships inside Codex (
install-skill-from-github.py) on the skill folders in the repo. We don't type the$skill-installerprompt you'd use in a Codex session, because that needs a signed-in session. The script is the one that prompt runs. - Then we ask Codex itself what it loaded, through its app-server: installed plugins, skills, hooks, and MCP servers with their tools and sign-in status. The result comes only from those answers.
Results
| Result | On the site | What it means |
|---|---|---|
| Codex loaded skills | Badge | Skills from the plugin or skill folder show up in Codex's skill list. |
| Codex loaded hooks | Badge | The plugin's hooks show up in Codex's hook list. |
| MCP server started | Badge | Codex started the MCP server and it listed its tools. |
| MCP server asks for sign-in | Badge | Codex added the server and it asks you to sign in to the service (for example with OAuth) before its tools load. We never sign in. |
| Needs a local install or app | Note, no badge | The server's command isn't installed in a clean setup, or it connects to an app on your own computer. |
| Needs an API key | Note, no badge | The server needs a key or other setup before it starts. |
| Needs ChatGPT sign-in | Note, no badge | Codex offers it only after a ChatGPT sign-in (15 official plugins today). Our checker never signs in. |
| No install command | Note, no badge | There's no one-line Codex install command, so there's nothing to run. |
| Install fails or nothing loads | Hidden | The listing is taken off the site until a later check passes. |
The checker runs every night on new and changed listings, rechecks every badge at least once a week, and does a full pass over every listing once a week.
How the checker is isolated
Installing a plugin can run its code, so the checker is boxed in on several levels:
- Its own user. It runs as a separate user with no admin rights. It can't use sudo and has no access to the website files or anyone else's files on the server.
- A locked-down service. The operating system starts it with extra limits: it can't gain new privileges, the system folders are read-only, home folders are hidden, it can only write to its own work and cache folders, and it has caps on memory, CPU, number of processes and run time.
- A fresh sandbox for every listing. Each listing is installed inside its own bubblewrap sandbox: the system is read-only, the home folder is an empty temporary one, and the server's private folders and settings files (passwords, keys, mail and alert settings, web server config, other checkers' data) simply aren't there. Each sandbox has a time limit and is thrown away afterwards.
- Internet only. The sandbox can reach the public internet (it has to download plugins), but not the server's own services, the private network, or the cloud provider's metadata address.
- A self-test before every run. From inside a sandbox, the checker tries to read the server's secret files, write to system folders and connect to local and private addresses. If any of those succeed, it checks nothing and the site shows no badges.
- Only the result leaves the sandbox. The sandbox hands back one small JSON result. We parse it strictly, keep only known fields, and escape everything before it appears on the site. Nothing else a plugin produces is reused.
- A pinned Codex. The checker uses a fixed Codex CLI version installed from a lockfile with integrity hashes. It never updates itself; moving to a new version is a deliberate, reviewed change.
What the check doesn't do
- We don't sign in to anyone's service and we don't run listings in a live Codex session. A badge says it installs and loads, not that it does what it promises.
- We don't review code. Plugins with hooks or MCP servers run code on your computer, so read community listings before you install them.
Every check is public
checks.json lists each listing's latest check: date, Codex CLI version, the commit we checked and what loaded. Each listing also has its own record with recent history at /checks/<listing>.json, linked as "proof" next to its badge. Dates are in UTC.
Spotted a wrong result? Email hello@codexstacks.com.
The Stacks team