Getting your repo found and correctly listed

Everything below is what Klondike's own discovery script actually checks - not guesses about how a "skill store" might work in general, but the literal signals this crawler looks for, found by inspecting real repos that were missing or misclassified.

1. Get discovered at all

Klondike finds candidates via three signals, unioned:

• A skill.json file (anywhere, but locale/en-us/skill.json is checked first) containing a pip_spec field.
• The GitHub topic ovos or openvoiceos set on the repo.
• ovos appearing literally in the repo's own name.

The single biggest lever if your repo isn't showing up at all: add the ovos or openvoiceos GitHub topic. Several real, actively-maintained repos (including some of OVOS's own) were invisible to this store simply because they had zero topics set and aren't skills, so no skill.json existed either.

2. Get classified correctly

Component type is derived from formal declarations, not guessed from a description:

• Skill - has a skill.json.
• Plugin (OCP, TTS, STT, PHAL, Pipeline, Persona, Solver, ...) - has a namespaced entry-point group declared in setup.py or pyproject.toml, e.g. entry_points={"opm.pipeline": "..."}. This is the same mechanism ovos-plugin-manager itself uses at runtime, so it's the most reliable signal - and a repo can have both a skill.json and a plugin entry-point at once (they aren't mutually exclusive); the entry-point takes priority for classification when both are present.
• Tool - no skill.json, no entry-points, but setup.py/pyproject.toml declares a real installable package (a name= field) - catches CLI clients and helper libraries.

3. Get your PyPI status verified

skill.json doesn't require a package_name field, but without one Klondike can't confirm PyPI status even when the package genuinely is published. Resolution order:

1. skill.json's own package_name field - most reliable, set this if you can.
2. setup.py/pyproject.toml's own name= declaration.
3. The repo's bare name, as a last resort - works only when it happens to match the actual PyPI package name.

4. Reach "Looks Complete"

Either of these gets you there:

• A confirmed manifest (from step 2) and published on PyPI and a real GitHub release.
• Already listed in OVOS's own upcoming Skill Store (not yet officially launched) - a maintainer reviewing and merging a submission there counts as a stronger signal on its own, since that store doesn't require PyPI or a release either.

Missing PyPI or a release doesn't exclude you - it shows as an "Incomplete" badge with the specific gap named, not a rejection.

5. Other things Klondike picks up automatically

• A LICENSE file - detected by GitHub itself, not read from skill.json. Missing one shows an explicit warning on the detail page.
• A real README - used as a description fallback when neither skill.json nor GitHub's own repo description have one, and scanned for Install/Setup/Configuration headings to surface any repo-specific setup steps beyond a plain pip install.
• A locale/<lang>/ folder (Skills only) - each subfolder becomes a language flag on your listing.
• Being tracked by OVOS Localize - gets you a "Translate" link automatically; nothing to add on your end beyond being in their own list.

6. Tested on OVOS

Every Looks Complete skill and pipeline plugin (not archived) is tested against OVOS's three release channels, and so is everything the OVOS installer installs on a channel, whatever its status here: an archived skill the installer still pulls in reaches users all the same, so it is tested and marked "still installed by OVOS" rather than hidden. The result shows as a small label per channel on the card, a section on the detail page, and a badge you can put in your README. A skill whose own golden utterances reach it also gets a quality label on the card: 🎯 Routes (level 3), or ⛏ Klondike Gold when they still do with the Klondike profile loaded. It is taken from testing, or from alpha (and says so) when testing has no result yet. The test overview shows the same results grouped the way a device gets them: the installer's defaults, its extra skills, and the Klondike profile, per channel.

• testing: what the OVOS installer installs by default ("recommended"), so what most new devices run. It weighs most.
• alpha: the installer's other choice, and the only one on a Mark 2/DevKit or macOS.
• stable: not offered by the installer today, but the slow, safe track a new stable release will land on.

• Level 1, installs: pip install of your package under the channel's own constraints file (constraints-<channel>.txt from OpenVoiceOS/OpenVoiceOS, the same file the OVOS installer uses, fetched live). This catches dependency conflicts with what a device on that channel actually runs. If the channel pins your package itself, the version it pins is the one tested.
• Level 2, loads: the package is booted in MiniCroft with every language it ships. A skill passes when it loads without errors; what it registered is recorded (intents per language, fallback, common query, OCP or other provider registration), so provider skills without intents are not failed. A language that registers no intents while others do is flagged. A pipeline plugin passes when its pipeline stage loads.
• Level 3, routes (skills): your own golden utterances are sent through a normal intent pipeline with the skills a default OVOS install has loaded next to yours, and each one must reach your skill (and the intent the row names). This is what catches intent theft: a sentence that works in your own test suite, where your skill is alone, but that a default skill takes on a real device.
  • Default skills: read live from ovos-installer with its default choices (profile "ovos", skills on, extra skills off), which today installs ovos-core[skills-essential,skills-internet,skills-audio]. The channel's constraints decide their versions. The pipeline order is the installer's too. The detail page lists what was loaded. A stage of that pipeline that does not load on a channel (its plugin is missing something) is left out of the routing, as it is on a device there, and the test overview says so for that channel.
  • Golden files: test/end2end/golden_utterances*.jsonl at the git tag of the version tested (v1.2.3 or 1.2.3), one JSON object per line with utterance, lang, skill_id and intent_label (or expected_intent); rows with "needs_manual": true are skipped. The same format ovoscope golden runs. A skill that answers OCP's search (a @ocp_search handler) gets "play …" and "start …" sentences through OCP, not through its intent: OCP picking your result counts as reaching your skill. Add "intent_type": "ocp" to a row that must go that way; keep its intent_label, which ovoscope golden still checks against your intent files. Rows from your default branch are not used: they may name intents the released version does not have.
  • Score: level 3 is reached when at least 80% of the counted en-US rows reach your skill; the label always shows the numbers, e.g. ✓ 14/14 golden. A skill without en-US golden utterances does not reach level 3. A miss says who took the sentence (a default skill, another of your intents, or nobody). A sentence taken by another tested skill that is not a default skill is shown as a collision and not counted: which skills are tested together is decided by the test run, not by users.
  • Languages: each language is a boot of its own with every default skill trained, so a run routes en-US plus the languages most skills in it have rows for; the rest is listed as not routed that run. Which those are depends on the other skills in the run, not on yours, so only en-US decides level 3 (and so the channels compare the same thing). The other languages are shown on the detail page per language, as language coverage.
• Level 3, routes (pipeline plugins): a pipeline plugin has no golden utterances of its own to reach; what matters is what it does to everyone else's. In the Klondike test every sentence is traced to the pipeline plugin that matched it, and for each plugin the test counts the sentences it reaches (took and handed to the skill they belong to) and the ones it takes (took from the skill they belong to; the overview lists a few). A pipeline plugin of the Klondike profile earns ⛏ Gold when it reaches some and takes none. This needs OVOS to say which plugin matched a sentence, which ovos-core 3 does (alpha today); on the other channels it shows as not measurable rather than as a guess. Level 1 and 2 are as above: it installs, and its stage loads.
• Stop check (skills that keep going): after its rows, one sentence your skill took is sent again (en-US). If your skill is still busy after its first speech or playback (reading a story, counting, a metronome), "stop" is said in that session, the way a user would, and your skill must go quiet within two seconds and let its handler finish. The label then reads ✓ 14/14 golden · stops, or ✓ 14/14 golden · doesn't stop when it went on talking or playing, or its handler never returned; the detail page shows what it said after "stop". A skill that answers and is done (most skills) has nothing to stop, and nothing is shown. To pass: implement stop() (or stop_session()) so it ends what your handler is doing, and check for it in loops that speak or play.

Level 3 on a well-equipped install: the Klondike profile

A normal install is only half the picture: people add skills. So level 3 is also run against a second, larger baseline, the Klondike profile: the installer's default skills, plus the installer's extra skills (off by default), plus a short curated list kept in compat/klondike-profile.toml, with pipeline plugins placed where their README says. It gives two results:

• Every skill against the profile: a second line in the detail page's Routing block, "Routing on a well-equipped install", and the ⛏ Klondike Gold label on the card when at least 80% of its golden utterances still reach it. A sentence taken by a profile skill that a normal install doesn't have is shown as an overlap: useful to a user, who can pick one. It does not change the level or the badge.
• The profile against itself: all profile skills in one core, with all their golden utterances. A profile skill's detail page lists who takes its sentences and whose it takes.

Two alternatives outside the profile (two calculators, say) are never tested against each other: that is intended. A curated entry has to pass level 2 on the channel (otherwise it is left out there, and listed below), cover a function nothing else in the profile covers, need no API key or special hardware, not be desktop- or GUI-only, and speak English at least. OpenVoiceOS first, then andlo, then others. To propose one, open a pull request against the TOML file.

What passes in each profile, Default, + Extra and + Klondike, per channel: 📊 Test overview by install profile.

The labels at a glance

Recommended order: the store's default sort. testing counts most (ten times alpha), stable three times alpha: a package that works on testing ranks above Looks Complete ones not tested yet, and one that fails on testing ranks below them. Among working ones, measured golden routing ranks above "loads", by the share of rows that route. Ties go to stars.

The channel stack is installed by OpenVoiceOS/ovos-test-harness (its channel_compat install, pinned by commit), so a result means the same thing there and here. Each result lists the exact ovos-core, ovos-workshop, ovos-padatious and ovos-bus-client versions it was tested against, and the full pip freeze is published as compat/stack-<channel>.txt.

When it runs: nightly, but a package is only re-tested when something that matters changed: a new release, a change to the channel's constraints, the languages it ships, or the test harness itself. A new release is picked up the same night (PyPI is checked directly). To test now, use 🧪 Request test on the detail page (at most once a day per entry).

Badge: the detail page shows a ready-to-paste snippet once your package has been tested. It looks like this:

[![OVOS alpha](https://img.shields.io/endpoint?url=https://andlo.github.io/ovos-klondike-mercantile/badges/<your-skill-id>/alpha.json)](https://andlo.github.io/ovos-klondike-mercantile/detail.html?id=<entry-id>)

Three grey results are not failures: needs config (your own load error says it needs an API key, account or identity first; the message is shown), not supported (the channel cannot run this kind of package at all, e.g. stable's ovos-core has no loader for third-party pipeline plugins), and needs device (loading waits in your own code for something only a device provides, such as network or GUI ready signals; the detail page shows the exact line). A failure caused by the test setup rather than your package (PyPI unreachable, the test driver unable to boot on a channel) is never shown as a ✗: it is kept out of the results and retried. Tests run in a job without any secrets or write access, since they install and import packages from PyPI. A result is information, not a verdict: if it looks wrong, open an issue.

7. Test your skill on a real device

Some skills can't be judged in our tests: they need an API key or an account (needs config), device services (needs device) or hardware. You can test them yourself on a configured device and publish the result. It shows on the card as ✓ alpha · maintainer, in its own colour so it is never mistaken for our own result, and it ranks above untested entries and below our own tests.

1. A device on the channel. The OVOS installer lets you pick testing (its default) or alpha; raspOVOS and ovos-docker images follow alpha unless changed. In ovos-tui-client, Ctrl+P → OVOS: Release channel shows which channel the device runs (worked out from its versions, whatever made the install) and how to change it on each kind of install. One device per channel, or switch.
   
   Make it the channel, not just close to it. An install is not always exactly the channel it was made for: the installer resolves in separate batches (on alpha with pre-releases allowed for everything), so a device can end up below the channel's versions, with third-party betas, or with plugins that conflict with the core. A report from such a device says little about the channel. After the installer, let ovos-tui-client (0.3.0a6 or newer) make it the channel, on the device, in the OVOS venv as the user that owns it:

   ~/.venvs/ovos/bin/pip install "ovos-tui-client>=0.3.0a6"     # no --pre: that would take betas of everything
   ovos-tui --set-channel testing --dry-run    # what it would change (or alpha, stable)
   ovos-tui --set-channel testing

   or in the TUI: the Set channel: <channel>… buttons in OVOS: Release channel, or Ctrl+P → OVOS: Set channel: <channel>… (the same as --set-channel), which show the dry run first and ask. It builds the device's stack the way our tests build theirs: the channel's packages by name under its live constraints, without --pre; ovos-core, ovos-workshop, ovos-bus-client, ovos-plugin-manager, ovos-config and ovos-utils then locked; everything else the channel names upgraded next to them; pre-releases nothing asks for moved to final releases. It never downgrades the core: a package that would need that is reported and left as it is. Restart OVOS afterwards. Running it again is safe, and the way to keep a device on the channel as it moves. See ovos-tui-client's release channels.
2. The released version, configured. Install it under the channel's constraints and the core lock the last --set-channel left, so it can't move the device off the channel:

   cd "$(ls -d ~/.cache/ovos-tui-client/set-channel/<channel>/* | tail -1)"
   ~/.venvs/ovos/bin/pip install -c constraints.txt -c lock.txt your-skill==<latest release>

   lock.txt holds the core versions, so pip refuses rather than quietly changing ovos-core or ovos-workshop for your skill. If it refuses, your skill doesn't fit the channel: that is the result, not something to force. Then add the keys or hardware it needs.
3. Utterances. Your golden file (test/end2end/golden_utterances_<lang>.jsonl) if you have one. If not, ovoscope generate writes a first draft from your .intent files to edit.
4. Run the test on the device with ovos-tui-client (0.2.0 or newer), in either of two ways. Both save the same ovos-test-report/1 file (….report.json, next to a readable .md) with the installed versions, your skill's version and the channel.
   • In the TUI: start ovos-tui, run Ctrl+P → Test: <your skill> - All, then Ctrl+P → Test: Save result…. A window asks for the channel (already filled in, with how it was found), a note on the setup ("needs an OpenWeather key", "Mark II"), and whether to include OVOS's replies: leave that off. Save and copy puts it on the clipboard; if your terminal doesn't pass that on, Save and show shows it to copy by hand (and Test: Show last result shows it again later). Over ssh, or in a terminal that doesn't pass the copy on (GNOME Terminal), use Save and share: it prints a short link to open the report in your own browser, and an scp command to fetch the file.
   • Headless, e.g. from a script:

     ovos-tui --run <your skill_id> --report report.json --notes "needs an OpenWeather key"

     --channel is optional; if you give it, it must match what the device runs. Don't use --report-replies for a report you share.
   Klondike's report link. When ovos-tui asks Where do you submit test reports? (or later: Ctrl+P → Settings: Skill store report link, or --submit-url), paste:

   https://andlo.github.io/ovos-klondike-mercantile/detail.html?skill={skill_id}#report={report_fragment}

   The page ovos-tui shares then has an Open detailed page button that opens your skill's page here with the report filled in and checked. ovos-tui submits nothing itself: you look at the result and submit it from this page. The report travels after the #, which browsers never send to a server. No link? Upload the .report.json file in Submit a test report instead.
   
   Klondike checks the channel again from the versions, so this works on any install (the OVOS installer, raspOVOS, Buildroot, Docker, your own venv). Never edit the report by hand.
5. Check it with Submit a test report on your skill's detail page: it says whether the report is valid for the channel as it is today and, if not, exactly why.
6. Optional: a Klondike-profile device. A report from a device that runs the Klondike profile is the most real version of that test, and shows as such next to ours. Install the profile for your channel into the OVOS venv under the channel's constraints and the lock (the requirements link in the profile's box above):

   cd "$(ls -d ~/.cache/ovos-tui-client/set-channel/<channel>/* | tail -1)"
   curl -fsSLO https://andlo.github.io/ovos-klondike-mercantile/compat/klondike-profile-<channel>.txt
   ~/.venvs/ovos/bin/pip install -c constraints.txt -c lock.txt -r klondike-profile-<channel>.txt

   run ovos-tui --set-channel <channel> once more so everything the profile added follows the channel too, set the pipeline from its mycroft.conf pipeline link, restart OVOS, then run

   ovos-tui --run <your skill_id> --report -

   Klondike compares the report's installed skills and pipeline with the profile and, if something is missing, says what ("3 profile skills missing: …"). ovos-tui-client doesn't know about the profile: it reports what is on the device, and Klondike decides.
7. Commit it as test/reports/testing.json, test/reports/alpha.json or test/reports/stable.json. Klondike picks it up on its next visit to your repo (within about 2 days, see How often does it refresh?), or within 3 hours if you use 🔄 Request update on the detail page.

When a report counts: it is for your latest release; its core versions (ovos-core, ovos-workshop, ovos-padatious, ovos-bus-client, ovos-plugin-manager) are what a channel runs today: allowed by that channel's current constraints (read live, every time) and on the same minor version the channel installs now (alpha's constraints only set minimums, so the second part is what makes an old alpha report stale). The channel is the one the versions match; a report whose versions match no channel, or more than one, doesn't count. Then: the skill loaded, and at level 3 at least 80% of the utterances reached it. A report that no longer counts stays visible, greyed out, as "tested on an older version or channel". A report with private data in it (IP addresses, e-mail addresses, paths under a home folder, things that look like keys) is not published at all; the detail page says so. The test tool is expected to leave host names and settings out in the first place.

Users can submit reports from their own setups too, through the same box. Those show as confirmations ("✓ alpha · 4 users") and as honest "doesn't work for me" reports with the setup, and weigh less than our own tests or your report. If you submit one yourself for your own skill, you are pointed here instead, since yours belongs in your repo.

Forks

Forks are excluded by default (to avoid random personal test-forks), except for a short list of orgs already confirmed as genuine first-party parts of the OVOS ecosystem (OpenVoiceOS, NeonGeckoCom, OscillateLabsLLC, TigreGotico, JarbasHiveMind, smartgic). If your org publishes real, maintained forks and isn't on that list, ask and it can be added.

How often does it refresh?

New repos get picked up within a few runs of appearing (the crawler runs every 3 hours). Already-known repos refresh on a rotation, not every run: with roughly 60 refreshed per run out of ~960 known repos total, a full rotation takes about 2 days - so as a rule of thumb, a change you make today (new topic, added package_name, archived the repo, whatever) should be reflected here within about 2 days, not immediately. To have your repo re-checked sooner, use 🔄 Request update on its detail page: it is picked up by the next crawl, within 3 hours.

The tests run on their own schedule: nightly, and only for packages where something that matters changed (see 6. Tested on OVOS). 🧪 Request test on the detail page tests one entry right away, at most once a day.