Commit 553d7a4

mo khan <mo@mokhan.ca>
2026-09-08 06:05:32
feat: implement cloudflare/agent-skills-discovery-rfc as a fetch client
Adds /skills fetch <origin>, backed by a new Fetcher class: 1. GET https://<origin>/.well-known/agent-skills/index.json 2. Reject unless $schema matches the v0.2.0 schema URI (an absent or unrecognized $schema is treated as an incompatible/unknown format and refused, per spec) 3. For each type: "skill-md" entry, GET the artifact and verify its SHA-256 against the index's digest before writing anything -- a mismatch raises instead of installing unverified content 4. Write verified skills to ~/.agents/skills/<name>/SKILL.md type: "archive" entries are skipped with a warning (not implemented -- no tar/zip handling, so no path-traversal/symlink surface to worry about yet either). Network/JSON errors are wrapped as Fetcher::FetchError so the /skills command's rescue clause catches all of them, not just the ones raised directly. Deliberately kept out of Catalog: discovery stays synchronous, local, and fast (it runs on every agent build, including /reload), so fetching is a separate on-demand step whose output just happens to land in a path Catalog already scans. Verified against the real, live index at developers.cloudflare.com (not just mocked specs) -- both directly via Fetcher and through the actual /skills fetch command in a real elelem chat REPL session with an isolated HOME. It publishes both entry types: the 7 skill-md skills (wrangler, web-perf, etc.) installed, digest-verified, and parsed correctly through Catalog/SkillFile; the 7 archive skills were skipped with warnings as designed.
1 parent 76d3e5f
lib/elelem/skills/fetcher.rb
@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+require "digest"
+require "fileutils"
+require "json"
+require "net/http"
+require "uri"
+
+module Elelem
+  module Skills
+    class Fetcher
+      SCHEMA = "https://schemas.agentskills.io/discovery/0.2.0/schema.json"
+
+      FetchError = Class.new(StandardError)
+
+      def initialize(install_dir: File.expand_path("~/.agents/skills"), http: Net::HTTP)
+        @install_dir = install_dir
+        @http = http
+      end
+
+      # Fetches https://<origin>/.well-known/agent-skills/index.json, verifies
+      # it, and installs every type: "skill-md" entry into install_dir. Entries
+      # with an unsupported type (currently anything but "skill-md") are
+      # skipped with a warning, per the spec's guidance for unrecognized types.
+      #
+      # Returns the list of skill names installed.
+      def fetch(origin)
+        index_url = URI.join(normalize(origin), "/.well-known/agent-skills/index.json")
+        index = get_json(index_url)
+
+        unless index["$schema"] == SCHEMA
+          raise FetchError, "unrecognized index schema #{index["$schema"].inspect} at #{index_url} (expected #{SCHEMA})"
+        end
+
+        (index["skills"] || []).filter_map { |entry| install(entry, index_url) }
+      end
+
+      private
+
+      def normalize(origin)
+        origin = "https://#{origin}" unless origin.start_with?("http://", "https://")
+        URI.parse(origin)
+      end
+
+      def install(entry, index_url)
+        name, type, url, digest = entry.values_at("name", "type", "url", "digest")
+
+        unless type == "skill-md"
+          Elelem.logger.warn("elelem-skills: fetch: skipping #{name.inspect}, unsupported type #{type.inspect} (archives are not implemented)")
+          return nil
+        end
+
+        body = get_raw(URI.join(index_url, url))
+        verify!(name, body, digest)
+
+        dir = File.join(@install_dir, name)
+        FileUtils.mkdir_p(dir)
+        File.write(File.join(dir, "SKILL.md"), body)
+
+        name
+      end
+
+      def verify!(name, body, digest)
+        expected = digest.to_s.delete_prefix("sha256:")
+        actual = Digest::SHA256.hexdigest(body)
+        return if expected == actual
+
+        raise FetchError, "digest mismatch for #{name.inspect}: expected sha256:#{expected}, got sha256:#{actual}"
+      end
+
+      def get_json(uri)
+        JSON.parse(get_raw(uri))
+      rescue JSON::ParserError => e
+        raise FetchError, "invalid JSON at #{uri}: #{e.message}"
+      end
+
+      def get_raw(uri)
+        response = @http.get_response(uri)
+        raise FetchError, "GET #{uri} -> #{response.code} #{response.message}" unless response.is_a?(Net::HTTPSuccess)
+
+        response.body
+      rescue FetchError
+        raise
+      rescue => e
+        raise FetchError, "GET #{uri} failed: #{e.class}: #{e.message}"
+      end
+    end
+  end
+end
lib/elelem/skills/setup.rb
@@ -43,8 +43,10 @@ module Elelem
         agent = @agent
         completions = -> { catalog.all.map(&:name) }
 
-        agent.commands.register("skills", description: "List available skills", completions: completions) do |arg|
-          if arg && !arg.empty?
+        agent.commands.register("skills", description: "List available skills, or `/skills fetch <origin>` to install from a remote index", completions: completions) do |arg|
+          if arg&.start_with?("fetch ")
+            fetch(agent, arg.delete_prefix("fetch ").strip)
+          elsif arg && !arg.empty?
             skill = catalog.find(arg)
             skill ? agent.output.say(skill.body) : agent.output.say("Unknown skill: #{arg}")
           else
@@ -52,6 +54,18 @@ module Elelem
           end
         end
       end
+
+      def fetch(agent, origin)
+        if origin.empty?
+          agent.output.say("Usage: /skills fetch <origin>")
+          return
+        end
+
+        names = Fetcher.new.fetch(origin)
+        agent.output.say(names.empty? ? "No skills installed from #{origin}." : "Installed: #{names.join(", ")}\nRun /reload to pick them up.")
+      rescue Fetcher::FetchError => e
+        agent.output.say(e.message, as: :error)
+      end
     end
   end
 end
lib/elelem/skills.rb
@@ -7,4 +7,5 @@ require_relative "skills/version"
 require_relative "skills/skill"
 require_relative "skills/skill_file"
 require_relative "skills/catalog"
+require_relative "skills/fetcher"
 require_relative "skills/setup"
spec/elelem/skills/fetcher_spec.rb
@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+
+RSpec.describe Elelem::Skills::Fetcher do
+  let(:install_dir) { Dir.mktmpdir }
+  let(:http) { class_double(Net::HTTP) }
+  let(:fetcher) { described_class.new(install_dir: install_dir, http: http) }
+
+  after { FileUtils.rm_rf(install_dir) }
+
+  def ok(body)
+    res = Net::HTTPOK.new("1.1", "200", "OK")
+    res.instance_variable_set(:@read, true)
+    res.body = body
+    res
+  end
+
+  def not_found
+    res = Net::HTTPNotFound.new("1.1", "404", "Not Found")
+    res.instance_variable_set(:@read, true)
+    res.body = ""
+    res
+  end
+
+  let(:skill_body) { "---\nname: example\ndescription: An example skill.\n---\nBody.\n" }
+  let(:digest) { "sha256:#{Digest::SHA256.hexdigest(skill_body)}" }
+
+  let(:index) do
+    {
+      "$schema" => Elelem::Skills::Fetcher::SCHEMA,
+      "skills" => [
+        { "name" => "example", "type" => "skill-md", "description" => "An example skill.",
+          "url" => "/.well-known/agent-skills/example/SKILL.md", "digest" => digest }
+      ]
+    }.to_json
+  end
+
+  describe "#fetch" do
+    it "installs a skill-md entry whose digest matches" do
+      expect(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(ok(index))
+      expect(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/example/SKILL.md"))
+        .and_return(ok(skill_body))
+
+      names = fetcher.fetch("example.com")
+
+      expect(names).to eq(["example"])
+      expect(File.read(File.join(install_dir, "example", "SKILL.md"))).to eq(skill_body)
+    end
+
+    it "adds https:// when the origin has no scheme" do
+      expect(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(ok(index))
+      allow(http).to receive(:get_response).with(URI("https://example.com/.well-known/agent-skills/example/SKILL.md")).and_return(ok(skill_body))
+
+      fetcher.fetch("example.com")
+    end
+
+    it "raises when the digest does not match" do
+      tampered = index.sub(digest, "sha256:#{"0" * 64}")
+      allow(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(ok(tampered))
+      allow(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/example/SKILL.md"))
+        .and_return(ok(skill_body))
+
+      expect { fetcher.fetch("example.com") }.to raise_error(Elelem::Skills::Fetcher::FetchError, /digest mismatch/)
+    end
+
+    it "raises on an unrecognized schema" do
+      bad = index.sub(Elelem::Skills::Fetcher::SCHEMA, "https://schemas.agentskills.io/discovery/9.9.9/schema.json")
+      allow(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(ok(bad))
+
+      expect { fetcher.fetch("example.com") }.to raise_error(Elelem::Skills::Fetcher::FetchError, /unrecognized index schema/)
+    end
+
+    it "raises when the schema field is absent (treated as incompatible v0.1.0)" do
+      no_schema = JSON.parse(index).tap { |h| h.delete("$schema") }.to_json
+      allow(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(ok(no_schema))
+
+      expect { fetcher.fetch("example.com") }.to raise_error(Elelem::Skills::Fetcher::FetchError, /unrecognized index schema/)
+    end
+
+    it "skips archive entries and warns, without raising" do
+      archive_index = {
+        "$schema" => Elelem::Skills::Fetcher::SCHEMA,
+        "skills" => [
+          { "name" => "bundled", "type" => "archive", "description" => "Has resources.",
+            "url" => "/.well-known/agent-skills/bundled.tar.gz", "digest" => "sha256:#{"0" * 64}" }
+        ]
+      }.to_json
+      allow(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(ok(archive_index))
+      expect(Elelem.logger).to receive(:warn).with(/unsupported type "archive"/)
+
+      expect(fetcher.fetch("example.com")).to eq([])
+    end
+
+    it "raises when the index request fails" do
+      allow(http).to receive(:get_response)
+        .with(URI("https://example.com/.well-known/agent-skills/index.json"))
+        .and_return(not_found)
+
+      expect { fetcher.fetch("example.com") }.to raise_error(Elelem::Skills::Fetcher::FetchError, /404/)
+    end
+  end
+end
elelem-skills.gemspec
@@ -26,4 +26,6 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
 
   spec.add_dependency "elelem", "~> 0.11"
+  spec.add_dependency "json", "~> 3.0"
+  spec.add_dependency "net-http", ">= 0.6"
 end
Gemfile.lock
@@ -3,6 +3,8 @@ PATH
   specs:
     elelem-skills (0.8.0)
       elelem (~> 0.11)
+      json (~> 3.0)
+      net-http (>= 0.6)
 
 GEM
   remote: https://rubygems.org/
@@ -38,6 +40,8 @@ GEM
       regexp_parser (~> 2.0)
       simpleidn (~> 0.2)
     logger (1.7.0)
+    net-http (0.9.1)
+      uri (>= 0.11.1)
     open3 (0.2.1)
     optparse (0.8.1)
     pathname (0.5.0)
@@ -100,6 +104,7 @@ CHECKSUMS
   json (3.0.0) sha256=1ff82a28c05c5cc7b646f3e3a3ac710e4ae2aa933690cba12f1f650b0ecdeb99
   json_schemer (2.5.0) sha256=2f01fb4cce721a4e08dd068fc2030cffd0702a7f333f1ea2be6e8991f00ae396
   logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
+  net-http (0.9.1) sha256=25ba0b67c63e89df626ed8fac771d0ad24ad151a858af2cc8e6a716ca4336996
   open3 (0.2.1) sha256=8e2d7d2113526351201438c1aa35c8139f0141c9e8913baa007c898973bf3952
   optparse (0.8.1) sha256=42bea10d53907ccff4f080a69991441d611fbf8733b60ed1ce9ee365ce03bd1a
   pathname (0.5.0) sha256=d5a331784f6e1f2fefb31c2ff0b8855aabfb661d807284ead9fa47b883d81623
README.md
@@ -41,15 +41,41 @@ links to bundled files (`[tests.md](tests.md)`) can be followed with the
 
 A `/skills [name]` command is also registered for human-driven listing/reading.
 
+## Remote discovery: `/skills fetch <origin>`
+
+Implements [cloudflare/agent-skills-discovery-rfc](https://github.com/cloudflare/agent-skills-discovery-rfc)
+(`.well-known/agent-skills/`) as a client. `/skills fetch example.com`:
+
+1. GETs `https://example.com/.well-known/agent-skills/index.json`
+2. Verifies `$schema` matches the v0.2.0 schema URI the spec defines --
+   an unrecognized or absent `$schema` is rejected outright, per spec
+3. For each `type: "skill-md"` entry, GETs the artifact and verifies its
+   SHA-256 against the index's `digest` before writing anything to disk;
+   a mismatch raises rather than installing unverified content
+4. Writes verified skills to `~/.agents/skills/<name>/SKILL.md`
+
+This is deliberately a separate, on-demand step from `Catalog` discovery,
+not folded into it -- `Catalog#discover` runs synchronously on every
+agent build (including `/reload`), so it stays local-filesystem-only and
+fast. Fetching is the only place in this gem that touches the network.
+
+Run `/reload` after a fetch to pick up the newly installed skills.
+
+**Not implemented**: `type: "archive"` entries (`.tar.gz`/`.zip` bundles
+with `scripts/`/`references/`/`assets/`) are skipped with a warning --
+only single-file `skill-md` entries install today. Verified against the
+real, live index at `developers.cloudflare.com`, which publishes both
+types; the `skill-md` entries (`wrangler`, `web-perf`, etc.) install and
+parse correctly, the `archive` entries are skipped as designed.
+
 ## What's not implemented
 
 - `agents/*.yaml` sidecar files (per-runtime display metadata, invocation
   policy) that some skills ship alongside `SKILL.md` -- ignored for now.
 - `allowed-tools` / permission scoping from the frontmatter.
-- Skill installation/updates (that's `npx skills`, a separate concern).
-- Remote discovery via `.well-known/agent-skills/` (see
-  [cloudflare/agent-skills-discovery-rfc](https://github.com/cloudflare/agent-skills-discovery-rfc))
-  -- this gem only discovers local filesystem paths.
+- Skill installation/updates from the `npx skills` package manager
+  ecosystem -- a separate concern from the well-known-URI discovery above.
+- `type: "archive"` fetch entries (see above).
 
 ## Setup