company_dns

company_dns V4.0.0-pre

A Rust server for company firmographics and industry-classification lookups: the V3 API’s URLs and answers, plus semantic and hybrid industry-code search, an experimental SQL endpoint, and built-in API documentation and UI. It combines what docs/plans/go-duckdb-rewrite.md, docs/plans/edgar-backend.md and experiments/ validated individually; the original build is in docs/plans/v4-server-prototype.md.

Status: pre-release. It runs on a staging environment and passes its API test suite there; V3 is still the live service. The plan for moving it to production is docs/plans/v4-production-cutover.md; the documents plan is docs/plans/v4-documentation-update.md. Data terms (and a pending permission for ISIC) are in the root README.

Version labels. The code reports 4.0.0 everywhere (/health, the OpenAPI document, every crate; one value in Cargo.toml). The documents call this release V4.0.0-pre until it is released; dropping “-pre” is a documents-only change.

Deploying it: three layers

1. Run the container (general). The image is self-contained (binary, UI, data and embedding model inside; no network needed to start) and runs as a non-root user on a read-only filesystem. Take a tag from the package page:

docker run --rm -p 4000:4000 -e COMPANY_DNS_REPLICAS=1 ghcr.io/miha42-github/company_dns/company_dns_v4:<tag>
curl http://localhost:4000/health

Every setting is an environment variable (see the Settings reference). Set COMPANY_DNS_REPLICAS to the number of copies you run; the rate limits are divided by it.

2. Run it on Kubernetes (a reference). k8s/staging/ holds standard objects (Deployment, Service, probes, resources, anti-affinity, ConfigMaps, an Ingress). Adapt, do not apply blindly: they assume Traefik as the ingress class (including a Traefik Middleware for the HTTP redirect), cert-manager with an issuer named letsencrypt-prod, this project’s host names, a SealedSecrets credentials file, and COMPANY_DNS_REPLICAS equal to replicas (set by hand).

3. How the maintainers deploy it (specific to our cluster). scripts/build-and-deploy-v4.sh and scripts/seal-secrets-v4.sh, guided by docs/v4-bringup.md. These are maintainer-focused today. The intention is to make them generic.

Already general Debt, to work through over time
The application and image: self-contained, non-root, /health, settings as environment variables or two optional mounted files, no Kubernetes dependency, a public image The scripts hard-code the registry and owner, the namespace, host and certificate names, the Traefik class and the letsencrypt-prod issuer; they assume SealedSecrets, a GitHub token in .env and microk8s kubectl. Move the constants into a config file or environment overrides
The manifest shapes (probes, resources, spread, ConfigMaps) The manifests assume Traefik, a named issuer and our host names. Kustomize overlays (a shared base plus per-environment changes) or documented substitutions
The deploy script’s pattern (preflight, tag applied at deploy time, rollback, build log) The default allow-list for the credentialed routes, and the rate limiter’s trusted-origin check (which earns the normal tier), name mediumroast.io. Make the trusted origins an explicit setting with no built-in default for outside deployments
The load and soak tools (perf_tests/) work against any URL COMPANY_DNS_REPLICAS must equal replicas by hand. Derive it where possible, warn loudly where not
  The verification path through the ingress address and the CoreDNS section of the bring-up guide are workarounds for our router. Label them as ours; the generic check is the public name
  svc_ctl.sh controls V3 only. Fold V3 and V4 into one controller, or retire it with V3

Layout

crates/
  cache/          general moka TTL+LRU cache (promoted from experiments/edgar-cache-spike)
  edgar/          edgarkit wrapper: firmographics fetch + cache, catalog ingest/query
                  (promoted from experiments/edgar-spike, edgar-index-query, edgar-cache-spike)
  sic/            US SIC data access: V3-parity lookups + similarity search
                  (promoted from experiments/df-spike, ic-similarity-service)
  wikipedia/      real MediaWiki/Wikidata client - infobox/claims parsing, V3's
                  corporate-suffix hint (restored, actually executed as REST calls),
                  429/503 backoff (promoted from experiments/wikipedia-spike)
  firmographics/  real EDGAR+Wikipedia merge (both sides real now)
  server/         the Axum binary wiring it all to HTTP routes

Requires DataFusion 55.x and edgarkit sharing one process — decided in go-duckdb-rewrite.md/edgar-backend.md’s status lines (2026-09-28), resolving the chrono-version conflict that forced the EDGAR spikes apart on DataFusion 42.

Running it

Build profiles. cargo build --release is the quick development build. For a shipped binary there are two named profiles, both with fat LTO, one codegen unit and stripped symbols (the DataFusion feature trim, no Parquet and no compression codecs, applies to every build):

cargo build --profile release-lean    # about 89 MiB stripped; no slower than the plain release build
cargo build --profile release-small   # about 62 MiB; same speed except scan-heavy SQL (about 28% slower)

Each takes several minutes to build. The sizes were measured on macOS arm64 and the choice between them is confirmed after testing on Linux (docs/plans/v4-release-to-staging.md, step 5). The binary lands in target/release-lean/ or target/release-small/.

Data directory. Every data file lives in one directory, named by COMPANY_DNS_DATA_DIR. For now, in development, point it at the repo’s top-level tmp/ (the same place the SIC files are staged by hand):

export COMPANY_DNS_DATA_DIR=/path/to/company_dns/tmp

If it is unset the default is that same repo-root tmp/, found from the source tree at build time, so it no longer depends on which directory you start the server from. In the container image it is the directory the files are copied into; a volume mounted there replaces them. Each file also still has its own override variable (below).

1. Ingest the EDGAR catalog (writes edgar_10x_catalog.feather into the data directory, not committed - docs/plans/v4-server-prototype.md §4, docs/plans/v4-deployment.md §3.2). With no arguments it pulls the last two years of completed quarters (8) from the SEC and rebuilds the catalog from scratch:

cargo run --release --bin ingest-edgar                       # last 2 years
cargo run --release --bin ingest-edgar -- --years 1          # last year
cargo run --release --bin ingest-edgar -- --from 2024Q1 --to 2025Q4
cargo run --release --bin ingest-edgar -- 2025 2             # one quarter

It needs network access to sec.gov. If any quarter fails nothing is written and the existing catalog is left untouched; an empty result is refused.

2. Get the SIC data - the Mediumroast-produced flat-and-embedded files, staged by hand in the data directory for now (the delivery mechanism from mediumroast.io is not confirmed): us_flat_embedded.feather, japan_rev13_flat_embedded.feather, nace_rev2_flat_embedded.feather, isic_rev4_flat_embedded.feather. Check a new file before use with python3 docs/plans/research/check_ic_feather.py <file>. See experiments/df-spike/README.md for background.

3. Run the server:

cd crates/server
cargo run --release

Env vars (all optional): COMPANY_DNS_DATA_DIR (above), and per-file overrides SIC_DATA_PATH, JAPAN_SIC_DATA_PATH, EU_NACE_DATA_PATH, ISIC_DATA_PATH, EDGAR_CATALOG_PATH, plus SIC_MODELS (all_minilm_l6_v2 default, per go-duckdb-rewrite.md §7.8), PORT (default 4000), COMPANY_DNS_QUERY_PARALLELISM (partitions per lookup query; default: the host’s CPUs, at most 4. The classification and EDGAR tables are read into memory once at start, and a lookup is split into a few partitions, not one per CPU: on these small tables more partitions cost more than they save), RUST_LOG/LOG_LEVEL_CONFIG_PATH (see “Observability” below).

Docker

The image is self-contained: the binary, the UI, the data files and the embedding model are all inside it, so a pod starts with no volume and no outbound access. Decided in docs/plans/v4-deployment.md section 3.2 and docs/plans/v4-release-to-staging.md step 6.

v4/scripts/docker-build.sh -t company-dns-v4:dev            # release-lean (about 89 MiB binary)
v4/scripts/docker-build.sh -t company-dns-v4:small -p release-small
v4/scripts/docker-smoke.sh company-dns-v4:dev               # starts with --network none, then runs the offline test layers

Like V3 (whose makedb.py builds its EDGAR database during the image build), the build prepares everything: it runs ingest-edgar against sec.gov (about 15 seconds, 8 quarters), so the build needs network access and the image is ready to run when it starts. --edgar data uses a frozen catalog from DATA_DIR instead.

What the build needs that is not in git (the script supplies both; override with DATA_DIR and MODEL_DIR):

Build context What it is Default
data the four classification .feather files (and edgar_10x_catalog.feather only with --edgar data) the repository’s tmp/, or COMPANY_DNS_DATA_DIR
model the fastembed cache holding all-MiniLM-L6-v2 (models--Qdrant--all-MiniLM-L6-v2-onnx) v4/.fastembed_cache; run the server once to create it

The classification files can instead come from a URL: v4/scripts/docker-build.sh --data-source url:https://HOST/DIR/ (a header file in DATA_AUTH_FILE is passed as a BuildKit secret). v4/scripts/fetch_data.py does the fetching, from dir:PATH or url:BASE, and checks each file’s SHA-256 against v4/data-manifest.json; when a new delivery arrives, check it with check_ic_feather.py, then update the manifest hash on purpose.

Gates run in the build, so a bad input fails the build and never becomes an image: the manifest hashes, check_ic_feather.py on every classification file, a row and quarter check on the EDGAR catalog, and a SHA-256 check of the model weights. The runtime is debian:trixie-slim (glibc), because the ONNX runtime fastembed uses has no musl build. The process runs as UID 10001. A volume mounted at /app/data replaces the data without a rebuild. The image carries THIRD_PARTY_NOTICES.md and the Redoc licence under /usr/share/doc/company-dns/. The ONNX Runtime is Microsoft’s official release (1.31.0, pinned by SHA-256, fetched from GitHub during the build) and is loaded at run time (ORT_DYLIB_PATH), so the image runs on CPUs without AVX2, such as the production nodes’ 2013 Xeons. docker-build.sh --ort download uses pyke’s prebuilt runtime instead, which needs AVX2. For local cargo run the default is still pyke’s runtime; cargo build --no-default-features --features ort-dynamic with ORT_DYLIB_PATH set uses the dynamic one. v4/scripts/cpu-check.sh reports a machine’s architecture and AVX2/NEON and which build fits; the server logs the same facts at start (CPU: ...) and refuses to start, with a message, if the runtime cannot run on that machine. The build needs network access (the running container does not).

Settings reference

Everything is an environment variable (no config file loader), plus three optional mounted files. All are optional; the server starts with the defaults. Set them with env: in a Kubernetes Deployment, environment: or env_file: in Compose, or -e on docker run. CPU and memory limits are not server settings: they belong to the container runtime (resources.limits in Kubernetes, cpus and mem_limit in Compose), and the lookup tables stay in memory whatever the limit (about 160 MiB idle, under 260 MiB peak measured at 0.5 CPU to 16 CPU).

Variable Default Effect
PORT 4000 Listening port.
COMPANY_DNS_DATA_DIR /app/data in the image Directory holding the four SIC files and the EDGAR catalog; per-file overrides: SIC_DATA_PATH, JAPAN_SIC_DATA_PATH, EU_NACE_DATA_PATH, ISIC_DATA_PATH, EDGAR_CATALOG_PATH.
SIC_MODELS all_minilm_l6_v2 Embedding model for similarity search. FASTEMBED_CACHE_DIR (set in the image) holds its weights.
COMPANY_DNS_QUERY_PARALLELISM host CPUs, at most 4 Partitions per lookup query.
COMPANY_DNS_REPLICAS 4 Set to the Deployment’s replicas. The fleet-wide rate limits are divided by it, because each pod enforces its own share. Wrong on purpose or by drift, the fleet allows more or less than intended. 1 to 64; anything else stops startup.
COMPANY_DNS_RATE_LIMIT_PER_MINUTE, COMPANY_DNS_RATE_LIMIT_BURST 200, 20 The fleet-wide per-IP quota for callers that identify themselves (a descriptive User-Agent, or a profile without its own grant). Each pod enforces its share (divided by COMPANY_DNS_REPLICAS, never below 1), so with many replicas keep the burst well above the replica count: 600 over 12 pods leaves 50 per pod, 20 over 12 leaves 1. 1 to 1000000; anything else stops startup. Profiles with a rate_limit grant in the rules file are unaffected.
COMPANY_DNS_RATE_LIMIT_ANONYMOUS_PER_MINUTE, COMPANY_DNS_RATE_LIMIT_ANONYMOUS_BURST 5, 1 The same for callers with no identifying User-Agent.
COMPANY_DNS_SHUTDOWN_DELAY_SECS, COMPANY_DNS_SHUTDOWN_GRACE_SECS 5, 15 What the server does on SIGTERM (what Kubernetes and docker stop send): keep serving for the delay, so the load balancer can drop the pod, then stop accepting, close idle connections, let running requests finish for at most the grace time, and exit. Delay 0 to 60, grace 1 to 120, in seconds. Their sum must stay under the pod’s terminationGracePeriodSeconds (30 in k8s/staging).
COMPANY_DNS_CORS_ORIGINS mediumroast.io, its subdomains and localhost Which origins a browser may call the credentialed routes (/V4.0/sql) from: a comma-separated list of exact origins (https://app.example.com), or * for any. Lookups answer any origin regardless. curl, SDKs and servers are not affected by CORS.
COMPANY_DNS_CREDENTIALS_FILE, COMPANY_DNS_RULES_FILE none Profiles for authenticated access (see “Profiles” below). Mount them; never bake them into an image.
COMPANY_DNS_SQL_ENABLED and COMPANY_DNS_SQL_* off The experimental SQL endpoint and its limits (see “Experimental: SQL endpoint” below).
RUST_LOG, LOG_LEVEL_CONFIG_PATH info, /etc/company-dns-log-level/log-level Log level at start, and the file watched for a live change (see “Observability” below).
ORT_DYLIB_PATH set in the image The ONNX Runtime library, for builds that load it at run time.

API docs

GET /openapi.json (raw spec), GET /docs (Swagger UI), GET /redoc (ReDoc) — matching V3’s FastAPI docs_url/redoc_url/openapi_url exactly. Built via utoipa/utoipa-axum — docs/plans/v4-openapi-docs.md.

Both pages stay in a light theme and load nothing from a third party: Swagger UI is embedded by utoipa-swagger-ui, and ReDoc is a vendored, pinned copy (2.5.4, MIT; crates/server/assets/redoc/) embedded in the binary and served at /redoc/redoc.standalone.js, with the system font stack (no CDN, no Google Fonts), so both work on an isolated network. Licences and notices: THIRD_PARTY_NOTICES.md.

Security

Per-client rate limiting, an inbound User-Agent gate, and request hygiene (timeout, body-size limit, tracing) — real, live, and past V3’s actual (undocumented) posture, not just matching it. Full design and live-verification record: docs/plans/v4-security-hardening.md.

Profiles: authenticated access

Callers can authenticate with HTTP Basic Auth: the profile id as the user name and a generated token as the password. Profiles live in two files, split by what they hold, both read once at startup (changing them means a restart; an invalid file stops the server; unknown keys are an error). With no credentials file everyone is anonymous.

1. Credentials (COMPANY_DNS_CREDENTIALS_FILE), like /etc/passwd: one profile:sha256-of-token per line, # comments allowed (credentials.example). This is the only secret file, and it holds only hashes, so a leaked copy is not a working credential. Keep it out of the repository and the image (a mounted Kubernetes or Docker secret in a deployment, a gitignored file locally).

TOKEN=$(openssl rand -hex 32)                           # give this to the client, once
printf '%s' "$TOKEN" | shasum -a 256 | cut -d' ' -f1    # put this after "profile:" in the file

2. Rules (COMPANY_DNS_RULES_FILE, optional), JSON with no secrets in it, so it can live in a ConfigMap or in git (rules.example.json): a defaults section and per-profile overrides. A profile starts from the defaults and an override changes only the settings it names:

{
  "defaults": {"rate_limit": {"requests_per_minute": 300, "burst": 30}},
  "profiles": {
    "mediumroast.io": {"rate_limit": {"bypass": true},
                       "sql": {"datasets": ["sic", "edgar"], "limits": {"max_rows": 500}}},
    "partner":        {"rate_limit": {"requests_per_minute": 600}}
  }
}

How inheritance works: objects merge key by key (so partner above keeps the default burst), a value or a list replaces (a datasets list is replaced, not appended to), and null removes an inherited setting ("sql": null takes SQL away, "max_rows": null drops one limit). bypass and a quota are alternatives, so naming one in an override replaces an inherited other. A profile with no entry just gets the defaults; an entry for a profile with no credential is an error (it catches typos).

The settings, all optional:

Setting Meaning
rate_limit: {"bypass": true} No rate limit on the lookup routes.
rate_limit: {"requests_per_minute": N, "burst": M} A quota of the profile’s own instead of the User-Agent tiers (burst defaults to N). Fleet-wide numbers, divided across replicas like the tier quotas (rate_limit.rs).
no rate_limit An identified caller gets the normal tier, whatever its User-Agent.
sql: {"datasets": [...], "limits": {...}} May use the experimental SQL endpoint, on those datasets, with optional limits of its own (below). No sql means no SQL.

Set it up, step by step

Start from the two templates in this directory, credentials.example and rules.example.json, and keep your real copies outside the repository (or under a gitignored path):

mkdir -p ~/company_dns-access && cd ~/company_dns-access
cp /path/to/company_dns/v4/credentials.example credentials
cp /path/to/company_dns/v4/rules.example.json   rules.json

  1. Make a token per profile and put its hash in credentials (replace the placeholder zeros on that profile’s line; rename or delete the example profiles):

    TOKEN=$(openssl rand -hex 32)                         # give this to the client, once; never store it here
printf '%s' "$TOKEN" | shasum -a 256 | cut -d' ' -f1   # paste this after "my-app:" in credentials
  2. Describe what each profile may do in rules.json: put anything most profiles share under defaults, and only what differs under that profile in profiles. Profile names must match the credentials file. Skip the rules file entirely and every profile is an identified caller with no special grants.

  3. Start the server pointing at both files (add the SQL flag only if you want the experimental SQL endpoint; it needs a profile with a sql setting):

    COMPANY_DNS_CREDENTIALS_FILE=$HOME/company_dns-access/credentials \
COMPANY_DNS_RULES_FILE=$HOME/company_dns-access/rules.json \
COMPANY_DNS_SQL_ENABLED=true \
cargo run --release -p company-dns-server

    Add RUST_LOG=info to see the profile names it loaded (the default level is warn; with SQL on you always get one warning line saying the endpoint is experimental). A mistake in either file stops the server with a message naming the profile and the problem.

  4. Try it with the token from step 1:

    curl -u my-app:$TOKEN http://localhost:4000/V4.0/na/sic/code/7372                   # a lookup, with the profile's rate limit
curl -u my-app:$TOKEN -H 'Content-Type: application/json' http://localhost:4000/V4.0/sql \
  -d '{"dataset": "sic", "sql": "select count(*) as n from sic_data"}'              # SQL, if the profile has it
  5. Deploying: mount credentials from a Kubernetes SealedSecret or a Docker secret, and rules.json from a ConfigMap or the image’s config, one pair per environment, then set the two env vars to the mounted paths (see docs/plans/v4-deployment.md). To try everything locally with throwaway profiles instead, use scripts/sql-try.sh (below).

No credential is anonymous (the User-Agent tiers apply). A wrong credential is a 401, never silently anonymous, and failed logins are throttled per source IP. Origin and Referer are ignored for identity. Whatever a profile ends up with is clamped by the server-wide limits, so these files can restrict a profile but never exceed what the host allows. Use TLS: the token travels with every request.

Experimental: SQL endpoint

Experimental. POST /V4.0/sql may change or be removed without notice. It is off by default, takes arbitrary SQL, and can use real CPU and memory, so it is closed to everyone who does not hold a credential you issued. Design and decisions: docs/plans/v4-sql-endpoint.md.

One read-only statement per request, against one dataset: sic (US SIC plus Japan SIC, EU NACE and ISIC when loaded) or edgar (the filings catalog), in DataFusion’s SQL dialect. The embedding (vector_*) columns are not exposed. select * from information_schema.columns lists tables and columns.

Turn it on (nothing below is set by default):

Variable Default Meaning
COMPANY_DNS_SQL_ENABLED off true registers the route. Without it the route does not exist (404) and is absent from /docs.
COMPANY_DNS_CREDENTIALS_FILE none, required when enabled Path to the credentials file (“Profiles” above), plus optionally COMPANY_DNS_RULES_FILE. The server refuses to start without a valid credentials file; it never starts open.
COMPANY_DNS_SQL_DEFAULT_ROWS / _MAX_ROWS 1000 / 10000 Rows returned by default / the ceiling a request’s limit is clamped to.
COMPANY_DNS_SQL_TIMEOUT_SECS 10 Statement timeout; the query is cancelled.
COMPANY_DNS_SQL_MEMORY_MB 256 Memory pool per dataset’s query context; no spilling to disk.
COMPANY_DNS_SQL_MAX_CONCURRENT 4 Queries running at once on this process; more get a 429, not a queue.
COMPANY_DNS_SQL_PARALLELISM 2 DataFusion partitions per query, so one query cannot take every core.

These are one global set of limits for every profile (per-profile limits are the next phase). Invalid values stop startup.

Credentials. The route needs a profile with a sql setting (see “Profiles” above): no credential is a 401, a profile without sql or without that dataset is a 403. Two layers of limits. SQL goes through the same rate limiter as the lookups, so the User-Agent tiers and a profile’s rate_limit grant apply to it, sharing the same buckets, except that a trusted Origin/Referer is never a bypass here (client-set headers). On top of that a profile can carry its own SQL limits in the rules file, each clamped by the server-wide values above (a mistake in the file can restrict a profile, never exceed what the host allows):

"sql": {"datasets": ["sic"],
        "limits": {"max_rows": 500, "timeout_secs": 5, "concurrency": 2,
                   "requests_per_minute": 30, "burst": 3}}

All five are optional. requests_per_minute and burst are fleet-wide numbers, divided across replicas like the other quotas; a profile’s concurrency is its own gate on top of the server-wide one.

curl -u my-app:$TOKEN -H 'Content-Type: application/json' http://localhost:4000/V4.0/sql \
  -d '{"dataset": "sic", "sql": "select count(*) as n from sic_data"}'

Try and test it locally: scripts/sql-try.sh creates throwaway profiles under .local/sql/ (gitignored), starts the server with SQL on and small limits, and runs a battery of checks (access, read-only, hidden vectors, limits, throttling) against it:

scripts/sql-try.sh start          # in one terminal
scripts/sql-try.sh check          # in another
scripts/sql-try.sh q sic "select * from sic_data limit 3"

Observability

Real structured logging (tracing, replacing raw println!), with a level that can be raised or lowered on a live, already-running process — no restart, no rollout. Full design and live-verification record: docs/plans/v4-dynamic-log-level.md.

Endpoints

Response shape. Every answer is the JSON envelope {code, message, module, data, dependencies}. Every answer also carries a top-level total: how many results data holds (a list gives its length, a V3-shaped dictionary the count it already carries (data.total, EDGAR’s data.totalCompanies, the SQL endpoint’s data.row_count), any other single record 1, no match or an error 0), so a client can read the count without looking inside data. It is added in one place (crates/server/src/total.rs), so no route can forget it. On /V3.0/, /V2.0/ and /v2/ it is the one deliberate addition to V3’s exact envelope (additive: data is unchanged; recorded in the differences ledger). /health, the OpenAPI document, the docs and the UI are not envelopes.

V3-parity: same URL shape with /V3.0/ changed to /V4.0/ (v4-server-prototype.md §5.2/§5.3/§10). Every one of them is also served at its original /V3.0/ path, and those paths answer in V3’s exact response shape (a dictionary keyed by code with a total, V3’s field names, messages and module strings), so an existing V3 integration keeps working; the /V4.0/ paths answer with V4’s shape (a list of matches, each with its parents). Both are documented separately in /docs (v4-openapi-docs.md §8):

GET /V4.0/na/sic/{description,code,division,industry,major}/{query}            (+ /V3.0/na/sic/...)
GET /V4.0/{eu,international}/sic/{section,division,group,class,description}/{query}   (+ /V3.0/...)
GET /V4.0/japan/sic/{division,major_group,group,industry_group,description}/{query}   (+ /V3.0/japan/sic/...)
GET /V4.0/na/companies/edgar/{ciks,detail,summary}/{company_name}              (+ /V3.0/na/companies/edgar/...)
GET /V4.0/na/company/edgar/firmographics/{cik_no}                              (+ /V3.0/na/company/edgar/firmographics/...)

EU NACE, ISIC and Japan SIC (the 15 per-system lookups) are matched against V3’s real responses: 16 of the 19 captured V3 production responses are identical, and the rest differ only in data (V4’s Japan and NACE files are the corrected ones, and the US division narrative full_description is V3’s text, read from the data file’s section_full_desc column). A no-match is a JSON 404 envelope (V3 answered with an HTML page). Tests: api_tests/ (python3 api_tests/run.py).

V2.0, the limited legacy set (V3’s shorter paths with no regional prefix), answered exactly as the /V3.0/ twin of each:

GET /V2.0/sic/{description,code,division,industry,major}/{query}
GET /V2.0/companies/edgar/{detail,summary,ciks}/{company_name}
GET /V2.0/company/edgar/firmographics/{cik_no}
GET /V2.0/company/wikipedia/firmographics/{company_name}
GET /V2.0/company/merged/firmographics/{company_name}

New, V4-only (§5.1), no /V3.0/ equivalent:

GET /V4.0/na/sic/similarity/{query}?model=all_minilm_l6_v2&k=10
POST /V4.0/global/sic/match          {"text": "..."} or {"chunks": [...]}   (default: US SIC only; "systems" to add more)
POST /V4.0/global/sic/map            {"from": "US SIC", "codes": ["3571"], "to": [...], "description": "..."}

POST /V4.0/global/sic/match is the first POST route: a company description in, a recommended set of 2-5 industry codes per classification system out, each with its hierarchy and the text segment that supports it (docs/plans/company-sic-match.md; the Company Explorer’s Industry Match tab). POST /V4.0/global/sic/map carries a chosen set of codes into the other systems by embedding similarity (no crosswalk tables are used).

Real, built 2026-09-28 (§8.1/§8.2: near-exact company/page title, same as V3; V3’s corporate-suffix hint restored and actually executed as a REST call, not just suggested). Also served at /V3.0/ and /V2.0/, and at the explicit /V3.0/.../v2/... URLs, which V3 documents as the same as the default:

GET /V4.0/global/company/wikipedia/firmographics/{company_name}   (+ /V3.0/..., /V2.0/company/wikipedia/..., /V3.0/.../v2/...)
GET /V4.0/global/company/merged/firmographics/{company_name}      (+ /V3.0/..., /V2.0/company/merged/..., /V3.0/.../v2/...)

Not carried forward: V3’s UK SIC endpoints (/V3.0/uk/...) and its legacy wptools-backed /v1/ Wikipedia and merged variants (V4’s client is a port of V3’s v2 backend specifically; no wptools equivalent exists).

Comparing against V3

python3 ../perf_tests/baseline.py --base-url <v3-url> --profile v3 --out v3.json
python3 ../perf_tests/baseline.py --base-url http://127.0.0.1:4000 --profile v4 --out v4.json
python3 ../perf_tests/compare.py v3.json v4.json

--profile v4 restricts the run to endpoints this server actually implements (perf_tests/baseline.py’s ENDPOINTS catalog, v4_path entries) — see v4-server-prototype.md §7 for why compare.py needed no changes to do this, just the right two reports.

This site is open source. Improve this page.