Yobibyte

Overview#

Most teams that want to use large language models do not want to become an MLOps shop. They want to deploy a model, get an OpenAI-compatible URL, point an application at it, and have something predictable handle GPU capacity, identity, autoscaling, observability, and billing. That problem space is what Yobibyte is built for — a managed consumption surface that turns 'I need a Llama 3.1 70B endpoint in the UK with a $5,500/month spend cap' into a deployed, observable, billed endpoint without anyone on the customer side touching a Kubernetes cluster.

Yobibyte is Yobitel's fully-managed AI-native platform service. Customers consume it as workspaces. Inside a workspace they declare `Inference` resources (deployed model endpoints), `FineTune` resources (managed adapter-training jobs), and `AIApplication` resources (first-party Yobitel applications such as MediQuery deployed into the workspace and configured against the customer's own data). They browse the Yobibyte marketplace for both vetted models and Yobitel AI Applications, federate identity from their existing OIDC provider, set per-resource and per-workspace spend caps, and pull a FOCUS 1.1 billing export into their FinOps tooling. The inference surface is OpenAI-compatible, so the standard `openai` Python and TypeScript SDKs work unchanged against a Yobibyte endpoint — only the base URL changes.

Compared with AWS Bedrock and Google Vertex AI, the operational shape is similar (you consume models as a service, you do not run clusters), but Yobibyte's differentiator is multi-region sovereignty pinning across Yobitel-operated regions. A workspace is bound at creation to UK (NCSC OFFICIAL alignment), EU (EU Data Boundary), or US (FedRAMP-equivalent), and admission refuses to place a workload outside that boundary regardless of capacity or price. Bedrock and Vertex AI are anchored to their parent cloud's region map; Yobibyte runs across Yobitel NeoCloud plus Yobitel-managed presence on partner cloud regions, with the same workspace API across all of them.

Yobitel Communications — a UK-headquartered AI infrastructure company and NVIDIA Inception partner — runs the platform end-to-end. Yobitel operates the runtime, the GPU fleet, the schedulers, the inference engines, and the upgrade cadence. The customer owns the data: models, datasets, fine-tuned adapter weights, and prompt/response logs live in customer-controlled object storage encrypted with customer-managed KMS keys. The customer never installs, upgrades, or operates Yobibyte itself.

Quick start#

The customer experience for getting an OpenAI-compatible endpoint live on Yobibyte has five steps and no Kubernetes. Sign in to the Yobibyte console with your corporate identity provider — OIDC federation is configured once per workspace, and the console will accept the SSO bounce on first login. Create a workspace and choose a sovereignty region at creation time: UK for NCSC OFFICIAL alignment, EU for the EU Data Boundary, or US for FedRAMP-equivalent controls. The region is bound to the workspace and cannot be silently changed afterwards.

Open the marketplace, find the model you want — say Llama 3.1 70B Instruct — and click Deploy. The deploy dialog asks for three things: replicas (minimum and maximum), a spend cap in your currency, and an optional autoscaling profile. Yobibyte fills in the serving defaults from the marketplace recipe attached to the model; you do not pick the inference engine, the tensor-parallel size, the quantisation, or the GPU SKU — those come from the recipe and you can override them later if you want.

Once the endpoint reaches Ready (typically 60–180 seconds for an in-region model, longer for the first deploy of a cold model), copy the endpoint URL and mint an API key in the workspace's Keys tab. The endpoint speaks the OpenAI Chat Completions and Embeddings API, so the standard `openai` SDK works against it unchanged — only `base_url` and `api_key` differ.

# Install the standard OpenAI SDK
pip install openai

# Point it at your Yobibyte workspace endpoint
export OPENAI_API_KEY="ybt_live_..."           # from the workspace Keys tab
export OPENAI_BASE_URL="https://acme.yobitel.app/v1"

python - <<'PY'
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="llama-3.1-70b-instruct",
    messages=[{"role": "user", "content": "Hello from Yobibyte"}],
)
print(resp.choices[0].message.content)
PY

Concepts#

Yobibyte exposes a small set of concepts that map to how customers think about AI workloads. The mental model is workspace at the top, with `Inference`, `FineTune`, and `AIApplication` resources and marketplace items inside it, and identity, sovereignty, and spend governing what is allowed to happen inside the workspace boundary.

• Workspace — the tenant boundary, the billing root, the OIDC anchor, and the sovereignty region binding. Every resource belongs to exactly one workspace; quotas, spend caps, audit logs, and the FOCUS billing export are scoped to it.
• Inference — a deployed model endpoint. The customer supplies model name, region, replica range, spend cap, and an autoscaling profile; Yobibyte selects the inference engine, GPU SKU, and serving flags from the marketplace recipe. The endpoint speaks OpenAI-compatible HTTP.
• FineTune — a managed adapter-training job. The customer supplies a base model, a dataset URI (s3://, azure://, or gs://), a fine-tune method (QLoRA / LoRA / full SFT), a hyperparameter preset or a small set of high-level knobs, and an output bucket with a KMS key. Yobibyte runs the job and writes the adapter back to the customer's bucket.
• AI Application — a Yobitel-published first-party application (MediQuery for clinical workflows, and the rest of the vertical AI Applications suite) deployed into a workspace. The customer configures the application — data sources, knowledge bases, user roles, branding — rather than the underlying models. The application composes `Inference` and `FineTune` resources internally; Yobibyte runs them on the customer's behalf, and the customer's interface is the application's configuration surface, not the model surface. Partner-published applications appear in the marketplace alongside Yobitel's own.
• Marketplace — a curated catalogue of two things: models and AI Applications. Model entries carry sovereignty, licence, and hardware-compatibility metadata, ranked using InferenceBench data, and ship with the serving defaults Yobibyte uses when the model is deployed. AI Application entries carry sovereignty, licence, industry/vertical, and integration metadata, with first-party Yobitel apps featured alongside partner-published apps. Customer-private models and customer-private application packages live alongside the curated catalogue but are scoped to the workspace they were uploaded to.
• Sovereignty Region — the compliance pin chosen at workspace creation. UK regions align to NCSC Cloud Security Principles and the OFFICIAL classification; EU regions sit inside the EU Data Boundary; US regions align to FedRAMP-equivalent controls. Admission refuses to place workloads outside the workspace's region.
• Spend Cap — a hard budget set per Inference and per Workspace. Yobibyte tracks accrued cost against the FOCUS-shaped cost stream and enforces at hourly granularity; when the cap is reached, replicas pause (they are not deleted) and a P2 alert fires on the configured channel.
• Identity — workspaces federate from any OIDC-compliant IdP (Okta, Microsoft Entra ID, Auth0, Keycloak, Google Workspace tested). SCIM 2.0 provisions users and groups. Permissions follow owner / maintainer / viewer plus a fine-grained resource-level set (`inference:deploy`, `finetune:read`, `workspace:billing:read`).

Inference resource reference#

Customers who prefer a declarative workflow can submit `Inference` resources directly via the Yobibyte API or commit them to Git. The shape below is the full customer-facing surface — everything Yobibyte needs to deploy and operate an endpoint on the customer's behalf. Engine selection, GPU SKU selection, and serving flags are NOT customer fields; they come from the marketplace recipe attached to the model name. Customers who need to override a recipe value can do so via the `overrides` block, but the defaults are correct for the vast majority of workloads.

apiVersion: yobibyte.yobitel.com/v1
kind: Inference
metadata:
  name: support-bot
  workspace: acme-uk
spec:
  model: llama-3.1-70b-instruct        # marketplace model name
  region: uk-london                     # must match the workspace region
  replicas:
    min: 1
    max: 6
  autoscaling:
    scaleMetric: tokensPerSecond        # tokensPerSecond | requestsPerSecond | concurrency
    target: 1800
  spendCap:
    amount: 5500
    currency: USD
    window: monthly
  accessControl:
    apiKeys: [ "support-bot-prod" ]
    allowedOriginCidrs: [ "10.0.0.0/8" ]
  observability:
    otelEndpoint: "https://otel.acme.com:4317"
    prometheusScrape: true

FineTune resource reference#

`FineTune` is the managed adapter-training surface. The customer points at a dataset and a base model, picks a method and a hyperparameter preset, and supplies an output bucket and KMS key — Yobibyte runs the job, ships metrics, and writes the resulting adapter back to the customer's bucket. The customer never picks the training framework or schedules nodes manually.

apiVersion: yobibyte.yobitel.com/v1
kind: FineTune
metadata:
  name: support-llama-v3
  workspace: acme-uk
spec:
  baseModel: llama-3.1-70b-instruct
  region: uk-london
  dataset:
    uri: s3://acme-ml/support-tickets-2025-q4.jsonl
    format: chat
  method: qlora                          # qlora | lora | full
  hyperparameters:
    preset: balanced                     # fast | balanced | quality
    epochs: 3
    learningRatePreset: standard
    rank: 64
  output:
    bucket: s3://acme-ml/adapters/support-v3
    kmsKey: arn:aws:kms:eu-west-2:111122223333:key/abcd-...
  spendCap:
    amount: 1500
    currency: USD

Marketplace#

The marketplace is where customers find what they can deploy into a workspace, and it covers two distinct catalogues that share a common shopfront: open-weight models, and Yobitel AI Applications. The same filters (sovereignty, licence, region, popularity, last-updated) apply to both; the result detail page differs in shape according to what is being deployed.

On the model side, every entry carries sovereignty metadata (which regions can host it), licence metadata (commercial use, redistribution, attribution), hardware-compatibility metadata (which accelerator families it fits), and an InferenceBench-sourced ranking on price per million tokens and tokens per second at representative input/output shapes. Filters narrow by sovereignty (only models that can be hosted in the workspace region), licence (only models cleared for commercial use), family (Llama, Qwen, Mistral, Phi, Gemma, etc.), and modality (text, embeddings, vision, multimodal, speech). The result list shows the InferenceBench rank, the per-million-token price band at default replicas, and the smallest GPU configuration the model fits on.

On the AI Application side, every entry carries sovereignty and residency metadata, licence and commercial terms, industry/vertical tags (clinical, financial-services, manufacturing, retail, telco, public-sector, etc.), integration metadata (which data sources, identity systems, and downstream tools the application natively connects to), and a deployment footprint (which Inferences and FineTunes the application provisions on deploy, and the spend-cap implications). Yobitel-first-party applications such as MediQuery are featured alongside partner-published applications; the deploy dialog asks the customer to configure the data sources, knowledge bases, RBAC, and branding rather than to pick models or serving flags.

Customer-private models and customer-private application packages live in the marketplace alongside the curated catalogues but are scoped to the workspace they were uploaded to. Bring-your-own model uploads go directly to a customer-owned object bucket; Yobibyte registers the entry and stamps the recipe metadata it needs to serve the model, but the weights stay in the customer's storage. Customer-built applications follow the same pattern: package + manifest land in the workspace, Yobibyte registers them, and the runtime is operated on the customer's behalf.

Customers do not interact with the internal recipe shape on either side. For models, the marketplace supplies sensible serving defaults that work for the listed model on the listed accelerators; customers can override individual fields (max context length, replica concurrency, sampling defaults) via the `overrides` block on the `Inference` resource, but engine selection and core serving flags are Yobibyte's responsibility. For applications, customers configure the application's surface (data, identity, RBAC, branding) and the Inferences and FineTunes the application provisions are operated by Yobibyte.

Sizing and capacity planning#

The table below summarises baseline sizing for the workload shapes most teams hit first. Treat figures as starting points; actual throughput varies with prompt length, output length, batch size, and quantisation. Validate against InferenceBench numbers for the exact model/GPU combination.

Sizing: workspace footprint by team size#

Workspaces map one-to-one with business units or product teams; sovereignty regions map to a Yobitel-operated region and isolation tier. The recommended ratios below come from production deployments at 10/50/200-engineer scale.

Limits and quotas#

Default workspace quotas exist to protect shared infrastructure during onboarding; almost every limit is raisable on request. Hard ceilings exist where the underlying primitive imposes one (e.g. Kubernetes ETCD object size limits).

Observability#

Yobibyte emits three telemetry streams for every customer workload. GPU metrics come from the DCGM exporter (an NVIDIA open standard) and cover SM occupancy, HBM usage, power draw, and NVLink throughput. Inference engine metrics come from the managed inference runtime — Yobibyte normalises them under a stable `yobibyte_*` metric namespace so customers do not have to chase engine-specific metric names across upgrades. OpenTelemetry traces cover the full request lifecycle from the workspace gateway through to the response.

All three streams are scrape-compatible with customer Prometheus and customer OTel collectors. Customers can scrape directly from the workspace, ship to their own OTel collector via the `observability.otelEndpoint` field on the `Inference` resource, or view the hosted Grafana with curated dashboards. The metric names are stable across engine upgrades — the inference runtime can change underneath without changing the customer-facing metric surface.

The PromQL snippet below is the alert most production deployments add first — it catches the 'replica is up, traffic is flowing, but tokens/sec has collapsed' failure mode that simple liveness probes miss.

groups:
- name: yobibyte-inference
  interval: 30s
  rules:
  - alert: InferenceTokensPerSecondDegraded
    expr: |
      avg by (workspace, inference) (
        yobibyte_inference_tokens_per_second_replica
      ) < 1000
      and
      sum by (workspace, inference) (
        rate(yobibyte_inference_requests_total[5m])
      ) > 0.5
    for: 10m
    labels: { severity: page }
    annotations:
      summary: "{{ $labels.inference }} tokens/sec/replica below 1k"
      runbook: https://docs.yobitel.com/runbooks/inference-throughput

  - alert: GpuHbmExhaustion
    expr: yobibyte_gpu_hbm_used_ratio > 0.97
    for: 5m
    labels: { severity: page }

Cost and FinOps#

Billing is per-resource and per-second for GPUs, per-GB-month for storage, and per-million-tokens for marketplace-hosted models. Every line item carries the FOCUS 1.1 columns (BilledCost, EffectiveCost, ListCost, ChargePeriod*, ServiceCategory, SubAccountId, Tags), so the export drops directly into a Cloudability/Apptio/Vantage pipeline or a customer-built BigQuery or Snowflake lakehouse.

Reserved and committed-use pricing is available for steady workloads; spot capacity is available at material discount for fault-tolerant fine-tunes and batch scoring. The table below is indicative for UK regions in mid-2026 — always re-validate against the live Omniscient Compute price feed before forecasting.

Security and compliance#

Identity federates from any OIDC-compliant IdP; tested integrations include Okta, Microsoft Entra ID, Auth0, Keycloak, and Google Workspace. SCIM 2.0 is supported for user and group provisioning. RBAC follows the standard owner/maintainer/viewer triple plus a finer-grained resource-level permission set (e.g. `inference:deploy`, `finetune:read`, `workspace:billing:read`).

Tenant isolation is a customer choice exposed per `Inference`. `shared` runs workloads on multi-tenant nodes with cgroup and MIG isolation — the right default for most workloads. `dedicated` pins the workload to dedicated nodes and is billed per node-hour rather than per pod-hour; choose this when a regulatory profile requires single-tenant compute or when a workload's noisy-neighbour sensitivity justifies the price. `confidential` runs the workload on NVIDIA H100/H200 confidential-compute mode with TEE attestation, so encryption keys never leave the GPU; choose this for the most sensitive data classes.

Data residency is enforced at admission via policy-based admission. A workload labelled `compliance=ncsc-official` is rejected if it targets a non-UK region; a workload labelled `compliance=eu-data-boundary` is rejected if it targets outside the EU; a workload labelled `compliance=fedramp-equiv` is rejected if it targets outside the US partner regions. Yobibyte will not silently spill workloads across compliance boundaries regardless of capacity or price.

• NCSC Cloud Security Principles — controls mapped per principle; OFFICIAL-tier UK regions audited annually.
• G-Cloud framework — listed under Cloud Software (Lot 2) and Cloud Support (Lot 3).
• SOC 2 Type II — annual third-party audit covering security, availability, confidentiality.
• ISO 27001:2022 and ISO 27017/27018 — current certificates available under NDA.
• GDPR / UK DPA 2018 — DPA, sub-processor list, and EU SCCs available; data residency enforced at admission.
• FedRAMP-equivalent — Moderate-baseline-aligned controls available via partner regions in the US.
• HIPAA — BAA available for healthcare workloads; encryption, logging, and access-control controls audited.
• CSA STAR Level 2 — published self-assessment plus third-party attestation.

Migration and alternatives#

Yobibyte is one option among several. Use the comparison table to decide when each fits.

Troubleshooting#

The errors below cover the failure modes seen most often during onboarding and the first weeks of production. The full runbook library is at docs.yobitel.com/runbooks.

Where Yobibyte fits in the Yobitel stack#

Yobitel has three principal platforms and one applications suite, and they sit in a clear stack. Omniscient Compute is the layer below Yobibyte — the vendor-neutral search and orchestration index that knows every SKU, every region, every price, every compliance tag. Yobibyte calls Omniscient at admission and reconciliation time so a workload's placement is always grounded in current capacity and pricing.

InferenceBench is the benchmark and economics layer that sits alongside Yobibyte rather than below it. The marketplace pulls model/GPU/provider scoring directly from InferenceBench, so the recipe attached to `meta-llama/Llama-3.1-70B-Instruct` is the same public ranking anyone can verify at inferencebench.io. Yobitel AI Applications — MediQuery and the rest of the vertical suite — sit on top of Yobibyte; they are first-party applications that use Workspace, Inference, and FineTune the same way any customer would.

Practically, a customer can adopt the stack at any layer. A FinOps team can use Omniscient alone to broker capacity. A platform team can adopt Yobibyte and let it consume Omniscient internally. An application team can consume an AI Application without ever seeing the layers beneath. The boundaries are deliberate, the APIs are stable, and the contracts are documented at each layer.