Skip to main content
Rase AI Intelligence Newsletter

AI infrastructure Product

KTransformers uses CPUs and GPUs together for model inference

KTransformers is a research framework for running large language models across CPUs and GPUs. [1]

What it does

For supported mixture-of-experts models, the project can assign different experts to CPU or GPU resources. [1]

Why it matters

This gives teams another hardware configuration to evaluate when a model’s memory needs and the available GPU capacity differ. [1]

Key dates

  1. Provisional KTransformers 0.0.1 package uploaded [2]
  2. Provisional KTransformers 0.7.2 package uploaded [3]

Sources and verification

Catalog evidence checked How we verify records

  1. Primary source ktransformers
  2. Primary source ktransformers 0.0.1, Python Package Index
  3. Primary source ktransformers 0.7.2, Python Package Index

Related profiles

AI infrastructure

Azure Cobalt 200 virtual machines

Microsoft Azure

The Arm-based CPU platform supports databases, applications and tool execution around AI models.

High confidenceEvidence checked

AI infrastructure

Sarvam Token Factory

Sarvam AI

It exposes hosted model inference through supported developer interfaces.

High confidenceEvidence checked

AI infrastructure

NVIDIA Rubin HPC

NVIDIA

It is designed to accelerate high-performance computing workloads in compatible systems.

High confidenceEvidence checked