Learning Efficient Convolutional Networks through Network Slimming, In ICCV 2017.
-
Updated
May 13, 2019 - Python
#
fast-inference
Here are 32 public repositories matching this topic...
Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.
-
Updated
Oct 12, 2024 - Python
JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Causal Parallel Tree Drafting
-
Updated
Aug 9, 2026 - Python
[ICML 2026] d3LLM: Ultra-Fast Diffusion LLM 🚀
inference diffusion efficient-algorithm post-training fast-inference diffusion-models large-language-models llm text-diffusion diffusion-language-models dllm
-
Updated
May 1, 2026 - Python
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
-
Updated
Dec 2, 2024 - Python
[NeurIPS'23] Speculative Decoding with Big Little Decoder
-
Updated
Feb 6, 2024 - Python
🔥 Blazingly fast ML inference server powered by Rust and Burn framework
rust high-performance deep http-server production-ready burn machine-le fast-inference ml-serving inference-ser
-
Updated
Jul 25, 2025 - Rust
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
-
Updated
Jul 16, 2024 - Python
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
attention-mechanism fast-inference sparse-neural-networks low-rank vision-transformer efficient-transformers sparse-attention efficient-vision-transformers
-
Updated
Jul 4, 2023 - Python
AudioMuse-AI-DCLAP is a lightweight, high-speed distilled version of LAION CLAP, designed for fast and efficient text-to-music search
python deep-learning pytorch music-search music-information-retrieval clap distillation onnx fast-inference audio-embedding edge-ai huggingface cross-modal-retrieval transformer-models text-to-audio model-distillation audio-retrieval lightweight-model multimodal-ai sonic-analysis
-
Updated
Jun 17, 2026 - Python
AI-powered legal assistant for Brazilian lawyers, built with Groq to deliver fast, accurate insights and document support.
-
Updated
Sep 17, 2025 - TypeScript
An implementation of the encoder-decoder transformer for SMILES-to-SMILES translation tasks with inference accelerated by speculative decoding
-
Updated
Jul 19, 2025 - Python
Fast Forward-Only Deep Neural Network Library for the Nao Robots
-
Updated
Jun 6, 2019 - C++
Claude Code on Taalas's silicon-baked Llama 3.1 8B (~14,500 tok/s, ~1,000x faster than Opus). A tiny Anthropic-API proxy with tool translation, best-of-N reliability, and a safety guard. 🦞🏎️
-
Updated
Jun 15, 2026 - TypeScript
Verification of the effect of speculative decoding in Japanese.
-
Updated
Mar 4, 2024 - Python
A self-contained CUDA inference engine for LiquidAI/LFM2.5-8B-A1B (hybrid conv + GQA-attention MoE, 8.5B params, 1B active) targeting a single RTX 3060 (12 GB). No Python, no frameworks at runtime: a single .cu engine + a header-only byte-level BPE tokenizer.
c open-source ai cpp gpu cuda inference nvidia moe token rtx inference-engine fast-inference 3060 large-language-models llm liquidai lfm25
-
Updated
Jun 7, 2026 - Cuda
Reproducibility Project for [NeurIPS'23] Speculative Decoding with Big Little Decoder
-
Updated
May 30, 2024 - Python
Mamba-3 in a single file of pure PyTorch. Official mamba_ssm API, nothing to build, runs wherever PyTorch runs.
deep-learning pytorch no-dependencies ssm mamba single-file state-space-models fast-inference sequence-modeling linear-attention long-context mamba-3
-
Updated
Aug 30, 2026 - Python
AudioMuse-AI-SAE implement the Sparse autoencoders (SAE) for DCLAP
python deep-learning music-search music-information-retrieval clap onnx fast-inference audio-embedding edge-ai cross-modal-retrieval text-to-audio audio-retrieval sparse-steer-retrieval
-
Updated
Aug 28, 2026 - Python
Multilable fast inference classifiers (Ridge Regression and MLP) for NLPs with Sentence Embedder, K-Fold, Bootstrap and Boosting. NOTE: since the MLP (fully connected NN) Classifier was too heavy to be loaded, you can just compile it with the script.
python nlp bootstrap nn transformers python3 classification ridge-regression learning-curve boosting classifiers mlp-classifier mlp-networks fast-inference ridge-regression-model ridge-classifier embedders classifiers-comparison
-
Updated
May 5, 2025 - Python
Add this topic to your repo
To associate your repository with the fast-inference topic, visit your repo's landing page and select "manage topics."