pretraining

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Llama中文社区，实时汇总最新Llama学习资料，构建最好的中文Llama大模型开源生态，完全开源可商用

General technology for enabling AI capabilities w/ LLMs and MLLMs

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and latent space reasoning.

Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).

[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation

Official Repository for the Uni-Mol Series Methods

Recent Advances in Vision and Language PreTrained Models (VL-PTMs)

EntitySeg Toolbox: Towards Open-World and High-Quality Image Segmentation

X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image captioning, video captioning, vision-language pre-training, visual question answering, visual commonsense reasoning, and cross-modal retrieval).

Pretraining and inference code for a large-scale depth-recurrent language model

【ICLR 2024🔥】 Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites

Official repository for "OpenWAM: An Open, Modular Exploration Towards Systematic World–Action Model Pretraining".

Official Pytorch Implementation of: "ImageNet-21K Pretraining for the Masses"(NeurIPS, 2021) paper

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.