Self-Distillation of Hidden Layers for Self-Supervised Representation Learning

Export BriefOpen in Build LoopConnect with Author
View PDF ↗
PDF Viewer
Open Full PDF

BUILDER'S SANDBOX

Build This Paper

Use an AI coding agent to implement this research.

OpenAI Codex
OpenAI CodexAI Agent

Lightweight coding agent in your terminal.

Claude Code
Claude CodeAI Agent

Agentic coding tool for terminal workflows.

AntiGravity IDE
AntiGravity IDEScaffolding

AI agent mindset installer and workflow scaffolder.

Cursor
CursorIDE

AI-first code editor built on VS Code.

VS Code
VS CodeIDE

Free, open-source editor by Microsoft.

Estimated $9K - $13K over 6-10 weeks.

See exactly what it costs to build this -- with 3 comparable funded startups.

7-day free trial. Cancel anytime.

Discover the researchers behind this paper and find similar experts.

7-day free trial. Cancel anytime.

References (80)

[1]
VL-JEPA: Joint Embedding Predictive Architecture for Vision-language
2025Delong Chen, Mustafa Shukor et al.
[2]
WavJEPA: Semantic learning unlocks robust audio foundation models for raw waveforms
2025Goksenin Yuksel, Pierre Guetschel et al.
[3]
Disentangling the Factors of Convergence between Brains and Computer Vision Models
2025Jos'ephine Raugel, Marc Szafraniec et al.
[4]
DINOv3
2025Oriane Sim'eoni, Huy V. Vo et al.
[5]
Many-Two-One: Diverse Representations Across Visual Pathways Emerge from A Single Objective
2025Yingtian Tang, Abdulkadir Gokce et al.
[6]
Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning
2025S. Venkataramanan, Valentinos Pariza et al.
[7]
Audio-JEPA: Joint-Embedding Predictive Architecture for Audio Representation Learning
2025Ludovic Tuncay, Etienne Labb'e et al.
[8]
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
2025Mahmoud Assran, Adrien Bardes et al.
[9]
Perception Encoder: The best visual embeddings are not at the output of the network
2025Daniel Bolya, Po-Yao Huang et al.
[10]
Enhancing DNA Foundation Models to Address Masking Inefficiencies
2025Monireh Safari, Pablo Millán Arias et al.
[11]
Cluster and Predict Latents Patches for Improved Masked Image Modeling
2025Timothée Darcet, Federico Baldassarre et al.
[12]
Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning
2024Shentong Mo, Shengbang Tong
[13]
T-JEPA: Augmentation-Free Self-Supervised Learning for Tabular Data
2024Hugo Thimonier, José L. Costa et al.
[14]
Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking
2024Zijian Dong, Ruilin Li et al.
[15]
M3-JEPA: Multimodal Alignment via Multi-gate MoE based on the Joint-Embedding Predictive Architecture
2024Hongyang Lei, Xiaolong Cheng et al.
[16]
Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation
2024Alain Riou, Stefan Lattner et al.
[17]
ColorMAE: Exploring data-independent masking strategies in Masked AutoEncoders
2024Carlos Hinojosa, Shuming Liu et al.
[18]
Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects
2024Michael A. Lepori, Alexa R. Tartaglini et al.
[19]
Self-supervised predictive learning accounts for cortical layer-specificity
2024Kevin Kermani Nejad, Paul Anastasiades et al.
[20]
Revisiting Feature Prediction for Learning Visual Representations from Video
2024Adrien Bardes, Q. Garrido et al.

Showing 20 of 80 references

Founder's Pitch

"Bootleg is a self-supervised learning method that enhances feature extraction by predicting latent representations from multiple hidden layers."

Self-Supervised LearningScore: 4View PDF ↗

Commercial Viability Breakdown

0-10 scale

High Potential

2/4 signals

5

Quick Build

0/4 signals

0

Series A Potential

0/4 signals

0

Sources used for this analysis

arXiv Paper

Full-text PDF analysis of the research paper

GitHub Repository

Code availability, stars, and contributor activity

Citation Network

Semantic Scholar citations and co-citation patterns

Community Predictions

Crowd-sourced unicorn probability assessments

Analysis model: GPT-4o · Last scored: 3/16/2026

Explore the full citation network and related research.

7-day free trial. Cancel anytime.

Understand the commercial significance and market impact.

7-day free trial. Cancel anytime.

Get detailed profiles of the research team.

7-day free trial. Cancel anytime.

Related Papers

Loading…

Related Resources