Issues

• Verify evals on Papers with Code

  AnswerDotAI/ModernBERT#254 · Filter by author NielsRogge opened on Sep 14, 2026

• missing file yamls/main/modernbert-base.yaml

  AnswerDotAI/ModernBERT#251 · Filter by author fengzi456258 opened on Dec 3, 2025

• StopIteration error during training with modernbert-large-context-extension.yaml configuration

  AnswerDotAI/ModernBERT#249 · Filter by author mmichall opened on Oct 23, 2025

• Resuming takes too much time with load_path activated

  AnswerDotAI/ModernBERT#246 · Filter by author dtamayo-nlp opened on Oct 8, 2025
  6

• Continue pretraining on my dataset

  AnswerDotAI/ModernBERT#245 · Filter by author Keramatfar opened on Aug 23, 2025
  1

• First layer residual is inconsistently normalized

  AnswerDotAI/ModernBERT#243 · Filter by author lgienapp opened on Aug 16, 2025

• Question about masking - why mask on unpadded sequence?

  AnswerDotAI/ModernBERT#242 · Filter by author yjoonjang opened on Aug 14, 2025
  1

• _pad_token attribute?

  AnswerDotAI/ModernBERT#241 · Filter by author ceferisbarov opened on Aug 11, 2025
  2

• Usage of GTE-ModernColBERT or ModernBERT embedding model for storing file chunks and vector embeddings in a vector store

  AnswerDotAI/ModernBERT#240 · Filter by author ampulsora opened on Aug 2, 2025
  4

• ValueError: Invalid pattern: '**' can only be an entire path component

  AnswerDotAI/ModernBERT#237 · Filter by author amrzv opened on Jun 8, 2025

• question to implementation for ModernBertForMaskedLM vs ModernBertForSequenceClassification

  AnswerDotAI/ModernBERT#235 · Filter by author geraldstanje opened on May 22, 2025

• Pretraining with FlexBert but can't export to HF

  AnswerDotAI/ModernBERT#234 · Filter by author asafam opened on May 21, 2025
  2

• grad_norm Nan question

  AnswerDotAI/ModernBERT#233 · Filter by author 937739823 opened on May 21, 2025
  1

• Pretraining Throughput — Unexpected Drop in Speed

  AnswerDotAI/ModernBERT#228 · Filter by author ebrarkiziloglu opened on Apr 30, 2025
  2

• Training Tokenizer and pretraining

  AnswerDotAI/ModernBERT#226 · Filter by author Thiago-Reis-Porto opened on Apr 17, 2025
  1

• How to continue to train ModernBert base/large in a specific domain?

  AnswerDotAI/ModernBERT#222 · Filter by author BernieVA opened on Apr 2, 2025
  2

• Multi-GPU Pretraining is slower than single GPU training?

  AnswerDotAI/ModernBERT#220 · Filter by author leannmlindsey opened on Mar 31, 2025
  5

• Is there a config file for ModernBERT-Large?

  AnswerDotAI/ModernBERT#217 · Filter by author leannmlindsey opened on Mar 24, 2025
  1

• NaN rotaty_emb weights in exported pretrained models

  AnswerDotAI/ModernBERT#215 · Filter by author ahxxm opened on Mar 20, 2025
  5

• Special tokens masking candidates

  AnswerDotAI/ModernBERT#212 · Filter by author jihobak opened on Mar 19, 2025
  1

• HuggingFace loss implementation

  AnswerDotAI/ModernBERT#206 · Filter by author megatron6000 opened on Feb 26, 2025

• IndexError: tuple index out of range in input_ids during pre-training

  AnswerDotAI/ModernBERT#199 · Filter by author ebrarkiziloglu opened on Feb 13, 2025
  3

• BackendCompilerFailed

  AnswerDotAI/ModernBERT#198 · Filter by author farrokhsiar opened on Feb 12, 2025
  4

• Question about MLDR Evaluation Metrics in ModernBERT Paper

  AnswerDotAI/ModernBERT#193 · Filter by author WoutDeRijck opened on Jan 31, 2025
  1

  

• Can't find yamls/main/modernbert-base.yaml in repo

  AnswerDotAI/ModernBERT#192 · Filter by author KennethEnevoldsen opened on Jan 28, 2025
  2