ChemBERTa is useful to investigate when your starting data are SMILES strings and a labeled molecular endpoint. A SMILES language model learns patterns in a textual representation; it does not acquire experimental validation simply by reading a large corpus. This guide is for researchers preparing a first checkpoint-specific comparison, with Python, molecular parsing and supervised learning experience.

Identify the model before loading it

The catalog’s ChemBERTa resource points to the original bert-loves-chemistry repository. The original ChemBERTa and ChemBERTa-2 paper describe related research, while the later ChemBERTa-3 repository provides a training/benchmarking framework. Their names do not identify one interchangeable set of weights.

For a concrete example, DeepChem/ChemBERTa-77M-MLM has a configuration declaring RobertaForMaskedLM. It is a masked-language checkpoint, not a ready-trained predictor for your solubility or toxicity endpoint. The hub page currently supplies no substantive model card or licence declaration in its API metadata; record this gap before using or redistributing the weights. DeepChem’s code licence cannot fill it. Do not interpret “77M” in a model name as a measured parameter count: the ChemBERTa-2 study varies pretraining corpus size up to 77 million compounds.

Record repository ID, immutable revision, tokenizer files, architecture, pretraining objective, downstream head and training corpus provenance. The DeepChem implementation exposes its own task/configuration interface; a class name in that library is not proof that a hub checkpoint is already fine-tuned.

Make SMILES processing explicit

Preserve the raw string and record ID before parsing. Decide how salts, disconnected components, isotopes, tautomers and stereochemistry are handled. A canonical string is a representation decision, not evidence that two records have identical experimental conditions. Missing labels and conflicting measurements need separate flags.

Use the tokenizer shipped with the selected checkpoint. Character count is not token count. Check token lengths with special tokens included and inspect the actual tokenizer/model limits before choosing a maximum length. Silent truncation can remove a ring closure or other chemically significant text. Group alternative SMILES for the same molecular identity before splitting, and generate randomized SMILES only inside the assigned training partition. This is an editorial evaluation protocol, not a claim that every ChemBERTa implementation applies that augmentation.

Minimal input audit

The following example uses a fixed hub revision retrieved during source review. It downloads configuration/tokenizer assets and parses an illustrative public string; it neither trains a property head nor produces a property value. Code status: not executed. Install separately reviewed RDKit and Transformers versions and preserve the resulting environment lock before running.

from rdkit import Chem
from transformers import AutoConfig, AutoTokenizer

model_id = "DeepChem/ChemBERTa-77M-MLM"
revision = "ed8a5374f2024ec8da53760af91a33fb8f6a15ff"
raw_smiles = "CCO"
mol = Chem.MolFromSmiles(raw_smiles)
if mol is None:
    raise ValueError("SMILES parse failed")
prepared = Chem.MolToSmiles(mol, isomericSmiles=True)
config = AutoConfig.from_pretrained(model_id, revision=revision)
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
encoded = tokenizer(prepared, add_special_tokens=True, truncation=False)
print({"raw": raw_smiles, "prepared": prepared,
       "token_count": len(encoded["input_ids"]),
       "tokenizer_limit": tokenizer.model_max_length,
       "position_table_size": config.max_position_embeddings,
       "architectures": config.architectures})

A successful audit should preserve molecular identity and expose token length without truncation. It does not establish chemical coverage, property accuracy or rights to use the weights. Keep invalid records in a rejection table rather than silently shrinking the dataset.

Freeze splits before fitting a property model

First define the endpoint, units and measurement conditions. Resolve identity duplicates with a documented policy, then save train/validation/test record IDs and their hashes. Choose a random split only if that question matches deployment; scaffold separation probes transfer between structural families, while time-based separation may better reflect prospective collection. The DeepChem splitter source documents random and scaffold mechanisms. Scaffold splitting is not a guarantee of prospective generalization.

Fit label scaling, learned feature transforms, hyperparameters and early stopping using training/validation data only. Preserve missing-label masks in multitask learning. Compare a trained downstream head, optionally fine-tuned encoder, and an appropriate descriptor/fingerprint baseline on exactly the same endpoint and records. A freshly initialized regression head requires training; its numerical output is not meaningful just because loading succeeds.

Investigate pretraining overlap separately from supervised label leakage. Presence in an unlabeled corpus is not automatically label leakage, but unknown overlap limits claims of wholly unseen chemistry. Report what could actually be checked rather than asserting zero contamination.

Evaluate the endpoint and inspect failures

For regression, report MAE or RMSE in the original target scale, including how any transformation was inverted. For imbalanced classification, consider precision–recall alongside ROC metrics and report class counts; some metrics are undefined for a one-class test subset. Fix classification thresholds on validation data. Keep per-record predictions, rejected-input counts, split sizes, seeds and repeated-run variation.

Inspect unusually long strings, rare elements, stereochemical pairs and large residuals. A language-model score or embedding is not a calibrated uncertainty interval. Improvement over a same-split baseline is evidence for that experiment, not a universal ChemBERTa ranking. This article reports no benchmark scores.

Reproducibility and acceptance boundary

Accept an evaluation packet only when raw IDs align with predictions, split membership is immutable, normalization and tokenization are recorded, and downstream weights are identifiable. Keep model/configuration hashes, package versions, hardware, seeds, stopping criteria and asset-use evidence together. A passing input audit, a completed training run and scientific validation are three different outcomes.

The linked general property/QM9 Guides cover broader endpoint design; this article supplies the molecular-text and checkpoint decisions those introductions do not resolve. Proposed automation could package the audit and evaluation records, but no new Recipe or CARS execution integration is supplied here.

Related ChemAI Atlas resources and workflows

ChemBERTa · RDKit · DeepChem

Evaluating Molecular Property Prediction Models · Evaluating Molecular Property Prediction on QM9 · Choosing between Chemprop and DeepChem · Molecular Graph Learning: PyG, e3nn, and DGL-LifeSci

Chemical Data Cleaning Stack

CARS

Sources and evidence status

Sources inspected on 2026-10-18. Primary documentation supports capability/API statements. Audit protocols are editorial proposals; no scientific validation or measured performance is reported.