TissueGPT

TissueGPT is a language model for tissue-engineering text. It is a fine-tuned version of Microsoft’s BioGPT, trained further on research articles (titles, abstracts and full texts) from the tissue-engineering literature.

What it is for

• Summarising biomedical and tissue-engineering papers
• Generating domain text, for example when drafting a background section
• Supporting research on scaffolds, the extracellular matrix (ECM) and related topics

It is a research tool, not for clinical use. Like any language model it can produce fluent but wrong statements, so its output should always be checked by an expert.

Training

Two rounds of fine-tuning with Hugging Face Transformers on an NVIDIA A100 80GB GPU: 3 epochs at a learning rate of 5e-5 (fp16), then 2 more at 3e-5 (bf16). Validation perplexity fell from 10.39 after the first round to 9.32 after the second.

Try it

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Saeed/TissueGPT")
model = AutoModelForCausalLM.from_pretrained("Saeed/TissueGPT")

inputs = tokenizer("The extracellular matrix plays a critical role in tissue engineering because", return_tensors="pt")
output = model.generate(**inputs, max_length=50)
print(tokenizer.decode(output[0], skip_special_tokens=True))

The model and its training dataset are on Hugging Face under CC BY 4.0, with the citable DOI 10.57967/hf/8917.