Instructions to use microsoft/BioGPT-Large-PubMedQA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/BioGPT-Large-PubMedQA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="microsoft/BioGPT-Large-PubMedQA")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("microsoft/BioGPT-Large-PubMedQA")
model = AutoModelForCausalLM.from_pretrained("microsoft/BioGPT-Large-PubMedQA")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use microsoft/BioGPT-Large-PubMedQA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/BioGPT-Large-PubMedQA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/BioGPT-Large-PubMedQA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/microsoft/BioGPT-Large-PubMedQA

SGLang

How to use microsoft/BioGPT-Large-PubMedQA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/BioGPT-Large-PubMedQA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/BioGPT-Large-PubMedQA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/BioGPT-Large-PubMedQA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/BioGPT-Large-PubMedQA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use microsoft/BioGPT-Large-PubMedQA with Docker Model Runner:
```
docker model run hf.co/microsoft/BioGPT-Large-PubMedQA
```

Usefulness?

by tomkr000 - opened Mar 23, 2023

Discussion

tomkr000

Mar 23, 2023

Has anyone found these very useful? Did you do any fine-tuning, and if so, on what/how?

Skier8402

Mar 28, 2023

It is for some questions that I have asked on idiopathic migraines. However, I would really like a link to reference the publications that answers my question. I haven't done fine tuning yet. Still learning about these models still.

Rufus9322

Mar 28, 2023

About the question answering task, I don't know of any real use cases?
When will we use bioGPT Q&A?

Skier8402

Mar 30, 2023

I think this will come in handy for researchers doing literature review about their area of interest for example parasite infection. Start using it now they are so many things you can ask:

incidence of disease in some places
most common ancestry in clinical trials
tell your doctor about it
etc

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment