Brev.dev ? NVIDIA NGC ????? ???? ? ?? ???? GPU ??? AI ????? ??

Reading Time: 4 minutes

Brev.dev? NVIDIA NGC ????? ????? ?????, ????? ? Jupyter Notebooks? ???? AI ??? ??? ? ?? ??? ????.

Brev.dev? ???? ???? ??????? ??? ??? UI? ?? ???? NVIDIA GPU? ?????? ?? ??? ? ????. ?? ??????? ????? ???? GPU? ?????, Brev CLI? ?? ???? ??? ???? ??????, ???? ?? SSH? ??? ? ????.

NGC ????? ????, ??, Jupyter Notebooks ? ??? GPU ??? ?????? ?? ?????. ? ????? AI ??? ?? ? ?? ????? ????? ??????? ?? ?? ??? ?????.

??? ?????

? ??? ?????? GPU ????? ???? ?? ?? ??(IDE)? ??? ? ?? ????, ?? ????? ?? ??, ??? ??? ?????, SSH ? ?? ?? ??? ??? ?????. ?? ?? ? ??? NGC ????? ??? ?????? ???? ??? ? IDE? ???? ??? ? ????.

? ???? ?? ??? ??? ????.

??? ??: ????? ?? ??? ?? ???? ?? ?? ??? NVIDIA AI ?????? ??? ??? ? ????. ?? ?? ?? ?? ?? ?????? ?? ??? ?????, ? ?? ???? 2~3? ?? ??? ? ?? ???.??
???? ?? ??: Brev? API? ????? ?????, ??? ???? ????, ????? ? ???? ???? ??? ?? ?? ????? ??? ???. ?? ?? ?? ????? NVIDIA ?????? ?? ??? ? ????. Brev? ????? ???? ?? ??? ??? ?? ??? ? ?? ?? ??? ?????.
???? ?? ????: Brev? ?? ?? ???? ??? Verb? ?? GPU? CUDA ? Python? ????? ???? ??? ??? ???? ??? ??? ??? ???.
??? ????: Brev? CLI ??? ???? SSH ?? ??? ???? ??? IP ?? PEM ??? ???? ??? ?? ??? ??? ???? ??? ? ????.

Mistral Jupyter Notebook ?? ??

? ????? ?? ?? ??(LLM) ?? ??? ??? ?? Jupyter Notebook? ??? ???????. ?? NVIDIA NeMo? ???? ??-?? ??? ?? Misral 7B? ???? ???? ????? ??? ?????.

NVIDIA NeMo? ???? ???? ??? AI? ??? ? ?? ?? ? ?? ??????. ???? ??? ????, ??, ?? ?? ??(RAG), ????? ?? ??? ???? ?? ??? ?? ?? ??? AI? ?? ?? ????? ??? ??? ? ?? ???.

Brev? ??? ?? ??? ???? NeMo? ?? GPU? ??? ????? ??? AI ??? ???? ? ????. NeMo? ??? ?? ????? ??? Brev? ???? ?? ????? ???? AI ???? ??? ? ????. ?? ??? ??? ?? ????? ??? ?? ?? ??? GPU ???? ??? ??? ??? ? ????.

1??: ?? ?? ??

?? NGC ?????? ???? ???????. Brev? ???? ?? ?????? ????? ??? ?? ??? ??? ? ????.

NGC ???? ????? ??? ?? ??? ?????. Brev?? ???? ?? ????? ??? ?????(?? 1).

*?? 1. NGC ????? Mistral Jupyter Notebook*

?? ?? Deploy Notebook ??? ?????(?? 2). Brev? ?? ???? ?? ??? ????? ???? ?????. ? ?? Deploy Notebook ??? ?? ???? ???.

????? ?? ? ??? ?? ??? ?????. JupyterLab? NVIDIA A100 Tensor Core GPU?? ???? Access Notebook? ?????.

2??: ?? ?? ??

????, Mistral 7B? ????? ?????? .nemo ???? ?????. ?? ??? ???? Hugging Face?? ??? ???? ?? ????? ?????.

!pip install ipywidgets !jupyter nbextension enable --py widgetsnbextension !mkdir -p models/mistral7b

import huggingface_hub from huggingface_hub import login TOKEN = "" login(TOKEN)

huggingface_hub.snapshot_download(repo_id="mistralai/Mistral-7B-v0.1", 
local_dir="models/mistral7b", local_dir_use_symlinks=False) !python
/opt/NeMo/scripts/nlp_language_modeling/convert_hf_mistral_7b_to_nemo.py 
--in-file=models/mistral7b --out-file=models/mistral7b.nemo

? ??? NeMo? nlp_language_modeling ?? ????? ???? Mistral ??? .nemo ???? ????? NeMo ?????? ?? ??? ??? ? ????.

3??: ?? ?? ??? ??

? ????? PubMedQA ?????? Mistral 7B? ?? ?????. PubMedQA? ??? ?? ?? ??? ?/???/??? ???? ???? ????. PubMedQA?? ??? ???? ??? 1K, ???? ???? ?? 61.2K, AI? ?? ??? 211.3K?? QA ???? ? ????.

?? ??? NeMo? ?? ???? ????? ?? ??(PEFT)? ?? ?? PubMedQA ???? .jsonl ???? ?????. ?? ??? ???? ??? ? ?? ????? ???? ??? ?? ???? ???.

!git clone https://github.com/pubmedqa/pubmedqa.git !cd pubmedqa/preprocess && python split_dataset.py pqal

import json
 
def write_jsonl(fname, json_objs):
    with open(fname, 'wt') as f:
        for o in json_objs:
            f.write(json.dumps(o)+"\n")
 
def form_question(obj):
    st = ""
    st += f"QUESTION:{obj['QUESTION']}\n"
    st += "CONTEXT: "
    for i, label in enumerate(obj['LABELS']):
        st += f"{obj['CONTEXTS'][i]}\n"
    st += f"TARGET: the answer to the question given the context is (yes|no|maybe): "
    return st
 
def convert_to_jsonl(data_path, output_path):
    data = json.load(open(data_path, 'rt'))
    json_objs = []
    for k in data.keys():
        obj = data[k]
        prompt = form_question(obj)
        completion = obj['reasoning_required_pred']
        json_objs.append({"input": prompt, "output": completion})
    write_jsonl(output_path, json_objs)
    return json_objs

test_json_objs = convert_to_jsonl("pubmedqa/data/test_set.json", "pubmedqa_test.jsonl")
train_json_objs = convert_to_jsonl("pubmedqa/data/pqal_fold0/train_set.json", "pubmedqa_train.jsonl")
dev_json_objs = convert_to_jsonl("pubmedqa/data/pqal_fold0/dev_set.json", "pubmedqa_val.jsonl")

?? ???? ???????. ???? ? ??? ?????? ?? ?? ??? ?? ?????.

4??: ???? ??

GPU ??, ???, ?? ?? ?? ??? ? NeMo ?????? ???? ???? ?????? ??????. ?? ??? ??? ???? ??? ????, ???? ????? ???, Experiment Manager? ????, ?? ??? Megatron GPT ??? ?????. ??? ??? ???? ??? ????? ?? ??? ?????. ? ??? LoRA ? PEFT? ?? Mistral? ?? ?? ??? ?????.

from nemo.collections.nlp.models.language_modeling.megatron_gpt_sft_model import MegatronGPTSFTModel from nemo.collections.nlp.parts.megatron_trainer_builder import MegatronLMPPTrainerBuilder from nemo.collections.nlp.parts.peft_config import LoraPEFTConfig from nemo.utils.exp_manager import exp_manager   trainer = MegatronLMPPTrainerBuilder(cfg).create_trainer() exp_manager(trainer, cfg.exp_manager)   model_cfg = MegatronGPTSFTModel.merge_cfg_with(cfg.model.restore_from_path, cfg) model = MegatronGPTSFTModel.restore_from(cfg.model.restore_from_path, model_cfg, trainer=trainer)   model.add_adapter(LoraPEFTConfig(model_cfg)) trainer.fit(model)

NVIDIA NeMo ????? ?? ??? ? ??? ?? ??? ??? GitHub? NVIDIA/NeMo-Framework-Launcher? ?????.

5??: ?? ? ?? ??

?? Mistral 7B ??? ?? ??????? ?? ??? ???? ??? ??????? ??? ? ????? ?????.

trainer.test(model)

Trainer.test(model) ??? ??? ??????? ??? ??? ???? ??? ??? ?????. ???? ??? ???(??? ?? ? PubMedQA ?????? ?? ??)? ?? ??? ?????. LoRA ???? Mistral? ?? ???? ? ??????? ?? PEFT ??? ?? ??? ???? ?? ??? ???.

?? NeMo? ???? Mistral 7B? ????? ?? ??????.

?? ????? ?? ????? ??? ?? ?? ????? ????? Brev? ???? NVIDIA ?????? ???? ??? NVIDIA ?????? ??? ??? ?? ? ????. ?? ??? ?? NGC ?????? ?? ??? ???? ?? ???? ???? AI ?? ? ?? ???? ??? ???.