Skip to main content

Finetune API

CREATE​

Endpoint: /finetune/create

Description​

This method is used to create a new finetuning task.

Request​

  • Method: POST
  • Headers:
    • Content-Type: application/json
  • Body: 
    {
      "job_name": "string",
      "model": "string",
      "data_path": "string",
      "user_dataset": "string",
      "hf_token": "string",
      "project_name": "string",
      "push_to_hub": false,
      "username": "string",
      "repo_id": "string",
      "use_spot": false,
      "cloud_providers": [],
      "wandb_key": "string",
      "artifacts_storage": "string",
      "autotrain_params": {
        "use_peft": "lora",
        "quantization": "fp4",
        "mixed_precision": "fp16",
        "disable_gradient_checkpointing": true,
        "use_flash_attention_2": true,
        "lora_r": 0,
        "lora_alpha": 0,
        "lora_dropout": 0,
        "lr": 0.00003,
        "batch_size": 1,
        "epochs": 1,
        "train_subset": "string",
        "text_column": "string",
        "gradient_accumulation": 0,
        "max_model_length": 2048,
        "block_size": "-1",
        "torch_dtype": "auto"
      }
    }

Parameter Description​

  • job_name [optional]:: string : The name of the finetuning task.
  • model:: string : The name of the model to be used for finetuning.
  • data_path:: string : The path to the HuggingFace dataset used for finetuning.
  • user_dataset:: string : Dataset file name if you are using a Virtual Mount.
  • hf_token:: string : HuggingFace token.
  • project_name:: string : HuggingFace project name.
  • push_to_hub:: string : A boolean indicating whether to push the finetuned model to the Hugging Face model hub.
  • username:: string : HuggingFace username.
  • repo_id:: string : HuggingFace repository ID.
  • use_spot:: boolean : A boolean indicating whether to use spot instances for training.
  • cloud_providers:: list[string] : An array of cloud providers.
  • wandb_key:: string : Weights & Biases API key.
  • artifacts_storage:: string : Name of the Artifacts Storage.
  • autotrain_params[optional]:: object : HuggingFace autotrain parameters.
  • use_peft [optional]:: boolean : A boolean indicating whether to use PEFT for finetuning.
  • quantization [optional]:: string["int4", "int8"] : The type of quantization to use.
  • mixed_precision [optional]:: string["fp16", "bf16"] : The type of mixed precision to use.
  • disable_gradient_checkpointing [optional]:: boolean : A boolean indicating whether to disable gradient checkpointing.
  • use_flash_attention_2 [optional]:: boolean : A boolean indicating whether to use Flash Attention 2.
  • lora_r [optional]:: int : The value of LORA r.
  • lora_alpha [optional]:: int : The value of LORA alpha.
  • lora_dropout [optional]:: float : The value of LORA dropout.
  • lr:: float : The learning rate.
  • batch_size:: int : The batch size.
  • epochs:: int : The number of epochs.
  • train_subset [optional]:: string : The subset of training data to use.
  • text_column [optional]:: string : The column containing the text data.
  • gradient_accumulation [optional]:: int : The number of gradient accumulation steps.
  • max_model_length:: int : The maximum model length.
  • block_size:: int : The block size.
  • torch_dtype:: string : The data type for PyTorch.

Response​

{
  "success": true,
  "message": {
    "job_id": job_id
  },
  "warning": "string",
  "info": "string"
}

Example​

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "job_name": "test_job",
    "model": "meta-llama/Llama-2-70b",
    "data_path": "wikitext",
    "hf_token": "your_hf_token",
    "project_name": "test_project",
    "push_to_hub": true,
    "username": "your_username",
    "repo_id": "repoid01",
    "use_spot": true,
    "cloud_providers": ["AWS", "GCP"],
    "wandb_key": "your_wandb_key",
    "artifacts_storage": "your_artifacts_storage",
    "autotrain_params": {
      "use_peft": true,
      "quantization": "int8",
      "mixed_precision": "fp16",
      "disable_gradient_checkpointing": true,
      "use_flash_attention_2": true,
      "lora_r": 3,
      "lora_alpha": 6,
      "lora_dropout": 0.01,
      "lr": 0.01,
      "batch_size": 4,
      "epochs": 5
    }
  }' \
  https://api.example.com/finetune/create