Skip to main content

Finetune API

CREATE​

Endpoint: /finetune/create

Description​

This method is used to create a new finetuning task.

Request​

  • Method: POST
  • Headers:
    • Content-Type: application/json
  • Body: 
    {
      "job_name": "string",
      "ft_type": "CLM",
      "model": "string",
      "base_model": "string",
      "use_recipes": false,
      "artifacts_storage": "string",
      "data_path": "string",
      "user_dataset": "string",
      "cloud_providers": ["SCALEGENAI"],
      "gpu_type": "A10G",
      "gpu_count": 2,
      "autotrain_params": {
        "model": "string",
        "job_name": "",
        "project_name": "ScaleGen Project",
        "data_path": "data",
        "push_to_hub": false,
        "repo_id": "string",
        "username": "string",
        "hf_token": "string",
        "wandb_key": "string",
        "comet_ml_key": "string",
        "train_split": "train",
        "train_subset": "string",
        "valid_split": "string",
        "valid_subset": "string",
        "add_eos_token": true,
        "block_size": -1,
        "model_max_length": 0,
        "padding": "left",
        "trainer": "default",
        "use_flash_attention_2": false,
        "log": "none",
        "disable_gradient_checkpointing": false,
        "logging_steps": -1,
        "eval_strategy": "epoch",
        "save_total_limit": 1,
        "save_strategy": "steps",
        "auto_find_batch_size": false,
        "mixed_precision": "fp16",
        "lr": 0.00003,
        "epochs": 1,
        "batch_size": 2,
        "warmup_ratio": 0.1,
        "gradient_accumulation_steps": 1,
        "optimizer": "adamw_torch",
        "lr_scheduler_type": "linear",
        "weight_decay": 0,
        "max_grad_norm": 1,
        "seed": 42,
        "save_steps": 20,
        "eval_steps": 0,
        "load_best_model_at_end": true,
        "resume_from_checkpoint": "string",
        "user_checkpoint_dir": "",
        "neftune_noise_alpha": 0,
        "use_deepspeed": "string",
        "apply_chat_template": "zephyr",
        "torch_dtype": "auto",
        "use_torch_compile": true,
        "quantization": "nf4",
        "double_quantization": false,
        "use_peft": "lora",
        "lora_r": 16,
        "lora_alpha": 32,
        "lora_dropout": 0.05,
        "init_lora_weights": "gaussian",
        "use_rslora": false,
        "adalora_init_r": 12,
        "adalora_target_r": 8,
        "llama_adapter_len": 128,
        "llama_adapter_layers": 8,
        "target_modules": "string",
        "merge_adapter": false,
        "model_ref": "string",
        "dpo_beta": 0.1,
        "max_prompt_length": 128,
        "max_completion_length": 0,
        "prompt_text_column": "string",
        "text_column": "text",
        "rejected_text_column": "string",
        "use_unsloth": false
      }
    }

Parameter Description​

  • job_name [optional]:: string : The name of the finetuning task.
  • ft_type:: string : Specifies the finetuning type (e.g., "CLM").
  • model:: string : The name of the model to be used for finetuning.
  • base_model:: string : The base model on which finetuning is performed.
  • use_recipes:: boolean : A boolean indicating whether to use predefined recipes for finetuning.
  • artifacts_storage:: string : The path or location to store artifacts.
  • data_path:: string : The path to the dataset used for training.
  • user_dataset:: string : Dataset file name if you are using a Virtual Mount.
  • cloud_providers:: list[string] : An array of cloud providers.
  • gpu_type:: string : The type of GPU to use for training.
  • gpu_count:: int : The number of GPUs allocated for the job.

AutoTrain Parameters​

  • model:: string : Model used for training.
  • job_name [optional]:: string : Name of the job, if different from job_name in main parameters.
  • project_name:: string : Name of the project (e.g., "ScaleGen Project").
  • data_path:: string : Path to the data.
  • push_to_hub:: boolean : Indicates whether to push the trained model to a hub.
  • repo_id:: string : Repository ID for pushing to the hub.
  • username:: string : Username for authentication.
  • hf_token:: string : Hugging Face token for authentication.
  • wandb_key:: string : Weights & Biases API key.
  • comet_ml_key:: string : Comet ML API key.
  • train_split:: string : Dataset split for training (e.g., "train").
  • train_subset [optional]:: string : The subset of training data to use.
  • valid_split [optional]:: string : Dataset split for validation.
  • valid_subset [optional]:: string : Subset of validation data.
  • add_eos_token:: boolean : Indicates whether to add an End-of-Sequence (EOS) token.
  • block_size:: int : Block size for processing.
  • model_max_length:: int : Maximum model sequence length.
  • padding:: string : Padding type ("left" or "right").
  • trainer:: string : Type of trainer (e.g., "default").
  • use_flash_attention_2 [optional]:: boolean : A boolean indicating whether to use Flash Attention 2.
  • log:: string : Logging level.
  • disable_gradient_checkpointing [optional]:: boolean : Indicates whether to disable gradient checkpointing.
  • logging_steps:: int : Number of steps between logging.
  • eval_strategy:: string : Evaluation strategy (e.g., "epoch").
  • save_total_limit:: int : Limit for the number of saved checkpoints.
  • save_strategy:: string : Strategy for saving checkpoints.
  • auto_find_batch_size:: boolean : Indicates whether to find the optimal batch size automatically.
  • mixed_precision [optional]:: string["fp16", "bf16"] : Type of mixed precision to use.
  • lr:: float : The learning rate.
  • epochs:: int : The number of epochs.
  • batch_size:: int : The batch size.
  • warmup_ratio:: float : Warm-up ratio for learning rate scheduling.
  • gradient_accumulation_steps [optional]:: int : The number of gradient accumulation steps.
  • optimizer:: string : Type of optimizer (e.g., "adamw_torch").
  • lr_scheduler_type:: string : Type of learning rate scheduler (e.g., "linear").
  • weight_decay:: float : Weight decay for optimization.
  • max_grad_norm:: float : Maximum gradient norm.
  • seed:: int : Random seed for reproducibility.
  • save_steps:: int : Steps between saving checkpoints.
  • eval_steps:: int : Steps between evaluations.
  • load_best_model_at_end:: boolean : Indicates whether to load the best model at the end of training.
  • resume_from_checkpoint [optional]:: string : Path to resume from a saved checkpoint.
  • user_checkpoint_dir:: string : Directory for user-defined checkpoints.
  • neftune_noise_alpha:: float : Alpha value for noise tuning.
  • use_deepspeed:: string : Enables Deepspeed for distributed training.
  • apply_chat_template [optional]:: string : Template for chat-based models (e.g., "zephyr").
  • torch_dtype:: string : Data type for PyTorch tensors.
  • use_torch_compile:: boolean : Enables torch.compile for optimization.
  • quantization [optional]:: string["int4", "int8"] : Type of quantization to use.
  • double_quantization:: boolean : Enables double quantization.
  • use_peft [optional]:: string : Type of PEFT to use (e.g., "lora").
  • lora_r [optional]:: int : The value of LORA r.
  • lora_alpha [optional]:: int : The value of LORA alpha.
  • lora_dropout [optional]:: float : The value of LORA dropout.
  • init_lora_weights [optional]:: string : Initialization method for LORA weights (e.g., "gaussian").
  • use_rslora [optional]:: boolean : Enables RSLORA.
  • adalora_init_r [optional]:: int : Initial r value for AdaLoRA.
  • adalora_target_r [optional]:: int : Target r value for AdaLoRA.
  • llama_adapter_len [optional]:: int : Adapter length for Llama.
  • llama_adapter_layers [optional]:: int : Number of adapter layers for Llama.
  • target_modules [optional]:: string : Specifies target modules.
  • merge_adapter:: boolean : Indicates whether to merge adapter weights.
  • model_ref [optional]:: string : Reference to the model.
  • dpo_beta [optional]:: float : DPO beta value.
  • max_prompt_length [optional]:: int : Maximum prompt length.
  • max_completion_length [optional]:: int : Maximum completion length.
  • prompt_text_column [optional]:: string : Column name for prompt text.
  • text_column [optional]:: string : Column containing the main text data.
  • rejected_text_column [optional]:: string : Column name for rejected text.
  • use_unsloth [optional]:: boolean : Enables Unsloth.

Response​

{
  "success": true,
  "message": {
    "job_id": job_id
  },
  "warning": "string",
  "info": "string"
}

Example​

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "job_name": "test_job",
    "ft_type": "CLM",
    "model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
    "data_path": "CyberNative/Code_Vulnerability_Security_DPO",
    "cloud_providers": ["SCALEGENAI"],
    "gpu_type": "A100_80GB",
    "gpu_count": 1,
    "autotrain_params": {
      "model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
      "job_name": "test_job",
      "project_name": "test_project",
      "data_path": "CyberNative/Code_Vulnerability_Security_DPO",
      "push_to_hub": true,
      "repo_id": "repoid01",
      "username": "your_username",
      "hf_token": "your_hf_token",
      "wandb_key": "your_wandb_key",
      "train_split": "train",
      "lr": 0.00003,
      "epochs": 1,
      "batch_size": 2,
      "model_max_length": 8192,
      "mixed_precision": "bf16",
      "warmup_ratio": 0.1,
      "gradient_accumulation_steps": 64,
      "weight_decay": 0,
      "max_grad_norm": 1,
      "optimizer": "adamw_torch",
      "lr_scheduler_type": "linear",
      "max_prompt_length": 256,
      "use_peft": "lora",
      "lora_r": 32,
      "lora_alpha": 64,
      "lora_dropout": 0.05,
      "finetuning_type": "dpo",
      "text_column": "chosen",
      "prompt_text_column": "question",
      "rejected_text_column": "rejected",
      "padding": "left",
      "seed": 42,
      "save_steps": 20,
    }
  }' \
  https://api.example.com/finetune/create