Count LoRA parameters

LoRA makes fine-tuning cheap because it trains two thin matrices instead of the full weight matrix.

What to build

A program that compares the weights of full fine-tuning with LoRA for one layer.

Requirements

  • Print full F, the number of weights of the layer.
  • Print lora L, the number of weights LoRA trains with rank r.
  • Print percent P, LoRA as a percentage of full, to two decimal places.

Input

One line with d_in d_out r.

Output

Three lines.

Constraints

1 ≤ r ≤ min(d_in, d_out) ≤ 100000.

Example

Input:

4096 4096 8

Output:

full 16777216
lora 65536
percent 0.39

Why it works

The full matrix has d_in times d_out entries, while two thin matrices of sizes d_in by r and r by d_out together have r times d_in plus d_out entries, which is far fewer when r is small.

Examples

Example 1 · Input
4096 4096 8
Expected output
full 16777216
lora 65536
percent 0.39
Example 2 · Input
100 50 2
Expected output
full 5000
lora 300
percent 6.00

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

A full layer has `d_in * d_out` weights.

Hint 2

LoRA of rank r trains `r * (d_in + d_out)` numbers; the percentage is 100 times LoRA over full.

Solution

Show one correct solution (try it yourself first)
d_in, d_out, r = map(int, input().split())
full = d_in * d_out
lora = r * (d_in + d_out)
print(f"full {full}")
print(f"lora {lora}")
print(f"percent {100 * lora / full:.2f}")

Lesson: Fine-tuning and alignment →