The augment step of RAG: retrieve the best chunks and assemble the prompt the model will see.
What to build
A program that retrieves the best chunks for a question and prints a grounded prompt.
Requirements
- Score each chunk by the number of different question words it contains. Keep at most k chunks with a score above 0, best first (ties by position).
- Print the instruction line
Answer the question using only the sources below. If they do not contain the answer, say you do not know. - Then each kept chunk as
[N] TEXT, or(no sources found)if none, thenQuestion: QUESTION.
Input
Line 1: k. Line 2: n. Then n chunks. Then the question. All lower-case words.
Output
The prompt.
Constraints
1 ≤ k ≤ 20; 1 ≤ n ≤ 1000.
Example
Input:
2
3
the library opens at nine
the cafe opens at eight
books can be renewed online
when does the library open
Output:
Answer the question using only the sources below. If they do not contain the answer, say you do not know.
[1] the library opens at nine
[2] the cafe opens at eight
Question: when does the library open
Why it works
The model can only use what is in its prompt, so retrieval chooses the most relevant chunks, and the instruction line keeps the answer tied to them.
Examples
2 3 the library opens at nine the cafe opens at eight books can be renewed online when does the library open
Answer the question using only the sources below. If they do not contain the answer, say you do not know. [1] the library opens at nine [2] the cafe opens at eight Question: when does the library open
1 2 x y z z
Answer the question using only the sources below. If they do not contain the answer, say you do not know. [1] z Question: z
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
Score each chunk by the number of different question words it contains; ignore chunks that score 0.
Hint 2
Sort by score (highest first, then by position) and keep at most k.
Hint 3
Number the kept chunks from 1 in that order.
Solution
Show one correct solution (try it yourself first)
k = int(input())
n = int(input())
chunks = [input() for _ in range(n)]
question = input()
q = set(question.split())
scored = [(-len(q & set(c.split())), i) for i, c in enumerate(chunks)]
kept = [i for s, i in sorted(scored) if s < 0][:k]
print("Answer the question using only the sources below. If they do not contain the answer, say you do not know.")
for number, i in enumerate(kept, 1):
print(f"[{number}] {chunks[i]}")
if not kept:
print("(no sources found)")
print(f"Question: {question}")