Spring AI实用技巧：通过游戏主题的元数据筛选RAG搜索结果。
Spring AI实现的RAG元数据过滤方案：按游戏名称对文档进行标记，让大语言模型从问题中选取对应值，再以此过滤向量搜索结果。
向量相似度可以找到相关文本，但仅靠相关性往往不够。一个商店可能存储着数千个关于不同主题的文本片段：产品、客户，或是本例中的桌游。当多个片段与某个问题的相似度都相当高时，如何确保检索结果仅来自真正相关的那部分内容？可以为每份文档添加元数据，并在搜索时根据这些元数据进行过滤。这样模型就只会看到你选择的那部分数据。
实验设置：三款小型游戏
起点是一个无需独立向量数据库即可实现RAG功能的Spring AI应用，该应用经过修改后可读取多个文件。演示文件是三款虚构游戏的纯文本摘要：Camp Bowwow、Frog Panic和Taco Truck。它们分别位于src/main/resources/documents/目录下。首先是Camp Bowwow，文件名为camp-bowwow.txt：
On each player's turn, they place a dog, food bowl, or toy into
one of the three play yards. The game ends when a player has
placed their last dog into a yard. Then each player who hasn't
played in that round gets to place one more item into a yard
until the last player so that all players have an equal number
of turns.
frog-panic.txt 中的内容为：
On their turn, a player starts by moving any flies on the board
to the same space as an adjacent frog (effectively the frog has
eaten that fly). Then the player can either place 1 fly adjacent
to a frog on the board as a distraction so that the frog won't
jump or they may place a frog of their player color onto any
empty space. If the frog is adjacent to another frog (even if it
is their own), the adjacent frog will jump two spaces in any
direction away from the frog that was just placed. The direction
of the jump is decided by the current player. If the frog jumps
off the board, then that frog is out of play.
The game ends when all players have placed all of their frogs
(not counting frogs that have jumped off the board). The player
with the most frogs remaining on the board wins.
taco-truck.txt 中的墨西哥卷饼车内容：
When it is a player's turn, they will draw a card from the
ingredients deck and add it to their hand. Then they may play
any ingredient card from their hand to add that ingredient to
one of up to three tacos in progress. Or, if they have a crunch
card in their hand, then they may play it to remove one
ingredient from any one opponent's taco. The first player to
completely build 3 tacos wins.
现在的导入配置必须加载所有三个文件：
@Configuration
public class RagIngestionConfig {
private static final Logger logger =
LoggerFactory.getLogger(RagIngestionConfig.class);
@Value("${rag.documents}")
Resource[] documentResources;
@Bean
@Order(-1)
ApplicationRunner load(VectorStore vectorStore) {
return args -> {
for (Resource documentResource : documentResources) {
var filename = documentResource.getFilename();
logger.info("Loading document from {}.", filename);
var reader = new TikaDocumentReader(documentResource);
var splitter = TokenTextSplitter.builder().build();
vectorStore.accept(
splitter.apply(
reader.get()));
}
logger.info("Document loading complete.");
};
}
}
与早期版本相比，差异很小。不再是单一的 String 路径，而是通过 rag.documents 属性传入一个 Resource 对象数组。load() 方法会遍历这些对象，将每个对象拆分成多个块，然后把它们写入向量存储中。
现在运行该应用并输入类似“我该做什么？”的提问，就会出现问题。由于三种摘要都描述了玩家的回合，因此其中任何一种的片段都可能被认为与向量存储相关。
没有过滤器会出什么问题
回复可能会描述某一种游戏，或者更可能的是混合了几种游戏的规则。有一次尝试得到的结果如下：
How can I help?
> What can I do on my turn?
- Draw a card from the ingredients deck and add it to your hand.
- Then you may either:
- Play any ingredient card from your hand to add that ingredient
to one of up to three tacos in progress, or
- Play a crunch card (if you have one) to remove one ingredient
from any one opponent's taco.
- On your turn you place a dog, a food bowl, or a toy into one of
the three play yards.
- The game ends when a player has placed their last dog; then players
who haven't yet played in that round get one more placement so all
players have equal turns.
其中的每一个事实都与问题相关，但混杂多种游戏会导致答案毫无用处。试想商店里有数百款游戏时的情况。这就是元数据过滤发挥作用的地方。
用标题为文档添加标签
为作演示，每份文档都会将游戏的标题作为元数据，这样检索就能通过该值锁定到某一款游戏。采集代码会稍作修改，以便将标题与数据块一同存储：
@Bean
@Order(-1)
ApplicationRunner load(VectorStore vectorStore) {
return args -> {
for (Resource documentResource : documentResources) {
var filename = documentResource.getFilename();
logger.info("Loading document from {}.", filename);
var reader = new TikaDocumentReader(documentResource);
var splitter = TokenTextSplitter.builder().build();
var titleTag =
filename.substring(0, filename.lastIndexOf('.'));
vectorStore.accept(
splitter.apply(
reader.get().stream()
.peek(document ->
document.getMetadata()
.put("title", titleTag))
.toList()));
}
logger.info("Document loading complete.");
};
}
在文档到达分割器之前，代码会从其文件名中提取标题，并将其作为 title 键值存储在元数据中。对于 camp-bowwow.txt 来说，这意味着：
title = camp-bowwow
分割过程会携带元数据，因此每个生成的片段都会连同元数据一起存储到数据库中。嵌入模型依然能够提供语义相似度，而元数据则增加了另一个维度用于进一步筛选。这样一来，你不仅可以获取与问题相似的片段，还可以获取既与问题相似又属于特定游戏的片段。
确定问题涉及的是哪款游戏
接下来，应用程序需要判断适用的是哪款游戏。不过这里有个问题：存储的数值来自类似 camp-bowwow.txt 这样的文件名，这与人们通常书写的游戏名称并不一致。大语言模型可以通过将问题中的游戏名称转换为存储的数值之一来弥补这一差异：
@Service
public class TitleHelper {
private final ChatClient chatClient;
public TitleHelper(ChatModel chatModel) {
this.chatClient = ChatClient.builder(chatModel).build();
}
public String determineGameTitle(String question) {
var title = chatClient.prompt()
.user(userSpec -> userSpec
.text("""
Your job is to try to determine the title of a game from
the question asked.
The game choices are:
- camp-bowwow
- frog-panic
- taco-truck
- unknown
If the game's title isn't explicitly mentioned in the
question, or you don't recognize the game's title, then
say "unknown".
The question is:
{question}
""")
.param("question", question))
.call()
.content();
return title.equals("unknown") ? null : title;
}
}
请注意，TitleHelper并不会重用应用程序中的主ChatClient Bean。它会特意接收一个ChatModel并自行构建客户端。由于只是简单的分类任务，无需使用主客户端上的内存、RAG建议功能或其他高级特性，普通的ChatClient就足够了。
该功能会向模型提交一份与元数据值匹配的允许标题的封闭列表。如果模型无法识别这些游戏中的任何一款，它会返回unknown，而determineGameTitle()函数会将此值转换为null。
将过滤器应用于检索
在准备好该辅助工具后，ApplicationRunner Bean中的聊天循环只需进行少量修改即可：
@Bean
ApplicationRunner go(
ChatClient chatClient,
TitleHelper titleHelper) {
return args -> {
System.out.println("How can I help?\n");
try (Scanner scanner = new Scanner(System.in)) {
while (true) {
System.out.print("> ");
if (!scanner.hasNextLine()) break;
var input = scanner.nextLine();
if (input.isBlank()) continue;
var requestSpec = chatClient.prompt(input);
var gameTitle =
titleHelper.determineGameTitle(input);
var answer = requestSpec
.advisors(spec -> {
spec.param(
ChatMemory.CONVERSATION_ID,
"DEMO");
if (gameTitle != null) {
spec.param(
QuestionAnswerAdvisor.FILTER_EXPRESSION,
String.format(
"title == '%s'",
gameTitle));
}
})
.call()
.content();
System.out.println("\n - " + answer);
}
}
};
}
有三点发生了变化：
TitleHelper被注入到运行器中。- 它用于确定问题涉及的是哪款游戏。
- 找到对应游戏后，过滤表达式会被传递给
QuestionAnswerAdvisor，从而仅考虑那些title元数据匹配的文档。
该表达式本身很简短：
title == 'camp-bowwow'
关键在于过滤发生在何处。这并非在获取相关文档后要求大语言模型忽略无关文档的提示。过滤器实际上是对向量搜索本身进行限制，因此检索结果只会返回所选游戏的相关内容，模型从一开始就能获得聚焦的上下文。
当未检测到任何游戏时，就会跳过过滤器，搜索会涵盖所有内容。这种备用机制值得保留。错误的判断或缺失的识别不应导致流程崩溃，且未过滤时的行为与之前相同。
实际测试
启动应用程序，询问玩家在每轮中会做什么，这次要指定游戏名称：
How can I help?
> I'm playing Camp Bowwow. What do I do when it's my turn?
- On your turn you place one item - either a dog, a food bowl,
or a toy - into one of the three play yards.
(For reference: the game ends when a player has placed their last
dog; then any players who haven't yet played in that round each get
one more item turn in order until the last player, so all players
have equal turns.)
检索仅限于title值为camp-bowwow的片段，因此答案不会涉及Taco Truck或Frog Panic的规则。请尝试其他查询：
> I'm playing Frog Panic. What do I do when it's my turn?
- On your turn:
1. First, move any flies on the board onto the same space as an
adjacent frog (the frog eats that fly).
2. Then choose one action:
- Place 1 fly adjacent to a frog on the board as a distraction
(so that frog won't jump), or
- Place a frog of your color onto any empty space.
If the frog you place is adjacent to another frog (even your own),
that adjacent frog jumps two spaces away in any direction you choose.
If a frog jumps off the board it is out of play.
(For reference: the game ends when all players have placed all their
frogs; the player with the most frogs remaining on the board wins.)
同样的问题，不同的过滤条件，答案背后的文档集合也不同。
超越相似度
向量搜索擅长判断哪些文档与查询最为相似，但它无法单独体现所有与搜索位置相关的信息。元数据正是将这些额外知识引入检索过程的关键。
此处过滤条件仅为简单的游戏名称，但在实际应用中，可以用客户、产品、文档类型、类别、语言、日期、租户、安全范围或任何能缩小搜索范围的属性来标记文档。将语义相似度与元数据过滤相结合，能让检索更加精准，同时为模型提供更简洁、更相关的上下文。
在将此模式用于其他场景之前，有几点实用注意事项。请保持元数据值的一致性，并从已知的有限集合中选取，因为分类器只有在能够从固定列表中选择时才能正常工作。需提前确定当问题与任何条目都不匹配时应如何处理。此外，诸如租户或权限范围之类的安全相关元数据，应视为代码必须强制执行的硬性要求，而非要求模型去遵守的内容。
相关阅读
- Spring AI：利用Lucene和RRF的混合搜索获得更好的RAG结果 —— 在Spring AI检索器中将向量搜索与Lucene BM25结合，并通过互反排名融合技术整合排序结果，从而使RAG匹配既包含语义又包含精确术语的查询。