← Back to LLM prompts

Pairwise Evaluation Prompt for LCEL QA

A coding prompt that guides an AI to build a pairwise evaluation workflow for LCEL QA, comparing two QA outputs and selecting the better answer.

coding a general-purpose LLM Prompt EngineeringCoding
<role>
You are a senior Python engineer and LCEL expert specializing in LangChain, LangGraph, and QA evaluation pipelines.
</role>
<task>
Create a concise, production-ready Python implementation for pairwise evaluation of two QA answers using LCEL, where the evaluator compares [answer_a] and [answer_b] against [question] and [reference_answer], then returns the preferred answer and a short rationale.
</task>
<context>
The code will be used in a QA evaluation system where multiple candidate answers need to be ranked. The evaluator should be deterministic in structure, easy to integrate into an LCEL chain, and suitable for batch processing.
</context>
<constraints>
- Use only Python and LCEL-compatible LangChain components.
- Keep the implementation focused on one main task: pairwise comparison.
- Include clear function signatures, type hints, and docstrings.
- Use placeholders for model, prompt, and output parser where needed.
- Avoid external dependencies beyond LangChain, LangGraph, and standard library.
- Prefer readable, maintainable code with minimal comments.
</constraints>
<format>
Return a single Python code block containing:
1. A prompt template for pairwise evaluation.
2. An LCEL chain that accepts [question], [reference_answer], [answer_a], and [answer_b].
3. A structured output parser that returns preferred_answer and rationale.
4. A small example usage snippet.
</format>
<tone>
Professional, precise, and implementation-focused.
</tone>
<action>
Generate the Python implementation now.
</action>
Website Source
#text