{"id":1689,"date":"2026-10-06T19:04:30","date_gmt":"2026-10-06T19:04:30","guid":{"rendered":"https:\/\/www.cs.ubbcluj.ro\/~meco\/large-language-models-as-judges-recent-advances-in-llm-based-evaluation-critique-preference-modeling-and-feedback-for-text-and-code-2026\/"},"modified":"2026-10-06T19:27:22","modified_gmt":"2026-10-06T19:27:22","slug":"large-language-models-as-judges-recent-advances-in-llm-based-evaluation-critique-preference-modeling-and-feedback-for-text-and-code-2026","status":"publish","type":"post","link":"https:\/\/www.cs.ubbcluj.ro\/~meco\/large-language-models-as-judges-recent-advances-in-llm-based-evaluation-critique-preference-modeling-and-feedback-for-text-and-code-2026\/","title":{"rendered":"Large language models as judges: recent advances in LLM-based evaluation, critique, preference modeling, and feedback for text and code (2026)"},"content":{"rendered":"<div class=\"entry-content\">\n<p>Artificial Intelligence Review<\/p>\n<h2>Authors<\/h2>\n<p>M. Nad\u0103\u015f<\/p>\n<h2>Abstract<\/h2>\n<p>Large Language Models (LLMs) are increasingly used as judges to evaluate, rank, and critique AI-generated text and code. This survey provides a comprehensive overview of recent advances (2020\u2013early 2026) in LLM-based evaluation, covering techniques, applications, and challenges across domains. We make three main contributions: (1) a unified taxonomy of LLM judging tasks spanning text (summarization, dialogue, factuality, safety) and code (correctness checking, code review, security analysis); (2) a systematic review of prompting strategies (zero\/few-shot, rubric-based, pairwise comparison, chain-of-thought) and advanced pipelines (ensemble judges, multi-agent debate, tool-augmented verification); and (3) an analysis of LLM judge quality, documenting systematic biases (length, position, self-preference) and their mitigations. We review practical applications including benchmark evaluation (MT-Bench, Chatbot Arena), data filtering, and reward modeling for RLHF\/RLAIF. Key challenges discussed include calibration, fairness, reproducibility, and adversarial robustness. We conclude with future directions emphasizing standardized protocols, uncertainty estimation, and human\u2013AI collaboration. LLM-based judging shows promise for scalable evaluation, but careful design and rigorous validation are essential to ensure these AI judges meet human standards of accuracy and fairness.<\/p>\n<h2>Citation<\/h2>\n<pre class=\"wp-block-preformatted\">@Inproceedings{Nad\u0103\u015f2026LargeLM,\n author = {M. Nad\u0103\u015f},\n booktitle = {Artificial Intelligence Review},\n title = {Large language models as judges: recent advances in LLM-based evaluation, critique, preference modeling, and feedback for text and code},\n year = {2026}\n}<\/pre>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Artificial Intelligence Review Authors M. Nad\u0103\u015f Abstract Large Language Models (LLMs) are increasingly used as judges to evaluate, rank, and critique AI-generated text and code. This survey provides a comprehensive overview of recent advances (2020\u2013early 2026) in LLM-based evaluation, covering techniques, applications, and challenges across domains. We make three main contributions: (1) a unified taxonomy&hellip;<\/p>\n","protected":false},"author":6,"featured_media":1714,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[85,71],"_links":{"self":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts\/1689"}],"collection":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/comments?post=1689"}],"version-history":[{"count":2,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts\/1689\/revisions"}],"predecessor-version":[{"id":1715,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts\/1689\/revisions\/1715"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/media\/1714"}],"wp:attachment":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/media?parent=1689"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/categories?post=1689"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/tags?post=1689"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}