Building AI for Lawyers with Jacob Lauritzen, CTO of Legora
Jacob Lauritzen, CTO of Legora, recently discussed the engineering and business challenges of implementing artificial intelligence systems in the legal sector. The discussion highlights complexities such as evaluating large language models without proper traces, conducting million-cell tabular reviews, and the shifting landscape of the traditional billable hour business model. This insights-driven discussion provides a valuable technical deep-dive into how legal tech companies overcome the high-stakes accuracy requirements of the legal industry. It illustrates practical architectural hurdles like LLM evals and traces while shedding light on how automation is fundamentally disrupting traditional legal billing practices. The conversation covers technical nuances like LLM evaluations without execution traces, which can lead to unreliable quality scoring or 'vibes-based' assessment. Additionally, it addresses large-scale data challenges like processing massive tabular reviews required for comprehensive legal document analysis.
## BACKGROUND
LLM evaluations and traces are critical mechanisms used by software engineers to monitor, debug, and grade the performance of artificial intelligence applications by inspecting individual component outputs during execution. In the legal sector, tabular reviews refer to structured data extraction and document analysis methods that help lawyers parse large volumes of contracts and regulatory files efficiently.