~/AI BENCHMARK/google-and-artificial-analysis-launch-mka-benchmark-for-multimodal-ai-agents

Google and Artificial Analysis Launch MKA Benchmark for Multimodal AI Agents

Google has partnered with Artificial Analysis to introduce the Multimodal Knowledge Agent (MKA) benchmark, designed to evaluate AI agents on complex, real-world knowledge work tasks. Google's Gemini 4 Carbon secured second place in this benchmark while achieving task completion costs roughly six times lower than the first-place model. This benchmark shifts AI evaluation away from text-only tasks toward realistic, visually complex professional workflows that require reasoning across multiple steps and file types. It highlights a critical industry trend: balancing high performance with cost-efficiency for enterprise-grade autonomous agents. The MKA benchmark includes 200 private tasks spanning 10 industries and 83 professions, running on a standardized open-source agent framework called Stirrup. Evaluation is conducted using an automated judge system that inspects files and verifies results against expert-crafted scoring rubrics.

## BACKGROUND

Multimodal AI agents are autonomous software systems capable of processing and reasoning across multiple data modalities—such as text, images, and structured spreadsheets—to execute complex workflows. Benchmarking these agents requires testing their ability to handle multi-step professional projects that typically take human experts hours to complete.

## REFERENCES

## KEYWORDS

#AI Benchmarks#Multimodal AI#Artificial Analysis#Gemini#AI Agents

$ subscribe --daily