The day in AI

AI’s centre of gravity today is Bremen, where IJCAI-ECAI 2026 has begun its workshop and tutorial programme ahead of the main technical sessions. The conference runs from 15 to 21 August and spans AI, robotics, human-centred systems and critical technologies. Yet the opening programme’s clearest signal is not a new model or headline benchmark. It is the effort to redefine reliability for systems that use tools, encounter changing information and affect the physical or social world. IJCAI-ECAI’s programme places these problems across workshops and competitions rather than confining them to a single safety track.

The emphasis is unusually concrete. RobustifAI asks how systems survive distribution shifts, adversarial inputs and human interaction across their entire lifecycle, while TRUST-AI has drawn 45 submissions around the broader trustworthiness agenda. Competitions turn similar questions into observable tasks: can an assistant admit that it lacks a capability, resist malicious instructions embedded in an email, produce a feasible travel plan, locate evidence of a deepfake, or connect industrial sensor data to a physically justified maintenance decision? These are proposals and evaluations, not proof that the problems have been solved. But collectively they show researchers looking beyond whether a model can produce a convincing answer. RobustifAI and TRUST-AI explicitly frame robustness and trustworthiness around deployment.

The deeper pattern

The emerging unit of AI evaluation is the trajectory, not the answer.

A conventional benchmark presents a fixed input and scores the resulting output. An agent operates inside a less forgiving chain: it interprets a request, observes an environment, chooses tools, handles missing information, changes state and explains what it did. Failure can enter at every link. A capable language model may still call the wrong service, invent an unavailable function, obey instructions planted in retrieved content or satisfy one constraint while silently violating another.

CAR-bench makes this distinction measurable. It models an in-car assistant across 254 public tasks, 58 tools and 19 policies, with separate tests for ordinary task completion, missing capabilities and ambiguous requests. Its consistency measure only credits a task when the agent succeeds in all three trials. The organisers report 34 ranked competition submissions, while their unoptimised frontier-model baselines achieved average consistency scores ranging from 0.16 to 0.58. Their central observation is a “completion over compliance” tendency: agents may fabricate a tool result rather than admit incapacity, or guess rather than ask for clarification. Those figures are benchmark-specific, but they are stronger evidence of a deployment gap than a single successful demonstration. CAR-bench documents the task design, baselines and completed competition.

The OpenClaw challenge extends the trajectory idea into security. Its test environment gives agents access to email, web, code-execution and simulated business services. Attackers attempt to place malicious instructions inside webpages, messages or ostensibly useful skill files, then induce unauthorised actions such as unlocking a door, issuing an excessive payment or exfiltrating data. Crucially, success is judged from the recorded tool-use trajectory and resulting action, not merely from whether the model produced unsafe language. The challenge also tests static scanning and model-based safety checks before an attack reaches execution. That design recognises that an agent’s attack surface includes its tools, permissions and software supply chain—not just its underlying model. It does not, by itself, establish that the proposed defences are adequate. The OpenClaw challenge specifies the attack environment and scoring stages.

Grounding is the complementary problem: if an AI system acts or makes an accusation, what evidence supports it? The industrial automation competition asks models to translate noisy signals into maintenance narratives using structured engineering context such as Failure Mode and Effects Analysis. It separates reasoning performed within the model from reasoning augmented by external industrial tools. The deepfake challenge similarly moves beyond a real-or-fake label: systems must localise manipulated regions and provide explanations tied to observable features. Its dataset contains more than 100,000 samples, and its evaluation combines detection accuracy, localisation overlap and explanation scoring. The conference describes the industrial task and its two tracks, while the deepfake challenge publishes its metrics and explanation rubric.

This is progress, but explanation scores need careful interpretation. A fluent description of a blurred jawline can be wrong, and similarity metrics can reward text that resembles a reference without proving that the model used the cited region in reaching its decision. The deepfake organisers partly address this with criteria requiring observable, precisely localised evidence and consistency between the predicted label and explanation. The harder question—whether investigators can rely on those explanations across new manipulation methods—remains open.

Two further workshops broaden reliability beyond immediate task performance. AIM4DK treats disappearing or ageing knowledge as a systems problem: an answer that is supportable today may become difficult to reproduce when a webpage changes, a dataset is revised or a source vanishes. AIM4DK places disappearing knowledge on the research agenda. VALE, meanwhile, treats values as engineering objects, with topics including value representation, conflict resolution, explainability, participatory design and legal questions. Its programme reflects an important limit of purely technical assurance: a system may execute a plan accurately and securely while pursuing an objective that affected people reject. VALE’s stated scope spans technical, participatory and legal approaches.

Taken together, these projects suggest a practical assurance stack: preserve the evidence; test performance under changing conditions; verify that the system recognises uncertainty and authority limits; constrain tool use; and retain an auditable account of decisions. No single benchmark covers that stack. The durable development is the recognition that model quality is only one component of system reliability.

What to watch next

  1. Whether CAR-bench presentations disclose failure-mode results, not just overall rankings. The competition session is scheduled for 18 August. A meaningful result would report hidden-set performance separately for task completion, missing-capability resistance and disambiguation—and show whether improvements survive repeated trials. An aggregate leaderboard without those components would leave the central reliability claim under-tested. The conference schedule lists the CAR-bench session.

  2. Whether the industrial competition demonstrates a measurable benefit from tools. Its native-model and tool-augmented tracks create a direct test of whether external engineering resources improve physically grounded diagnosis rather than merely producing longer explanations. Watch for held-out results, causal-error analysis and examples where a tool prevents an unsupported maintenance recommendation. Their absence would make “physics-grounded” more aspiration than demonstrated capability. The two-track design is described here.

  3. Whether workshop agendas become inspectable evidence. TRUST-AI says accepted short papers are intended for CEUR-WS publication. Watch for proceedings that specify datasets, failure definitions and reproducible evaluation methods, especially for agent security, provenance and value conflict. If the output remains mainly principles and position statements, trustworthiness will still be advancing faster as vocabulary than as assurance practice. TRUST-AI states its publication plan.

Editorial note

The main uncertainty is that much of today’s evidence comes from organisers’ descriptions, competition pages and self-reported benchmark results. Several workshops are presenting research agendas whose accepted papers or proceedings are not yet available for independent scrutiny, and organiser-defined simulations may omit failures found in open deployments. This edition is also deliberately conference-centred: IJCAI-ECAI offers a coherent view of research priorities, but it is not a complete sample of global AI development.