Skip to content

Read-only version: For security reasons, the Research Information System (FIS) has been temporarily replaced by this read-only version. All public content and full texts remain available; login, editing, submission and statistics are not available. Questions: fis@uni-bamberg.de

Home › Publications › Organising AI for safety : Identifying structural vulnerabilities to guide the design of AI-enhanced socio-technical systems

Publication

Organising AI for safety : Identifying structural vulnerabilities to guide the design of AI-enhanced socio-technical systems

Gazos, Alexandros; Kahn, James; Kusche, Isabel; u. a. (2025): Organising AI for safety : Identifying structural vulnerabilities to guide the design of AI-enhanced socio-technical systems, in: Safety science, Amsterdam [u.a.]: Elsevier, Jg. 184, Nr. 106731, S. 1–13, doi: 10.1016/j.ssci.2024.106731.

Details

Faculty/Chair
Faculty/Professorship
Author
Gazos, Alexandros
Kahn, James
Büscher, Christian
Götz, Markus
Title of the Journal
Safety science
ISSN
0925-7535
1879-1042
Publisher Information
Elsevier : Amsterdam [u.a.]
Year of publication
2025
Volume
184
Issue
106731
Language
English
DOI
Abstract

Artificial intelligence is increasingly being integrated into socio-technical systems. Existing design principles for ethical, safe and trustworthy AI tend to be highly abstract and focus on AI systems in isolation. They have rarely considered the adverse effects on safety that may emerge from interactions between AI and other technical components. Organisational theories of safety take such emergent outcomes of interactions between entities in socio-technical systems into account. They offer guidance on how to identify structural vulnerabilities in socio-technical systems enhanced by AI, and how to organise the design and operation of such systems for safety. In this paper, which is the result of a collaboration between sociologists and computer scientists (AI consultants), we conduct an analysis that can support the process of designing AI-enhanced autonomous systems in order to avoid structural vulnerabilities. It builds on organisational theories of safety and derives five key descriptors from them, the examination of which can guide the design of AI-enhanced systems. We demonstrate the utility of the descriptors by applying them to proposals for AI-enhanced critical functions in advanced microgrids. We discuss these proposals from the research literature on microgrids and review their effects on structural vulnerabilities. We then explore the implications that go beyond the example of advanced microgrids and propose steps for reviewing and reflecting on structural vulnerabilities that AI controllers may introduce into socio-technical systems.

GND Keywords
Künstliche Intelligenz
Microgrid <Energietechnik>
Systementwurf
Keywords
Socio-technical system
Organisation
Safety
Microgrids
Artificial intelligence
Design
DDC Classification
300 Social sciences, sociology & anthropology
RVK Classification
MS 4855
Peer Reviewed
yes
International Distribution
yes
Open Access Journal
no
Type
Article
URI
Activation date
2024-12-19

Permalink: https://fis.uni-bamberg.de/handle/uniba/105485 · Data as JSON · Cite: BibTeX · RIS

Data as of: 2026-11-16

Restricted read-only access: login, editing and submission are temporarily disabled.