Human in the Loop 4.0
How can we harness the potential of automation while retaining human oversight? And what does successful automation look like in practice? In their paper “Targeted Automation and Sustaining Human-AI Learning,” Kai Hoberg (KLU), Christina Imdahl (Eindhoven University of Technology), and William Schmidt (Emory University), make the case for targeted automation.
Balancing Efficiency and Continuous AI Improvement
“Our research shows that companies should take a selective approach to automation. Rather than automating all decisions, they should deliberately retain human involvement in those decisions where people can help improve the algorithm. At the same time, sufficient human decision-making must be maintained to ensure that the algorithm can adapt to changing conditions,” says Kai Hoberg, Professor of Supply Chain and Operations Strategy at Kühne Logistics University (KLU).
This allows companies to strike a balance between short-term efficiency and continuous AI improvement, helping to ensure long-term success.
A Four-Step Approach to Targeted Automation
To put this into practice, the researchers developed a four-step approach to targeted decision automation.
- Identify where human input adds value: First, they analyze when human corrections to algorithmic recommendations lead to better decisions.
- Predict the value of human expertise: Based on these insights, the model developed by the researchers learns to predict when human expertise is likely to improve decision outcomes.
- Automate decisions selectively: Decisions where human intervention is expected to offer little additional benefit are automated, while those where human expertise can add value remain in human hands.
- Continuously improve the algorithms: Finally, human corrections are fed back into the system to refine and improve the algorithms over time.
How Human Expertise Improves Algorithms
“The crucial point is that it is not simply about leaving difficult decisions to humans, but about predicting which decisions actually allow humans to contribute to improving the system,” emphasizes Christina Imdahl, who earned her doctorate at Kühne Logistics University and is now a professor at Eindhoven University of Technology.
This ensures that valuable human expertise continues to feed into algorithm development while still allowing for a high level of automation. “In our empirical application, we were able to automate around 84 percent of decisions,” she says.
Professor Hoberg advises business leaders to take a gradual approach to automation, starting by establishing how they can measure the added value of human intervention.
“This provides a basis for making targeted decisions about which tasks to automate and where human expertise is still needed.”
Humans and AI as Partners in a Dynamic System
For their study, the researchers analyzed an extensive dataset covering more than 475,000 order-related decisions processed over a four-year period at a major commercial vehicle manufacturer with annual revenues exceeding four billion euros. Their aim was to identify the circumstances in which humans outperformed the algorithm.
However, Hoberg, Imdahl, and Schmidt were not primarily concerned with the traditional question of “Who performs better: humans or AI?”
Instead, they sought to develop a model that views humans and AI as partners within a dynamic system. The goal is to relieve employees of repetitive tasks that add little value while ensuring that AI continues to evolve and improve through ongoing human feedback.
More information:
Paper: Imdahl, C., Schmidt, W., & Hoberg, K. (2026). Targeted Automation and Sustaining Human-AI Learning. Production and Operations Management, 35(4), 1520-1538. https://doi.org/10.1177/10591478251381899
Photo by Igor Omilaev on Unsplash
