Exploring the Social Science of Human-Autonomy Interaction

Throwback Thursday: The Ironies of Automation

If I have seen further, it is by standing on the shoulders of giants
— Isaac Newton, 1675

Don't worry, our Throwback Thursday doesn’t involve embarrassing pictures of me or Arathi from 5 years ago.  Instead, it is more cerebral.  The social science behind automation and autonomy is long and rich, and despite being one of the earliest topics of study in engineering psychology, it has even more relevance today.

Instead of re-inventing the wheel, why don't we look at the past literature to see what is still relevant?

In an effort to honor that past but also inform the future, the inaugural "Throwback Thursday" post will highlight scientific literature from the past that is relevant to modern discussion of autonomy.

Both Arathi and I have taught graduate seminars in automation and autonomy so we have a rich treasure trove of literature from which to draw.  Don't worry: while some of the readings can be complex and academic, in deference to our potentially diverse readership, we will focus on key points and discuss their relevance today.

The Ironies of Automation

In this aptly titled paper, Bainbridge discusses, back in 1983(!), the ironic things that can happen when humans interact with automation.  The words of this paper ring especially true today when the design strategy of some companies is to consider the human as an error term to be eliminated:

The designer’s view of the human operator may be that the operator is unreliable and inefficient, so should be eliminated from the system.
— Bainbridge, pp. 775

But is this design strategy sustainable?  Bainbridge later wisely points out that:

The second irony is that the designer who tries to eliminate the operator still leaves the operator to do the tasks which the designer cannot think how to automate.
— Bainbridge, pp. 775

The paper then discusses, how, under such an approach, many unintended problems arise.  The ultimate irony, however, is that the implementation of very high levels of automation (including eliminating the driver in a self-driving car) will ultimately lead to a higher workload burden for the "passenger."

A more serious irony is that the automatic control system has been put in because it can do the job better than the operator, but yet the operator is being asked to monitor that it is working effectively.
— Bainbridge, pp. 775