Updates from Lukas Murdock
Does Ubiquitous Computing Need Interface Agents?
July 27, 2025
Does Ubiquitous Computing Need Interface Agents?
Mark Weiser, Head of Computer Science Lab, Xerox PARC
Agents92.tioga, October 6, 1992
Ubiquitous Computing vs. Interface Agents
- something altogether different, even opposed, to interface agents: ubiquitous computing (also known as embodied virtuality)
- outline of this talk:
- what is ubiquitous computing
- compare and contrast to interface agents
Context of work: Xerox PARC
-
A different kind of research in two ways
- Reverse technology transfer: PARC transfers research ideas to universities to pursue.
- We deploy and use our discoveries.
-
Over twenty years of pioneering basic research
- Individuals change, innovation continues
- Steady Xerox commitment to long-range research
- Laboratories ranging from physics to anthropology, all working together.
-
Connection to Xerox
- PARC technology in nearly every product
- Research a vital part of The Document Company
This is a talk about metaphors and world views
-
All people see and understand the world as filtered through a world view
- The world view itself is not seen
- Because the world view filters everything, it is hugely influential on what we do
- Some metaphors become ingrained as world views
-
Metaphors are extremely influential
- Plato’s metaphors of ideal forms and knowledge as viewing shadows in a cave influencing 2,000 years of theories of knowing
- Rousseau’s “Noble Savage” influence in the French revolution, morals, cruel experiments on children
- The metaphor of the “master molecule” delayed biochemical understanding
People are most effective and authentic when they are fully engaged, mind and body, in the world
- Examples:
- flow of the athlete in the groove
- effortless use of pencil, paper and language when writing
- “feel for the organism” of the great biologist
- effortless 65 MPH driving of the experienced driver (while talking, reading roadsigns, …)
-
This is a basic characteristic of humans, across all cultures
- Polynesian navigation between islands depends upon attunement with currents, wind, and weather
- African tailor apprenticeships via peripheral participation depend upon learning by engagement
- Technologies should enhance this ability to engage, to “flow” with life and work
The most powerful technologies are invisible: they get out of the way to let the human be effective
-
Electricity
- Electric motors hidden everywhere (20-30 per car)
- Electric sockets in every wall and portably available through batteries
- Integrated, invisible infrastructure
-
Literary Technology
- Continuously surrounding us at many scales
- Used trivially and profoundly
- Literally visible, effectively invisible
How to do invisible computing?
-
Integrated computer systems approach
- invisible, everywhere, computing
- named “ubiquitous computing” in April 1989.
- Invisible: tiny, embedded, attachable, …
- Everywhere: wireless, dynamically configurable, remote access, adapting, …
Goals of ubiquitous computing (ubicomp)
-
Ultimate Goal
- invisible technology
- integration of virtual and physical worlds
- throughout desks, rooms, buildings, and life
- take the data out of information, leaving behind just an enhanced ability to act
-
Phase I
- tabs, pads, and boards
- hundreds of computers per person
- wireless networks
- location-based services
- shared meeting applications
Using a computer should be as refreshing as a walk in the woods.
ubicomp phase 1: Ubiquitous I/O devices
-
Post-it note-sized palmtop computers
- One hundred per person per office
- Always have one on you, wirelessly connected
- Small touch-sensitive display screen
- Scatter around the office like postit notes
-
Notebook-sized computers
- Ten per person per office
- Stylus-based input primary
- Near megabit wireless comm bandwidth
- Can support multi-media when “tethered”
-
Wall displays
- Large ones used as shared display surfaces (replaces whiteboards)
- Replace physical bulletin boards, etc.
- Lots of bandwidth available because they’re plugged into the wall.
Ubicomp is Situated Computing
-
Makes use of simple shared context
- space
- time
- proximity
- affordances (Norman)
-
Participation in the context
- is physical
- is out here with us
- is in many small places, including trivial ones
Encouraging result: new science from exploring Ubicomp
- theoretical computer science: network security, caching over slow networks, …
- operating systems: scalable to wristwatches, user-extensible O.S.’s, reliable without redundancy, low power O.S.
- user interfaces, hardware and software: gestures, two-handed input, pie-menus, unistroke alphabets
- networking, hardware and software: radio, infrared, mobile protocols, inbuilding wireless LANs, multimedia protocols over varying bandwidth
- computer architecture, hardware and software: postit-note computers, low power O.S., multimedia pad computers
Some Ubi-Examples
-
Activity-based Information Retrieval
- like filing assistant for physical documents
- uses events, time, context, who
- tracks things by badge, and video shape
- just indexing, no “agent”
-
Physical Retrieval
- book or document beeps with answer
- screens (active, custom, signs) on walls direct you to right shelf, or right clothes, …
- Newman and Lamming, EuroPARC
Agent Premise?
- When is a program an agent?
- Many excellent talks in the morning stretched to cover agents
- Papert: syntonic and non-true/false schooled thinking
- Dertouzos: EForms, G.S.S.s
- Kay: handheld machines used everywhere
- Negroponte: shared context
- Is a human the right model for the ideal computer?
- An appealing panacea, and so dangerous
- Better: make the computer more like things of which we are unaware: eyeballs, hands, …
“I think of a personalized computer as something like a well-trained, long-standing English butler – someone intimately aware of your idiosyncrasies, your habits, your friends, your goals, and who you deal with.”
- You talk to it
- It watches us and learns our needs
- It has “knowledge”, is “aware”, or has a personality
- It is an assistant
Example of the different points of view: Pilot’s Assistant
The goal of the pilot’s assistant is to enhance the ability of the pilot to fly the airplane
-
Interface agent: like a co-pilot
- watching the planes systems and the situation
- offering advice, answering questions
-
Ubicomp: like a simpler, better airplane
- the arrangement of controls, displays, windows, seats, etc.
- the ability to act is enhanced by the total system
- no locus of expertise
-
For example: being alerted of a potential collision
- agent: “collision, collision, go right and down”
- ubicomp: background presentation of airspace information for continuous spatial awareness, as in everyday life. You’ll no more run into another airplane than you would try to walk through a wall.
-
Myth 1: voice recognition is important for human/computer interaction
- Better: voice may help a little, sometimes
-
voice recognition is to computer as typewriter is to paper
-
Myth 2: people know what they want, and a smart assistant could help them get it
- Better: people are opportunists, muddling through, never doing exactly the same thing twice
-
Myth 3: people interface with the world
- Better: people dwell in the world, so they and the world together are a functioning whole, neither alone.
- “User interface” embodies a type error: it names a boundary that is instead a union.
- Is marriage a spousal interface problem, improved by a better GUI? Is ecology a plant/animal interface problem, improved by plants with more MIPS?
-
Myth 4: hierarchical organization is helpful
- Better: success in leading comes from understanding, coaching and enabling, not commanding
- “Master molecule” theories (slime mold, DNA) are not true, instead more complex interaction theories, with no locus of control, are biologically realistic
- People at the top of hierarchies, e.g. presidents, have little control over the bureaucracy
-
You talk to it
- but what you can say is no different from what you can type: i.e. you lisp at it
- Myths 1 and 3
-
It watches us and learns our needs
- but has no human context for needs
- Myths 2 and 3
-
It has “knowledge”, is “aware”, has a personality
- must Artificial Intelligence be solved first?
- Myths 2 and 3
-
It is an assistant
-
It does not go far enough
- it stops at the notion of an assistant
- much better is the intuitive or anticipatory computer, that needs no commands
- aim for an extension of our body, or integration of mind/body/world
-
It keeps the computer in the foreground
- personal computer is the wrong idea, intimate computer even worse
- invisible computer is best
-
It stays within an old paradigm
- breakthroughs less likely, because area well studied
-
It obsessively fascinates
- the human-like machine to which we give life
- the perfect, all-powerful, slave
- be careful when appealing to ancient prejudice
Some different design principles
| Interface Agent |
Ubiquitous Computing |
| single locus of information about me |
distributed, partial information by place, time and situation |
| command the computer |
what computer? |
| personal, intimate, computer |
personal, intimate people |
| filtering |
breathing, living, strolling |
| user interface |
no boundary between you and machine |
| DWIM do what I mean
|
WIWYHIAFI when I want your help I’ll ask for it
|
| I interact with agent |
I interact with the world |
- A good butler is also invisible
- Invisibility is more poorly focused than agents, less amenable to PhD theses and point products
- People want a personal butler, not just a better life
- Ubiquitous computing is harder, because it requires complete new systems thinking
- An intelligent agent is a better model for some things, like information filtering and controlling access to me
Summary
- Ubiquitous computing avoids metaphors of hierarchy, power, and control, and myths about what it is to be human
- Ubiquitous computing emphasizes metaphors of life, interaction with other people, invisibility, and is leading to new discoveries in computer science
“Using a computer a computer should be as refreshing as taking a walk in the woods.”
Two kinds of issues
-
Desired end state
- invisibility vs. explicit interaction
-
Means for achieving either end state
- distributed affordances vs. focused expertise
Does Ubiquitous Computing Need Interface Agents?
July 27, 2025
Does Ubiquitous Computing Need Interface Agents?
Mark Weiser, Head of Computer Science Lab, Xerox PARC
Agents92.tioga, October 6, 1992
Ubiquitous Computing vs. Interface Agents
- something altogether different, even opposed, to interface agents: ubiquitous computing (also known as embodied virtuality)
- outline of this talk:
- what is ubiquitous computing
- compare and contrast to interface agents
Context of work: Xerox PARC
-
A different kind of research in two ways
- Reverse technology transfer: PARC transfers research ideas to universities to pursue.
- We deploy and use our discoveries.
-
Over twenty years of pioneering basic research
- Individuals change, innovation continues
- Steady Xerox commitment to long-range research
- Laboratories ranging from physics to anthropology, all working together.
-
Connection to Xerox
- PARC technology in nearly every product
- Research a vital part of The Document Company
This is a talk about metaphors and world views
-
All people see and understand the world as filtered through a world view
- The world view itself is not seen
- Because the world view filters everything, it is hugely influential on what we do
- Some metaphors become ingrained as world views
-
Metaphors are extremely influential
- Plato’s metaphors of ideal forms and knowledge as viewing shadows in a cave influencing 2,000 years of theories of knowing
- Rousseau’s “Noble Savage” influence in the French revolution, morals, cruel experiments on children
- The metaphor of the “master molecule” delayed biochemical understanding
People are most effective and authentic when they are fully engaged, mind and body, in the world
- Examples:
- flow of the athlete in the groove
- effortless use of pencil, paper and language when writing
- “feel for the organism” of the great biologist
- effortless 65 MPH driving of the experienced driver (while talking, reading roadsigns, …)
-
This is a basic characteristic of humans, across all cultures
- Polynesian navigation between islands depends upon attunement with currents, wind, and weather
- African tailor apprenticeships via peripheral participation depend upon learning by engagement
- Technologies should enhance this ability to engage, to “flow” with life and work
The most powerful technologies are invisible: they get out of the way to let the human be effective
-
Electricity
- Electric motors hidden everywhere (20-30 per car)
- Electric sockets in every wall and portably available through batteries
- Integrated, invisible infrastructure
-
Literary Technology
- Continuously surrounding us at many scales
- Used trivially and profoundly
- Literally visible, effectively invisible
How to do invisible computing?
-
Integrated computer systems approach
- invisible, everywhere, computing
- named “ubiquitous computing” in April 1989.
- Invisible: tiny, embedded, attachable, …
- Everywhere: wireless, dynamically configurable, remote access, adapting, …
Goals of ubiquitous computing (ubicomp)
-
Ultimate Goal
- invisible technology
- integration of virtual and physical worlds
- throughout desks, rooms, buildings, and life
- take the data out of information, leaving behind just an enhanced ability to act
-
Phase I
- tabs, pads, and boards
- hundreds of computers per person
- wireless networks
- location-based services
- shared meeting applications
Using a computer should be as refreshing as a walk in the woods.
ubicomp phase 1: Ubiquitous I/O devices
-
Post-it note-sized palmtop computers
- One hundred per person per office
- Always have one on you, wirelessly connected
- Small touch-sensitive display screen
- Scatter around the office like postit notes
-
Notebook-sized computers
- Ten per person per office
- Stylus-based input primary
- Near megabit wireless comm bandwidth
- Can support multi-media when “tethered”
-
Wall displays
- Large ones used as shared display surfaces (replaces whiteboards)
- Replace physical bulletin boards, etc.
- Lots of bandwidth available because they’re plugged into the wall.
Ubicomp is Situated Computing
-
Makes use of simple shared context
- space
- time
- proximity
- affordances (Norman)
-
Participation in the context
- is physical
- is out here with us
- is in many small places, including trivial ones
Encouraging result: new science from exploring Ubicomp
- theoretical computer science: network security, caching over slow networks, …
- operating systems: scalable to wristwatches, user-extensible O.S.’s, reliable without redundancy, low power O.S.
- user interfaces, hardware and software: gestures, two-handed input, pie-menus, unistroke alphabets
- networking, hardware and software: radio, infrared, mobile protocols, inbuilding wireless LANs, multimedia protocols over varying bandwidth
- computer architecture, hardware and software: postit-note computers, low power O.S., multimedia pad computers
Some Ubi-Examples
-
Activity-based Information Retrieval
- like filing assistant for physical documents
- uses events, time, context, who
- tracks things by badge, and video shape
- just indexing, no “agent”
-
Physical Retrieval
- book or document beeps with answer
- screens (active, custom, signs) on walls direct you to right shelf, or right clothes, …
- Newman and Lamming, EuroPARC
Agent Premise?
- When is a program an agent?
- Many excellent talks in the morning stretched to cover agents
- Papert: syntonic and non-true/false schooled thinking
- Dertouzos: EForms, G.S.S.s
- Kay: handheld machines used everywhere
- Negroponte: shared context
- Is a human the right model for the ideal computer?
- An appealing panacea, and so dangerous
- Better: make the computer more like things of which we are unaware: eyeballs, hands, …
“I think of a personalized computer as something like a well-trained, long-standing English butler – someone intimately aware of your idiosyncrasies, your habits, your friends, your goals, and who you deal with.”
- You talk to it
- It watches us and learns our needs
- It has “knowledge”, is “aware”, or has a personality
- It is an assistant
Example of the different points of view: Pilot’s Assistant
The goal of the pilot’s assistant is to enhance the ability of the pilot to fly the airplane
-
Interface agent: like a co-pilot
- watching the planes systems and the situation
- offering advice, answering questions
-
Ubicomp: like a simpler, better airplane
- the arrangement of controls, displays, windows, seats, etc.
- the ability to act is enhanced by the total system
- no locus of expertise
-
For example: being alerted of a potential collision
- agent: “collision, collision, go right and down”
- ubicomp: background presentation of airspace information for continuous spatial awareness, as in everyday life. You’ll no more run into another airplane than you would try to walk through a wall.
-
Myth 1: voice recognition is important for human/computer interaction
- Better: voice may help a little, sometimes
-
voice recognition is to computer as typewriter is to paper
-
Myth 2: people know what they want, and a smart assistant could help them get it
- Better: people are opportunists, muddling through, never doing exactly the same thing twice
-
Myth 3: people interface with the world
- Better: people dwell in the world, so they and the world together are a functioning whole, neither alone.
- “User interface” embodies a type error: it names a boundary that is instead a union.
- Is marriage a spousal interface problem, improved by a better GUI? Is ecology a plant/animal interface problem, improved by plants with more MIPS?
-
Myth 4: hierarchical organization is helpful
- Better: success in leading comes from understanding, coaching and enabling, not commanding
- “Master molecule” theories (slime mold, DNA) are not true, instead more complex interaction theories, with no locus of control, are biologically realistic
- People at the top of hierarchies, e.g. presidents, have little control over the bureaucracy
-
You talk to it
- but what you can say is no different from what you can type: i.e. you lisp at it
- Myths 1 and 3
-
It watches us and learns our needs
- but has no human context for needs
- Myths 2 and 3
-
It has “knowledge”, is “aware”, has a personality
- must Artificial Intelligence be solved first?
- Myths 2 and 3
-
It is an assistant
-
It does not go far enough
- it stops at the notion of an assistant
- much better is the intuitive or anticipatory computer, that needs no commands
- aim for an extension of our body, or integration of mind/body/world
-
It keeps the computer in the foreground
- personal computer is the wrong idea, intimate computer even worse
- invisible computer is best
-
It stays within an old paradigm
- breakthroughs less likely, because area well studied
-
It obsessively fascinates
- the human-like machine to which we give life
- the perfect, all-powerful, slave
- be careful when appealing to ancient prejudice
Some different design principles
| Interface Agent |
Ubiquitous Computing |
| single locus of information about me |
distributed, partial information by place, time and situation |
| command the computer |
what computer? |
| personal, intimate, computer |
personal, intimate people |
| filtering |
breathing, living, strolling |
| user interface |
no boundary between you and machine |
| DWIM do what I mean
|
WIWYHIAFI when I want your help I’ll ask for it
|
| I interact with agent |
I interact with the world |
- A good butler is also invisible
- Invisibility is more poorly focused than agents, less amenable to PhD theses and point products
- People want a personal butler, not just a better life
- Ubiquitous computing is harder, because it requires complete new systems thinking
- An intelligent agent is a better model for some things, like information filtering and controlling access to me
Summary
- Ubiquitous computing avoids metaphors of hierarchy, power, and control, and myths about what it is to be human
- Ubiquitous computing emphasizes metaphors of life, interaction with other people, invisibility, and is leading to new discoveries in computer science
“Using a computer a computer should be as refreshing as taking a walk in the woods.”
Two kinds of issues
-
Desired end state
- invisibility vs. explicit interaction
-
Means for achieving either end state
- distributed affordances vs. focused expertise