What We're Thinking About and Working On
Thoughts on learning, AI-assisted programming, and our research agenda
Vision
History of Computing Education
Seymour Papert developed constructionism as a theory of learning to support his vision that computers would permeate the lives of all people to augment their cognition and empower them to learn and solve problems through programming. Clearly, this has not happened. In fact, I’d argue that many of our attempts at inspiring students to explore computer programming has just reinforced the popular perception that computer programming is difficult and for nerds.
In high school, we make our computer science courses laughably simplistic and focus on arbitrary topics that are of no interest to the students nor the teachers: representing integers in binary, memorizing internet protocols, and learning rudimentary code syntax. Similarly, students (often non-CS majors) in introductory programming courses are bombarded with overly formal programming syntax and a vast array of scattered concepts that are seldom connected to what makes computer programming interesting and useful: solving problems and building cool things[1].
Many have speculated about why we have failed (not cared about?) Papert’s vision of using computers to support constructionism. However, I feel that our failure to broadly acculturate computer programming is primarily due to our failure to demonstrate the benefits of computer programming for augmenting human cognition. Indeed, I had no idea that the computer could be much more than a word processor and a means to completing a goal before I started frequently using AI. This is actually a pretty complex topic---that is, the balance between accessibility and self-reliance---that I will fully address in a future essay and will introduce in the next section.
Is Accessibility Always Positive?
I envision a world where every person—whether their interests lie in technology, farming, education, or medicine—feels empowered and excited to use computers to augment their cognition. This is precisely the objective of Papert Lab.
Although personal computers had been around since the 70s, the release of the Macintosh in 1984 made the computer more accessible to people who had previously been unable to interface with the device. By deliberately deciding to structure the Macintosh around an intuitive graphical user interface (GUI) instead of a command line interface (CLI), as had previously been the case, consumers were able to purchase and use the computer without having a background in computer programming. I need not argue that personal computers have since become ubiquitous in the home and office, and it takes little effort to see all of the ways in which non-computer programmers rely on computers everyday; document processing, browsing, communicating, applying, and now, interfacing with generative AI chatbots. Needless to say, increasing the accessibility of computing has been transformative to both programmers and non-programmers.
However, although the GUI made computing more accessible to more people, it also masked the divide between the haves (technical) and the have-nots (non-technical). What is particularly problematic now is that because non-technical people can still use computers and computer programs (thanks to the internet), non-technical people are not cognizant that they lack technical skills and are at a disadvantage; their deficiencies are unknown-unknowns.
I worry that AI may perpetuate and exacerbate this pattern, particularly because people who are proficient at using Claude Code/agentic coding tools already give someone a significant creative and productivity advantage over someone not using AI agents. The same phenomena that happened with the GUI (masking skill deficiencies) is happening now, whereby the foundation model companies are trying to make agentic coding more accessible (e.g. Claude Code -> Claude Cowork), but by doing so, the technical divide will once again be masked and the less-technical people will be left behind, since using the Claude Code CLI and knowing how to design specifications will likely continue to outperform chatbot interfaces that utilize algorithmic prompt optimization.
Reliance on Programmers; We are Being Programmed
Although the dissemination of computers into the lives of nearly all working professionals and students (in the Western world) has indeed been transformative and has empowered a large fraction of the global population, non-technical people have been dependent on the programs that computer programmers code. The alternative, I suppose, is spending hundreds of hours learning an arsenal of programming languages to design programs that improve or modify how they interface with the computer, but this requires an immense amount of foresight that many people do not possess. Let’s call this second group of people who design their own programs “end-user programmers.”
Neither of these options are particularly ideal, primarily because 1) creating and manipulating software empowers students to learn, experiment, and feel that they belong within a culture of designing computer programs to augment human activities; 2) learning a rigid programming language is challenging and unnatural for many people, discouraging those who would otherwise enjoy designing computer programs; 3) even when university students take an introductory programming course, as is the case in many undergraduate programs, they feel that what they’re learning (e.g. lists, loops, conditionals, object-oriented programming) is disconnected from the software systems that they use every day; 4) programmers and software engineers are often so far removed from the domain for which they are coding software, meaning that the programs they write often include inherent deficiencies that would only be obvious through a posteriori knowledge. This is one benefit of being an end-user programmer!
These four problems—GUIs inhibit constructionist learning and experimentation through program creation, learning programming language syntax is too rigid and discouraging, introductory CS courses feel disconnected from real programing, and the large gap between who experiences the problem and who implements the code—are the current subject of Papert Lab’s research on specification design.
AI-Assisted Programming Pedagogy
SWEs, but Not Really
Computer science researchers like to highlight that a recurrent theme in computer science is abstraction; as we have developed new programming languages, they have come to increasingly mirror natural language. As programming became more accessible, a growing number of students and professionals wanted to learn how to engineer computer programs as a means to enabling them to apply programming to their own profession or interest; again, we call these people end-user programmers (EUPs) (Ko, 2011).
One attribute of EUPs is that they can code without including the necessary components of a computer program that are required when it is widely distributed (e.g. redundancy, security). Large numbers of people wanted to use their own technical expertise in conjunction with an ability to build things through computer programming languages like Python and Java. An offshoot of EUPs are conversational programmers (Cunningham, 2021), who desire technical fluency but do not need to be able to generate complicated, production-level code.
Either way, for both EUPs and conversational programmers, building software tools has previously required programmers to learn a non-trivial amount of a(n) programming language(s), which is 1) too large a barrier for many people; 2) prevents the more important skill of designing a system due to the more tractable skill of memorizing and generating strings of formal syntax. In fact, even seemingly trivial tasks like writing an automated email script or crawling a web page---both easily describable in natural language---were obfuscated by unfamiliar code syntax.
So... Now What???
So naturally, with the advent of AI-assisted coding agents, (at this point, AI writes 100% of my code), I wonder how these tools 1) remove the barrier that code syntax has imposed on anyone trying to program a computer for a domain other than computer science; 2) change the learning objectives of introductory and intermediate computing courses, which contain large numbers of non-CS majors from other subjects. These two questions, and more saliently, their implications, are precisely the short-term guiding research questions that Papert Lab is actively investigating.
It turns out that much of the existing computer science curricula is incompatible with the skills needed for AI-assisted programming. We[2] suppose that it is vital that we teach people to specify the behavior of a system to control AI coding agents. We refer to this process as specification design, a term we developed after reviewing requirement engineering and design literature spanning the past 50 years for our most recent submission. We define specification design more concretely in the following section.
Specification Design
We define specification design as
The process of using a system’s structure to probe and iteratively refine specifications based on the behavior of the system.
Specification design involves using a scenario to iteratively probe the design of a system and articulate the implicit design decisions into language-based specifications. In a typical computer science course, even theoretical ones at universities with strong CS departments, students are usually given a well-defined problem and asked to implement it using code and test it using unit testing. These tests are often seemingly random that do not intentionally probe the code’s behavior. Even when students are asked to resolve edge cases, these problems are usually just poorly-specified problem descriptions that intentionally omit design decisions. This can be a helpful skill in some contexts, and indeed some work is being done to apply this to a GenAI context, but it is important to recognize that this is different from designing a specification that is not known by the designer and using scenarios to enlighten tacit information about the specification. It’s like the difference between simply playing hide and seek with a pre-determined set of rules vs. discovering and creating the rules as you play.
To reiterate: specification design refers to the skills that, when coupled with a minimum threshold of domain knowledge, enables a user or student to effectively engage with an AI programming agent. We are still discovering what all of these essential skills are through empirical studies, but our current theory is that specification design requires abstraction traversal, prototyping and testing different scenarios, and specification articulation. These skills are not usually learned in computer science courses, particularly within novice and intermediate ones, which is what we’re concerned with. Thus, we must investigate what precisely makes up specification design, which skills are more esoteric than others, and how we can design pedagogies that scaffold the acquisition of these skills.
How Undergraduates Probe Scenarios
One critical aspect of specification design is the ability to recognize the objects and their attributes embedded within a scenario, and to use these structural components to iteratively design and articulate the intended behavior of the system. Since this is a novel task, we feel that it is important to first understand how novices (undergraduate students with some prior programming experience) use scenarios to elicit specifications when designing a system. We do this through one-on-one participant interviews using a think-aloud protocol, whereby participants are asked to verbalize all of their actions. We also developed two in-house tools for conducting this study and the qualitative analysis, which have now been used by more than 15 participants and will be used for post hoc qualitative analysis. The objective of this study is to extend existing studies of scientific discovery and learning from anomalous data to this task, which will allow us to develop pedagogical paradigms that support novices in this aspect of specification design.
We claim that imbuing an intuition for engaging in specification design within novices and non-CS majors will address the four problems outlined in the previous sections. Particularly, we believe this will enable otherwise non-technical people to build and learn with computers and computer programs, make computer programming more like design than engineering, make intro CS courses more helpful across domains, and mitigate the gap between the end-user of a program and its designer (notice how we refer to the creator of programs as designers and not software engineers).
Thoughts on Learning Science
Theories of Learning
Our work is motivated by the belief that situated learning (which will be the topic of a future essay)---requiring the learner to acquire new skills as a means to completing a desired task---is critical in designing learning paradigms that engage a student and embed them within the culture that they’re learning.
Further, our research also embraces constructionism, whereby we believe that instead of the computer “being used to program the child,” we should investigate learning paradigms where the “child programs the computer,” which enables them both to interface with the computer as a standalone device and use the computer as a means of learning other concepts (Papert, 1980).
Particularly germane to our work, however, is our belief that the purpose of formal instruction is to guide the learner through a process of abstraction and assimilation (of seemingly unrelated concepts), which is only possible if a student has thoroughly explored the problem and solution spaces and made hypotheses about them before being guided through the process of abstraction.
The general purpose of Papert Lab is to explore the theoretical and practical implications of this belief, which is most obviously illustrated in the domain of computer science education; hence why our initial research is in this domain.
Concretizing These Theories
We have found that AI-assisted programming enables students to experiment with their own programs without having to learn a formal programming language before they learn abstract principles of computing, which may or may not be communicated through traditional programming languages. How the learner learns through abstraction and applies these principles will depend greatly on their domain interest; if the student is interested in security, networking, or low-level hardware, it probably makes sense for them to express their ideas in a formal language like C or Rust, just as any good mathematician uses formal mathematical notation when explaining new theories (although they often omit the formality during exploration and experimentation). However, if the domain does not require the use of a formal programming language---which will be the majority of all careers and hobbies---it likely makes more sense for the learner to express their intent through natural language or a medium that is more suitable to their domain (e.g. papers for researchers, Figma mockups for designers, etc.). Regardless, we argue that the common thread between designing computer programs to augment all of these domains is specification design, which is the subject of our ongoing research in computing education and human-computer interaction.
We feel that developing an inductive understanding of how novices engage in specification design for AI-assisted programming will allow us to derive more general principles within the learning and cognitive sciences, with the ultimate goal of using these findings to improve (a) how we learn, (b) artificial intelligence model architecture, (c) how we interface with computers and artificial intelligence.
While our research into abstract learning science principles is preliminary and under-defined in nature, we are beginning to develop a thesis surrounding how we learn through scenarios and experimentation. Further, we are exploring how a learner’s ability to develop the right scenarios and ask the right questions determines the development of their mental model for a given problem space. We map this onto previous research on scientific discovery. This theory is a direct reflection of our work with AI-assisted programming education, and we are hopeful about its realization, evaluation, and future implications in cognitive science literature and pedagogy design.
Agenda
Papert Lab recently submitted a paper on specification design to a leading computing education conference, and we plan to publish an additional four papers this summer that address some of the aforementioned topics. Looking forward, we plan to conduct larger user and cognitive task analysis studies for journal publications, as well as deployments in a large introductory computing course at a large public R1 university. Our research involves a combination of empirical, qualitative, quantitative, formative, and retrospective studies in conjunction with position papers and literature reviews.
If you’re curious to learn more about our specific research agenda, please don’t hesitate to reach
Papert Lab is an applied cognitive science research nonprofit that uses experiments to model how humans solve problems and explores the implications for designing computational mediums that augment memory, learning, and curiosity. We do this through designing intelligent tutoring systems and exploring new interaction paradigms between artificial intelligence systems and humans. If you’d like to support our mission, please consider donating.
All donations are tax-deductible and are directly used to support research expenditures.
Please email hudson[at]papertlab[dot]org about collaborating, venture philanthropy, and individual donations or grants.
Thanks to Emergent Ventures and Foundry10 for supporting this work through grants. Thanks to Tyler Cowen for reading a draft of this essay. Thanks to David Smith and Annapurna Vadaparty for your continued support and mentorship.
I am in no way arguing that memory is a useless skill; in fact, our discussion group explored mimetic mediums to augment memory just this weekend!↩︎
Because I’m now referring to the actions and principles of Papert Lab as a broader organization instead of my own observations, I will now shift the writing to use a collective subject.↩︎
 | A guest post by
|