🧭Steering LLM personality with vectors, not prompts

I’ve been spending sometime with representation engineering, a mech interp sub-field [?] that asks: what if you could find “directions” inside an LLM’s internal activations that correspond to personality/mood, and then nudge the model along those directions at inference time? Without fine-tuning/re-training, that is.

The foundational idea comes from the Representation Engineering paper (Zou et al., 2023), which showed that concepts like honesty, fairness, and harmlessness correspond to linear directions you can extract and manipulate. Theia Vogel made this very accessible with the repeng library and a wonderful blog post that walks through training vectors for things like happiness and creativity. More recently, Lu et al.’s the Assistant Axis mapped out a full “persona space” using 275 character archetypes, and found that the main dimension separating different personas captures a spectrum from “assistant-like” to “role-playing.”

Read more

Indulge me in a thought experiment: imagine concepts in your head as points scattered across a vast, invisible landscape. In this landscape, each concept has coordinates that describe its features. Now, apply this landscape framework to the concepts of fruits. Apples might occupy a region of marked by moderate sweetness, crisp texture, and reddish hues. Lemons would exist elsewhere, sharp-tasting, brightly colored, positioned far from apples on a landscape defined by taste, texture, and color. What feels intuitive to us - that apples are closer to pears than to lemons - becomes precise and measurable within this geometric framework.

It seems that concepts have an innate structure that enables day-to-day, seemingly trivial cognitive tasks possible: comparison, categorizing, analogizing.

This is precisely what cognitive scientist Peter Gärdenfors1 suggests: our mind organizes knowledge geometrically. Concepts are defined in multidimensional spaces composed of different dimensions or features - sweetness, color, texture, and much more - sometimes even unnamed. These shapes, constantly shifting and evolving as we learn from new experiences, are the cornerstones of our mind.

Dimensions: the Building Blocks

At the heart of Gärdenfors’s framework are “quality dimensions” – the basic ways in which we discern similarity and difference. Consider the dimensions of color: hue, saturation, and brightness. Or for sound: pitch and loudness. They are not necessarily scientific properties - and yet they are fundamental to our perceptual model of this world.

We don’t just do this with natural phenomena - we do it with everything, including ourselves. Take personality for example: we distill the messiness of human traits into dimensions like introversion–extroversion or openness. Tests like the Myers-Briggs or the Big Five plot you along a few chosen axes, locating you in a “personality space” where your position can be compared to others, as if mapping the psyche in coordinates.

Similarity, then, is defined by proximity within these maps; the closer two representations, the more alike they are perceived.

These dimensions coalesce into “domains”, coherent spaces dedicated to particular kinds of qualities. The color domain encompasses its related dimensions, distinct from, say, the domain of spatial relations. Within these domains, individual objects are located as points, their coordinates determined by their specific values along the constituent dimensions. Crucially, properties are not merely labels but are represented as regions within these domains. Gärdenfors argues that the “natural” properties that anchor our thought and language tend to occupy well-behaved, often convex, regions within these spaces. Imagine the property “red” as a defined area within the color space, encompassing the various shades we recognize as red.

This geometrical approach seems to have more explanatory power: it provides an intuitive account of how we judge similarity, a cornerstone of concept formation and categorization. Learning what constitutes a “bird”, for instance, can be seen as carving out a region in a multidimensional space defined by features like size, shape, color, and characteristic sounds. Furthermore, conceptual spaces offer a foundation for a cognitive semantics, where the meaning of words is rooted in these embodied, perceptual structures, potentially illuminating the mechanisms of metaphor as structured mappings between domains. This framework makes me hopeful for a more sophisticated artificial intelligence systems capable of nuanced perception, analogical reasoning, and a more human-like understanding of the world.

Neurobiological Foundation

Is this geometric framework just a convenient metaphor, or does the brain literally use geometric codes?

Interestingly, the theory of conceptual spaces finds neurobiological plausibility in the brain's inherent use of spatial organization and vector-like representations. Evidence from topographic maps in sensory cortices demonstrates that the brain maintains spatial relationships corresponding to stimulus features, suggesting domain-specific geometrical layouts2. In “the Organization of Learning”, Gallistel argues that the nervous system represents stimulus properties using vectors in anatomical spaces, where dimensions of these spaces correspond to stimulus dimensions. The tensor network theory of Pellionisz and Llinas also aligns with the idea of the brain as a "geometrical object" in its sensorimotor functions3.

Conceptual Spaces in Artificial Minds

But conceptual spaces aren't limited to biology. Artificial intelligence and language models also appear to “think” geometrically.

Google’s recent research4 offers an intriguing glimpse into how closely human brains and LLMs might be thinking alike - at least when it comes to language. In the study, researchers found that neural activity in the brain’s language centers during natural speech maps surprisingly well onto the internal representations used by LLMs. Essentially, when we process words and sentences, our brain seems to be navigating a kind of high-dimensional space that mirrors the one an AI uses to make sense of language.

This finding dovetails beautifully with the theory of conceptual spaces. If both our brains and LLMs organize language and meaning as positions in a geometric space—where distances between points reflect relationships and associations - then perhaps geometry really is essential to understanding our mind. It feels like a compelling convergence: humans and machines, each drawing mental maps of meaning, not with definitions or rules, but with spatial intuition.

That convergence isn't just intellectually satisfying - it may also be practically useful: instead of teaching AI every aspect of the physical world from scratch, we might just align their internal maps to our reality. The AI, in other words, has already sketched out a fairly accurate landscape—it just needs calibration.

Uncharted Maps

In the book, Conceptual Spaces, Peter Gärdenfors also argues that scientific breakthroughs often come not just from new data, but from the introduction of entirely new conceptual dimensions - new ways of slicing up reality. Just as adding “temperature” to our understanding of gases led to thermodynamics, expanding our conceptual framework is often what enables deeper explanations and predictions. Scientific terms are more than just labels - they represent the birth of new cognitive domains, the psychological equivalent of adding new axes to the space in which we understand the world.

LLMs are trained to learn to organize information in high-dimensional embedding spaces. While their internal dimensions aren’t directly interpretable like “color” or “mass,” they might function in a similar way - carving up the space of meaning along statistically derived lines. As the model clusters concepts and learns latent dimensions, it may be constructing its own internal domains, not unlike how science forms new fields. This raises a fascinating possibility: could LLMs, in exploring their own latent spaces, stumble upon novel conceptual structures - relationships or distinctions we haven’t yet formalized? While they lack grounding, understanding, or intention, their ability to generate new patterns and recombine ideas suggests they might be performing, in their own alien way, something not unlike the early stages of scientific discovery.

If LLMs organize knowledge geometrically - positioning meanings in high-dimensional space - then we’re not just training systems that talk. We’re building maps of human thought, drawn in coordinates. And once you have a map, you can do much more than describe the territory. You can edit it, expand it, explore it.

I found myself crawling on a web, high above the ground, sensing vibrations from other wriggling web-bound creatures. But I was still human. A human attending an art exhibition. A human in a colossal sphere. A human at The Shed in New York City, where people (for some reason) suspend themselves in midair on a giant net, and experience gusts of artificial wind.

As I stretched out further, I could tell which were from my friend sitting near me, or someone a bit further away. I closed my eyes and reopened them to check if my purely tactile judgement matched my visual read. As I got better at my little exercise, I feel the web had become my eyes, my ears, and in some inexplicable way, an integral part of me.

Internalization of the World

Studies1 show our mind has a mental map of the space around us within our reach: certain brain cells fire only when something enters this range. In addition to that, there are neural maps of the space occupied by the body, and the space beyond our reach.

This body schema of ours is not fixed - over time, it can be stretched with experience. experiments2 show monkeys trained to use rakes to reach food beyond their reach otherwise, the neurons that map the hand and arm, as well as the space around them, change their firing patterns to include the rake and the space it can reach. Basically, the monkey’s neural representation of its reach is changed by its tool use.

I spent half an hour in the art installation’s taut web. The Japanese macaques were trained for three weeks to use rakes. Many of us spend hours on a daily basis using the mouse, the screen. Have we yet grown new senses of embodiment in the virtual reality that is the ubiquitous computational interfaces?

In virtual reality, we can redraw our body schema to have infinite reach. We can all become spider people.

Externalization of the Mind

Our mind extends beyond internalizing the external world - it also externalizes internal processes. We often use tools like notes, sketches, and even language itself to offload memory and reasoning tasks.

Consider this thought experiment. Otto, who relies on a notebook due to memory impairment, and Inga, with normal memory. When meeting at MOMA, Otto checks his notebook while Inga recalls from memory.

Andy Clark and David Chalmers argue3 their cognition is equivalent, suggesting the mind uses the most efficient method, whether internal or external, as long as the function is fulfilled. However, in the real world, would you prefer Otto's notebook method or Inga's recall for accessing information?

What’s missing in Clark and Chalmers’ rather functionalist view of the coupling between the mind and the tool is the phenomenology: how does using the tool actually feel? Is it convenient or cumbersome? Despite achieving virtual the same result, the experience of accessing information may differ significantly, thus impacting our choices.

When I have an idea I want to revisit in the future, my first instinct is to jot it down on paper. This "instinct" is actually my brain's quick calculation of the action's cost versus its likely future benefit. In the past, I relied on my memory, which often failed me. Interestingly, the perceived inconvenience of typing on a keyboard or screen still outweighs that of writing on paper. When choosing between flipping through a notebook and recalling from memory, our mind is economical with cognitive costs yet indifferent to the medium.

So, extending our mind with technology is possible: we just need to pitch a cognitively cheaper technology to our brain so that will become the default route.

From Technology to Mindware

Even the default routes might not be the best one that’ll ever be. A notebook is a great technology but not a great piece of mindware. To operate with the mind, a mindware needs to run at the mind’s speed, and feels like a natural extension of the body. The mind’s speed means close coordination and tight temporal coupling4 between the external interaction and the internal process. To find certain information in a notebook, we take it out, flip through its pages, read and search. From the moment we want that information to the moment we find it, it’s way longer than recalling from memory, which takes place typically within milliseconds.

Similarly, the mouse is a good technology, so good it’s still one of the dominant ways we interact with a computer. Yet it is still not a mindware since it doesn’t feel completely like a part of the body: it requires a flat surface, limits movement to 2D, and creates a disconnect between more nuanced hand motion and on-screen actions.

If these are not mindwares, then what is?

Speech for internal dialogues. Sketching (as opposed to looking up in a notebook, which involves physically flipping pages and visual search). We can certainly do more, with gestures, in an embodied and intuitive way.

Imagine swiping through holographic menus, resizing 3D models with a pinch, or manipulating data visualizations with natural hand movements – all without an input device as a middle person between human and computer. All of this is possible - and is happening - through machine learning models that recognize our gestures, as well AR/VR that weaves the space our gestures act on.

The Power of Externalizing Thoughts

Sometimes, putting our thoughts into the outside world can be more helpful than we might think at first. For example, when we sketch something we know well or play with a physical model, we might grow a novel way to interpret it. Or think when playing Scrabble, people tend to shuffle the tiles to prompt word recall, instead of simply doing that inside their head.

In one study5, researchers looked at how people understand ambiguous images (like pictures that can be seen in two different ways, such as the Duck/Rabbit illusion or the Necker Cube). When people were asked to imagine these tricky pictures in their minds, they couldn't see the different possible interpretations, failing to see the ambiguity. But when they drew the pictures themselves, they could suddenly see both interpretations. This suggests that the pictures in our minds tend to stick with one interpretation, while drawings we make can help us see things in new ways.

AI-enabled Mindwares

An interesting thing to ask is, what type of mindware we can make to externalize thoughts?

We are certainly not short of tools to do this (think sketches, diagrams, writing) - but can they be faster? More tightly coupled with our thinking processes? The way speech is coupled with internal dialogues?

To achieve this speed, we need automation. Speech to text only records existing, organized thoughts - for the first time in history we have a tool at our disposal that somehow managed to detect not just patterns on a word-by-word level, but layers of interpretable meanings, taking the forms of embeddings and features. More interestingly, there are ways to reduce the mindboggling number of dimensions to what we are used to - 2D or 3D. We can construct a thinking space from a space that is already enriched with our patterns of meaning, hence is capable of representing our thoughts in a way that makes sense to us. The space is fluid, ready to learn new things and be molded as we think with them.

Imagine:

1. A dynamic concept graph consisting of nodes, each representing an idea, and edges showing the hierarchical structure among them.

   • LLMs generates the hierarchical structure automatically but the structure is editable through our gestures as we see fit

   • attract and repulse in force between nodes reflect the proximity of the ideas they contain

   • nodes can be merged, split, grouped to generate new ideas

2. A data landscape where we can navigate on various scales (micro- and macro views).

   • each data entry turns into a landform or structure, with its physical properties (size, color, elevation, .etc) mirroring its attributes

   • apply sort, group, filter on data entries to reshape the landscape and look for patterns

   …

   I will dive more and showcase some prototypes in part 2.

Next & Part 2: Prototyping A New Species of Interfaces

The fluid mind stretches far beyond our biological brain. From feeling vibrations in a spider's web to exploring data with gestures, we have always relentlessly, uncontrollably integrating tools as extensions of our cognitive processes. Mindware - technologies operating at thought-speed - further blurs internal cognition and external representation. LLM-enabled interfaces, coupled with embodied, gestural controls, offer a glimpse of a future where thoughts become tangible entities.

Our journey into the realm of extended cognition is only starting. In part 2, I’ll showcase a series of experimental prototypes of how these ideas could inspire a new species of interfaces materializing thoughts into tangible, explorable entities.

Nodal points are the search results and our interests are the search algorithm. Like other computation my cognition performs, I sense the usefulness of the search and yet have a hard time describing it. A list is always a good first try:

• systems that displays open-endedness, or emergent behaviors: evolution, human and machinic creativity, artificial life;

• artworks that are complex systems themselves, a continuity between a variety of intelligent life forms, inanimate objects and technologies;

• tools to think about and play around with complex systems; to think the unthinkable.

I guess a running theme is “systems”. Especially those that evolve, learn, adapt; those that surprise me; those that seem to almost have their own mind.

I strive to learn more about this type of systems (as a researcher), and create them (as an artist and engineer). I want to gaze at them and be fascinated. One can be fascinated by one’s creation, right?

“Nodal Points Digest” is the type of posts that I started writing mostly for myself: it is a liminal space between the raw, unprocessed Twitter/Instagram/Are.na bookmarks and the final works they inspire. Writing about the nodal points forces me to understand better not just them, but also why I’m drawn to them. This newsletter, a space that I grow and share my nodal points and, hopefully, the fruits they bear, could potentially be one of your nodal points. Just as yours could be one of mine. A constellation of minds connecting.

I am warmed by this vision.

Deeper AI Art

I’ve been slowly catching up this series: MIT Lectures: Deep Learning for Art, Aesthetics, and Creativity

In one lecture Joel Simon talks about how, as an aspiring sculptor, he picked up a 3D modeling software to do digital sculpting and yet it didn’t feel quite right:

Conceptually something was wrong. And I realized these are digital attempts to mimic something analog. But they weren’t truly computational. It wasn’t sculpting in the medium of computer.

Then it struck me: prompted image generation, which represents most of the AI art we see, isn’t really creating in the medium of AI, either.

This series of talks are from people trying to explore AI’s own medium.

Take Joel Simon’s “Dimensions of Dialogue” for example. Two neural networks are challenged to communicate with each other by inventing new writing systems. It is a problem already solved by humans independently, in various times and ways. How would artificial neural networks approach this problem? What are some invariants between these alphabets that shed some light on the essential traits of why languages work?

We can also venture into the space of qualia construed by our linguistic structure. In “the Latent Space of Colors”, I extracted all the colors listed on Wikipedia and obtained each color’s embeddings through one of the OpenAI models, making a semantic color space purely from human languages. It seems to support the claim that large language models encode not just relational information, but also those relations that are grounded in our perceptual space.

AI is the most interesting, complicated tool we’ve ever invented, the closest thing we’ve ever made to a human mind. And we are at a time that it’s at our liberty to peek deeper into the layers of neurons in the architecture, to spin up instances of minds and do fantastical philosophical experiments that previously are only "thinkable” - that’s wild.

Different Forms of Cognition

Greg Egan’s new book, “Morphotrophic”, imagines a world where the relationship between us and our cells is much more keenly felt: a girl lost her limbs after some of her “cytes” relinquished their allegiance by melting away from her body; regular body “swapping” practitioners shuffle their cytes hoping that can keep them strong and ever flourishing; cytes (not neurons) can contain a whole person’s consciousness.

In acknowledgement, Egan cites Michael Levin as one major inspiration: “it should be possible to induce the formation of metazoan-like bodies in an otherwise unicellular organism…Similarly, it may be possible to induce dissolution of an entire metazoan body by appropriate changes of bioelectric dynamics.”1

Indeed, “Morphotrophic” reads like a speculation of Levin’s research where cellular intelligence is acknowledged, albeit still yet to be fully understood, and harnessed to create outcomes that human have long been dreaming of: regeneration, longevity, even shapeshifting. In Levin’s works, ample examples of regeneration in real life from other species: flatworms that can regenerate and grown into an anatomically complete worm from any parts; through metamorphosis, caterpillars dismantle their brains but still retain prior behavioral memories as butterflies. Stories like this stretch our definition of conventional intelligence: brainless, neuronless, just cells and their networks display cognitive traits like goal-directed actions and memory.

In another paper “Biology, Buddhism, and AI: Care as the Driver of Intelligence”, Levin et al. propose a frame work of intelligences that liberate the agents from biology-based evolutionary lineage. My belief is being able to recognize intelligence in various forms will be one of the biggest progresses we make. To simulate a system where people can experiment with truly extending their definition of intelligence, I made the installation “The ${self} Is A Computational Boundary”: virtual creatures, driven by other humans, AI and simulated collective intelligence, surrounds the viewer. The viewer observes, guesses and responds to the ecosphere with their body movement.

LLMs as Borgesian Fiction Machines

In his paper “Borges and AI”, Bottou offers an alternative perspective on LLMs to treating them as sentient and intellectual threats: they are fiction machines that do not concern either intention or truth, only narrative necessity:

The ability to recognize the demands of a narrative is a flavour of knowledge distinct from the truth. Although the machine must know what makes sense in the world of the developing story, what is true in the world of the story need not be true in our world. Is Juliet a teenage heroine or your cat-loving neighbour? Does Sherlock Holmes live on Baker Street? [8] As new words are printed on the tape, the story takes new turns, borrowing facts from the training data (not always true) and filling the gaps with plausible inventions (not always false). What the language model specialists sometimes call hallucinations are just confabulations.

Ok, so what do we do with fiction machines? What’s the use of stories any way?

As children we start our learning through fairy tales; in school we learn through stories lessons of every subject, including science. We receive our news through stories. We tell stories with our family and friends. Peter Winston, a student of Minsky’s, even goes as far as arguing storytelling is the one thing that separates human intelligence from that of other primates.

We even tell stories to ourselves: our inner dialogues are constant conversations within our mind, taking on the form of self-reflection, rehearsing future events or rehashing past ones.

LLMs weave living fictions that can be actualized.

A few thoughts for experiments:

• prompts for cognitive reframing, a psychological technique that identifies and changes how someone views the world around them - this could be augmented by LLMs. There is already evidence that LLMs made better reappraisals than humans when reframing negative scenarios;

• collective lore-making: a lore is constantly being written and rewritten by LLMs while members of the lore community decide which part to take;

• or, just for the sheer fun, forks out as many as possible paths from a single point and lets your mind be constantly bombarded by the infinity of the possible worlds.