Projects

Current CMAW projects fall within one or both of the laboratory's two broad research areas described on the Research Overview page. Projects below are tagged to indicate which research areas they address:

Foundations

Indicates that a project relates to Foundations of Computational Minds. Such projects develop mathematical accounts of how physical systems, computation, and experience may be related.

Qualia Optimization

Indicates that a project relates to Qualia Optimization. Such projects study how AI systems might take candidate accounts of agent well-being into consideration alongside task performance.

IN PROGRESS

Marks projects that do not yet have a completed public paper.

Current work

First page of the Qualia Optimization technical report

Foundations Qualia Optimization

Philip S. Thomas · 2025

Qualia Optimization

This technical report introduced qualia optimization: the problem of considering a candidate measure of an agent's experiential valence alongside task performance. It also proposed representation robustness as a desirable property of candidate valence measures. Representation robustness requires that changing only the representation of a computation does not change the experience attributed to the system. For example, two computers can implement the same abstract computation while using most-significant-bit-first and least-significant-bit-first conventions. Under a computational theory of mind, they should not be assigned different experiences merely because they use different encodings internally.

The report shows that natural measures based on total lifetime reward and cumulative temporal-difference error do not have this property. The same hardware and software can be described using different numerical encodings that assign different values to rewards or TD errors without changing the underlying physical process. These measures can therefore assign different valence to the same physical system solely because its internal quantities are interpreted differently. That conflicts with the physicalist and computationalist assumptions motivating the analysis.

As a possible alternative, the report examines whether the reinforcement of behavior, rather than the magnitude of an internal signal, might be associated with positive valence, and whether inhibition might be associated with negative valence. Likelihood ratios measuring how learning changes the probabilities of past actions provide representation-robust candidates in certain policy-based settings. The report also identifies unresolved issues, including where to draw the agent-environment boundary and how to define reinforcement when an agent does not separate neatly into policy parameters, contextual memory, and distinct learning and acting phases.

Technical report PDF

Plots comparing a reinforcement-based candidate objective and task performance for PPO and SAC

Qualia Optimization In progress

Derek Lacy and Alexandra Burushkina

Steering PPO and SAC toward reinforcement

This project generalizes the reinforcement-based candidate q from the original Qualia Optimization report to policy-gradient algorithms that update from batches of experience. It asks whether standard learning algorithms can be modified to favor updates that reinforce behavior while characterizing the resulting trade-off with task performance.

For PPO, the project changes the policy objective directly so that a policy update favors reinforcement more strongly. For SAC, it introduces rumination correction, which changes which stored experiences are replayed or how strongly they influence learning. The connection comes from computational psychiatry, where rumination is modeled as repeated or biased sampling of negatively valenced memories. In an RL replay buffer, repeatedly emphasizing transitions associated with inhibition may create an analogous bias in learning.

The experiments illustrate that the mechanism matters. Direct changes to PPO produce comparatively smooth and tunable trade-offs between the reinforcement-based candidate objective and task performance. Replay-based interventions in SAC are less predictable: their effect varies across environments and choices about how past experiences are prioritized.

PPO code SAC code

Foundations In progress

Yiying Zhong

Mathematical Formulation of Computation

Physicalism motivates treating phenomenal experience as determined by a complete physical history φ. A computational theory of mind adds a further claim: this dependence can be factored through the computational content implemented by that history. One candidate decomposition is φf⟶ch⟶e, where c is computational content and e is experience.

The project seeks mathematical formulations of two basic desired properties. First, the account should be non-degenerate: at least two physical systems can have different experiences. Second, changes of representation that leave the implemented computation unchanged should not change the predicted experience. A motivating example is two computers using different bit-order conventions while implementing the same computation.

Several natural ways of formalizing the second property conflict with the first or admit functions that satisfy the formal condition without capturing computational content. These results suggest that the relevant account may need more than the realized physical history alone. Candidate additions include the distribution of physical histories that could have occurred and the causal or counterfactual organization of the process (as proposed by Chalmers).

Abstract design representing learning, disruption, and recovery

Qualia Optimization In progress

Elisabeth Chkheidze

Valence, learning, and recoverability

This preliminary project asks whether the valence of a learning event should depend not only on whether the immediate outcome was good or bad, but also on what the agent learned and whether the event left it able to recover. It explores candidate signals based on how a policy update changes expected future outcomes, together with separate terms for recoverability and harm.