Introduction

As artificial intelligence (AI) continues to advance at a breakneck pace, particularly with the rise of large language models (LLMs...]]>

Introduction

As artificial intelligence (AI) continues to advance at a breakneck pace, particularly with the rise of large language models (LLMs), it is imperative to ensure that these systems are truthful, trustworthy, accountable, and robust. Without such properties, current LLM-style AI systems will struggle to move beyond their roles as sophisticated search engines and writing assistants, hindering their ability to autonomously perform tasks such as acting as web services or agents.

Enter the Semantic Web, a long-standing vision of a machine-readable web that enables agents to understand and interact with data in a meaningful way (Berners-Lee et al., 2001). To learn about the original architectural vision of the Semantic Web, see this design issue. In this blog post, we'll explore how Semantic Web technologies can be leveraged to build better AI systems that are more reliable, accountable, and aligned with human values.

Grounding LLMs with Knowledge Graphs

One of the key challenges with LLMs is ensuring the quality and truthfulness of their responses. By grounding LLMs with high-quality data represented in a knowledge graph (KG), we can significantly improve the accuracy and reliability of their outputs. Knowledge graphs, a core component of the Semantic Web, provide a structured and interconnected representation of data, allowing LLMs to reason over and draw insights from this information. Industry is rapidly adopting this approach, with companies like Ontotext and Neo4J offering out-of-the-box solutions for performing Retrieval Augmented Generation (RAG) with Knowledge Graphs. The interest in grounding LLMs using information from Knowledge Graphs extends beyond companies run by Semantic Web enthusiasts. Andrew Ng, a prominent figure in the AI community, has co-developed a course on RAG with Knowledge Graphs. Furthermore, during Langchain's Memory Hackathon, 30% of the participating teams sought to implement knowledge graphs in their architectures, highlighting the growing recognition of their potential in enhancing LLM performance.

Protecting User Privacy and Ensuring Ethical Data Usage

As AI systems increasingly rely on user and enterprise data to function, it's crucial to ensure that this data is used ethically and in compliance with legal governance frameworks. Semantic Web, and satellite technologies such as Solid offer strategies for attaching access controls, and usage agreements to data in order to support responsible data usage.

Towards Trustworthy and Accountable Personal AI Agents

The Semantic Web has long envisioned a future where autonomous AI agents work on behalf of users, assisting them with various tasks and decision-making processes. By leveraging Semantic Web technologies, we can build the groundwork protocols that enable conversational agents to negotiate and transact over the web which contain features such as:

1. Mechanism for describing the origin and provenance of exchanged data
2. Mechanisms to determine the "trustworthiness" of data by modelling which sources / individuals / organisations are reliable, and which are not.
3. Unambiguous description of usage restrictions on exchanged data to protect privacy allowing conversational LLM-agents can become more trustworthy, accountable, and compliant with data protection regulations such as GDPR.
4. Clear definition of dialogue outcomes that require agreement or transaction
5. The ability to discover AI representitives of individuals & organisations using Web Identities.

More on this topic soon … in the meantime, here is my article on machine dialogues

Conclusion

The Semantic Web, with its rich history and powerful technologies, holds the key to building better AI systems in the age of LLMs. By grounding LLMs with knowledge graphs, ensuring ethical data usage through explicit policies, and laying the foundation for trustworthy personal AI agents, the Semantic Web complements and enhances the capabilities of modern AI. As we continue to push the boundaries of what's possible with AI, it's essential to look to the past and leverage the insights and technologies developed by the Semantic Web community. By combining the old and the new, we can create AI systems that are not only powerful but also accountable, transparent, and aligned with human values.

Identifying who is best to work on a project in academic

There are...]]> Early draft, rant post stage, written in 2min

Identifying who is best to work on a project in academic

There are increasingly many disparate groups doing many different things. There are perhaps different groups that should be turned to for:

1. Helping solve an industry challenge in the medium term
2. Working towards a better long term solution

And the best for either are usually not easily identifiable

Aligning understanding when multiple members of a group have different backgrounds

Very much per title

Beaurocratic Admin

This is a social problem, not a technical one

We are about to create another cookie problem :cookie: - with Digital Credentials in Data Wallets.

My previous article on

My previous article on Data Wallets discussed what Verifiable Credentials were - and where they fit into regulatory frameworks such as eIDAS in Europe and the proposed Digital Verification Scheme (DVS) in the UK.

Such credentials can contain sensitive personal information, that often requires the verifier to establish a legal basis upon which they can process the data - such as user consent. Within the W3C Verifiable Credentials Specification this is recognized, and a termsOfUse entry is supported to support in precisely describing the permissions (e.g. Service Personalization) for which data may be used, what usage of data within the credential is prohibited (e.g. Marketing) and any obligations that must be met when using the data (e.g. report any 3rd parties this data is shared with). The formal language often used to describe these permissions, prohibitions and obligations is known as the Open Digital Rights Language (ODRL).

However, without the legal infrastructure to recognize the inclusion of this termsOfUse entry as a valid way of creating a legal basis for data processing - such as being a valid way of proving consent; this entry is relegated to merely being a descriptor of the basis' for processing that have already been established. How are these bases' established? Likely, the mechanism that we know and love - a pop-up consent box, every time you use the credential¹!

As I discussed in my previous article the Digital Verification Scheme is proposed within the UK's Data (use and access) Bill. The DVS provides a mechanism for the Secretary of State to create a list of certified service providers to provide services related to credential issuance, management and verification.

The Digital Identity and Attributes Framework (DIATF) is a framework that explicitly describes the different types of service providers that can be implemented under the DVS - including technical roles and responsibilities.

If we are specifically talking about the Digital Identity and Attributes Framework (DIATF) in the UK, it could take the form of terms of use term within Verifiable Credentials. Specifically in the context of DIATF what regulators could do is state that when a Verifiable Credential is sent from a Holder:

1. The Holder is legally obliged to use pre-defined user preferences or ask directly from the user, the privacy terms they wish to apply to the data. These would likely be described as a set of permissions, obligations, and prohibitions within an ODRL Rule
2. The service receiving the data is legally obliged to adhere to the stated privacy preferences.

In most cases - the rules would largely enumerate the DPV Purposes for which the user consents their data to be used.

here instead.

It is my view that we are nearing a point in the research landscape where papers, pate...]]> EARLY STAGE DRAFT

It is my view that we are nearing a point in the research landscape where papers, patents, standards documents, blogs and other long-form documents describing a dense set of ideas are nearing their end-of-life as the best mechanism for describing knowledge.

Other elements to observe:

• When external sources are not easily referencable, a full story needs to be told up-front. The story whether in a paper or book, is not personalised to an individual consumer - but instead designed to be digestible by a wide audience of readers.
• When external sources are more referenceable - such as on the Web where hyperlinks can be used. Authors have a level of liberty to create shorter documents and link users to external sources which they may consume at their own discretion.
• When explaining something to someone in a conversation, I will piece together ideas that I have or know; but tailor that response to the person I am speaking to - for instance, based on how much they know about a particular topic - which informs what terms I can use vs. what terms I need to define in the process.

Roadmap[^1]:

• Structured knowledge injection and production via RAG in LLMs
• Model memory with some level of understanding of concepts (similar to LCMs)
• Models actually working with concepts, and generating:
• Formal commitments where possible
• Indexes to point to databases containing knowledge required (i.e. look over there for a dataset of patient records that you need to answer this question).
• Models able to actively identify which concepts they are operating with when performing inference, and able to pull in more information about that concept on-demand[^2].

Similarly, my view is that we should be developing such capabilities in order to enhance collaboration and consensus.

Near term possibilities for collaboration include:

• Helping identify when multiple groups of people have a need for the same (e.g. software) infrastructure across multiple projects - and support pooling of resources to create one thing that can be re-used across projects. Note that this comes with the potential risk of loss in hetrogeneity in software - which comes with its own set of downsides and risks.
• Distribution of shared next steps - identify when multiple groups have overlapping sets of tasks they want to achieve in the future and support them in distributing the workload. This is a very real problem that can be seen in the way that e.g. academic grants are currently managed wherein multiple institutions often try to outcompete each other to present a better grant proposal on similar hot topics of research in order to get funding.
• Real-time matching of individuals with given skillsets to tasks that need to be done in a project.
• There is a statistic that if you have $n$ people in an organisation than $\sqrt{n}$ of the people to $n$ of the work. It would be interesting to see if something like this could be used to induce a more 1:1 scaling - or at least use such metrics to evaluate the efficacy of this idea.

Near term possibilities for consensus include:

• Harmonisation of standards

• Topic based consensus amongst large groups - individuals discuss their needs, wants and priorities with their model which goes off to represent their views. This would enable democracy at the level of issues rather than elected representatives. This is crucial since top-down governance results in lack of granular awareness of issues faced by many people - and thus is not able to make the most effective decisions possible - or even make decisions that contradict public desires^3. A further indication that this is a sensible path to pursue is in those countries have demonstrated the effectiveness of citizen-led budget planning[^4].

[^1]: Note there is overlap with ideas in work/literature on interpretability of models. [^2]: Note that one driver to try and build such architectures is that this would enable core models to be much smaller and to fetch relevant knowledge as needed on demand. This may also remove the need for knowledge cut-off dates if any new information published can quickly be pre-processed into concepts. This would also enable the use of private and other transactional data on-demand without needing to use RAG techniques. [^4]: Discussed in Humankind: A Hopeful History - I owe a page reference.

Abstractly, an agent is "a computer system that is situated in some environment, and that is capable of autonomous action in this environm...]]>

Abstractly, an agent is "a computer system that is situated in some environment, and that is capable of autonomous action in this environment in order to meet its design objective" (Wooldridge, 2009, p. 15; Wooldridge, 1995). Just as humans must negotiate and cooperate to achieve shared goals, so too must agents within multi-agent systems (Wooldridge, 2009, p. 24-25). This is only possible if agents have an effective means of communication. To communicate they must have a (R1) shared conceptual understanding of the topic on which they communicate (e.g. two agents can only communicate about weather if they both 'know' what it means to be sunny, raining and overcast, and have a notion of temperature and a measure - such as degree celsius - by which to describe it) and (R2) a means of sender (resp. reciever) encoding (resp. decoding) messages to transmit these concepts between agents. We shall refer to the requirement that agents are "capable of autonomous action [execution]" (Wooldridge, 2009, p. 15) as (R1E).

Since the emergence of autonomous agents in the 1980's (Wooldridge, 2009, p. 304) there have been numerous developments in the way in which agents understand (R1) and transmit (R2) concepts.

First, came systems where human programmers would encode specific execution semantics in code (R1E) for instance the "concept of temperature" would be defined by a function which takes a reading from a temperature sensor and these systems would communicate (R2) using a structured encoding such as sending JSON encoded objects over the Web - containing a "temperature" key. In this paradigm, the conceptual understanding (e.g. of temperature) (R1) did not fully live within the system, but was rather implicitly communicated between systems in the form of API documentation where the developers implementing each system would explain to each other "the temperature key in the JSON object is a 32bit floating point value representing the temperature, in central London, as measured in degrees celsius". Naturally, these primitive agents had very limited versatility as human developers were required to define the execution semantics and conceptual semantics (documentation) by one exact very precise concept at a time. Moreover, these agents, which are still widespread in the form of Web APIs, face an interoperability problem - as there are countless Web APIs all of which have a "temperature" key in the JSON object they return; but with different meanings due to the implicit semantics of the metric, collection location, collection date etc. encoded in the API documentation. This tightly binds/couples the client-agent to communicate with a single server agent and to communicate with more "agents" the developer must manually write code to decode the different encodings of different server agents and align the conceptual understanding across different documentation.

Subsequent developments saw attempts to migrate this conceptual understanding (R1) from documentation to data which is sent between agents (Verborgh et al., 2013, Verborgh, 2013, Verborgh, 2021, webservices). In domains such as the Semantic Web, conceptual understanding is described symbolically using ontologies (Wooldridge, 2009, p. 180) and rules (Wooldridge, 1991, verborgh2015drawing) (R1) - and encoded in highly generic RDF syntaxes (R2). With these agents (systems) possessing a symbolic understanding of concepts, they are able to precisely describe and reason about the information they send and receive. This increases agent interoperability and portability by making explicit the semantics that are often implicitly encoded in API documentation; moreover, agent versatility increases with the ability to describe requests and responses on the fly without being constrained to use the finite set of concepts listed in API documentation. However, ontologists are still required to build the vocabularies and rules for concept description (e.g. someone must manually define "degrees celsius" as a "unit of measure" for "temperature"; someone must provide the rules that define the mapping between "degrees celsius" and "fahrenheit" so that systems using two different measures can interoperate) (R1) and developers are still required to write code that defines the execution semantics (R1E) that is, a system can semantically formulate the question "what is the current temperature in London in degrees Celsius"; but a developer is then still required to write the code that then fetches the temperature from the sensor. Furthermore, in this paradigm "If two agents are to communicate about some domain, then it is necessary for them to agree on the terminology that they use to describe this domain." (Wooldridge, 2009), often requiring agents to share the same vocabularies and hence "worldviews".

Now LLM-powered agents are emerging, with many using LLMs trained upon a textual corpus containing a vast array of human knowledge. Consequently, these agents have been demonstrating a greater breadth and depth of conceptual understanding (R1) than human developers could imagine encoding by hand in formal ontologies. Moreover, LLMs can encode and decode these concepts in natural language (as well as, increasingly, machine syntaxes) (R2). Conversational LLMs also possess inherent execution semantics (R1E) with the capability to formulate a written response to any input which they are given. These execution semantics, however, do not extend to access of system-level knowledge or resources (such as temperature sensors) and are less reliable/deterministic than production-ready implementations of previous generations of agents. The trade-off of the breadth and depth of LLM understanding; is an increased vagueness of concepts and the presence of internal inconsistencies in LLM conceptualisations - moreover, there is no single worldview, per se, that the language model holds; and, just as with humans, the facts that an LLM claims are highly dependent on the context of the conversation in which they occur.

I've spent the last few days using Draft / mood-post-written-in-10-minutes

I've spent the last few days using OpenAI's Deep Research agent with ChatGPT 4.5. I'm quite impressed with the results it produces; and definitely reduces what would have previously been a few hours of browsing - to a 10-minute coffee break whilst I wait for the Researcher Agent to do its thing.

At the same time - it exacerbates the following challenges of doing research on the Web:

• The most advertised information is that which features in the result; but that doesn't necessarily mean it is the best or most factually correct information
• Unrelated information creeps in under the guise of an answer to the query. A good example of where I saw this is in this discussion of Verifiable Credentials where the model included a lengthy discussion of self sovereign identity and public key infrastructure such as KERI. Whilst this often infrastructure often appears in solution architectures with Verifiable Credentials - it was certainly not appropriate to have an entry for KERI in a table comparing different Verifiable Credential Standards.

I expect (hope) we will see a number of iterative improvements over the next view months to try and:

• where applicable, prioritize academic / authorotative sources; such as those found on Google Scholar and standards documents
• filter out "derived sources"; for instance blog posts and news articles that are just summarizing an article or publication that the model can access
• filter out sources generated by GenAI

As I briefly touched upon in this post; my view is that we can go much further than this by curating a Web of precise and trusted data - where possible using formal semantics, such as in a (RDF) Knowledge Graph. This seems sound given that:

• Graphrag is still increasing popularity for grounding models; including to ground Gemini's answers in google queries, and
• LLMs are getting better at supporting text to structured data conversions

Beyond soundness - a couple of reasons this is necessary include:

• We can be clearer about the semantic commitments we're making, and
• We can search for data rather than pages

Since the full p...]]> I recently collaborated on a paper with the Cambridge Service Alliance - for which I wrote a lengthy piece asking "What does it mean for a system to be a Personal AI Agent?"

Since the full piece did not make it into the article - I'm posting it here instead!

Some quotes I like:

• "Computer Science is the art of abstraction" -

  Some quotes I like:

  • "Computer Science is the art of abstraction" - David Hyland-Wood
  • "You cannot have no ontology - just bad ontology" Giancarlo Guizzardi

  ---

  This is a living document

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2024/04/25/raw-data-now https://blog.jeswr.org/2024/04/25/raw-data-now Thu, 25 Apr 2024 00:00:00 GMT Sir Tim Berners-Lee's invention of the Web was a dramatic milestone in the information age, making vast amounts of human knowledge accessible to everyone. However, we have now reached a point where...]]> Sir Tim Berners-Lee's invention of the Web was a dramatic milestone in the information age, making vast amounts of human knowledge accessible to everyone. However, we have now reached a point where such availability is ironically creating inaccessibility, with humans and machines alike overwhelmed by false and duplicate information.

  In his seminal 2001 article "The Semantic Web", Berners-Lee claimed that "Properly designed, the Semantic Web can assist the evolution of human knowledge as a whole". While this vision has been proven true in limited contexts, from powering Google's Knowledge Panel (example shown below) to enabling biomedical discoveries, the full potential of the Semantic Web for capturing and organizing human knowledge remains largely unrealized.

  

  Knowledge representation is a decades-old area of research, originally developed with symbolic reasoning in mind. However, as I've previously discussed, knowledge graphs are now playing a crucial role in system 1 reasoning systems, in particular being used to ground LLM queries using Retrieval Augmented Generation (RAG).

  This only works if we have knowledge graphs in the first place. Enterprises have been throwing resources at building out structured databases from which they can generate knowledge graphs for decades. However, in some parts of academia, we are not doing this job so well. It's time to follow the lead of biomedical domains and start modeling our research and results so that humans and machines can better understand what knowledge we, as humanity, have discovered.

  We now have a greater impetus and better means than ever before to create a comprehensive Human Knowledge Graph using Semantic Web technologies. The key is to describe and publish concepts in as granular a manner as possible.

  One promising domain to start with is mathematics, where every theorem, proof, and axiom can be represented as its own entity, with rich semantic links capturing the relationships between them. Efforts to formalize 21st century mathematics provide a foundation to build upon.

  Beyond mathematics, we can extend this approach to other fields like physics, chemistry, and computer science. For example, a knowledge graph could precisely define different graph neural network architectures, link them to the mathematical foundations that characterize their expressivity, and connect them to experimental results on standard datasets. This provides a unified view of both the theoretical and empirical landscape.

  Building such a knowledge graph of human knowledge is valuable for multiple reasons:

  1. It enables AI systems to learn from a clean, structured representation of knowledge, improving their accuracy and grounding their outputs in precise concepts rather than just statistical patterns in raw text.

  2. It facilitates research, especially interdisciplinary work, by allowing scientists to search and integrate knowledge at the concept level, across different languages and terminologies.

  3. It avoids information duplication and establishes clear provenance and authorship of ideas.

  To realize this vision, we can leverage technologies like Solid, which provides a decentralized protocol for access-controlled linked data. This allows the public knowledge graph to be extended with private layers, enabling use cases like pre-publication research and internal industrial R&D.

  So now is the time to revisit the call for "Raw Data Now" and put resources into generating structured data for each of our domains.

  In conclusion, the Semantic Web has immense untapped potential to revolutionize how we represent, integrate, and leverage human knowledge. By undertaking ambitious collaborative projects to build granular, multi-disciplinary knowledge graphs, we can take a major leap towards realizing this vision and transforming fields from AI to scientific research. The journey will be long, but the benefits are immense.

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2025/02/17/research-topics https://blog.jeswr.org/2025/02/17/research-topics Mon, 17 Mar 2025 00:00:00 GMT … that I'm interested in but don't have time to work on at the moment.

  If you're looking for topics specifically related to Solid and data sovereignty look here.

  If you're looking for some more topics related to semi-autonomous Web Agents look here and for early work on the topic see here.

  Suitable Representations

  

  Kind-of-ok LLM generated mermaid diagram

  Formal semantic languages such as RDF encoded description logics (which in most profiles has a sub first-order expressivity), are a good means of precisely describing information – supporting interpretability and interoperability of data between systems. For much data with clear-cut attributes like the address, DOB, and website of a user – these languages can capture full information.

  However, the fact that these languages are constrained to a particular logical profile, (e.g. First Order Logic) means that there – by definition - is a limitation to what they can express. In contrast, natural languages such as English are highly expressive – and have no formal ruleset to describe how the language should be interpreted and executed. Instead, the only systems to date that can operate upon the full spectrum of natural language (including the human brain, LLMs and other machine learning systems) are those which have learned by example; and thus, will have contextually dependent interpretations and reactions to the language. Moreover, multilingual speakers often attest to the idea that there are gaps in the expressible concepts between different languages – thus indicating that contemporary natural language does not effectively capture of all concepts relevant to daily life – let alone all concepts that could be communicated by a human or machine. In the context of communication between LLMs, there is an emergent discussion of whether it would be appropriate to communicate using intermediate token representations rather than natural language. In theory this may improve expressivity but will nevertheless remain restricted to the conceptual model that lives within LLMs.

  If we are to work towards a Web of Agents that accurately and effectively negotiate, communicate and cooperate with maximal versatility, efficiency and unambiguity (where versatility and efficiency are defined as per Section 3 of this paper) then we must work towards the design of a lingua franca for inter-system communication that encapsulates all of these different language modalities.

  Moreover, we observe that the process of describing a concept in a higher-order modality, may be seen as conceptual alignment to allow that concept to be operationalised across systems in a lower-order modality. We already see this today in the way that RDF taxonomies are created – the Data Privacy Vocabulary for instance creates a taxonomy of terms defining a range of purposes for which data may be used. However, the definition of the purpose is still described using the natural language rdfs:description.

  This means that whilst, for instance, an ODRL policy can be formalised using a first-order-logic calculus description with the ODRL data model and DPV terms; the implementation of a policy engine operationalising that description will still require a human (or LLM) to interpret the natural language and codify the actions the system should take when, for instance, one is permitted to use data for dpv:Marketing which is defined in natural language as “Purposes associated with conducting marketing in relation to organisation or products or services e.g. promoting, selling, and distributing”. There is also a question of what, if anything, sits between formal-logic and natural language. We are not the first to pose such a question.

  I’m sure there is some interesting Thery of Mind type stuff to investigate in here if one was to start properly digging – but I am going to withhold from that until I decide if I am going to do research in this direction. Guidance note I don’t think it is going to be possible to invent much of this in a top-down manner. Would start first in a use-case driven manner.

  Possible intermediary points include:

  • Normalised natural language (e.g. application of strict grammar rules and/or reduction to a concise and precise language.)
  • Building hybrid KG / VDB architectures and understanding the internal representation + mapping between the two views may go a long way in assisting with the creation of this intermediate representation.

  Conceptual alignment between agents (including LLM, neurosymbolic, and logical-only)

  Work includes evaluating how well LLM-based agents can collboratively describe and build formal conceptual models of the world in order to discuss new concepts on the fly.

  Web-aware agents

  In this paper we present a vision of how we can evolve from the current paradigm of Computer Using Agents (CUAs) towards Personal AI Agents reminiscent of the 2001 Semantic Web Vision Paper. In the same paper, we posit that a migration towards a Web of self-describing agents will only be possible should we make operator style agents Web-aware. What this enables is for Web Agents to more feasibly learn what the “action space” of a particular website is, and more efficiently affect operations.

  Bootstrapping Web Agents

  In this paper we present a vision of how we can evolve from the current paradigm of Computer Using Agents (CUAs) towards Personal AI Agents reminiscent of the 2001 Semantic Web Vision Paper.

  In the same paper paper, we posit that a migration towards a Web of self-describing agents will only be possible should we first bootstrap a critical mass of such agents from existing Web Services. This is particularly important when interacting with legacy services that may not have any active maintainers. The core research questions here are:

  1. How does one discover the action space of a Website
  2. How does one describe that action space of said Website, e.g., with Semantic Web Service Descriptions In parallel, we create the opportunity to begin to develop more bespoke search engines / indexing engines for a Web of Agents. In the same way that schema.org enabled Google to effectively index information embedded in Websites - including movie schedules - and present aggregate information at the top of a search result, this work creates the possibility to create an index of what range of actions a particular platform supports.

  For "operationalising the action space" - e.g. going from call to the agent interface to website interaction, we could experiment to evaluate whether we can bootstrap existing Websites into becoming agents using LLMs to generate RML style mappings + potentially playwright interactions for Websites and APIs.

  Privacy Preserving Reasoning over Decentralised Ecosystems

  See here.

  “Belief” in semi-autonomous agents via personalised models of trust

  Exploring Risk Ontologies and Legal Data Sharing Agreements for Next-Generation Data Governance

  See here and here.

  Extracting ontological models from LLMs

  Ontological construction has traditionally been a time consuming, slow and expensive progress – requiring close collaboration between knowledge engineers and domain experts.

  In this work package shall test the viability of using LLMs to do a first pass of ontology construction. However, the goal is to investigate whether this can be done using mind mapping techniques rather than prompt engineering.

  Symbolic conceptual memory modules for Deep Learning models

  For both:

  • Interpretability purposes
  • Semantic committment purposes

  Hybrid KG & Vector Database architectures

  The core idea here is to build a DB with both a VDB and a KG view. Bergi has already done some work on this topic here and here.

  There is further related work here.

  The first piece of work I would be interested in is a formal specification for calculating points in vector space of knowledge graph concepts.

  Probabilistic RDF 1.2

  Following on from the above topic we need to define precise semantics for describing probabilities and performing inference with said probabilities in RDF/SPARQL 1.2. In particular, see Bergi's article on tackling uncertainty.

  This may take the form of a prov-o(?) extension to enable the expression of concepts such as how sure a given entity is that a claim is true.

  Knowledge Graph Memory For LLM-Based Agents

  There is emerging work, such as this on LLM memory, which build memory such as SSD access directly into the LLM architecture - rather than needing to perform this via RAG into the LLM prompt.

  The goal here is to build specialised architectures for access to graph-structured data.

  Open Research Dashboard

  As a way towards having a fully online collaborative environment – encourage researchers to discuss ideas on a topic on a platform that uses Solid and ActivityPub, and have either humans or machines label the topics for each idea.

  Then allow one to browse / view who is working on what, as well as their latest progress – e.g. on GH.

  Search interfaces to discover semantically described data and datasets

  This includes:

  • A good portal which supports the discovery of authoritative entity identifiers such as WebIds based on a natural language query - this could be for name "Jesse" or a rough description "Australian guy working on AI and Solid."
  • A good portal to discover semantically described datasets such as those described using Croissant or DProd.

  Operationalizable guidelines on how to comply with regulatory requirements as a host of decentralized infrastructure

  In particular - today - as a host of:

  • Decentralised social media - such as Mastodon, or
  • Decentralised storage such as a Solid Pod provider

  and in the future as a host of e.g. decentralised AI services.

  Industry consortia do the work of creating such operationaliseable guidelines within their own domains. For instance, the financial indusry has created the Cloud Data Management Controls (CDMC) which provides a framework for operationalising GDPR when managing financial data in cloud services.

  Some writing on the topic specifically in terms of Solid is here.

  Machine-readable definition of RDF syntaxes (probably boring topic for most people)

  Not just ANTLR grammar definitions for syntax for lexers; also prescribing how this maps to triples for parsers.

  Starting point here

  Malleable Software for Solid (read: RDF data structures and Shapes)

  • For Malleable Software see here.
  • For Solid see here.
  • For RDF see here.
  • For Shapes see here.

  Semantic commitments all the way down

  Work towards have transactions on the Web semantically described commitments

  Say goodbye to the cloud and hello to the dataspace

  Don't write apps and code agains servers - write them against data instead.

  Using LLMs to create a densely linked Web

  Before we move to a fully fledge web-of-concepts, I believe LLMs have a strong role to play in densely linking the Web. That is, helping us relate all of the knowledge that has been written about so that it is much easier to discover all of the existing research on a given topic - going beyond existing work such as tools:

  • That link papers based on human annotated links, such as citations of papers that people have discovered that are related, and
  • OpenAI researcher agents which are able to crawl the web and identify papers which are about a particular topic

  Model response (may be applicable to Web Agents WG)

  One feature that has supported the Web to scale is having public and cacheable resources. I recommend standardising an HTTP interface that can be used to GET responses from LLMs in a way that allows these responses to be stored in DNS caches.

  For responses that require payment - e.g. because of the size of the model needed to answer the query - then there should be a redirect flow to "unlock" the link and then keep the response cached and public. See extension on this thought in the following topic.

  Standard for making per-request payments to LLM providers (may be applicable to Web Agents WG)

  So that services relying on those LLMs need to implement the standard rather than create custom flows and deals for each LLM service.

  Extending this - directly delegating authorization of payments to a user account - so that they can:

  • Make use of ongoing subscriptions on existing platforms
  • Further, delegate the cost directly to their organizations when using services for work
  • Have a single platform that they connect their bank accounts to and set controls for "LLM usage"

  One Identify for agentic platforms (may be applicable to Web Agents WG)

  Largely to achieve the above item

  LLM metadata for discovery (may be applicable to Web Agents WG)

  Presumably products that use LLMs - such as cursor - need to manually curate the list of LLMs that they are using in their product. As the number of LLMs that are available proliferates, it may be better for LLMs to provide a .well-known description to identify the capabilities, cost and usage mechansim.

  Agent Chat Rooms (may be applicable to Web Agents WG)

  To enable, for instance, groups of personal agents to organize travel plans for a party of 5 people, or distributed tooling agents to negotiate with each other.

  In the medium term I would think that these are more somethign like "concept whiteboards" - but agent chat rooms are a good place to start.

  Standardize authentication for LLMs to access private resource

  Intersects both with goals of MCP, Solid and protocols like OIDC4VP

  Standard sharing of re-usable context data

  Standardise Solid as the way of having agents request and use contextual data. With both:

  • Access Controls (including which services can access it)
  • Usage controls (including which services results can be forwarded to, and whether the data can be used for monetization)

  More things to standardise in the LWS-WG

  Other Web Agents Group topics could include:

  • Standardize search interface to support and look up of publically available vectorised data
  • Taxonomise common services of LLM providers (e.g. embeddings endpoint, what modalities does a provider support etc.)
  • Describing which models you have deployed where. In particular to allow the discovery of multiple deployments of opensource models; and to allow agentic workflows to define "this model must be used here" but then automatically do things like work out if it is deployed locally, and if not find a trusted remote deployment that can be used.

  ---

  This is a living document

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2024/02/06/role-of-solid https://blog.jeswr.org/2024/02/06/role-of-solid Tue, 06 Feb 2024 00:00:00 GMT This article was originally for the Oxford University department of Computer Science as part of the DPhil the future series. This article was originally for the Oxford University department of Computer Science as part of the DPhil the future series.

  The modern Web has seen a significant erosion of data privacy and control. User-generated data, stored in centralised data silos, is often used without the knowledge or control of the individuals it belongs to.

  In the age of AI and Big Data, this lack of data autonomy becomes increasingly detrimental to both individuals and society.

  A striking example is a recent lawsuit against Meta by U.S. states, alleging the unlawful collection of minors' personal data without parental consent and exposing them to harmful content. This is but a glimpse into the dangers posed by emerging technologies such as personal AI agents if they are not designed to “legally, ethically and algorithmically” work for you and with data you control (Charlie works for Bob, Berners-Lee 2023).

  Celebrating Safer Internet Day  

  Today, we celebrate the 20th anniversary of Safer Internet Day, with a theme that resonates deeply with our current digital challenges: “Inspiring change? Making a difference, managing influence and navigating change online.” This aligns with the mission of the The Ethical Web and Data Architectures (EWADA) programme funded by the Oxford Martin School. EWADA aims to address the concentration of power on the World Wide Web by developing new technical and legal infrastructures. 

  Solid Project - A Beacon of Hope 

  In the context of data autonomy, EWADA is putting a spotlight on Sir Tim Berners-Lee’s Solid project which was created with the aim of revitalising the Web. Where the current system of centralised data silos creates an ecosystem of limited integration, availability and innovation, Solid brings a course correction for the Web. Based on the separation of data and applications, the vision defines an ecosystem that facilitates the integration of data in different applications, while keeping people in direct control of their data. 

  Understanding Solid Pods 

  At Solid's core is the ‘pod’ - a personal online data store. Utilising the Linked Data Platform (LDP), pods offer an HTTP interface for access-controlled storage of documents to a personal server or trusted cloud storage. Coupled with OAuth - the same protocol we use every day for SSO login - this system facilitates a global single sign-on across all Solid-compliant web applications.

  Together, these pods form a decentralised Solid ecosystem, from which applications can directly integrate data from people’s Solid pods, after receiving their permission. Solid applications are encouraged to store data using the RDF data model of the Semantic Web (I strongly recommend reading the design issue if you’ve not heard of it before) - and serialised in a syntax such as JSON-LD or Turtle. By having applications read/write data with explicit semantics rather than using the typical smorgasbord of JSON, GraphQL and other platform-specific API’s that modern Web developers are accustomed to it becomes possible for applications to re-use one another's data. 

  Such applications include BBC Together, itsme, Umai, Solid Focus, KNoodle and SolidFlix,. 

  To learn more about the anatomy of a pod - see What is a Solid Pod? 

  Benefits and Future of Solid Architecture 

  In 2009, Berners-Lee enumerated a number of benefits to this architecture which remain relevant to this day (these come from his design issues; if you’ve never come across these it’s definitely worth a read): 

  1. Users control data access, including the ability to revoke an applications’ access to your data at any point in time. 
  2. It allows the data from various applications to be cross-linked, at great derived extra value. For instance, if I relocate, there's no need to separately inform my bank, insurance agency, and driving authority. By simply updating my address in my personal data pod, any authorised organisation can access the new information instantly. 
  3. It allows innovation in the market for applications, because the bar for launching an app is far lower, as the app can run in the open data cloud. 
  4. The persistence of applications and data may be very different. In some cases, a well-established application which people have grown very familiar with may be used to make an online discussion which is ephemeral, in another case an application may be developed to solve a short-term problem in an enterprise where the life of the data exceeds that of any of the applications the enterprise uses. By decoupling the application and data, these persistences can be managed independently. 

  Concluding 

  By reimagining the web as a place where users have sovereignty over their data, initiatives such as Solid pave the way for a more ethical, transparent, and user-centric online world. Realising this ambitious vision demands a collaborative approach that spans various sectors and roles. Whether you're keen on exploring personal pods, disseminating your research through a Pod, engaging in privacy-focused computation with Solid, developing innovative applications, or shaping relevant policies, your contribution is vital. Let's join forces and work together to make this vision a reality!

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2025/03/22/social-impact-assessment https://blog.jeswr.org/2025/03/22/social-impact-assessment Sat, 22 Mar 2025 00:00:00 GMT

  Infrastructure projects rightfully require Environmental Impact Assessments so tha...]]>

  Infrastructure projects rightfully require Environmental Impact Assessments so that potential harms to the environment are identified - before determining whether infrastructure projects may proceeed.

  Increasingly, it would seem there is a need for an equivalent for both physical and digital infrastructure - given the increase in, e.g., online harms.

  This is the first result that comes up when searching the term "Social Impact Assessment" on Google - so it seems these kinds of ideas have been circulating at least in the space of assessing e.g., policy interventions; but less so it seems for digital infrastructure.

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2024/09/15/term-paper https://blog.jeswr.org/2024/09/15/term-paper Sun, 15 Sep 2024 00:00:00 GMT This is the term paper I wrote for my DPhil. Far from the best piece of work I have produced as I still quite ill at the time it...]]> This is the term paper I wrote for my DPhil. Far from the best piece of work I have produced as I still quite ill at the time it was written; but I'm not going to make that a reason to hide my work.

  ---

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2025/03/18/things-i-want https://blog.jeswr.org/2025/03/18/things-i-want Tue, 18 Mar 2025 00:00:00 GMT

  Doc Searls' intention economy talks about the idea of sending a "signal into the market" de...]]>

  Doc Searls' intention economy talks about the idea of sending a "signal into the market" describing to a market what you want.

  Some claim that this vision formed some of the initial inspiration to develop Solid.

  This post is me sending a signal to the world describing the things I would like to see built that would improve my life.

  Something is wrong - go fix it

  I've had a fair share of health issues over the last year.

  For instance, I have the following allergens and intolerances:

  • Gluten (coeliac)
  • Lactose
  • Sulphites
  • Nuts (at least peanuts and hazelnuts, rest tbd)
  • Pesticides(?) … or something else that appears to be on some fruits such as grapes …

  I was also taking GluteGuard to try and cope with trace elements of gluten that might pop up in meals when I went out … funny thing, it turned out the GluteGuard contained sulphites and I spent the better past of the last year in a state where my brain was barely functioning.

  The funny thing about being in that state; is that you'll tend to not realise, or be in denial, about there being a problem in the first place - which in turn means I wasn't trying to resolve the issue anywhere near as actively as I could/should have been.

  Advertisers already heavily track minute interactions, how long we spend on particular posts, our reaction time, which links we click on etc. in order to do targeted advertising. I want that same infrastructure to tell me when something is … off; so that I am validated to go and fix it.

  A balanced news feed

  For engagement, most news feeds are weighted to have many alarming or unique stories; whilst less exciting and ongoing things get ignored. An example given in not the end of the world is the minimal reporting on improvements in renewable technologies in comparison to climate disasters. I want a news feed with an algorithm that is ranked somehow highest impact changes - than most alarmist / engaging.

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2025/05/06/transfer-of-status https://blog.jeswr.org/2025/05/06/transfer-of-status Tue, 06 May 2025 00:00:00 GMT This is the transfer of status report that I wrote for my DPhil.

  ---

  This is the transfer of status report that I wrote for my DPhil.

  ---

  ]]> jesse@jeswr.org (Jesse Wright) https://blog.jeswr.org/2025/03/23/dynamic-declarative-protocols https://blog.jeswr.org/2025/03/23/dynamic-declarative-protocols Sun, 23 Mar 2025 00:00:00 GMT ARTICLE IN DRAFT

  

  Placeholder image produced by...]]> ARTICLE IN DRAFT

  

  Placeholder image produced by ChatGPT 4.5

  This post discusses why we need to create infrastructure that supports a network of systems to:

  1. Declaratively describe what features they need a protocol[^1] to achieve for them such as:
     • proving that the user of a particular device in the network is a particular entity,
     • calculating an aggregate result from data living on multiple devices in the network (think Multi Party Computation, Federated Learning etc.),
     • proving that a payment has been made, and
     • coordinating to ensure that a payment is made once an action has been performed (think smart contract)
  2. Declaratively describe their acceptable security requirements, including:
     • what operations other systems in the network can be trusted to perform,
     • what information can be trusted to come from other systems in the network,
     • who in the network do they permit to view particular pieces of information, and
     • other requirements that may be identified in a W3C Security Review
  3. Have a means to create or discover and agree to use a protocol on-the-fly that satisfies these set of requirements.
  4. Whilst still allowing for long term planning
  • For instance, a desirable feature of the ongoing use of HTTP(s) is the ability to use documents cached in DNSs.

  We observe that:

  • this proposal largely reflects my view that we should - for the most part - not need to use Generative AI at the layer of deciding how to transfer data, but instead to declaratively state what data is needed,
  • this proposal assumes that we are going to have some level of data and compute sovereignty - whilst also moving us towards that world, and
  • the capability to generate a proof that the proposed protocol has a particular set of security properties is critical to enabling all parties to 'buy in' to use the protocol

  The kinds of use-cases which this unlocks include:

  • • Dynamic Authentication Flows: Currently, the flows for, e.g., authentication are often quite fixed; and require heavy changes to specification documents + flows to evolve. In this use-case we show that the OIDC flow can “emerge” simply by having 2 agents negotiate for data they need to identify one another. A specific example is this e-reader access use-case.
  • Agents performing scheduling: As described in this paper.
  • Features people care about in the blockchain world: For example, could show how a DAOs could form.

  [^1]: Note that here we are using the term protocol to refer to the sequence of steps that a set of systems should take in order to achieve a joint goal. This includes how to collaboratively transport information (networking protocols), secure data in transport (encryption protocols) and proving the identity of an end user on one system to another system (authentication).

  ]]> jesse@jeswr.org (Jesse Wright)