The news feed of www.michelepasin.org

What does an LLM need to know about your data? Climbing the semantics ladder

Fri, 06 Nov 2026 00:00:00 +0000

Give an LLM a spreadsheet and ask it a question. How much better does it do if you also tell it what the data means? And does it matter how you tell it: a Markdown glossary, an OWL ontology, a set of rules?

I spent the last couple of weeks running experiments to find out. The short version: written-down meaning helps a lot, but only where the meaning is local to your data. The format it comes in barely matters to the model. It matters a great deal to the people who have to maintain it.

Everything is on the Semantics ladder site: the data, the questions, the ontologies, every run and every answer.

Where this started

The trigger was Andrea Splendiani's article Ontologies for AI in practice / part I. He took a family-tree question and walked it up a ladder: raw data, then data plus definitions, then curated data, then rules, checking at each step what the LLM got right.

His conclusions were clear:

Simple text definitions give the biggest bang for the buck, while minor data curation has little impact. Putting the data in RDF helps people understand it, by surfacing assumptions that mislead humans and LLMs alike. A well-curated knowledge graph doesn't necessarily change the answers, but it improves trust and consistency. And ontologies give LLMs an edge by telling them what things mean in a specific context.

I liked the idea a lot, and wanted to see whether it holds up with more questions, more repeats and a domain where the meanings are less obvious than "parent" and "sibling". So I built a small lab around it.

The set-up

Two invented domains, so that no model could answer from memory:

Credit ratings. Companies, their debt instruments and rating actions, with the cryptic codes real extracts have (LTIDR, RWN, WD).
Research analytics. Works, authors, institutions and citations, shaped like OpenAlex.

Each domain hides traps of the kind an analyst knows without saying. A rating holds until the next action, so the latest row isn't the answer for a past date. An outlook change isn't a downgrade. Two records with the same LEI are the same company. A preprint and its published version are one work. A record from the QA system is a test, not a real issuer.

Some of these are general knowledge, which a good model has from training. Others are local: rules this particular dataset sets for itself, which no model could guess. That distinction turned out to be the most important one.

Each domain has about 48 questions, graded by code against keys computed from the data wherever possible. I asked them to Claude Sonnet and Haiku, five repeats each, under a ladder of conditions. The main ones:

csv: the data only.
rdf: the same facts as RDF triples, with no ontology.
csv+padding: the data plus an unrelated glossary of the same length, on the same subject.
csv+markdown and csv+ontology: the data plus the definitions, as a Markdown glossary or as an ontology in Turtle.
csv+axioms: the data plus OWL and SHACL axioms, with all the prose removed.
executed: the data after the rules have been run on it, with no definitions.

The padding condition matters more than it looks. A longer prompt full of domain words might help on its own. If it did, any gain from the ontology could be length, not meaning. The Method page has the full list, including the querying conditions I come back to below.

What helps: meaning that's written down

Here are the scores on the credit data:

And on the research data:

With the definitions, in whatever form, both models land between 96% and 100% in both domains. Without them, they get 78–84% on credit and only 45–57% on research.

The padding adds nothing, so it's the meaning that does the work. And the gain tracks how local the domain is. Sonnet already knows most credit conventions, so the definitions only fill a few gaps (+16 points). The research data has more conventions of its own, such as which affiliation gets the credit or whether a retracted work counts. Without them the model is guessing, and the definitions are worth +41 to +53.

Two more things stood out. The weaker the model, the more the definitions are worth: with them, Haiku catches up with Sonnet. And RDF on its own never helped. It holds exactly what the CSV holds, in a more verbose form, and sometimes cost a few points.

What doesn't matter: the format

This is the result I expected to be most controversial.

Markdown, SKOS in Turtle, and the same definitions in both forms all scored within a point or two of each other. More surprisingly, the axioms alone (OWL classes and SHACL rules, with every comment and definition stripped out) did about as well as the prose: 99% on credit and 97% on research, for both models. The model reads logic perfectly well.

So the model doesn't care what format the meaning comes in. But people do. The glossary has to come from somewhere. It has to be written once, kept consistent, versioned, owned, checked against the data, and shared by everyone who uses it. That's what a knowledge graph is for.

In the lab, the Markdown is generated from the Turtle, so the two can never drift. I think that's the pattern in miniature: keep the meaning in RDF, and render it as whatever the model reads best.

What can go wrong: ontologies have opinions

Two small findings worried me more than any score.

First, one sentence in the credit ontology called the foreign-currency rating "the headline rating". A company with three ratings was asked "is it investment grade?", and the honest answer is "which one?". With that sentence, Haiku answered a flat "No" in most repeats, quoting the sentence back. With one neutral rewording, it got it right five times out of five.

Second, an axiom is followed to the letter, mistakes included. In an early version I declared test records a subclass of entity, while the prose said they describe no real entity. With the axioms alone, the model counted them as entities, every time. The error was mine, and it reproduced it faithfully.

An ontology carries its authors' emphasis as well as its meaning. If you give one to a model, you need to review it and test it against the data, the way you'd test code.

Read the data, or query it?

The obvious objection: nobody pastes real data into a prompt. The model should write SQL or SPARQL, and let the database do the counting. So I tried that too. The model never sees the rows: it gets a schema, writes queries for up to eight rounds, and answers from the results.

At the sizes I tried, reading held up better than I expected. At 300 companies, with the definitions, reading scores 95% and querying 97–99%. Querying pays off mainly on long lists, where one missed item fails the answer.

The research data tells the same story. With the definitions, SQL scores 96% against 98% for reading. Without them, both get about half the questions right (52% and 57%).

So the definitions mattered just as much when querying: +19 points for SQL on credit, +45 on research. A query is only as good as the model's grasp of what the columns mean. Querying doesn't remove the need for an ontology.

What about graphs? I added ownership chains four to six levels deep, the kind of question a graph store should win. It didn't:

With the definitions, SQL got every path question right in every session. SPARQL got 42% of them on rdflib and 60% on GraphDB.
The SPARQL failures were mostly about how the model worked: paging through whole classes, or giving up after a few rounds.
The research data showed the same thing. SPARQL with the ontology scored 97–98% in four sessions out of five, then collapsed to 17% in the fifth, returning almost no answers. SQL never dropped below 96%.
With the chains precomputed into an ancestors table, SQL got every path right, every time.

The graph's advantage is following links of any depth at query time. Once the rules have computed the chains, a plain table does as well. And in my runs, the difference between SQL and SPARQL was less about accuracy than steadiness.

At ten times the size (850 research works, too big to read comfortably), the picture shifts:

Reading with the definitions drops to 87–89%, because the model keeps the meaning but loses the arithmetic. Querying with the definitions does no better. What works is querying data where the rules have already run: 92% for SQL over the executed tables, and 91% for SPARQL over the graph after its SHACL rules.

So querying is about size, and the ontology is about meaning. They solve different problems.

Rules beat reading, if you keep the receipts

The executed condition, where the rules have already been applied (duplicate records merged, test records dropped, hierarchies rolled up), got most of the way without any definitions: +15 to +19 points on credit, +34 to +46 on research.

But it fell short in an instructive way. Merging a preprint into its published version means you can no longer ask about the preprint. Tables that apply the rules without stating them can't answer "why?". Curation decides what the data can still answer. A store that keeps the records it merged, and the rules it ran, answers more.

A pinch of salt

These are early results. They're measured carefully, but the scope is small.

There are two invented domains, with about 50 questions each and ontologies of about 50 terms. Only two models were tested, both Claude, and the datasets are mostly small enough to read. I also wrote the questions myself. So the findings are indicative, not general. They show what happened here, and suggest what might hold elsewhere. The Conclusions page lists where to dig next.

The biggest gap is size. A real enterprise ontology has hundreds or thousands of terms, and a production dataset is far too big to read. At 2,000 terms, the definitions alone would fill most of a context window, before any data.

That's where I think knowledge graphs come into their own, and not as a format for the model to read. They're where the meaning is organised and stored as it grows and changes over time: versioned, owned, checked against the data, shared across teams. And they make the hard part tractable: deciding which bit of the ontology should become context for a given question.

A graph knows which concepts a table's columns use, what those depend on, and which rules touch them. Selecting the right slice from that is a retrieval problem, and the structure is what makes it cheap. It's also where the rules can run, upstream, so the model queries results instead of re-deriving them.

What I'd take away

Most of this agrees with Andrea's conclusions. Simple definitions were the big win here too, and the gain came from meaning specific to the data's context. RDF on its own didn't help the model, but that fits his point: its value is to the people who build and check the data, not to the LLM. The one difference is curation. Running the rules over the data, rather than tidying it up a little, did change the answers, especially once the data grew.

Write the meaning down, especially the local rules. That's where the gain is: +15 to +50 points, depending on how much of your data's meaning is its own.
Don't worry about the format you give the model. Do worry about the format you keep it in.
Review your definitions like code. A slanted sentence or a wrong axiom will be followed faithfully.
Run the rules upstream when the data gets big, and let the model query the result.

The model reads Markdown just fine. But someone has to keep that Markdown true as the data and the business change. I think that's still the job of a knowledge graph, and probably a bigger one than before.

The code, data and every run are on GitHub. Thanks to Andrea Splendiani for the article that started this. I'd love to hear from anyone who has tried something similar, especially at real enterprise scale.

Ontospy: three releases in a day, after nearly four years of silence

Mon, 12 Oct 2026 00:00:00 +0000

Today I released three versions of Ontospy: 2.2, 2.3 and 2.4. The previous release was in November 2022.

Ontospy is the open source Python library and command line tool I've been maintaining since 2013. It reads RDF vocabularies and ontologies, lets you inspect them, and generates documentation for them as HTML, Markdown or interactive D3 charts.

A confession

I let this project go stale. Between a demanding day job and family life, Ontospy kept slipping to the bottom of the list.

Meanwhile people kept using it, and kept writing in. Bug reports, feature requests, pull requests with real work in them. The oldest open issues dated back to 2017. One pull request, a nice idea for cleaner class trees, had been waiting since 2019. By last week the library didn't even install on recent versions of Python.

That kind of backlog is heavy to look at. Each item on its own is small. Together they need days of context-switching I never had.

What changed

This time I worked through the whole lot with Claude Code. The three releases went out on a single day.

It runs again (2.2)

Support for Python 3.12 and later, which removed an API Ontospy relied on
A proper test suite with pytest, and continuous integration on Python 3.10 to 3.14
Packaging fixes: dependency versions are now actually enforced on install
A crash when documenting class instances, which had been reported three times

Community contributions (2.3, 2.4)

Custom templates: you can now override any of the documentation templates with your own (thanks to @jgsogo)
Better Markdown output (thanks to @rvesse and @fmeynadier)
Namespace prefixes declared in a file now win over rdflib's defaults
Ontospy can load an existing rdflib graph
Non-redundant class trees for reasoned ontologies, from that 2019 pull request (thanks to @floatingpurr)

Correctness (2.4)

Properties without a domain or range now inherit them from their superproperties, as RDFS intends, instead of showing owl:Thing
Properties typed as a subclass of owl:ObjectProperty are recognised, which matters for ontologies like Brick
A property's type no longer depends on alphabetical order (Pizza's hasBase used to show up as an owl:FunctionalProperty)

Large ontologies (2.4)

This is the one I'm happiest about. Someone reported that Ontospy crashed on FOVT, a large biology ontology, and suggested raising Python's recursion limit. The real cause was elsewhere: the code walking class hierarchies didn't keep track of where it had been, so cycles and heavy multiple inheritance made it recurse forever.

With that fixed, FOVT went from not finishing after 26 minutes to loading in under two.

Documentation and examples

gendocs no longer wipes the output folder before writing to it
URLs in metadata are links, and rdfs:seeAlso gets its own section
The examples site is regenerated with the new version, and its index now shows what each ontology is about

In numbers: 28 issues closed, every open pull request dealt with, 45 commits. Three issues are still open, on purpose: a licensing question, a SHACL feature that needs a contributor, and general performance on very large ontologies.

How it actually worked

It wasn't magic, and I didn't just press a button.

Claude did the legwork. It reproduced each bug before touching the code, wrote a failing test first, then the fix. It dug into causes I'd have guessed wrong: the FOVT crash wasn't where the original report, or my own notes, pointed. When a change touched the generated documentation, it rebuilt the docs before and after and compared them, to make sure nothing else moved.

I made the decisions. Should an inherited domain be shown as declared, or marked as inferred? Should the examples index be a table or a set of cards? Every change came to me as a diff before it was committed, and the public steps (merging, tagging, publishing to PyPI) stayed mine.

That split felt right. The judgement calls are the interesting part of ontology engineering anyway. The context-switching was the part I never had time for.

Back to it

Doing this reminded me how much I enjoy this kind of work: structuring knowledge, and building tools that help other people do it.

Next on the list are performance on very large ontologies, better support for Jupyter notebooks, and a cleaner documentation site. If you use Ontospy, now is a good time to open an issue. This time I promise to answer.

pip install -U ontospy

See also: the changelog, the documentation and the source code.

Update, 14 October: version 2.5

Two days later, 2.5 is out too, with a few things that came up while regenerating the examples:

The documentation now lists only the namespaces an ontology actually uses, rather than all of rdflib's thirty or so defaults. For Pizza, that's 5 prefixes instead of 30.
Queries to SPARQL endpoints send only the prefixes they need. On DBpedia, that cut a typical query from 29 PREFIX lines to 11.
Ontology metadata is sorted, and nested details like an ontology's editors are shown properly, instead of as cryptic node ids.

I also refreshed the examples site. It now covers 32 ontologies, adding widely used standards like PROV-O, DCAT, SKOS, GeoSPARQL and FIBO, and dropping some dated ones.

Book Summary: The Mom Test

Sun, 11 Oct 2026 00:00:00 +0000

Ask your mom whether your business idea is any good and she'll say yes. Not because it is, but because she loves you. That's the starting point of The Mom Test by Rob Fitzpatrick. The trouble is that everyone behaves like your mom. Show people an idea and they'll be polite. They'll say it sounds great, that they'd totally use it, that you should let them know when it launches. None of that is data. It just feels like data.

The book was published over 10 years ago (2013), but it's still one of the most useful short books on product management I've read.

How to pass the Mom Test

The fix is simple, and it fits in three rules:

Talk about their life, not your idea.
Ask about specifics in the past, not opinions about the future.
Talk less, listen more.

A question passes the Mom Test if even someone who loves you can't give you a misleading answer to it. "Would you pay for this?" fails. "When did this last happen to you, and what did you do about it?" passes. People are bad at predicting what they'd do, and pretty good at describing what they did.

The rest of the book is about the habits around those rules. How to deflect compliments, anchor fluff back to real events, and dig beneath feature requests to the problem behind them. How to keep conversations short and casual rather than turning them into formal interviews. And how to end a meeting with something real: a fact, a commitment (time, reputation or money), or a next step. If a meeting "went well" but nobody gave up anything, it probably didn't.

Why I liked it

It's short, practical and a bit funny. More importantly, it's honest about how easy it is to fool yourself. Several of the "signs you're just going through the motions" hit close to home: you talked more than they did, you got an unexpected answer and it didn't change anything, you weren't scared of any of your questions.

That last one stuck with me. If none of your questions could hurt your plan, you're not really testing it.

A field guide

To make the ideas stick, I turned my notes into a one-page Mom Test field guide. It has the three rules, a little quiz on whether a question passes the test, the bad-data patterns and how to handle them, and a before/during/after checklist for customer conversations.

It's meant to be skimmed five minutes before a call, which is roughly how long it takes, according to Fitzpatrick, to find out whether a problem is real.

A Dream of Venice, with Claude as Editor

Thu, 10 Sep 2026 00:00:00 +0000

My children can read Italian. Most of their classmates can't. Back in July 2023 I wanted to read the book Un sogno a Venezia by Štěpán Zavřel to their class at school, so first I had to put it into English. Recently I have picked up that project again and used Claude to review it — it's impressive what AI can do with language translation.

Un sogno a Venezia means "A Dream in Venice." A boy called Marco falls asleep worrying about a drawing he owes his teacher. A little mermaid taps on his window and takes him down to see Venice as it will be one day: on the seabed. They swim through St Mark's Square and into the golden basilica. They pass the Doge's Palace, where an octopus guards the empty rooms and plays cards against himself. Children play hopscotch with the fish. It is strange, and gentle, and a little sad.

Zavřel was a Czech illustrator who spent much of his life in a small town in the Veneto region (Italy), where the foundation named after him still champions children's book illustration.

There is no English edition, so I translated it myself and built a small website for it: zavreldream.michelepasin.org. Each page is one of Zavřel's paintings, full screen, with the text beside it. You can read it as a slideshow, as one long scroll, or as a book with pages that turn. Nothing clever underneath — a Django app that produces static HTML, and the code is on GitHub.

From A Dream in Venice by Štěpán Zavřel.

The city really is sinking

The premise isn't invented. Venice was built on millions of wooden piles driven into the mud of a lagoon, and it has been settling into that mud ever since. For a stretch of the twentieth century it settled faster, because factories on the mainland pumped groundwater from under the city for decades, until Italy banned the practice in the 1970s.

The bigger threat now is the sea rising to meet it. Measured against the water, Venice sits roughly 25 cm lower than it did in 1900, and the gap keeps widening. The high tides that flood St Mark's Square — the acqua alta — were once rare. Now they come dozens of times a year. A set of mobile barriers called MOSE can seal the lagoon against the worst surges, but it was built for today's sea, not the one we are heading towards.

Zavřel drew the book in 1975. In it, the children are told to imagine Venice underwater and to paint what they see. Fifty years on, that reads less like an art lesson and more like a forecast.

Translating with AI in the loop

Here is the part I didn't expect: translating with an AI in the loop is properly addictive.

I did the first pass myself, the honest way, one sentence at a time. Then I handed it to Claude to check. It started pulling at threads I couldn't see any more, because I had been staring at them too long. And once it found one, I wanted it to check the next page, and the next.

The easy things to catch are the italianisms — phrases that are grammatical in English but that no English speaker would choose. When Marco says the drowned city can't be real, the mermaid answers "Ma sì". I had written "But yes, replies the little mermaid." Claude's version: "Oh yes it is!" replies the little mermaid. Elsewhere my characters kept "arriving in front of" buildings — a word-for-word lift of arrivare davanti a — where English just says "coming to" them.

The harder things are in the dialogue. Italian punctuates speech loosely: the tag folds in with commas, the quotation marks are often dropped. Carry that habit across and you get:

"It's not possible! he exclaims, it's not possible!"

when English wants:

"It's not possible!" he exclaims. "It's not possible!"

In one line, the sea-children beg Marco to bring help. I had crushed two sentences together and lost a word along the way: "We've been waiting for so long come and save!" Claude put it back: "We have been waiting so long for you to come and save us!" I know exactly what the Italian says. I wrote the broken version anyway, and I read past it a dozen times.

I want to be clear about what this is and isn't. The AI did not translate the book. The wording is mine, and I threw out plenty of its suggestions. It is closer to having an editor who has read the page forty times and is still paying attention on the forty-first.

What writing becomes

I keep wondering what this does to writing over the next few years.

Not translation as a profession — that is already turning over. I mean the small daily act of getting a sentence to say what you meant. A comment on a pull request. A message to a colleague you don't want misread. A children's book you are turning into English so a room full of kids can follow along.

We are heading somewhere the distance between "roughly what I mean" and "exactly what I mean" costs almost nothing to close, for anyone, at any time. Mostly that is wonderful. It is also a little strange. Some of my worst sentences are the ones that taught me the most, and I don't know what teaches you that once the friction is gone.

The class got their story in the end, though. That part is done.

Reading Naples '44 as a database

Wed, 02 Sep 2026 00:00:00 +0000

I recently read Norman Lewis's Naples '44, and it stayed with me more than most books do.

Naples '44 is Lewis's diary from the year he spent as a British intelligence officer in Naples, from the Allied landings of September 1943 to October 1944. It's thin — a few paragraphs per entry. It's also one of the best things I've read about what military occupation looks like from street level. Hunger, the black market, bombs, and in between all of it the ordinary theatre of Neapolitan life: a saint's blood that has to liquefy on schedule, a requisitioned piano, the city aquarium emptied to feed an official banquet.

Part of why it landed is that I've spent time in many of the places he describes. Reading stories of that much suffering and misery, set in streets I know, and less than a hundred years ago — that does something to you.

So I decided to take the book apart and build a small website out of it: naples44.michelepasin.org. Partly to hold on to what the book is about, as an interactive index I can come back to. Partly as a test of how far I could get using Claude to do the work.

Taking notes by talking to Claude

I didn't read Naples '44 on paper. I listened to it on Audible, on walks and commutes, and I took notes by talking to Claude as I went — telling it what had just happened, which places and people I wanted to keep track of, what a passage reminded me of.

That turned out to be a genuinely new way to take notes on a book. I've always meant to annotate as I read, and I've always been too lazy to actually do it: stopping to write breaks the flow, and you can't type on a walk. Doing it by conversation removes the friction. It's less like keeping a commonplace book and more like going through the book with someone next to you — you can argue with the text, ask "wasn't that the same officer from the Salerno chapter?", and let the running index build itself out of the back-and-forth.

By the end, that conversation had turned into a small structured dataset.

The unit is the entry. There are 108 of them, each with a date, a short summary in my own words, and a set of tags. I kept track of themes (11 of them — the evil eye, Allied conduct, the Camorra, survival), places (129), recurring people (37), and a running list of saints, feasts and superstitions (31). Lewis keeps circling back to the same handful of subjects, so most of the tags fell out of the book on their own.

What surprised me is how much the tagging changed the listening. Deciding whether an entry is "about" the black market or "about" Allied conduct — when it's plainly about both — forces you to say what the entry is really doing. Naming a place means deciding whether two vague references point at the same street. It's interpretation, not extraction — and having something to talk it through with makes you do more of it, not less. The dataset is a record of my reading of the book, not a neutral index of it.

The timeline

The main view is a circular timeline. The inner ring holds Lewis's 108 entries in date order. The outer ring marks 20 events of the wider war — the Anzio landings, Monte Cassino, the liberation of Rome — on the same axis.

The point of putting them together is the juxtaposition. While the Allied armies are stuck below Cassino through the winter, Lewis is writing about a woman who rents out her daughters, or the etiquette of a Camorra funeral. The military history and the view from one desk in Naples run in parallel, and you can see where they touch.

Hover an entry and you get its summary and tags. The theme filters light up every entry that touches a subject, so you can pull the "evil eye" thread through the whole year in one move.

A page for everything

The other half of the site is flatter. Every place, person, saint and theme has its own page — a description, a note on what it's doing in the book, and a chronological list of every entry it appears in. Places also get a map: 114 of the 129 are geocoded, so you can see Lewis's Naples as a scatter of points, most of them within a few streets of each other.

This is the part I keep coming back to in my own work. People of Medieval Scotland, Wittgensteiniana, and now this — they're the same move. Take a text, pull out the entities, give each one a stable URL and a page that gathers every mention. It's an old idea, basically an index with hyperlinks, but it still produces something you can't get by reading front to back.

How fast this now goes

The plumbing is nothing clever. It's a Django app with a SQLite database and an admin interface I use to edit the data, and then the whole thing gets wget'd into static HTML and served from GitHub Pages. No server running anywhere. I've written about this pattern before — it's my default now for small sites that are really just a database with a nice front end. The visualisation is D3, vendored, no CDN. Source on GitHub.

What's new is the speed. A few years ago, building this as a Django application — models, admin, importers, templates, the timeline — would have taken me weeks, maybe months of evenings. This took a few days, working with Claude.

The judgement didn't go away. Someone still has to decide what counts as an entity, whether two mentions are the same person, what a theme means, and whether each summary is faithful to the book. I went over every one of those summaries. But the mechanical work — the models, the migrations, the import scripts, the geocoding, the D3 boilerplate — mostly just happened, at a pace that still surprises me.

I'm planning to package the workflow into a skill: point it at a text and a list of entity types, get back a navigable site like this one. More on that once it exists.

The book is still the point

If anything the site is an argument for reading Naples '44 slowly. Every summary is deliberately a paraphrase, never Lewis's own words — partly for copyright, and partly because doing the compression — even with an assistant to help — is what makes you look at a passage twice.

I built this mostly to find out what a book looks like once you've taken it apart. The answer, at least for this one: a year, a few hundred fragments, and the same dozen preoccupations coming round and round.

Five new ways to visualize Wittgenstein's Tractatus

Mon, 17 Aug 2026 00:00:00 +0000

I've spent a bit more time this year on Wittgensteiniana, my long-running side project that turns Wittgenstein's Tractatus Logico-Philosophicus into a set of experimental visualizations. Five of them are worth writing up properly: Depth-Rhythm, Silence Spiral, Heartbeat Strip, The Ladder, and Concept Web.

The starting point for all of them is the same: the Tractatus isn't a normal book. It's 526 numbered propositions, arranged in a strict decimal hierarchy - 1, 1.1, 1.11, 1.12 - where each sub-proposition is a comment on the one above it. That structure is unusual enough that it invites visual treatment on its own, before you even get to the content. So each of these views takes one property of the text - its nesting depth, its length, its ending, its recurring vocabulary - and turns it into a shape.

All five now show the full text in three editions side by side: the 1922 Ogden translation, the 1961 Pears & McGuinness translation, and Wittgenstein's German original. Click into any proposition and a reading panel opens with all three, plus Previous/Next buttons and left/right arrow-key navigation to step through in reading order.

Depth-Rhythm

Rationale. The numbering isn't decorative - it's the argument's scaffolding. A proposition like 6.3751 is four levels deep in commentary on proposition 6.

Depth-Rhythm plots reading order against that nesting depth as a timeline, so you can see, at a glance, where Wittgenstein digs in with long chains of elaboration and where he surfaces back to a bare top-level claim.

Features. Every proposition plotted as a point, positioned by depth; click any point to open the reading panel with all three translations; Previous/Next and arrow-key navigation through the whole book.

Silence Spiral

Rationale. The book famously ends with proposition 7: "Whereof one cannot speak, thereof one must be silent."

Silence Spiral takes that ending literally as a destination - every proposition sits on a spiral that tightens as it goes, converging on 7 at the centre. It's a way of visualizing the whole text as a single movement toward its own limit.

Features. Two-pane layout: the spiral on the left, the reading panel on the right. Dot size is driven by each proposition's word count rather than its position on the spiral, so length becomes visible independently of place in the sequence. Click a dot to read; Previous/Next and arrow keys to move through.

Heartbeat Strip

Rationale. The Tractatus has seven top-level propositions - seven "chapters," in effect - and they don't all move at the same pace. Some are terse, single-line claims; others build through dozens of nested sub-propositions.

Heartbeat Strip renders each chapter as its own EKG-style pulse strip, with spike height mapped to word count, so the pacing of the argument becomes a literal rhythm you can read off the page.

Features. Seven pulse strips, one per chapter, on a warm paper background with a deep green accent. Reading panel below the chart with all three translations; Previous/Next and arrow-key navigation across the flattened reading order spanning all seven strips.

The Ladder

Rationale. Wittgenstein describes his own propositions, near the very end of the book, as a ladder: you climb it, and once you've used it to see the world rightly, you throw it away.

The Ladder takes that metaphor at face value. You scroll upward through a ladder of rungs - one per proposition - and the rungs you've passed dissolve behind you, exactly as the book says they should.

Features. A full-page gradient background running from cool slate at the top to pale blue-white by proposition 7, with a faint dot-grid texture; each rung sits on a dark translucent card so the text stays legible across the whole gradient. Click a rung to open a large overlay with all three translations, Previous/Next buttons, and Escape to close. Up/down arrow keys move along the ladder - scrolling when the overlay is closed, paging through translations when it's open. The dissolve itself is continuous and scroll-linked: opacity and grayscale fade smoothly as a rung crosses out of view, rather than snapping off-screen.

Concept Web

Rationale. This one started life as a placeholder - a graph of propositions linked to whichever of 14 keywords happened to appear in the text. I've since replaced it with something better: a philosophical concept classification I actually did, by hand, studying the Tractatus years ago, encoded at the time in an old Lisp-based system called OCML and sitting unused ever since.

I extracted it, cleaned it up, and rebuilt Concept Web to run on it directly - 99 concepts, each with its own description, linked to the 466 propositions I originally read them into. It's no longer a keyword search standing in for interpretation; it's the interpretation.

Features. A three-panel explorer. On the left, a browsable list of the 99 concepts - hover one to preview the description I wrote for it back then (what "Silence" means in the Tractatus, say, and why); click to select it. In the centre, the force-directed graph itself - concepts as hub nodes, propositions as small satellites coloured by chapter, drag and zoom still work as before, but every edge now reflects an actual reading, not a text match. On the right, a browsable list of propositions - all 466 by default, narrowing to just the selected concept's propositions once you pick one. Click any proposition, in the graph or either sidebar, to open a full-screen reading overlay with all three translations side by side and Previous/Next to step through whichever list is currently active.

PS the data for this visualization is also available as RDF.

Conclusion

None of these are meant to replace reading the book - if anything, they're an argument for reading it more slowly.

The Tractatus keeps insisting on the limits of what can be said, and there's something fitting about trying to say a bit more about it anyway, just not in words...

AI Skills Are a Cottage Industry - And That Might Be Permanent

Tue, 16 Jun 2026 00:00:00 +0000

Everyone in my organisation is building their own AI skills. Prompts for summarising documents, triaging emails, drafting updates, analysing data. Each one slightly different. Each one optimised for the person who built it.

The obvious reaction is: we need to standardise this. Create a shared library. Avoid duplication. But I'm not sure that instinct is right - or even achievable.

The Python library analogy, and where it breaks

At first it looks a lot like the early days of Python. A massive proliferation of libraries, everyone solving the same problems slightly differently, and only gradually a market emerging where you could tell the mature, trustworthy packages from the abandoned proof-of-concepts.

And the numbers are striking. PyPI grew from 60,000 packages in 2015 to over 500,000 a decade later - today it lists more than 800,000. Most of them built by single individuals, not organisations. A huge long tail of abandoned or barely-maintained work, and a tiny fraction dominating actual usage. The ecosystem became navigable - but it never converged to a clean, curated set of canonical packages. The market never fully cleared.

But there's a critical difference. Writing a Python library required real technical depth. You had to understand APIs, packaging, dependency management. That friction was also a filter - it meant most libraries were written by people with enough rigour to make something coherent.

With AI skills, that bar is gone. Anyone who understands a problem and knows how to talk to an LLM can build a skill. That's genuinely exciting - domain experts can now build tools for their own domains without needing a developer. But it also means the signal-to-noise ratio will be far worse than PyPI ever was. The Custom GPT store gives a glimpse of what's coming: over 3 million custom GPTs were created within two months of launch, yet only around 160,000 were ever made public. The rest built privately, used once, or quietly abandoned. And unlike code, there's no agreed standard for what a good prompt even looks like - let alone how to evaluate one.

Trust, not quality

Here's what I think changes most fundamentally. With code, you can inspect quality. Read the source, check the tests, look at the issue tracker. With a skill, the source is a prompt - and even if you read it, there's no coding standard for prompts, no way to formally verify correctness, and the outputs are probabilistic anyway. The same skill can give you something sharp on Monday and something subtly off on Friday.

So you can't really evaluate a skill. You can only trust it - or not.

Trust built through repeated use, word of mouth, and the credibility of whoever built it. That's a very different relationship to tooling. It's closer to how you trust a colleague than how you trust a library.

This connects to something deeper. Michael Polanyi observed that "we can know more than we can tell" - that the most valuable knowledge is often the hardest to articulate. Skills are a kind of externalised tacit knowledge. Every refinement you make, every edge case you quietly handle, every implicit assumption about your workflow - it all folds into the prompt in ways you probably couldn't fully explain. And that kind of knowledge doesn't transfer cleanly. Hand someone the skill and you're not handing them the context that shaped it.

Skills resist generalisation

There's another dynamic at play. Skills are so malleable that you can refine them endlessly for your own use case - ask the LLM to adjust them, specialise them further, fold in your own context. The result is something exquisitely fitted to your niche. And unlike extending a Python library, you don't need to understand how it was constructed to modify it.

But that personalisation is also what makes skills hard to share.

This is niche construction in the biological sense - individuals adapting their local environment in ways that work for them, which is rational at the individual level, but which fragments the shared environment over time. With a skill, the tacit knowledge doesn't just surround the artifact. It lives inside it. The divergence compounds with every refinement.

So the ecosystem might not converge the way PyPI did. It might stay fragmented almost by nature - not because people aren't trying to share, but because the artifact resists generalisation.

The marketplace is forming - but the rules are different

There are early signs of infrastructure. GitHub repos cataloguing agent skills are accumulating stars. Dedicated marketplaces are emerging: SkillsMP alone indexes over 800,000 skills scraped from public GitHub repositories. Collections curated by practitioners who've earned credibility are appearing. Social proof is forming.

But look more closely and the PyPI parallel reasserts itself. SkillsMP's catalog size is the most misleading metric in the category - much of it abandoned experiments, half-finished tests, and duplicates, with no quality filter beyond having at least two GitHub stars. And the stakes of getting it wrong are higher than with a badly written library. The first large-scale empirical study of skills in the wild analysed over 31,000 skills from major marketplaces and found that 26.1% contained at least one security vulnerability - spanning prompt injection, data exfiltration, and supply chain risks. Skills execute with implicit trust. There's no pip audit equivalent. Anthropic's own documentation advises users to install only from trusted sources and thoroughly audit anything unverified.

That word again: trusted.

A starred skill repo isn't quite like a popular library. Forking a library produces something legible - another developer can read it, understand what changed, and evaluate the fork on its own terms. Forking a skill produces something that looks almost identical but behaves differently in ways that are hard to articulate. The non-determinism means two forks of the same skill can diverge silently, with no diff to inspect and no test suite to run. Stars travel with the original; trust doesn't automatically transfer to the fork.

The cottage industry might be permanent

The Python ecosystem partially converged - not through top-down standardisation, but through a few dominant packages pulling away on genuine merit, and experienced practitioners developing taste. But even there, the long tail never disappeared. PyPI is navigable, not clean.

I'm not sure the same convergence is coming for AI skills. The artifact itself resists generalisation. There's no deterministic behaviour to diff, no breaking change to announce, no changelog to read. Versioning a skill is a conceptual problem we haven't even started to solve - what does it mean for a skill to "change" if the output drifts rather than breaks?

So the cottage industry might not be a phase. It might be the natural state of this ecosystem.

If that's true, the question isn't how to standardise skills eventually. It's how to function well when fragmentation is the baseline. How trust networks replace registries. How taste develops without inspection. How knowledge survives when it lives in the practitioner, not the artifact.

What the data doesn't show: presenting at CLACSO's funding flows project

Sat, 25 Apr 2026 00:00:00 +0000

I was in Buenos Aires last week for the closing event of Tracking research funding flows in the Global South, a project run by CLACSO in partnership with CWTS/Leiden University and IDRC. I'd been invited to present on Dimensions as a funding data infrastructure - roughly fifteen minutes to explain what we collect, how it's structured, and where we stand on openness.

I came prepared to talk. I left mostly wanting to listen :-)

The question the project is asking

The core research question sounds simple: who funds what, where, in Latin America and beyond? But the fact that this is still a research question - that a multi-year project with serious institutional backing is needed to answer it - says something important about the state of global research infrastructure.

CLACSO represents 927 research and postgraduate centres across 56 countries. The researchers in that room weren't asking theoretical questions. They were asking why the funding landscape of their own region is so hard to see clearly from within it.

That framing stuck with me. Infrastructure is never neutral. What gets counted, and by whom, shapes what gets funded. The project is mapping a terrain that mainstream bibliometrics has largely ignored - not out of malice, but because the tools were built elsewhere and for other purposes.

What Dimensions actually holds

My job was to explain what Dimensions can contribute to that question.

The short version: we hold around 10 million grant records from approximately 800 funders worldwide, linked to 130 million publications and 1.5 billion citations. The linking is the important part - we can trace a grant from the funder through to the papers it produced, across institutions and borders.

For questions about international funding flows specifically, the approach is to query where funder country differs from recipient country. A Canadian grant (say, from IDRC) landing at a Mexican university shows up as an external flow. An EU Horizon grant to a Brazilian institution shows up the same way. In principle, you can build a reasonably detailed picture of how money moves across borders.

We also extract funding acknowledgements from the full text of publications using NLP. This is the second data stream, and it matters because a lot of funding - especially informal, bilateral, or smaller-scale support - never gets registered as a formal grant anywhere. It only appears in the acknowledgements section of a paper. If we can extract those at scale, we can see flows that would otherwise be invisible.

See the notebook Analysing Research Funding in Latin America - A tutorial using the Dimensions Analytics API for more real world examples.

The map has edges

Here's the honest part.

Both data streams have the same structural bias: they see the world through the lens of who's already in the registries and the indexed journals. That means they're reasonably good at capturing flows into Latin America from Northern funders - IDRC, NIH, Gates Foundation, EU Horizon. These organisations register their grants. Their grantees publish in indexed journals.

What we see much less clearly is domestic funding, South-South flows, and the informal support structures that keep a lot of regional science running.

Take Argentina. Over the past couple of years, the CONICET funding crisis has been serious - budget cuts, delays, researchers leaving. Against that backdrop, the question of how much Argentine science now depends on international funders isn't just academic. It matters for policy, for sovereignty, for what gets studied. We can begin to answer that question with Dimensions data. But we can't answer it completely, because the picture gets blurry exactly where domestic and informal flows dominate.

South-South flows - say, a Brazilian agency funding work in Colombia, or African-Latin American scientific partnerships - are largely invisible in the data. Not because they don't exist. Because the infrastructure wasn't built to see them.

See the notebook Latin America Research Funding Patterns- A tutorial using the Dimensions on Google BigQuery for more examples.

What I learned from the feedback

The researchers at this event had been using Dimensions - and pushing it hard. The feedback they gave wasn't the vague kind you get from people who've skimmed the documentation. It was specific. Funder X isn't linked correctly. Country Y's national agency has three different names in the data and they resolve to different entity IDs. The acknowledgement extraction misses a common Portuguese grant number format.

That's the kind of feedback you can only get from people who've actually tried to do real work with the data and hit the edges. It's uncomfortable and it's exactly what you need.

I also found the conversation about openness genuinely refreshing. The panel included colleagues from the Barcelona Declaration on Open Research Information, and there was a real dialogue about what open means in practice - not just in principle.

Dimensions occupies an honest middle position here: we are a commercial company but we've always actively engaged and supported the research community for example by releasing open datasets, or by giving them free access to our database for scientometrics projects.

I came back with a list of things to look at in our funder coverage. I also came back with a clearer sense of why the work matters.

That's not a bad outcome for fifteen minutes of slides.

Thanks to the CLACSO team, IDRC, and CWTS/Leiden University for the invitation. The project homepage is at clacso.org/fundingflows.

Dimensions as scientific data infrastructure: funding data, international flows, and open access

Sun, 12 Apr 2026 00:00:00 +0000

The Shifting Landscape of Research Analytics: From Data Access to Trustworthy Insights

Tue, 03 Mar 2026 00:00:00 +0000

I watched the "Vibe Coding with Open Alex" video recently, and it's been rattling around in my head since.

The barrier has dropped

A researcher who used to need a vendor's tools or a specialist analyst can now point an LLM at a CSV of publications and get something meaningful in an afternoon. NotebookLM, Cursor, ChatGPT with code interpreter - these tools have commoditised a lot of what used to be specialist craft. Knowing how to wrangle data, write the right pandas operations, build a decent visualisation: it's still a skill, but it's no longer a bottleneck.

The community is noticing. People at ISSI or STI who wouldn't have called themselves coders are now shipping notebooks.

What vendors still have

The traditional pitch - "we make the data accessible and usable" - is eroding. But it hasn't gone. The remaining advantages are real, just narrower:

Data quality and curation is still genuinely hard. An LLM can't conjure a correctly disambiguated author profile from messy affiliation strings. The judgement calls embedded in our data - what counts as the same person, how to handle name variants, how to classify access type - represent accumulated domain knowledge. You can't replicate that just by having a model and a raw corpus.
Coverage and timeliness still matter. The gap between Dimensions and something patched together from Unpaywall + OpenAlex + CrossRef is real, especially at scale and for recent data.
Trustworthiness as infrastructure. Institutions and funders need something they can cite in a report to government. That requires provenance, audit trails, a vendor who can be held accountable. An ad-hoc notebook doesn't cut it.

Where the value is moving

The role of a data vendor is genuinely shifting. Clients increasingly don't want a database to query - they want answers and frameworks.

That means becoming more opinionated: providing benchmarks, recommended metrics, pre-classified outputs. Less neutral data pipe, more methodology partner. The value-add is less about retrieval and more about knowing what to measure and why.

Something interesting is also happening with risk. Clients used to question their own capability to analyse data. Now many feel they have that capability - so the concern has shifted to whether the underlying data is trustworthy enough to build on. That's a subtle inversion, and it increases scrutiny on data quality. Pressure for some vendors, opportunity for others.

The vendors who thrive in five years will be the ones who made their data legible to AI workflows, not just to human analysts.

Event: Hayden Thorpe live in Pordenone

Mon, 27 Oct 2025 00:00:00 +0000

I went to see Hayden Thorpe for the first time the other night at Pordenone's Ex-Convento Live venue (Italy). It's quite funny because I had never heard of this musician before last week. I just ran into him on Spotify and immediately felt it was so introspective and inspiring, full of simple yet deep melodies. I loved the performance and am looking forward to seeing more of him!

Here's a brief excerpt:

A Thorpe Music Box

A little interactive experiment..

A lyric collage built from Hayden Thorpe's 2019 album Diviner. Phrases cut from all nine songs are scattered across the screen like torn newspaper clippings — move your mouse to illuminate them, click any fragment to read the full song on Genius.

The piece runs entirely in the browser with no server: layout, sound, and interaction are generated in pure HTML and JavaScript. Audio is synthesised on the fly using the Web Audio API — a soft paper-flick when you brush a phrase, and occasional minor-pentatonic tones drifting in from the edges as the collage quietly stirs around you.

Thorpe Music Box click to illuminate

↗ open full screen

The Dimensions API: a domain specific language for scientometrics research

Wed, 01 Oct 2025 00:00:00 +0000

We describe the Dimensions Search Language (DSL), a domain-specific language for bibliographic and scientometrics analysis. The DSL is the main component of the Dimensions API (version 2.12.0), which provides end-users with a powerful, yet simple-to-learn and use, tool to search, filter, and analyze the Dimensions database using a single entry point and query language. The DSL is the result of an effort to model the way researchers and analysts describe research questions in this domain, as opposed to using established paradigms commonly used by software developers e.g., REST or SOAP. In this article, we describe the API architecture, the DSL main features, and the core data model. We describe how it is used by researchers and analysts in academic and business settings alike to carry out complex research analytics tasks, like calculating the H-index of a researcher or generating a publications' citation network.

Enhancing the Accessibility of ORCID Public Data, now additionally hosted on Google BigQuery

Mon, 16 Jun 2025 00:00:00 +0000

ORCID is committed to openness, exemplified by the annual release of its Public Data File since 2012. This dataset, encompassing all public ORCID records, has been downloaded over 190,000 times and serves as a resource for analyzing research community dynamics, scientific migrations, collaboration networks, and ORCID adoption trends. However, the file’s substantial size poses challenges for users lacking advanced data management skills, hindering exploratory analyses

Performing Research Analytics at Scale: the Dimensions Reporting Platform

Mon, 02 Jun 2025 00:00:00 +0000

Performing Research Analytics at Scale: the Dimensions Reporting Platform

Mon, 05 May 2025 00:00:00 +0000

Event: Installation at Sydney Data Arena

Mon, 17 Mar 2025 00:00:00 +0000

My piece, "Dreamy Pianos - Study No 1 in C Minor," is part of a new sonic installation by Simon Porter, currently showcased at the University of Technology Sydney.

The installation took place on March 4, 2025, as part of the Digital Science Showcase within the UTS DataArena. The installation, titled Australian Institution Co-authorship Collaboration Patterns, was developed by Simon Porter and represents a network of co-authored researchers affiliated with Australian institutions from 2018-2023.

Each researcher has been color-coded by the scientific field code they are most associated with. Each researcher is depicted by a sphere and given a size based on the number of publications produced. More information is available on the installation's Figshare page.

Find out more about my piece: Dreamy pianos - Study No 1 in C Minor.

Alleanze Ingannevoli: Svelare il lato nascosto della ricerca

Tue, 28 Jan 2025 00:00:00 +0000

Introdotta nel mondo della ricerca nel 2024, la scientometria forense (Forensic Scientometrics o FoSci) è una nuova disciplina sviluppata per facilitare l'analisi dei dati di pubblicazione, delle reti di co-autorialità, delle collaborazioni istituzionali e altro ancora. Le tecniche FoSci permettono di portare alla luce aspetti della ricerca scientifica che indicano potenziali rischi, come la partecipazione occulta a reti di ricerca compromesse o i rapporti con individui o gruppi noti per la diffusione di produzioni scientifiche di dubbia qualità o fraudolente.

Unpacking OpenAlex topics classification

Fri, 27 Sep 2024 00:00:00 +0000

In this post I have taken a closer look at the classification of scientific disciplines in OpenAlex, a recently developed database of scientific works. The topics classification has been entirely generated computationally using a mix of citation clustering techniques and LLM-based labeling. The results, although not always so precise, are definitely worth exploring further.

Last week I went to the STI 2024 conference in Berlin, the annual European get together of experts in the area of research analytics and evaluation.

Lots of interesting talks but probably the thing that stroke me the most was the general excitement and sense of expectation about OpenAlex.

If you haven't encountered yet, OpenAlex is a 2022-released open database of research publications and other related content e.g. datasets, authors, journals etc.. developed by OurResearch.

Pretty much all of OpenAlex data and services and made available free of charge (with options to pay more for premium services) under a handy CC0 license.

Scientific topics

One thing that caught my attention is the topics classification in OpenAlex. This is as hierarchical classification of scientific topics that consists of various thousand categories organized into a 5-levels hierarchy: keywords, topics, sub-fields, fields and domains.

The gist of this approach, according to the OpenAlex team:

Topics

Works in OpenAlex are tagged with Topics using an automated system that takes into account the available information about the work, including title, abstract, source (journal) name, and citations. There are around 4,500 Topics. Topics are grouped into subfields, which are grouped into fields, which are grouped into top-level domains. This is shown in the diagram below, along with the counts for each.

Keywords

Our team put together a new implementation of keywords based on our Topics. There are currently over 26,000 keywords and we expect to add more as time goes on. [...] With our new topics system that was developed in coordination with CWTS, we came out with a list of 10 keywords for each topic. In order to assign keywords to works, we took the topics assigned to that work (at most 3 topics), pulled the keywords associated with those topics (at most 30 keywords, for now) and then determined the similarity of the keyword to the title/abstract using embeddings (and the BGE M3-Embedding model).

See the official data documentation for more, and in particular also worth looking into the paper OpenAlex: End-to-End Process for Topic Classification that describes the process that OpenAlex uses to assign topics to works. Note: this approach is itself taking advantage of publications clustering technology developed by Leiden CWTS group.

Exploring the topics hierarchy

I downloaded the topics to have a closer look. This jupyter notebook pulls the topics dataset and turns it into a format that can be more easily visualized, eg by the FoamTree chart.

See the results in full-screen on this page.

First impressions:

The classification is pretty impressive, especially considering it's been fully generated algorithmically via a combination of citation based clustering methods and generative AI.
The topics titles are generally pretty descriptive and in some cases surprisingly specific too EG see Zebrafish as a Model Organism for Multidisciplinary Research and Automated Currency Recognition and Authentication (wow - who would have guessed that's a thing!).
The hierarchy also makes sense to me, of course modeling scientific knowledge is quite an opinionated piece of work and there is no right or wrong way to do it, but in general I couldn't find any parent-child relationships that stood out as wrong or completely inadequate.

One thing worth of notice is that all topics have either a Wikipedia or a Wikidata page (or both), which makes it pretty handy for cross-database analytics projects.

I also had a go at transforming the classification into an ontology or more precisely an RDF (SKOS) model, so more compatible with knowledge graph technologies like wikidata and schema.org. If you are interested, the results are here openalex-topics-rdf.ttl and here a summary of generated ontology documentation for it.

Now, on to the less so good bits..

I took at look at the actual publications that get classified within these topics, but the results are a bit spotty. This is just a very preliminary analysis.. as I basically ran a few spot checks of papers I am familiar with, and found them in surprising 'places'.

For example, looking at my publications page on OpenAlex:

The main topic Semantic Web and Ontology Development is pretty spot on.

But it turns out that I have done work in two areas that I wasn't quite expecting: Management and Reproducibility of Scientific Workflows and Statistical mechanics of complex networks. I suppose it could be argued that my work is tangential to those areas?

Looking at other publications, the classification can be instead plainly wrong:

On the New Perspectives on Medieval Scotland publication, Scotland and Wales seem to have gotten mixed up. Or maybe is it because the citation network of Welsh historian is also correlated to Scottish history?

I wondered if these clusters might be more representative of groups of researchers / publications connected through an emerging topic, which may not be established or even known to them as a distinct area. Fascinating question..

Another example is this paper which is about HCI and databases, but instead is classified as NLP:

Conclusions

Without a doubt, an automatic classification of science topics is a hard problem to solve. So, despite the sometimes patchy assignment of topics to works, the fact the OpenAlex team did it all using citations-based clustering is still pretty impressive. I honestly hope there will be future iterations where we see this progressing further!

The enthusiasm around OpenAlex is justified, I believe. OpenAlex has has a pretty good API, good coverage of publications data (see this arXiv report) and a committed startup team developing it. All good ingredients that contribute to making the research community feel this may be a stepping stone towards a more open, free and reliable research data infrastructure.

I’m genuinely sympathetic towards initiatives like this. An ecosystem with open Research metadata makes science progress faster. I’ve had my fair share of experience with such projects while I was working on SN Scigraph.

Long term, no one knows whether this can be a sustainable enterprise. Data quality, funding, the need to keep innovating and for users' uptake are surely among the main challenges that OpenAlex has to deal with.

Digital Science's Daniel Hook recently outlined a model for thinking about innovation where both open data and commercial enterprises can - and should - coexists. That makes a lot of sense to me, so I'd like to think of a future where such synergies do become real .. and I guess that this little learning exercise about OpenAlex is a step in that direction.

Dimensions: Calculating Disruption Indices at Scale

Fri, 06 Sep 2024 00:00:00 +0000

Evaluating the disruptive nature of academic ideas is a new area of research evaluation that moves beyond standard citation-based metrics by taking into account the broader citation context of publications or patents. The "CD index" and a number of related indicators have been proposed in order to characterise mathematically the disruptiveness of scientific publications or patents. This research area has generated a lot of attention in recent years, yet there is no general consensus on the significance and reliability of disruption indices. More experimentation and evaluation would be desirable, however is hampered by the fact that these indicators are expensive and time-consuming to calculate, especially if done at scale on large citation networks. We present a novel method to calculate disruption indices that leverages the Dimensions cloud-based research infrastructure and reduces the computational time taken to produce such indices by an order of magnitude, as well as making available such functionalities within an online environment that requires no set-up efforts. We explain the novel algorithm and describe how its results align with preexisting implementations of disruption indicators. This method will enable researchers to develop, validate and improve mathematical disruption models more quickly and with more precision, thus contributing to the development of this new research area.

Livecoding: Techno jam

Fri, 05 Jul 2024 00:00:00 +0000

This piece plunges into a heavy, atmospheric soundscape driven by the collision of overlapping basslines and raw industrial noise.

The melody is provided by a synth line that continuously varies a diatonic interval within a minor scale, creating a sense of subtle suspense and emotional depth. The livecoding environment allows this melodic movement to be modified in real-time, enabling dynamic, expressive shifts in the harmonic texture.

Crucially, the percussion—specifically the kick drum and snare—is not sequenced conventionally. Instead, these elements are activated only when the synth's oscillator exceeds a defined threshold, linking the rhythm directly to the intensity of the melodic line (which is also driving the filter's cutoff and resonance). This creates a highly reactive, breathing relationship between the melodic and percussive layers.

Listen with a good pair of headphones!

Techniques

Rhythmic patterns

Interesting rhythm patterns can be simply generated by using non standard beats.

Eg here we have a 5/3 downbeat, with the hi-hat playing at each 1/4. This results in two distinct syncopated beats every bar.

(at 5/3 1/4 (playk *kit-hhat3* 90))

Synth line

Generated using a varying diatonic interval on a minor scale. Which can be modified in real time for nice effects etc..

(play bass (mkint C2 (oneof 0 3 4 5) 'm ) 90 dur)
(mcc *cutoff* osc1)
(mcc *reso* osc2)

Oscillators driving the DrumKit

The kick drum and snare are activated only when the synth oscillator (driving the cutoff and resonance) goes beyond a certain threshold.

This allows to build up momentum semi automatically.


;; cyclical drums
(if> osc1 60 
  (at 1 1/2 (playk *kit-kick* 90)))

(if< osc1 60 
  (at 10/3 1/4 (playk *kit-snare* 90)))

Credits

Livecoding software: Extempore
Sounds: Ableton Live
- syn1: Analog Lab V / Kick Glitch Hits
- syn2: ZebraHZ / Chordination
- syn3: Kontakt / Ethereal Earth / Trumpet Fireflies
- syn4: Police Radio Sample

Full source on GitHub: 2024-07-techno-beat.xtm