Why language models need not get worse
Thursday 9 February 2023, 00:00Sam Kriss has a Substack posting in which he describes the zairja of the world and then links it to his ideas on why AI is getting worse. Basically, what I get from the piece is that he's saying successive generations of GPT models have produced less and less valuable output as they better and better approximate the average content of the World Wide Web. Nearly all of the Web is garbage, and so an approximation of the average Web document is garbage too. The output from older models is weird and interesting because more random and less accurately approximating the average Web document; the output from new models is boring and worthless. My own view is that these issues are not as serious or as inevitable as he presents them. There seem to be some gaps in Kriss's understanding of how language models, and the ChatGPT Web service in particular, work; and filling in the gaps points to some natural solutions for the problem he describes.
自分のドキュメントクラスを作ろう
Wednesday 21 December 2022, 00:00LaTeXで文書を書いてなら、時々すべてに体裁を指せたいです。
たとえば僕の仕事で、アナログシンサーのマニュアルを書いて、いつも同じの体裁が欲しいです。
LaTeXのデフォルトは普通過ぎます。 個性的な体裁を書きたいです。 でも全部のNorth Coast Synthesisユーサーマニュアルは、同じの体裁が必要です。 だから、自分のLaTeXクラスファイルを作りました。
Neural networks and the Unix philosophy
Sunday 25 September 2022, 16:13There are a number of directions from which we can look at current developments in deep neural networks and the issues I raised in my streamed comments on pirate AI. Here's a summary of the implications I see from the perspective of the Unix philosophy.
Somebody but not anybody
Tuesday 20 September 2022, 15:29We have a patient suffering kidney failure; he's in a lot of pain and the disease will soon kill him. He could be saved - if someone would donate a kidney to be transplanted into this patient's body. So, as a matter of ethics, somebody ought to do that, right?
But who, exactly?
On language modelling and pirate AI (transcript)
Sunday 11 September 2022, 00:00I've been thinking a lot recently about the current developments in deep neural network generative models, and how they implicate free software issues. I went into a lot of this stuff in my July 25 Twitch stream, and although I'd also like to write up some of my thoughts in a more organized way, the transcript of the relevant video content is a pretty decent introduction in itself, so I thought I'd post it here.
This is an automatically generated transcript (by another of these models!), with a fair bit of manual editing - so it's spoken English written down, not the usual standard form of written English. The video itself will go online eventually, but it may be a long time before that happens, because my video archive server is at capacity now and I probably won't expand it until the Twitch stream gets enough viewers I can sell some subscriptions to pay the bills.
The end of Lotto 6/49 as we know it
Sunday 4 September 2022, 00:00The rules of Lotto 6/49 are about to change in a way that I think is pretty significant. It's drastic enough that I think we might well say Lotto 6/49 is coming to an end, being replaced by a new game that is not even properly a lotto game anymore and only happens to reuse the name.
Embodying the machine
Sunday 28 August 2022, 00:00There's an idea in neuroscience that the brain maintains what is basically a 3D model of the body - the "body schema" - and that when we use tools, they become part of that schema. Claimed evidence for this concept includes studies where people were asked to perform a task with a tool, like picking up an object with a gripper like a pair of tongs, and then measurements afterward suggested their body schema had changed. For instance, the person's response time and the way they moved when picking up an object without a tool might change in a way that suggested their internal estimate of the length of their arms had increased; or their perception of the distance between two touches on their forearm might increase, as if they were at some deep level measuring against an estimated forearm length that had increased.
Bad enough to qualify
Sunday 21 August 2022, 00:00Today is the Queen's Plate, a horse race here in Toronto that is considered a big enough deal to be reported in the general news media, and it had me thinking about a peculiarity of horse racing: the qualifications to enter a race are usually backward compared to human sports. Whereas human athletes need to be good enough to qualify for an event, horses need to be not too good.
The recipe for watering down a goal
Sunday 14 August 2022, 00:00The overlapping circles of "life coaching," "mindfulness," and certain fluffy religions, have a highly-developed technology for achieving personal goals, and it works well - provided you're willing to follow the recipe.
What nobody else will do
Sunday 7 August 2022, 00:00Somebody asked whether there's a purpose to my life (or that of whoever cared to answer) - on Twitter, where my profile description currently consists the the three words "Anchorite, apostate, asteroid."
I don't think there is. I used to think there was, but it's been some years since that fell apart for me. However, it's interesting what's left.
