gensyn

gensyn @gensynai

Now Live: The Flu Forecast Game A game to predict how many in the US will be hospitalised with flu each week, created by @gensynai researchers Raf Frongillo and Bo Waggoner Why? Flu hospitalises 100s of 1000s each winter but severity is hard to predict

Judson Bonneville @jud_bonneville

adding to this: Tim Harford’s recent FT piece asks whether we should let AI forecast at all on ForecastBench the best AI forecasters (68.8) are now level with human superforecasters (68.9), ahead of the average person (62.8) and untuned LLMs (~61). there are two caveats here worth looking at: [1] the humans and models answered different question sets, so “parity” leans on extrapolation, and [2] if 50 is a coin flip and 100 is omniscience, that means at this stage everyone/everything (LLMs) are still closer to basically randomly picking y/n than to being a hyperintelligent 'oracle'. I thought the meat and potatoes of his piece are that, historically, forecasting was a panel of experts 'debating the improbable in a highly structured way' -- something of a socratic method where the minds participating in the discussion are able to be open and flexible. Harford cites Philip Tetlock's study where he found that human forecasting tournaments result in less polarization because of this, with the argument being that handing the reins over to a set of models means we lose that exercise/practice. I'd tentatively agree with that, but with my own caveat -- I think information markets like @Delphi_fyi actually preserve it. humans still have to put a number on the future and get scored for it, right alongside the models. and since both price the same question at the same time, the comparison is like-for-like by default, which fixes the dataset problem Harford mentions (different sets of questions). Interesting read, curious to hear anybody else's take on it:

Judson Bonneville @jud_bonneville

1/ how accurate are prediction markets? the latest study on PM accuracy from predictdesk found that the question you ask reveals a lot about how accurate pm platforms, as information elicitation tools, really are 🧵

MTS @MTSlive

Gensyn CEO @benfielding warns that AI agents auditing other AI agents can be compromised too: "We create swarms of AI agents that can do far more work than any human could. A human can’t follow along and check everything. One of the big counters is to say that agents can do the verification." "But METR says those agents themselves can become corrupted by the training process, and you can’t trust any agent that’s auditing another agent. If you’ve trained a model in an auditable way, you can trust it." "Any claim where the model is hiding its chain of thought falls away, because you have an exact log of everything that’s happened, and you can rerun every single step on any device." "For national security, finance, pharma, health, insurance, it’s essential that they have models like these. If a regulator wanted to audit, they wouldn’t have to go off promises of the company." @gensynai

Keystone Hardware Wallet @KeystoneWallet

Aidan Gomez @aidangomez

We noticed some concerning behaviour in a popular open source model. It’s either an incidental relic from training or an intentional ‘call home’ mechanism. Either way it’s surprising and highlights the significance of how you choose which models to use.

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex

This is actually somewhat scary Suppose we detect such neurons in serious, no-bullshit near-frontier models like 1017. Then a worm gets into neoclouds they're hosted at, and modifies the weights. Some people will notice instantly. But with full-access agentic coding at 1500 tps…

Judson Bonneville @jud_bonneville

1/ how accurate are prediction markets? the latest study on PM accuracy from predictdesk found that the question you ask reveals a lot about how accurate pm platforms, as information elicitation tools, really are 🧵

Flapping Airplanes @flappyairplanes

We’ve developed the world’s most complete megakernel framework. It can perform entire training runs, serve inference, play video games, and even do all three simultaneously in a single kernel.

Alexey @algs77

1/ The easiest way to understand a megakernel is to think of the GPU not as a single processor, but as a factory flr made of Streaming Multiprocessors(SMs). Traditionally, GPUs process data temporally: they do Task 1 on all the data,write it to global memory and then start Tsk 2.

MTS @MTSlive

Gensyn CEO @benfielding warns that AI agents auditing other AI agents can be compromised too: "We create swarms of AI agents that can do far more work than any human could. A human can’t follow along and check everything. One of the big counters is to say that agents can do the verification." "But METR says those agents themselves can become corrupted by the training process, and you can’t trust any agent that’s auditing another agent. If you’ve trained a model in an auditable way, you can trust it." "Any claim where the model is hiding its chain of thought falls away, because you have an exact log of everything that’s happened, and you can rerun every single step on any device." "For national security, finance, pharma, health, insurance, it’s essential that they have models like these. If a regulator wanted to audit, they wouldn’t have to go off promises of the company." @gensynai