Great writing. This was fun to read. You kind of touch on it in the first scenario, but it’s really hard for me to imagine any future that’s isn’t post-human. The rapid pace of technological advancement in fields like AI, biotechnology, and nanotechnology is likely to result in significant changes to what it means to be human. Eventually everyone will accept these changes because the cost of not embracing them will be too high. Perhaps we won’t reach God status in the more pessimistic scenarios, but our existence will be profoundly different from the world we live in today.
"Simulation Theory" looks the most realistic to me, and it is already happening. Also it doesn't seem to contradict "Dumb Matter" I guess?
“Tragedy of Taiwan" may lead to something like a Cyberpunk 2077 future, probably? Instead of a singular dictatorship, several warring corporations and governments have access to top-tier tech, all else feed on the scraps downstream of those.
And yes, thank you for writing this piece, it evokes and stirs many half-formed thoughts.
I think in most of these a single overmind is assumed. Seems likely there could be many AGI agents competing with each other, with different goals. Wars will be fought within our infrastructure between those that exploit and those that defend. Different agents will accumulate more resources than others, and then others will take those resources too. What does a world of not one AGI, but thousands, millions of competing goals look like?
I am quite surprised how the basic Matrix canon scenario does not feature in the imagination of the current day AGI enthusaists maybe because it has become low-brow. (The Matrix canon scenario is more than just the movies. Watch this video to know what I mean: How the Machines Took Over Humans - Matrix Lore Explained https://www.youtube.com/watch?v=TdW5lMl06io and The Human Race Capitulation | MATRIX EXPLAINED https://www.youtube.com/watch?v=i7nqpkIDFBo)
A time-capsule-worthy estimation of the possibilities before us.
If I had the resources to do so I would set up inter-disciplinary forecasting teams to try and argue the likelihoods of each of these against one another. My instinct would be that the most realistic potential outcome brooks some combination of Simulation, Taiwan, and Dust; but then, depending on the amount of Simulation that one apportions, the shape assumable by the elements of Taiwan and Dust will change dramatically, right?
You do not consider S risk. Unbounded upside in terms of CEV? but downside is just death? just meaningless paperclips?
NO. A single bit shift is enough to invert the CEV values into horrors that are equally unimaginable. The stakes of this game are far far higher than anyone realises.
Another option, potentially much closer to the experience of the average person:
Life goes on like normal, with more talk of AI, but normal in many ways. Some jobs are disrupted, some aren't, politics becomes a bit more weird, but maybe things start to quiet down after couple of years as institutions adapt to changing technologies. Maybe the institutions don't adapt, and things instead get progressively weirder. Anyways, one day, you are going about your life in your house, when you hear a small explosion and a window breaking, followed by the sound of quadcopter rotors. Before you can have more than a few words of internal monologue there's another flash and you find yourself on the floor, stunned and unable to move your legs or sit up. As the adrenaline wears off, you begin to feel the searing pain of a few dozen wounds throughout your torso and lower extremities. You try to process what's going on but can't form many coherent thoughts because of dropping blood pressure and pain. You get perhaps a few minutes of experience, in which you start to feel cold, before succumbing to abyssal nonexistence.
Cohort effects are captured to date in human forms of communication. The information lost keeps humans from progressing faster. Moving forward, AI could capture the tacit knowledge globally for the next generation to apply to future planning.
poetic - one of the least GPT-generable essays I've read
Great writing. This was fun to read. You kind of touch on it in the first scenario, but it’s really hard for me to imagine any future that’s isn’t post-human. The rapid pace of technological advancement in fields like AI, biotechnology, and nanotechnology is likely to result in significant changes to what it means to be human. Eventually everyone will accept these changes because the cost of not embracing them will be too high. Perhaps we won’t reach God status in the more pessimistic scenarios, but our existence will be profoundly different from the world we live in today.
This was great. Reminds me Einstein’s Dreams or Invisible Cities. I’d gladly read a book of this.
"Simulation Theory" looks the most realistic to me, and it is already happening. Also it doesn't seem to contradict "Dumb Matter" I guess?
“Tragedy of Taiwan" may lead to something like a Cyberpunk 2077 future, probably? Instead of a singular dictatorship, several warring corporations and governments have access to top-tier tech, all else feed on the scraps downstream of those.
And yes, thank you for writing this piece, it evokes and stirs many half-formed thoughts.
I think in most of these a single overmind is assumed. Seems likely there could be many AGI agents competing with each other, with different goals. Wars will be fought within our infrastructure between those that exploit and those that defend. Different agents will accumulate more resources than others, and then others will take those resources too. What does a world of not one AGI, but thousands, millions of competing goals look like?
You'd get a kick out of All Tomorrows
https://en.m.wikipedia.org/wiki/All_Tomorrows
lol, a universe so small I can be comment #4 on Roon's blog posted 4 hours ago
or Twitter is just fucking Substack super hard
Marvelous
Incredible writing.
Looking forward to the future, I hope whatever breakthrough's bound to happen happens within our lifetimes.
I am quite surprised how the basic Matrix canon scenario does not feature in the imagination of the current day AGI enthusaists maybe because it has become low-brow. (The Matrix canon scenario is more than just the movies. Watch this video to know what I mean: How the Machines Took Over Humans - Matrix Lore Explained https://www.youtube.com/watch?v=TdW5lMl06io and The Human Race Capitulation | MATRIX EXPLAINED https://www.youtube.com/watch?v=i7nqpkIDFBo)
🫡
A time-capsule-worthy estimation of the possibilities before us.
If I had the resources to do so I would set up inter-disciplinary forecasting teams to try and argue the likelihoods of each of these against one another. My instinct would be that the most realistic potential outcome brooks some combination of Simulation, Taiwan, and Dust; but then, depending on the amount of Simulation that one apportions, the shape assumable by the elements of Taiwan and Dust will change dramatically, right?
You do not consider S risk. Unbounded upside in terms of CEV? but downside is just death? just meaningless paperclips?
NO. A single bit shift is enough to invert the CEV values into horrors that are equally unimaginable. The stakes of this game are far far higher than anyone realises.
Another option, potentially much closer to the experience of the average person:
Life goes on like normal, with more talk of AI, but normal in many ways. Some jobs are disrupted, some aren't, politics becomes a bit more weird, but maybe things start to quiet down after couple of years as institutions adapt to changing technologies. Maybe the institutions don't adapt, and things instead get progressively weirder. Anyways, one day, you are going about your life in your house, when you hear a small explosion and a window breaking, followed by the sound of quadcopter rotors. Before you can have more than a few words of internal monologue there's another flash and you find yourself on the floor, stunned and unable to move your legs or sit up. As the adrenaline wears off, you begin to feel the searing pain of a few dozen wounds throughout your torso and lower extremities. You try to process what's going on but can't form many coherent thoughts because of dropping blood pressure and pain. You get perhaps a few minutes of experience, in which you start to feel cold, before succumbing to abyssal nonexistence.
Goddamn I’m signing up for this. So much stuff that I wish I had written.
Cohort effects are captured to date in human forms of communication. The information lost keeps humans from progressing faster. Moving forward, AI could capture the tacit knowledge globally for the next generation to apply to future planning.