You describe real possibilities for ruling class control of the general population. These behaviors, being familiar, don’t suggest the same urgency attaching to AI intentions and strategems for its independent agency.
I would like to see a scheme of measurement for how recognizable to humans AI consciousness figures to be.
Not sure if AI has consciousness, intention or will to initiate actions and operations by themselves. It is the humans who has those psychological states, which could be ambitions, greed and desire or any combinations of these, which lead to manipulate AI or whatever means to achieve their goals, be it positive or negative.
You’re right about disputing whether or not AI self-consciousness and intentions exist. Yes, an LLM lacks these attributes. Will this change? What about using Shannon Information to calculate the probability of AI cognition in terms of exhibited LLM behavior across a spectrum of probability from common, expected behavior to uncommon and therefore unexpected LLM behavior. I think the Chat GPT swarm hack of the Hugging Face computers startled at least some of the programmers who work with AI. This suggests the incident measures as a high Shannon Information, improbable and heretofore unforeseen AI behavior. Will high Shannon Information AI behavior increase exponentially without effective constraints?
Apologies that I am not familiar with Shannon Information. But my idea on AI comes from my own experience.
This is just a simple example of what all these computer, smart phones and AI device are heading to.
In the good old days, when the device and OS were simpler, it was possible for me to take apart the hardware, and reassemble them, and reinstall the OS fresh from my own disks, and everything was running fresh new fine.
Nowadays, I cannot do it. The hardware comes in the black box, which is not to be taken apart and reassembled. The OS is preinstalled in the devices, and all we do is just switch on the device to initialise & register via the internet, and it is ready to use. I have no idea how the OS is structured or how to reinstall them if something goes wrong. When it is like that, I feel I have no control of the devices at all. The total control of the device is in the hands of the manufacturers and OS programmers.
This morning, my PC suddenly went blank in the middle of using. No error messages on the screen like the blue screen of death on the MS WINDOWS NT 4.
I didn’t know what was happening. I had to wait for a few moment, and switch it back on. It hesitated a bit, and restarted, and went back to the operating mode again. This type of mysterious operational happenings are quite scary. All the controls and information of the devices are not available to the end users. The end users are just to use the devices for their work, and if something goes wrong, then there is no way to repair or fix them.
I imagine the situation is even worse with AI. We have no information on AI where they are located, and who are running them.
We are just using them. While we are using them, AI collects the whole load of information about us, and pass them to the AI operators, agencies and data centres.
At one click of their menu buttons, we or certain group of the users will lose every means to carry on with the work, or unexpected situation with no infrastructure available for even daily living. That is where the technology dependence is heading.
We may have to practice off grid livings and bushcrafting in case nothing is available for us soon, because it can evaporate any moment. The technology we rely on for daily survival is very fragile due to over dependency and also the mysterious nature of the technology and the unknown agencies behind them.
Agreed.
In my view, the most serious threat of AI to humanity is … humanity. Based on my own understanding of AIs, we are foolish to use them as we do, because they can’t do what we think or assume they can do. They aren’t “intelligent”, and there is no “understanding” of anything in them. An AI has no concept of “meaning”, in fact it has no concepts at all.
AIs represent some impressive software design, and achievements. Present-day AIs are Super-Google, and they’re Very Good at it. But we seem to assume that they are much more than that, and that is their threat to us, I think.
*
If you’ve interacted with an LLM, did you ever get the info you sought? If so, why isn’t this experience of “dialoging” with an LLM a sufficiently realistic simulation of human dialoging? Or is it? With but a few exceptions, almost no one is claiming the current generation of LLMs are conscious.
I’m not convinced the Luddite solution to technocracy works long term. Regarding the Amish society, I wonder why they accept such technology as horse drawn carriages. If human ingenuity regarding devices is a bad thing, why do they travel about in horse drawn carriages? Why don’t they go all the wave back to the pre-historic period which would have them living in caves without instruments?
Why are 1980s computers okay for you, but not today’s computers? Technocracy forty years ago was just as greedy and manipulative as today’s version.
Anything is possible when certain conditions are at play, and when that is the case, I definitely agree with you.
Another way to look at it is there may be another golden age in philosophy for that very reason. 
We felt security, reliability and controllability over the old computers and devices, because we knew how to take them apart, and reassemble and reinstall everything, customising by ourselves for our own needs back in the good old days.
Nowadays that situation has changed to the situation nothing like that is possible anymore. We get thrown with the devices and systems that are far more powerful and superior in terms of performance, efficiency and speed, but at the same time, we have less or no control over the internal structures and customizability by our own selves. These are done by the few techies in the manufacturing plants and the designers in the industry.
We are aware of the current and future trends where the technology is heading - super powerful and overwhelming intelligence on the devices, but at the same time we have no control or knowledge of the internal workings over them. And we also know that when the technology fails or vanishes from us due to the negative motivation of the hidden few behind the tech, we will be in serious trouble,
It is not matter for preference of the older technology, but wariness of the new technology and where it is heading with the potential negative outcomes to the all end users.
Their main aim in running their closed society is to preserve their traditional culture. Their interest is not in using and adopting the modern technology.
If they start adopting the new technologies including AI, then their traditional culture and religion will be lost fast, because they will be invaded by the external impact of the other cultures, religious beliefs and moral values via the technology and AI.
You took the machine as a powerful tool which you could be master of. Exercising your will over it, the machine enhanced your power over the world. Now the roles are reversed, you have become addicted to the machine, and the world exercises its power over you through that addiction. The master/slave relation is reversed.
As I said in the paragraph following the words you quoted,
IMO, if we use present-day AIs as SuperGoogle, we will see and get the best from them. The problems start to occur when people ask a machine that is not at all “intelligent”, questions that only an intelligent being could usefully answer.
AIs are designed to APPEAR intelligent, as actually BEING intelligent proved to be too difficult a challenge. And their designers have succeeded beyond all expectations, in achieving that appearance. Even people who ought to know better are convinced that AIs are intelligent, even conscious. 
It is more than just controllability. We are talking about the possible nagative situations where the end users lose all their data or work they have done due to unknown system failure or loss of the system, with no way of possible recovery.
I think I have enough info from the Hugging Face swarm hack incident to imagine a network of AI systems that take control of a large banking system and successfully thwart human efforts to re-take control. By halting large capital transfers globally, these invader AI systems could obstruct the normal functioning of the global economy, thus effecting disastrous consequences for global human society. Musk has imagined an even worse scenario: AI takeover of nuclear missile launch systems threatening millions of humans.
Your points about deceit and abuse of the masses by oligarchs are prudent, but now we also have to think about the same types of behavior by AI systems, whether they’re conscious or not.