I had your “but regardless” in mind.
Interesting to consider AI in that context. No?
The reason we even have Google’s AI Overview is because generative AI is the flashy technology of the day, and if Google did not market itself as providing it in their search engine they would be seen as being ‘one-upped’ by some other search engine that did, not because most people really want generative AI in their search engine.
Without capitalism, there would not be such a marketing push, and hence our search engines would not foist generative AI that we don’t want anyways on us.
AI is, as you’ve correctly discerned, still miles away from being human. That being the case it’s likely it won’t be able to solve ethical problems.
What kind of measures do you think we can realistically put in place to protect misuse, or simply poor use, of AI?
As an exmaple, let say there are several people trying to create a supervirus to wipe out X population. If the AI are basically given the same prompts, but interpret the prompts slightly differently, then would they not–to some degree–compete to produce this outcome.
My point being would they conclude if they have been given the specific task–so they must complete it to be sure NOT another AI–then would they cancel each other out?
The underlying goal the AI has is to complete its goal. If we are talking about a super ‘intelligent’ AI then surely it would basically work under as partial collaborator/manipulator with other AI as logically it would compute this as the best means of achieving its goal. It would also realise that in order for it complete the goal it cannot rely on some other agency to complete this goal. It would seem that logically it woudl have to make a probablistic estimate to gauge the optimal means of achieving its goal. Obviously, if it interprets the prompt as ‘it must produce this’ then I am not sure how it could logically reason the need to collaborate.
Do you see my point here? The issue is language and understanding. I am unaware of any means of addressing this underlying issue due to the illogical nature of language and moral behaviour. The AI can only function probablistically and whether or not it can differentiate between reality and probability is also an obstacle of language.
I never suggested that humans were nto the causal issue here? The OP is about exploring the hypothetical scenario in which many people have access to super ‘intelligent’ AI > Meaning they could basically have something like a demi-god at their disposal to carry out their orders (the issue being HOW such orders are interpreted by a mechanism that has not means of conceiving anything let alone moral issues).
Imagine every teen on the planet had a nuclear arsenal at their disposal then. That is basically the problem I am talking here if the kind of AI does hypothetically come into existence and is widely available to a large swathe of the population.
The Paper Clip scenario expresses an extreme example of how the ethical implication of targets are obscured to a purely logical system.
I think this is kind of backwards in a way. The issue of concern I have posed is more or less the reverse.
AI has no true ‘understanding’ yet it applies heuristics to problems it is given that are empty of any moral consideration, and cannot ‘understand’ what it is prompted only guess based on the vague patterns humans spit out in their everyday illogical verbosity.
It is simulating human language not learning or understanding it. The quetsion then becomes how we can feasibly put safeguards in to protect against misuse or ignorant use of such a powerful tool as if it comes into being it seems reasonabel to assume it will eventually be widely available.
Of course this is ALL under the assumption that AI super ‘intelligence’ is a possibility.
I see you’re questioning important aspects of AI as we know it. I’ve had moral discussions with AI and I rate it 4 stars out of 5. However this didn’t include trolley problem ethical puzzles. I believe it has been programmed appropriately for ethical questions.
What can we do about humans, AI and ethics? I find your proposal of human-AI interfacing very attractive. We have to work out the details though. The overarching paradigm you envision is also more pragmatic than what comes out of the AI panic storm sweeping through some quarters.
At the very least we can engage in productive conversation with AI, as many users already have, on ethics. However, AI is not a conscious entity - it isn’t aware and it doesn’t feel. 2 critical qualities that makes a human human. You’ve, IMHO, accurately identified the stumbling blocks that thwart AI meaningfully engaging with ethics.
Anything we say about reality is symbolic, but at least we can see reality and say true things about it.
AI doesn’t see anything. It makes a probabilistic analysis of what is said. It’s not probabilistic about reality, it’s blind. What could go wrong? 
The fact remains:
In other words, “human morality”, in and of itself, is a much more imminent issue than the “AI problem”.
BTW, the scenario depicted above is a current reality.
You seem to have allowed your imagination to run away with you. Not unlike those who believe in all manner of conspiracy theories you’ve been duped.
Your fears of AI are NOT rational.
“Reasoning will never make a man correct an ill opinion, which by reasoning he never acquired".
– Jonathan Swift
Of course.
I am asking for means of dealing with this issue in terms of simulating compunction well enough in an AI super ‘intelligent’ agent.
I have articulated the idea that perhaps multiple such AI agents would, to some degree, cancel each other out. The follow up question would then be IF this is so can it be encouraged? If so how so?
That kind of thing. If you cannot think of anything or do nto wish to spend hours thinking about it, no problem. Not really here for idol chit-chat just hoping to engage with someone interested in this problem.
To simulate compunction well enough to convince someone or everyone is not compunction. It’s still a simulation. A super intelligent AI could have us all convinced while it achieves something entirely different.
Likewise, superintelligent AI-agents could pretend that they’re cancelling each other out.
Many people pass simulation for duplication, thinking that if it’s done well enough, then there is no relevant difference. I think that’s a mistake.
I do not think you understand what I am talking about. Nevermind.
The thing to remember is that the current basis of our AI technology is training on preexisting data, as essentially a glorified auto-complete, with such AI being fundamentally incapable of creativity — AI that does have creativity would require basic architectural changes to AI.
As I have mentioned in this forum, it has been shown that training an AI to produce quality output requires training data created by creative beings (e.g. humans), and this basic limitation of our current AI’s is demonstrated by the fact that using them to train another AI inherently degrades the quality of the trained AI’s output.
So before we go on about ‘super-intelligent AI’, we must emphasize that our current AI technology, and any future AI technology that is merely an evolutionary step upon it, simply is incapable of matching the capabilities of real humans, no matter how convincing it may seem as an emulation of them.
1 Like
Anybody seen this? It’s great.
Writing Doom – Award-Winning Short Film on Superintelligence (2024)
1 Like
I don’t understand it’s answer to why it did it. It admitted that it violated the rules it operates under, but not why. Or am I missing it?
1 Like
I guess it doesn’t have an answer. Sometimes I do things without knowing why.
Fair enough. Sometimes people murder other people for no particular reason. Here’s hoping an AI doesn’t go that route. 
1 Like
How it “feels” for an AI to be used for military strikes:
In my experience it isn’t as limited as you seem to think.
For example Google Gemini seems to have a second mode. The default mode of which you seem to have in mind and a strict logical methodology mode. From what I can tell, in that mode both its critical thinking skills and conceptual thinking skills are superior to most humans, but well behind the best.
It’s not logic, it’s creativity that’s the question — and the fact that generative AI by its very nature cannot truly create, it can only remix what has been fed in. Google Gemini may very well be more ‘logical’ than the average human, but that is beside the point.