Why do some people think that AI could NEVER feel emotion? What exactly speaks against that?
For emotional feeling states to exist, there must first be consciousness. Feelings are subjective states that are experienced and expressed by a conscious subject. If there is no consciousness, there is no subject and no such thing as subjective states.
Life Before Consciousness
And for consciousness to exist, there must first be life. It’s a widespread myth, especially among fans of AI (or at least AI-based sci-fi), that any sufficiently advanced computer will magically become sentient and self-aware. (And then, for some reason, it will decide to enslave or exterminate humanity.) And behind this is another widespread misconception: that consciousness is just a function of intelligence.
Consciousness Beyond Intelligence
Consciousness is a process by which a living creature continually experiences and determines its existence. The conditions of its existence tend to vary from one moment to the next — as do the personal meanings of its subjective experiences and the significance of its external perceptions. A conscious being is thus motivated by its experiences and perceptions to deliberately adjust and control its subjective states through voluntary behavior, often in cooperation with others. Intelligence adds to the sophistication of perception and behavior, but intelligence alone does not give rise to consciousness per se. No amount of complexity can turn a set of algorithms into a conscious subject-agent with the existential needs and states of a living being.