Gilthan wrote:Starglider wrote:Connectionist designs are for the most part self-obfuscating at the low-level; you may remove some options for malicious deception but it's a moot point since you've made white box analysis nearly impossible.
If your AI design is connectionist and not logic-based, if white-box analysis isn't even an option, what choices do you have?
Simple; don't build a connectionist AI. Go and design a more sensible one instead.
I am arguably oversimplifying here; 'symbolic' and 'connectionist' are somewhat fuzzy terms and in theory there should be a huge spectrum of possible designs in-between. In practice though, not so much; up to about the mid 90s AI was almost completely partitioned into designs based on symbolic logic (though sometimes with fuzzy rather than Boolean propositional logic), and designs that a massively parallel mass of links and units with 'emergent structure'. There has been a lot of pontificating about how the first should 'emerge' from the later - see Marvin Minsky's 'Society of Mind' for the classic layman-accessible treatment of that - but no one has really demonstrated it in a working program. Some recent AGI designs do blur the boundaries, but they're still in the minority - even models that abstract low-level neural details, e.g. Hierarchical Temporal Memory (and the various Sparse Distributed Memory schemes that preceeded it) are still definitively connectionist and thoroughly opaque.
An obvious difference is that while all symbolic logic people admit that connectionist intelligence is possible (since humans are connectionist - though in the very early days, a few researchers insisted that all human conscious thought was really symbolic logic), quite a lot of connectionists think that connectionism is the
only way to do intelligence. The spectacular failures of symbolic AI by the end of the 80s seems to have allowed them to win most sci-fi writers over their side - the fact that their designs are supposedly 'more like humans' allowing them to handwave their own just-as-serious failures. People in this camp dismiss work on formal Friendliness as irrelevant because they believe the prerequisite (transparent general AI) is impossible. I am not inclined to relate the whole list of nonsensical arguments supporters attempt to use to dismiss logic-based AI, so I will restrict myself to the ones with the most merit; the 'brittleness' and 'shallowness' of symbolic approaches.
Brittleness mostly occurs with spoonfed knowledge that the AI can't generalise or change, although in some rare cases it can be a limitation on learning performance (very few symbolic systems have significant structural learning ability) where learning must be unnecessarily repeated. Connectionists don't run into the former problem only because it's nearly impossible to spoonfeed nontrivial connectionist systems. The latter problem is also unfair, because existing connectionist systems have their own severe limitations on what can be learned, but focusing on series complexity (i.e. simple functions only) rather than 'fluidity'. Brittleness is significantly mitigated through use of probabilistic logic and concurrent use of multiple candidate representations and heuristics (combined into probability distributions). It can be eliminated by sufficient reflective metdata about how and why AI elements are designed, such that the AI can purposely modify those elements at will using the same logic as action planning. This is fiendishly hard to implement - it's similar to research projects to make operating systems based on pervasive self-modifying code, but several orders of magnitude more complex - but at least the design concept for it can be strongly supported in theory. Connectionists have
no strongly supported theories for how to make their designs work - all they have are neat-sounding ideas and 'well the brain is connectionist so something like this must work'.
'Shallowness', or the 'empty symbol problem' refers to the fact that classic symbolic AI treats a word with a few attached rules as an adequate model of a complex concept. To be fair, originally that was the only way to do anything even vaguely intelligent looking on the hardware of the time. If you look at say Schank's classic work on scripts and plans in the 70s, it's essentially a big list of regular expressions implemented in Lisp, that take real sentences, simplify them into a standard structure, then create new sentences by looking at word patterns. Faking intelligence like this was actually a very impressive feat of 'knowledge engineering' (as creating expert systems used to be known), but it's incompatible with any serious learning algorithm and completely unable to model sublties, physical situations - anything you can't capture in a few sentences of restricted vocabulary. Searle famously whined about it in the Chinese Room paper and early connectionists swept in with NNs that seemed to do real learning and declared it all irrelevant.
Again, the persistence of this problem stems mainly from the fact that very little real work has been done to overcome it. Cycorp remain the standard bearers for the old vision of symbolic AI, whose workings look almost entirely like English sentences. Others latched onto the 'grounding problem' and mainly viewed it as 'how can we write interface code that bridges the vision code or the NNs to the classic symbolic logic code'. Not much work has been done interfacing symbolic AI to sophisticated physics engines - most of the people who do it are games programmers. Some psychologists have written good treatments of how rich symbols and adaptive level of detail of detail should work (e.g. various papers by Lawrence Barsalou, though the details are different in AI vs humans), which hardly anyone paid attention to. I'm being a little unfair here, in that rich representations make spoonfeeding harder, which means you really need to crack learning and dynamic hypothesis generation at the same time - which comes back to one of the central issues of AGI, that it does not decompose well into subproblems.
Most modern connectionists - certainly the ones trying to promote their personal takes on AGI - like to stereotype symbolic AGI as stuck in the 80s - not that they even bother to read Schank's papers and look at his proposed long-term improvements. There are no good reasons to persist with any form of connectionism other than slavish brain emulation. When I was at the SIAI we used to jokingly call emergent methods (encompassing genetic programming and connectionionism) 'the dark side' - because it
seems quicker and easier, well suited for the impatient who want to brainstorm cool AGI ideas without having to wrestle the hard, unforgiving logic. It was black humor though, because the destructive potential is quite real.
I know you prefer logic-based AIs, but it may turn out that connectionist methods become easier with future increase in hardware performance, while logic-based AGI development remains highly dependent on hypothetical brilliant insights of its programmers
Oh undoubtedly. When I say 'people should not use emergent and opaque methods', that is my opinion and the opinion of pretty much everyone studying formal Friendliness theory. It clearly isn't what most people are actually doing, just like your entirely-theoretical safety proposals.
The most straightforward brute force way of getting AI, with the fewest requirements for brilliant breakthroughs by the researchers, would appear to be throwing enough money at emulating the intelligence of a neural cluster or a worm and then working up to more complex brains, one step at a time (following the usual cardinal rule of solving near-impossible problems: breaking down into simpler steps to master before moving on, not shooting for human intelligence directly).
General AI as a rule has proven fiendishly hard to decompose like that, and connectionist systems are not immune. In the late 80s and early 90s early successes lead to people confidently predicting that they'd have artificial dogs by 2000 and artificial humans by 2010. When they actually tried to write the code, most people got stuck at the worm level, the really good researchers made it up to insect level and then got stuck. In fact we did get Aibos, but they were built with conventional modular software engineering and control systems theory, without learning ability. The only 'brute force' technique that is
guaranteed to work is slavish imitation of biological systems, which is in fact what is happening. Throwing massive amounts of computing power at simulated evolution of connectionist systems should work, but there are so many variables and so much potentially relevant richness in real evolution that it's still pretty dicey even with massive computing power.
We understand and can confirm friendliness (or at least predictability, controllability) of existing connectionist entities in the form of humans and animals because:
No, you can't. You can check that they seem benevolent in simulation. If they're animal level and your design isn't prone to instability, then you can be fairly sure they'll act benevolently in reality. For human-level AGI, checking that they act nice in a box is already useless. Aside from the significant potential for deception, there is just no way to simulate reality well enough to have any confidence that a novel situation won't trigger a cascade phase change in the goal system. Since the system was made by human programmers, it clearly already has the basic level of competence needed for further self-improvement even without invoking all the special advantages AIs have at programming, and even if you somehow manage to have perfect barriers against it doing that in testing, such barriers will not exist out in the real world. That's without even considering all the people who will try to
deliberately break and pervert this thing as soon as it is published / put on sale / available for download from ThePirateBay.
Yet this is considering an AGI under the cardinal safeguard of not being superhuman in version 1.0.
I've already noted that this is almost impossible to enforce for anything except human uploads, or something very close to it. Even if you could enforce this restriction inside your development box, there is no way you can continue to enforce it in an AI embedded into sold products or even worse, open source. Finally without going superhuman, you have no way of knowing what futher instabilities superhuman reflection may reveal in the goal system (hint; if the system evolves from neuromorphic human-level AGI to rational/normative transhuman AGI, it'll almost certainly run into some serious ones).
Dealing with a limited AGI of only humanlike intelligence would make having presence in the real world a containable risk. (That's assuming appropriate precautions and no "we made its body with this wireless transmitter, to transmit a copy of itself able to self-modify exponentially and run on standard conventional computer hardware worldwide, with this nearby internet access point").
You're proposing massive efforts (with corresponding increases in cost and development time) to build an AI box of dubious reliability, but you can't point to any real problems that it convincingly solves. You might catch some simple problems in the narrow competence window between 'not a threat at all' and 'not a threat while in a box', but that does nothing to solve all the problems that will emerge later when it is inevitably let out of the box, and becomes capable of deliberative self-modification.
This is essentially why the 'AI box' argument is treated as a sovereign remedy by various nontechnical denizens of transhumanist forums, but is treated as a backup precaution at best by people serious trying to build Friendly AIs (of course, it is ignored as irrelevant by the 'oh but all intelligence is friendly by default' people).
Rather, it is easier to develop safe humanlike AGIs than safe superhuman AGIs at the start, for reasons including the simple fact that you're more likely to survive to have a second chance if you don't try the latter when inexperienced.
True in theory but not of much practical relevance. Most people envision a relatively slow climb in capability simply because they ignore self-enhancement and assume that researchers will have to do all the work. It is true that designing an AGI from the start as a seed AI, with a full reflective model and pervasive self-modifying code, decreases the competence threshold for and increases the speed of 'take-off' (that's half the point of such a design). However the gains in terms of being able to do white-box analysis and formal structural verification vastly outweigh the added risk. I would certainly be much more inclined to trust someone who bases their arguments for Friendliness on the later, since if nothing else it indicates that they have thought about the real details of the problem, not superficial empirical pallatives.
Once you have safe humanlike AGIs
Again, we have no mechanism for making a unsafe human-like AGI into a safe one. If it is actually human-level and neuromorphic, it will be no more use than another random human researcher. If it is transhuman or innately good at programming due to design, it is already a serious existential risk whatever your boxing precautions. Finally, teaching it the details of FAI and seed AI design (so that it can assist you) will inherently be giving it the knowledge to self-enhance into a normative design.
This comes back to the topic of empirical real world experience, though. Indeed, certainly you can't really inspect all of its complexity manually, but, like the easy verification that animals (a small subset of all possible connectionist entities) respond to food motivators, you may be able to test if your goal and control system is reliably working.
Now I am a big fan of empirical experience, compared to many researchers who prefer just to do theory, but that is in order to determine which structures have adequate performance, in both the problem-solving and conventional programming senses. Performance is something you can measure empirically without incurring any particular risks, and if you make mistakes it is no big deal - the AI is just a bit less capable than you hoped, until you fix the problem. Safety is not like that - you cannot 'verify' it by empirical trials, because empirical trials are inherently limited to a narrow set of circumstances. Even ignoring reflective stability issues (that tend to lie hidden until later levels of AI capability - as evidenced by the stability problems with Eurisko vs AM), you cannot verify that your trials have comprehensive coverage of the functional mechanism without doing a thorough white-box analysis of that mechanism. And guess what, if you can do the through white-box analysis you've already had to solve about half of the FAI problem, so why not just bite the bullet and do it properly?
Also, if you slow down the society of humanlike AGIs who were working in accelerated time, you can inspect samples of what they were doing and see if they developed a hostile secret conspiracy
Yeah, that will work about as well as the average oppressive tyrannical government. You can examine the representations they're passing back and forth, but if they're optimal they will be as opaque as the AI design itself. If you insist that the messages are in English, you still have no idea what might be stenographically encoded into the exact phrasing and word choice. You can trace the representations into the NN, and try to label activation patterns, but your guesses may be wildly off even without considering the potential for active obfuscation of activation patterns (which doesn't require direct self-modification; the inherent holographic nature of most connectionist designs means that it can be done with careful management of co-activation patterns). Also I'd note that researchers even vaguely qualified to do deep analysis of complex NNs, or just AGI cognition in general are very very rare. If you're doing some massive verification project, more likely you'll produce millions of words of transcript that will get scanned by bored grad students.
If your AGIs see you as an opponent, you got to stop the project immediately.
You're behaving like an enslaving tyrant (note; I've said before that AGI isn't necessarily slavery, but if you are making arbitrary black-box sentient AGIs then killing or modifying them if they don't do what you want, that is probably slavery and possibly murder to boot). If the AGIs do anything other than play along with you they're not exactly human-level intelligences. If you were stupid enough to try and replicate humanlike emotions, they may well start hating and resenting you (for some alien version of 'hate' and 'resent') in the process. Arguably the real lesson you are teaching them is 'if you don't trust someone, ensure you have total power and control over them, and kill or brain-modify them if they disobey'. So frankly you'd deserve your fate if the AGIs got out and enslaved humanity - shame everyone else would suffer for your stupidity, or more likely just die.
There must be mutual friendliness with monitoring only as a backup precaution, like police monitor the population but are mostly supported by that same population.
This has merit, but the division into separate 'AGI individuals' isn't necessarily a good way to do it. The whole individual personhood distinction is really a human concept that only holds for highly neuromorphic AGIs. The causality barrier between the internals of one AI instance and the internals of another is rather weaker for normative AGIs with exact copies of each other's core code, which explicitly try to converge to an optimal model of external reality, and which can pass around huge chunks of their mind as standard communication acts.
Through the history of the field, the estimate for adequate hardware has usually been 'just a bit better than the computers the researchers are using'.
That's a red flag suggesting commonplace wishful thinking.
True. In this FAQ I am trying to outline commonly held positions in the field, not just my own personal views.
My subjective answer is yes, a contemporary PC should be more than adequate for rough human equivalence using optimal code.
If your guess was accurate, that a contemporary PC could be human equivalence with optimal code, then a contemporary PC utilized at about 1/1000th efficiency ought to be able to match a rat, given that a rat's brain is 1/1000th the mass of a human brain.
It isn't that simple, basically because most of the things animals do (e.g. visual processing) parallelises very nicely and benefit less from massive serialism. Conversely, humans are the only animals who do very symbolic, abstract thought, and we do it very badly because neural hardware isn't well suited for it. A computer can solve a differential equation in a few nanoseconds - a human takes many milliseconds for something embedded into a reflex loop like catching a ball, many seconds or even minutes if doing it consciously as part of say designing an electric circuit.
Furthermore, animals do not demonstrate a close correlation between brain mass and intelligence. A cat is certainly not 20 times smarter than a rat; actually in problem solving terms, rats have consistently demonstrated better performance. A human is much more intelligent than a blue whale despite having a much smaller brain. Ravens are much more intelligent than horses etc. Clearly structure matters.
Or a modern PC programmed at 1/50th efficiency should match a cat's brain. Yet we haven't seen anything like that.
Even disregarding my earlier points, that would not confirm that the power available is insufficient, it may just confirm that no human has worked out how to program such a mind yet.
Is current programming being universally under 1% efficiency at using the CPU to its full potential capabilities a probable answer
IMHO, almost certain for existing attempts at human-equivalent AI. That said, PCs already do a huge range of sophisticated things a cat could never hope to do, so I question the validity of your comparison.
A hundred million MIPS would be like two thousand top-end modern PCs combined (like millions of PCs a couple decades ago)
True for CPU-only processing, but those algorithms are massively parallel and run fine on a GPU (in fact there has been a lot of effort to port them to such platforms recently). Modern GPUs put out about a teraflop of usable performance, so that's 25 workstations with 4 GPUs each. However
Those computer vision programs are not written in a neuromorphic manner, rather by conventional coding (which shouldn't be orders of magnitude inefficient)
you have no basis for 'shouldn't be'. There is no proof of optimality in these algorithms, except for some very low level signal processing, all they are is 'some researchers came up with the best method they could think of at the time' (computer vision is mostly black magic and voodoo). In actual fact nearly all such algorithms do a full frame analysis and then filter down for relevance, the same way human visual systems do. You can tell by the fact that the quoted computation requirement is a static figure, rather than being dependent on scene content. While they aren't slavishly neuromorphic in that they use pointer structures and high-precision maths in the 3D shape extraction, in information theory terms they are massively inefficient. An AGI would almost certainly design a selective sampling probabilistic algorithm, that very effectively focuses analysis effort on areas predicted to contain further useful information. Full-frame analysis would be limited to random checks for unexpected gross alterations in non-dynamic areas, with the occasional drift check for slow motion.
All of which is in any case irrelevant to my argument. Vision processing is something virtually all complex animals do. It's probably the hardest single task to optimise for serialism (though as I've just pointed out, I think there's massive potential even there). Thus connectionists love to latch on to it - largely because it's something their toy NNs can do and symbolic AI can't. Meanwhile, Schank's 1970s programs were convincingly analysing newspaper articles (see PAM) and writing passable Aseop's fables (see TALESPIN), faster than a human could, using the computing power of a 286. No connectionist system ever written can do these things (stastical text summarisation exists, but that isn't the same thing) - for that matter cat's can't either. There is no objective reason to consider visual processing to be a better example of the computational requirements of human thought than these abstract reasoning examples; at the very least you should interpolate between them (which will, incidentally, give you a rough figure close to my own best guess).
Morravec's argument is a tunnel-vision worst case - though a popular one because it's a great argument to use when you're asking the university board to fund a new supercomputer.
A top-end CPU package, LGA775, has 775 pins total, and, more to the point, modern CPUs are typically 64 bits (though with often 2-4 cores on modern CPUs). Consequently, although the input / output data streams are alternating at gigahertz frequency instead of a fraction of a kilohertz for human neurons, that factor of millions-of-times increase in serial input speed can be more than countered by how the brain has so many billions of neurons and trillions of synapses in contrast.
Why are you comparing pins to neurons? That is a complete red herring. Either compare transistors to synapes, neurons to logic gates, or pins to input nerves.
The number of transistors inside a modern CPU is enormous, but its overall setup only handles a comparatively simple input signal per clock cycle, without many millions or billions of input wires going into its CPU package at once.
Also irrelevant, since now you're disregarding serial speed. Data per packet is not relevant (not least because organic nervous system traffic isn't packetised and clock-rate isn't relevant beyond a single nerve - also the encoding scheme is less efficient but that's another issue), you should be comparing the total bandwidth of the I/O nerves connecting to the brain and the I/O connecting a computer to... something. This is where the comparison is useless even in principle. Most problems we want an AI to solve exist entirely within the computer (even in robotics, most tests are done with simulators because it's cheaper), so what is the relevant of the external connections? The real issues are in central intelligence.
Moravec's estimates make sense in context. If they're even close to valid, the limited results from AI research over prior decades become well explained
If Moravec was correct, all it would mean is that vision processing that is broadly structured like an organic brain (i.e. parallel rather than serial) would require a medium-sized supercomputer to run at human-equivalent speed and fidelity. Which we don't have. It also implies that the algorithm would run at 1/1000th speed on a normal PC... which we don't have, although frankly that's very hard to say for certain because high-level visual processing blends into central intelligence so much.