https://youtu.be/gvNpHQo0JIE

- Symbolic AI is the approach which may lead to an AGI which would be transparent. Transparent in the sense that behind the box is logic. Symbolic AI can be understood by humans and it's thoughts explained as just a bunch of "if...then..." statements, rules, logic, reasoning, all which can be explained to us. This is the type of AI we see with expert systems for example.
- Connectivity AI is the approach which may lead to a black box AGI. The connectivist approach is the approach behind Deep Mind, Deep Learning, and like a child this AI has to be trained by many examples. This AI works but because it is a neuro-network it is like a black box with magic inside and we simply measure the results to find that it's working. We cannot explain what it is thinking or even understand how it works.

1 Symbolic AI works and can explain every decision back to us
======

2 Connectivity AI works but cannot explain back to us why
=====

Of course both these statements are grossly simplified but the problem of black box computing does exist for approach #2 (connectivitst approach). It can generate a language which we can't even understand and speak to itself.  Symbolic AI is merely a symbolic manipulation where the logic and knowledge is open for all to see. The logic is perfect which gives it power but because it's logic there is a level of transparency you cannot get from the connectivist approach.

Can neuro-networks be analyzed? Maybe but it's not easy or automatic. It's at best also only an approximation. 




References
1. https://en.wikipedia.org/wiki/Connectionism
2. https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence
3. https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/?utm_campaign=add_this& utm_source=reddit&utm_medium=post
4. https://en.wikipedia.org/wiki/Physical_symbol_systems_hypothesis