Yes, God is all powerful.
Being all powerful is something that defies our imagination. We are afraid of even our own power, because we know we can get burned by it if we don't wield it responsibly. We imagine all kinds of demons and devils to somehow counterbalance God's power in our minds - but that lessens only our own power, not God's.
For us humans, power is the measure of our ability to affect, change and influence our inner and outer environments. Let's look at some examples: If you are physically strong, you can lift a lot of weight. If you are intelligent, you can solve problems fast. If you are disciplined, you don't let change affect you negatively. All these are different measures for power, but there is always something against which to measure that power: in this case weight, problems, or change.
Since God is all powerful, God doesn't have anything to measure his power against. If God is everything, then there is only God and nothing else. So, our human idea of power does not really apply to God.
For God, the question is not how to become powerful enough to create the world, but what kind of world to create? For this reason, the world has a say to how it wishes to be created - and that includes human individuals. We too are part of the world and share God's infinite power (as long as we learn to use it responsibly and with balance).
God's deepest and truest power is Love.