In this talk I will discuss some criteria for good codes and good qubits for quantum error correction. A good qubit has a highly structured noise channel and good codes are the ones that have special underlying symmetries with respect to this channel. For fault-tolerance it is also necessary that we are able to implement the code with physical operations that preserve the symmetries. I will highlight this deep connection between what makes a good qubit and good code using Rydberg atom qubits and surface codes.