Every Chao begins its life in an egg. Eggs come in all kinds of colours and patterns, indicating the breed of Chao that will hatch from it. Every single breed has a corresponding egg to go with it. Unfortunately, most of these are unobtainable without hacking. This is because most breeds must be created through mating, and mating always produces a normal-looking egg; therefore, the eggs for those breeds can normally never be seen.
The kinds of egg that are available without hacking are the Two-Tone Normal egg, all Mono-Tone colour eggs apart from Mono-Tone Normal, and all Jewel eggs. The Mono-Tone eggs can be purchased from the Black Market. The Jewel eggs can only be purchased from the Tiny Chao Garden; they can also be found in Sonic Adventure DX, but then you'd still need the Tiny Chao Garden to be able to transfer them over. The Two-Tone Normal eggs are what you start off with, but more can be bought from the Tiny Chao Garden for free.
When a Chao hatches from its egg, two halves of the eggshell will be left. One of these cannot be interacted with and will simply disappear when you leave the Chao Garden. The other piece will stay in your Chao Garden, and can be picked up and thrown around. It is actually a hat! If your Chao is able to wear hats, then you can give the eggshell to it and it will put it on. See the Hats page for more information. You can also simply take the eggshell to the Black Market and sell it; this way, every Chao that you hatch earns you some rings!