Each shard can be found in a different area of the Mirror World, except for Rainbow Route - the first area. Each area's Boss holds one shard, and will only relinquish it when defeated. Collecting all of them is necessary to reach the final area of the game, the Dimension Mirror, to locate and defeat Dark Mind.
When a mirror shard is collected, it will automatically be added to the frame of the Dimension Mirror. When all are collected, they will fuse back together into the whole mirror.