The outer protective layer surrounding the spore-bearing organ in many fungi.