The highest quality sweaters often come in natural fibers (or blends that improve wearability like stretch) and fully fashioned knits (you'll see the marks around the neck and arms to see the garment was knitted into shape). Take care of your sweaters by following care instructions, store them away from moths (in cedar chests or plastic bags) and keep them folded instead of on a hanger.
Cashmere is available in a huge array of price points, qualities and styles. Cashmere garment weight is often described in "plys," with two-ply having a finer hand and three-ply and up making up a chunkier or heavier knit. Check your care label, but generally cashmere can be handwashed with a very mild soap and laid flat to dry (only recommended after six or seven wearings). Expect to pay $100+.
Types of wool include merino (a finer fiber that's good if other wool makes you itchy), alpaca (strong, but soft), lambswool (fine and soft), shetland (usually describes a sweater made from coarser fibers) and mohair (lustrous and long-wearing). Keep your wool sweaters folded and launder according to instructions (some wools can even be machine washed). Often available for under $100.
Virtually seasonless and easy to care for, the cotton sweater can go from workplace to weekend in a flash. Look for beautiful colors and fun patterns (cotton yarns take color beautifully). Many styles can be washed (machine or by hand). Cotton is perfect for warmer climates where wool or cashmere would be too toasty. A budget buy: look for great cotton sweaters under $75