When I was a kid, my mom kept the ice cream cones in the oven, because the pilot light made it a little warmer, a little drier, and that kept them crispy. Now, I live in a dry climate where you can leave a box of crackers open, and it's highly likely that nothing bad will happen. We don't have humidity, and we don't have bugs. Those two things make me very, very happy.
That said, when you store stuff in a pantry, it's a good idea to store it properly. Keep it in a cool dry place, all that. And, depending on your climate, you may need well-sealed plastic or glass containers to keep your food fresh and critter-free. Initially, this can be a bit of an investment, but there is a lot you can do free or low-cost to keep your food safe from deterioration and/or infestation.
Before you store, take things like rice, beans and flour and give them a long weekend in the freezer. If there are critters (or eggs of critters) in there, this will keep them from doing damage. (I know - this is not a pleasant thought. Why do you think I grow so much of my own food?)
Gallon size zip top bags will hold lots of things in the same package you buy them in - just put the whole package in and seal it up. Keep in mind, however, that this is not mouse-proof. I know. Ew!
Save glass jars from things like spaghetti sauce or jelly. Clean them well and use them to store pasta, brown sugar, whatever you've got that needs a container.
Keep an eye out on freecycle, craigslist, or at thrift stores. I once answered a freecycle ad for 18 glass gallon jars. While I was on my way to the home to pick them up, he found 'a few more' and I ended up with 32 or so. (That's a lot of gallon glass jars, y'all!)
A 2 week supply, the recommended minimum that we keep on hand in case of emergency or natural disaster, doesn't really take up a ton of space, but if you decide to move beyond that to a more substantial pantry, you may run out of storage space in your kitchen. We certainly do, because we can things like applesauce and green beans for the year. If you have a basement, a few shelves downstairs can really help.
Some people store extra items in the back of closets, under beds, even under the couch. (It helps if the couch has a ruffle, lest your company see all your mini raviolis peeking out at them). I've even heard of people putting a row of canned goods behind the books on their bookshelves, or storing stacked containers in a row with a board on top, a fabric cover, and turning that into a behind-the-couch table. The point? If you look around, I bet you can find some space to put a few things so that your kitchen is not so over-crowded. Just don't forget where you put your pantry foods!