I don't know about anyone else, but even after doing a year+ of theory, I can't have a sheet of harmonic stuff thrown down in front of me and have it made sense enough for me to have it be useful on first glance (I'm also a pure drummer, meaning I have no proficiency at any other instrument besides mandatory piano dinking around for theory classes). That being said, if I had a couple hours, a pencil, and a copy of the tunes I could listen to, then I could mark it up and have it make sense. In general, I would pay more attention to the rhythmic stuff than anything else, perhaps cue in on some chords that indicate changes (by circling them, etc.), and basically mark it up with any other useful stuff.
I am nowhere near as good of a reader of drum music as others on here, but I have done it quite a bit; the one thing that I can say is that one of the perhaps hardest things that you would come up against if you are new to reading anything is simply keeping track of where you are in the music, which is why I still mark the heck out of my music whenever I have to read-- it is like a nice little safety net. Sometimes I'll have to take my eyes off of the sheet for a number of reasons and lo and behold, I have a moment of "crap, where am I?" hit me, so making the music 'your own' by writing in notes really helped me, especially when I was picking it up and trying it on live gigs-- I had road maps, marked up sheets, all over... just in case!
In general, I would think that a command of reading rhythms would be important, especially since you are a drummer and if you should know anything, I would say start with that. As far as I am concerned, hitting the wrong thing at the right time is better than hitting the right thing at the wrong time-- i.e. get the rhythms right and work on perfectly orchestrating the actual notes on the drums later.
A book that I would start out with is the Encyclopedia of Reading Rhythms by Gary Hess. It has all the basics and is simply written on one line (i.e. snare). Pick it up and set your metronome to 40 bpm and plug away nice and slow. Furthermore, I would second the books that everyone else has mentioned. I am kind of a instructional drum book whore, and I have way more books that I could ever get through; however, I like just picking up ones that are focused on playing certain styles and working through those-- i.e. not ones that are focused on independence, speed, etc., but have musical pieces of drum music written out. This way, you are see the notes, play them, and hear an actual musical phrase come out of them-- this helps connect the dots.
I would even suggest picking up some big band instructional books-- not that there is a thriving big band scene that you need to capitalize on, but big band drumming charts have two things that I find to be very helpful: 1.) you are forced to essentially interpret figures that are or have been derived from other instruments, and you then have to figure out how to orchestrate those figures on the drums, how to set them up, etc. It's an art knowing when to do what where and it makes you think of figures and orchestration ; and 2.) reading big band charts really helps you keep track of where you are in a piece of music, because you might have quite a few measures of "playing time" which means you have to keep yourself focused and on the page so you are in the right place when it is time to jump back into kicks, etc. One book that I don't have, but looks interesting is Big Band Drumming at First Sight by Steve Fidyk. Steve Houghton has some good books to check out too, if you want to get into charts, etc.
In general, I'd just pick up as music/drum music as possible, whether it be free online or having to buy actual books-- books with actual parts that are grooves are better, IMO, as you have to read those parts and make them feel good as opposed to just playing some independence exercise. That being said, no one ever sat me down and taught me how to read-- I just picked it up by asking people questions who knew how to read, questions about it figures, note values, chords, etc. I later went and did some schooling to fill in the gaps. That being said, I used to read tons of drumming exercise books and it eventually all soaked in, I guess. I also did little Jazz Combos where they used to hand out lead sheets, and I would have to fake my way through them until I knew the tunes well enough to get off-book. However, I music admit that even when I am reading a drum part that is written out, I fudge it. If I am playing in a setting where what is written is absolutely necessary and needs to be played, I keep it note-for-note. However, If I am doing a show (i.e. a musical), then usually the person who wrote the drum part is not a drummer and has generally written out what I would say is an idea of a part, but doesn't have the drummer-mind to really make it sweet. In those situations, I try to understand what they are getting at and blend it so that I am reading/playing the groove, but not every single note. What I am trying to say is that even when you get great at reading, in many situations it may be more about interpreting the ideas that are on the page, as opposed to mechanically plugging through it, trying for 100% accuracy-- we're not robots, and frankly, I think a person's time could be better spent on something else if they are doing that!
Therefore, I say in summary, it's a long process that involves just starting picking up stuff, doing and doing it and doing it. Starting with easy stuff and working your way up (duh), but just getting used to looking at it. It will sink in through osmosis
That being said, no one is perfect and it is a skill that takes constant improvement and work-- it also is one that gets rusty (Steve Gadd talks about that a lot)-- I say that it is not like riding a bike, as it is more like riding a unicycle-- you never really forget, but you need time to get back up to speed. If I haven't read for a while, my reading chops get horrid and I end up having to tap some embarrassingly simple stuff out to get my head back in the game when I pick up a page for the first time in a while.
Last but not least, here is Branford talking about how he considers playing 90% of the notes, a good night playing from the sheet, skip to 1:00: