In reply to munro:
It partially depends on how remote you want to be / how much you are bothered by doing the same thing as others. Karakol in the East is probably the most popular trekking destination and there is also some easy mountaineering in the area. As a result you won't be the only there. However, as information about remoter areas can be difficult to come by this doesn't have to be a bad thing. There are several agencies in Karakol who can help with organising transport to the start of your trek.
Ala Archa (just south of Bishkek) is used by many to acclimatise before heading to bigger peaks and is easily accessible. There are various peaks there of various difficulties.
Apparently there is good trekking near the Kazakh border as well, but didn't do this myself.
My main piece of advice would be to get all the maps you might need in Bishkek. We traveled 5hrs from Bishkek to another small town with a tourist office where we were told the only maps for that area were available in Bishkek.
Also, depending on what you want to do (especially if it is mountaineering in remote areas) be prepared to either get a company in Bishkek to arrange all the logistics for you or be prepared for all the issues related to traveling in a non-Western country (language, culture, unexpected events, etc.).