Ninh Binh probably won't be near the top of many people's wishlist for Vietnam, but we absolutely love it (the province, rather than the capital of the same name). Readily accessible from Hanoi, it is all about charming rural Vietnamese countryside, with limestone karst rocky outcrops surrounded by meandering rivers among the ricefields.
It isn't so much about "doing" in this area, more about just being there - a chance to see and experience a very peaceful rural Vietnam. Plenty of people just take a day trip from Hanoi, but you won't experience the tranquillity of the area if you don't spend a night or two here.
The Tam Coc river trip to visit the three caves is really beautiful, but can be a bit of a tourist trap - do the boat trip early in the morning or late in the afternoon to avoid the Hanoi day trippers and minimise the hassle.
A boat trip in the Vanh Long wetlands is a less touristy alternative, not as spectacular but always more peaceful. Both of these areas are also great for just meandering around on a bicycle.
The Cuc Phuong National Park is Vietnam's oldest and biggest park, and has an interesting Primate rescue center, and turtle conservation center. For a hike, you can hire an English speaking guide and get off the beaten track.