One of the ways to explore New York City is by bike, and there are numerous options to rent one. For example citi bike offers several hundred stations where you can take and return the bikes. I was also surprised by the fact that NYC has quite many paths for bikes!
Regarding the costs I’d recommend to calculate if walking and riding the subway wouldn’t be cheaper. When renting the bike $101 deposit is reserved on the card. 24 h rental in 30 min slots costs $24.95, you will be charged more in case of longer rides.
More information about New York here!