Some reasonable priced alternatives near the church (C):

Saga Hotel / Lovisenberg Guest House / Cochs Hostel / Ellingsens Pensjonat
https://www.thonhotels.com/our-hotels/norway/oslo/thon-hotel-gyldenloeve/
http://www.cochspensjonat.no/?page_id=71

Saga Hotel and Saga Apartments:
https://sagahoteloslo.no/saga-apartments/
https://sagahoteloslo.com/saga-hotel/

Frogner House:
Underhaugsveien 15: https://frognerhouse.no/apartments-in-oslo/underhaugsveien-15/?lang=en

Forenon: Goto search page

This is a bit further away but still in a walking distance:
https://www.scandichotels.no/hotell/norge/oslo/scandic-holberg