A quickly and somewhat crudely drawn layout map of the castle is now in the maps section for the game. Button's at the top of the page.
The ovoid section at the top is the lower courtyard, it is about 50 feet below the level you are on.
Like I said, the towers are ~600 feet apart, which should give you a sense of the scale.
So we’ve got about 36k square feet of interior footprint, plus however many floors that are inside, to deal with. How many total forces have we actually seen out and about during loading?
And the two towers we are involved with are the back two?
You've only seen about 30 people outside, but most of them have stayed outside, meaning there's probably a good host of them within.
On the other hand, it is about an hour after sunset. A good chunk of em are likely asleep.
As to total interior space, the keep is pretty tall, based on the windows you can identify three primary floors on and above the level you're on, and there's plenty of room for there to be floors below your position as well.
Yes, the two towers you are messing with are those at the bottom of the map.