In my experience, there are several boat-specific criteria more important than size in determining whether you will be able to safely and comfortably handle a boat.
Agree w/ many of the prior comments... my $1.02...
Are the sleeping arrangements the only point stopping you from longer cruises?
Why no agree to "suffer through it" to see if longer term cruising is really for you & the crew... w/ the understanding that if the answer is yes you can / will take action to correct it.
Agree on a written list of Musts & Wants (from Admiral, Capt & Crew)
and then go looking / shopping to see which best fits YOUR individual list realizing it won't necessarily be the same as others' list. Modify / confirm the list while on the 'extended' cruise
We had wide / safe side decks and no ladders on the Must side.
Wasn't sure about going back to a single but thrusters make that combo maybe the best of both worlds.
We decided newer & smaller was a plus for us but it was only the two of us.
Ease of Locking, docking & rafting up were a priority as was ability to single hand if/when necessary. The helm side door & wide decks have exceeded our expectations
I do think there are two types of boaters - those worried about handling the boat - and will always worry about it no matter what size - and those that have the attitude that they can figure it out w/ education / coaching / practice regardless of size.
I know a couple that has taken and completed every USPS course available and are still afraid to take the boat out - hires a Capt to take it to the other end of the lake every fall for lay-up???
It doesn't sound like you are in this situation but only you & the Admiral can honestly evaluate your situation.