Should a surf board be work?

A friend picked up a Hyperlite Quad which we were riding behind my boat. 09 22ve with a fairly large wave. The board is 59" (4,9") and most of us are in around 225lbs. I'm coming from a 5'6" broadcast so the board is a bit smaller and otherwise took a few minutes to figure out. I can ride it fine but, it does take a bit more work to stay in the pocket through. You have to pump the board a bit more and adjust your lean forward. The biggest difference I would say and was a general consensus with the rest was the board wanted to be ridden. It wasn't meant to just sit there and do nothing. It wants to be carved and moved around. We played around with the fin set up to where we all liked it a bit loose but still tracked well. This was the two larger fins in front and the two smaller fins in the back. I don't have a lot of experience with different surf or skim style boards so I don't always know what to expect and what others are doing on their boards. So ultimately, my question is it ok for a board to be a bit more work or are we ultimately riding a board that is too small for us? Again, we could all ride it and it was fun but we were all out of breath and had sore legs when we were done.

surf style boards that have more rocker like to be worked harder. When you start doing more tricks you will understand why.
There are definitely surf style boards that are good cruisers though

I'm 225 lbs on a 4'10" Doomswell Bigstep surf board... I can work as much as I want to on the board. I can ride it just chilling, or if I want to pump it and get into the wave, I can have sore legs and be out of breath. Never ridden the board you're talking about though. Is it a compression board, or foam core? I know going from the CWB tsunami I used to ride to the Doomswell, there was a HUGE difference in construction and weight of the board. That and the way it's designed all contributes.

Length of the board isn't necessarily the only factor. If you have someone that knows how to build a board to it's riders liking, a 4'6" board could be just as fast and easy to ride as a 5 foot board. The big box boards don't really take these things into account, they're building boards for "everyone" so to speak.