Comparing two backpacks, their difference are more than just size.
The front mesh pockets are different. The 35L model has one single big space, and it was held by the strap on top. So when you unload the stuff from the main compartment, the mesh pocket might fall down. The 55L model has two separators which separates the single big space into three parts, and these separators also hold the big mesh pocket in place. I consider it a huge improvement.
Another significant difference is on the opening of the main compartment. The 35L features a zipper on top, but the 55L does not. I don't find the zipper very useful, but it might squeeze some more space from the main compartment in extreme case. We usually fold the top a few times to close it.
Finally, the 55L has balance adjustment on top and it is very important too.
Having said that, if you want to go ultra-light, it's always nice to keep the size and weight low, and so 35L might be sufficient and a good limit. I bought the 55L in case I also want more comfort.