I assume when you say “large rubber washer” you have a part 2 or part 3 ball valve and you mean the diaphragm washer?
If you look at the washer you will notice a groove close to the entire washer diameter on one side.
The flat lip edge will be thinner on that face, if that makes sense.
This is the side that goes into the valve body, towards the white cone part.
The valve will have a groove to accommodate the lip, thus holding the washer and preventing it all pulling in.