Do you stop saying "X-O'clock in the afternoon" at a certain time, or is it based on if it is still sunny?

Like is six afternoon or evening. I think six is usually considered evening, and five afternoon, but during the summer it is sunny to 7:30 or so!