I don't think it's a big secret that Uber drivers have the option to rate their passengers in the same way passengers rate drivers. As a ride-sharing service, peer scoring is really the main driver of internal quality check and balance processes for Uber.
It's a little weird that everyone starts at a rating of 5 - that sets a generous starting position that everyone is good. And individual ratings start to bring that score down, although it's not as simple as basic arithmetic, I believe. If that were the case, a bad experience on your first ride as either a driver or a passenger could result in some pretty negative scores right out of the gate. And the fact that an Uber driver I rode with mentioned that passengers with scores of 4.4 and below were already considered "trouble" passengers, again the thresholds seem pretty high.
I asked the same driver that had shared the 4.4 threshold information what my score was - and he immediately answered that I'm a 4.8, which is still pretty good. Considering my odd hours and the fact that I've yet to tip and Uber driver, I guess being friendly and polite is good enough to keep my score relatively high. Then again, the few bad experiences I immediately provided feedback to customer service in order to get corrective action immediately - the customer is supposed to be right after all, yes?