Statistics for UI Testing

So, you've setup two copies of your web application, and don't know which is better. You know you have 10,000 users, and you randomly give 50 people the new copy, and see how that goes. 23 of them don't like it. Is that statistically significant?

Here's a quick article summing up all of statistics for user experience testing. It's long, but I'm bookmarking that one for the next time the numbers come out close to even, and I need to be able to explain what those numbers mean.