View Full Version : PG Top 100 rated pubs
I noticed the discussion about trying to produce a PG top 100 pubs in the 'Yet another list' thread and it reminded me of some stuff I'd done a while back for weighting ratings.
So here you go, a top 100 based on the weighted ratings of PG reviewers.
https://www.pubsgalore.co.uk/stats/pubs/top-rated/
It is undoubtedly skewed in all sorts of ways, but it might prove to be an interesting guide for good pubs that haven't been reviewed much.
One of the skews means that it is generally harder for pubs that have featured on a PG crawl (and have lots of ratings) to appear on the list, I need to work out how to tweak the algorithm without pushing it too far in their favour instead.
Wittenden
05-12-2018, 16:28
Interesting,Dave! Not quite sure how it works-it's been a long day-especially as I and others do not rate pubs numerically. 8/100-must get out more.
especially as I and others do not rate pubs numerically.
Yes unfortunately in this list it is only taking into account those who have rated numerically so it is lacking the opinions of several reviewers.
I do have an idea (well actually Conrad had the idea) for improving the concept of ratings but I can't quite work out how to do it.
Aqualung
05-12-2018, 16:53
I've a few questions.
What is a weighted rating?
Is there a lower limit to the number of reviews?
What happens if I press the add button?
There are some it says I haven't reviewed when I have. Bitter-Suite (https://www.pubsgalore.co.uk/pubreviews/86100/) Four Candles Alehouse (https://www.pubsgalore.co.uk/pubs/79092/)
Well, that's next year's challenge sorted then (after the Somerset 2019 Ale Trail that is).
I've a few questions.
What is a weighted rating?
It makes the ratings more uniform, so if someone only ever rates pubs a 10 then it will consider them more like a 5, effectively ignoring them.
Is there a lower limit to the number of reviews?
At the moment it is set to only show if there are ratings from 2 or more reviewers who have rated (I think) at least 50 pubs.
What happens if I press the add button?
It will just record that you have at some point visited the pub.
There are some it says I haven't reviewed when I have.
Ah damn, I ran a one off process a few years ago to say that any pubs reviewed had been visited. I thought it was an ongoing thing but clearly not.
Aqualung
05-12-2018, 17:39
Thanks for the answers.
It will just record that you have at some point visited the pub.
I don't like pressing unknown buttons in case it starts a nuclear war.
Ah damn, I ran a one off process a few years ago to say that any pubs reviewed had been visited. I thought it was an ongoing thing but clearly not.
That makes sense as the ones I had noticed are fairly recent.
Blackthorn
05-12-2018, 17:45
That is brilliant, I really like that. A proper list based on genuine reviewers rather than the opinions of some food/wine snob or whoever is doing the particular list in question. And presumably it will also get updated as ratings change. A very useful reference. Is there going to be a link added to the front page, I can see me checking this whenever I’m off to a new area?
Interesting that for some reason I hadn’t rated No.3, although I generally always do. A bit annoying too that I only photographed no. 71 a few weeks ago, and would have made the effort to stop if I’d known it was a “top 100” and had planned on going in no. 88 last week but abandoned the idea due to the very inclement weather. Again, would have made more of an effort if I’d known.
Aqualung
05-12-2018, 18:09
Am I missing something? By my reckoning the Great Western (https://www.pubsgalore.co.uk/pubs/39188/) has an average rating of 9.2 or 9.3. The Duke Of Cambridge (https://www.pubsgalore.co.uk/pubs/39093/) in Willenhall has two 9s making 9.0.
Edit: Similarly I can't see how the Albion in Conwy has missed out. Is it cos it's Welsh?
NickDavies
05-12-2018, 18:42
That is brilliant, I really like that. A proper list based on genuine reviewers rather than the opinions of some food/wine snob or whoever is doing the particular list in question.
I wouldn't set too much store by it, many are only reviewed by a couple of people, sometimes long ago.
That is brilliant, I really like that. A proper list based on genuine reviewers rather than the opinions of some food/wine snob or whoever is doing the particular list in question. And presumably it will also get updated as ratings change. A very useful reference. Is there going to be a link added to the front page, I can see me checking this whenever I’m off to a new area?
Interesting that for some reason I hadn’t rated No.3, although I generally always do. A bit annoying too that I only photographed no. 71 a few weeks ago, and would have made the effort to stop if I’d known it was a “top 100” and had planned on going in no. 88 last week but abandoned the idea due to the very inclement weather. Again, would have made more of an effort if I’d known.
It should get updated once a day, can't quite do it in real time as there are a lot of calculations going on. I wouldn't get too attached to the list as it stands, it is an interesting starting point but I am going to have to tweak the algorithm a bit.
I wouldn't set too much store by it, many are only reviewed by a couple of people, sometimes long ago.
Yes, very valid points, but without having tens of thousands of ratings coming in every year it is always going to be difficult.
Am I missing something? By my reckoning the Great Western (https://www.pubsgalore.co.uk/pubs/39188/) has an average rating of 9.2 or 9.3. The Duke Of Cambridge (https://www.pubsgalore.co.uk/pubs/39093/) in Willenhall has two 9s making 9.0.
Edit: Similarly I can't see how the Albion in Conwy has missed out. Is it cos it's Welsh?
Thanks for those examples. The way I have done it causes a bias against pubs with more ratings, also just spotted a problem with my maths. :muppet:
Aqualung
05-12-2018, 19:58
Just spotted a problem with my maths. :muppet:
Isn't that a line from one of the Dollars Spaghetti Westerns?
I agree with Blackthorn, it's a great idea.
Do you intend to have a map layer that shows where they are?
london calling
05-12-2018, 20:27
Cease and desist immediately.Just checked the London pubs featured and cannot believe they are anything like the top pubs in London.Hate to think someone would look up Pubs Galore and went to some of these pubs on this recommendation.
Ohh stats! I love stats.
So let's have a look at some basics
- Only include scores from reviewers who have reviewed over X is sensible
- The more reviews you have, the better level of certainty. Should there actually be two tables, those where the pub has =>4 and those where the pub has <4?
- Should reviews be weighted by age? In that, the older reviews carry less weight. So a review in the last year has a weight of 10, in the last two years 8 etc etc
- Mean, median or mode? :D
I've fixed my initial mistakes and the list has changed a fair bit.
Isn't that a line from one of the Dollars Spaghetti Westerns?
Really not sure, very long time since I've seen them.
Do you intend to have a map layer that shows where they are?
Certainly can do if you want.
Cease and desist immediately.Just checked the London pubs featured and cannot believe they are anything like the top pubs in London.Hate to think someone would look up Pubs Galore and went to some of these pubs on this recommendation.
Well we can try to improve it, but I can only work with the data I have.
Ohh stats! I love stats.
So let's have a look at some basics
- Only include scores from reviewers who have reviewed over X is sensible
I wrote that part years ago when I was playing with the first so had to check. It turns out I'm including anyone with 3 or more reviews, but they are weighted lower than someone with say 50 reviews.
- The more reviews you have, the better level of certainty. Should there actually be two tables, those where the pub has =>4 and those where the pub has <4?
At the moment there is a weighting to slightly downgrade ones with fewer reviews. I think it still biased too much towards ones with few reviews but I quite like that in terms of pub discovery.
- Should reviews be weighted by age? In that, the older reviews carry less weight. So a review in the last year has a weight of 10, in the last two years 8 etc etc
Good question. I had ruled that out when I was looking at this before, but I think the main reason was that I was trying to pull it all together into one insane SQL query. Could certainly do it.
- Mean, median or mode? :D
Toyed with all of the above in various ways, at the moment it is based on normalised reviews on a per user basis and then taking a weighted mean. Ideally I'd lean more towards a median of some sort I think, but trying to gain meaning from small data this is the best I can do.
I noticed the discussion about trying to produce a PG top 100 pubs in the 'Yet another list' thread and it reminded me of some stuff I'd done a while back for weighting ratings.
So here you go, a top 100 based on the weighted ratings of PG reviewers.
https://www.pubsgalore.co.uk/stats/pubs/top-rated/
It is undoubtedly skewed in all sorts of ways, but it might prove to be an interesting guide for good pubs that haven't been reviewed much.
One of the skews means that it is generally harder for pubs that have featured on a PG crawl (and have lots of ratings) to appear on the list, I need to work out how to tweak the algorithm without pushing it too far in their favour instead.
Hello Dave. You're looking well today.
Is there not a small problem with the evolution of a pub? Sometimes a pub's name changes but is otherwise the same, some pubs retain the same name but over time alter beyond recognition. The problem is that ratings don't always reflect the pub as it stands: a much loved boozer from 2008 is now a stripped out gastropub. Conversely, the awful theme pub serving Fosters is now a beautifully restored Victorian beauty - the rating a reviewer gave ten years ago may have no bearing on their verdict ten years on. If the https://www.pubsgalore.co.uk/pubs/23788/ had been reviewed ten years ago it would have been condemned, now it's almost universally praised yet the old ratings - if there were any - would still apply.
Blackthorn
05-12-2018, 21:11
Cease and desist immediately.Just checked the London pubs featured and cannot believe they are anything like the top pubs in London.Hate to think someone would look up Pubs Galore and went to some of these pubs on this recommendation.
Well, Ye Olde Mitre was in there, that’s one of my favourite London pubs. Not familiar with the others though, so can’t comment.
I've fixed my initial mistakes and the list has changed a fair bit.
Not sure what you’ve done with the algorithm, but from going out to sample the mixed delights of Moxley and two previously unvisited pubs in Bradley my percentage has shot up from 21% to 44% as some previous ‘No’s have changed to ‘Yes’s.
Is there not a small problem with the evolution of a pub?
It is a big problem and one of the reasons to resist making lists like this, or at least taking them too seriously.
One the suggestions Quinno makes is to reduce the relevance of ratings over time which would help with this issue. Once I work out how to actually do that I'll give it a go and see what sort of a list we end up with.
sheffield hatter
05-12-2018, 21:55
Like the idea (haven't looked at the list yet, so don't know if I'll like the results).
I agree with what Quinno and Tris said about older reviews, so it would be good if some sort of weighting for age (of review rather than reviewer, obviously) could be incorporated. (Edit: We appear to have been typing simultaneously.)
Slightly concerned about the maths failure. :evilgrin:
Hello Dave. You're looking well today.
Is there not a small problem with the evolution of a pub? Sometimes a pub's name changes but is otherwise the same, some pubs retain the same name but over time alter beyond recognition. The problem is that ratings don't always reflect the pub as it stands: a much loved boozer from 2008 is now a stripped out gastropub. Conversely, the awful theme pub serving Fosters is now a beautifully restored Victorian beauty - the rating a reviewer gave ten years ago may have no bearing on their verdict ten years on. If the https://www.pubsgalore.co.uk/pubs/23788/ had been reviewed ten years ago it would have been condemned, now it's almost universally praised yet the old ratings - if there were any - would still apply.
Hence my suggestion of weighting reviews by age, with a diminishing weight rate applied by year. There's never a perfect way of doing it though.
Hello Dave. You're looking well today.
Is there not a small problem with the evolution of a pub? Sometimes a pub's name changes but is otherwise the same, some pubs retain the same name but over time alter beyond recognition. The problem is that ratings don't always reflect the pub as it stands: a much loved boozer from 2008 is now a stripped out gastropub. Conversely, the awful theme pub serving Fosters is now a beautifully restored Victorian beauty - the rating a reviewer gave ten years ago may have no bearing on their verdict ten years on. If the https://www.pubsgalore.co.uk/pubs/23788/ had been reviewed ten years ago it would have been condemned, now it's almost universally praised yet the old ratings - if there were any - would still apply.
Perhaps restricting the reviews rated/considered to the last 10 reviews or the last 5 years, and resetting the tally to 0 if the pub closes and re-opens or the Type changes?
sheffield hatter
05-12-2018, 22:19
Is there not a small problem with the evolution of a pub?
It is a big problem and one of the reasons to resist making lists like this, or at least taking them too seriously.
Hence my suggestion of weighting reviews by age, with a diminishing weight rate applied by year. There's never a perfect way of doing it though.
Once I work out how to actually do that I'll give it a go and see what sort of a list we end up with.
A couple of examples: Swan & Rushes (https://www.pubsgalore.co.uk/pubs/20714/) has not been reviewed for four years, and I think I have read somewhere that it is now a pale shadow of its former self; The Leopold (https://www.pubsgalore.co.uk/pubs/14000/) has been mentioned in the context of the March 2019 visit to Portsmouth & Southsea - the last review in its pomp is 30 months old but two more recent reviews, reporting the closure and change of style, are without scores so presumably have no effect. These pubs are currently 82 and 89 respectively. (This is not meant as mere carping, just examples of the problem - and hopefully you'll get an idea of a fix from this.)
Hence my suggestion of weighting reviews by age, with a diminishing weight rate applied by year. There's never a perfect way of doing it though.
Indeed.
I also think that, as suggested earlier, a reviewer's rating shouldn't apply unless they have submitted a minimum number of reviews to help rule out 'spikes' in the ratings due to a person's personal grievance (which if they are a publican/manager) could have been submitted to discredit competition, or boost the rating of their own venture. You'd need to discount a fair number (say their first 10), just in case the reviewer manages a pubco chain. Won't stop Tim Martin though...:rolleyes:
sheffield hatter
05-12-2018, 22:36
A bit annoying too that I only photographed no. 71 a few weeks ago, and would have made the effort to stop if I’d known it was a “top 100” and had planned on going in no. 88 last week but abandoned the idea due to the very inclement weather. Again, would have made more of an effort if I’d known.
Interesting! There's a danger, is there not, of reviewers making special trips to "Top 100" pubs, which would then tend to reinforce the pubs' places in that top 100, thus making it more and more difficult for improving or newly discovered pubs to make their way up the list.
Probably best not to take the whole thing too seriously, as Dave has already mentioned.
Not sure what you’ve done with the algorithm, but from going out to sample the mixed delights of Moxley and two previously unvisited pubs in Bradley my percentage has shot up from 21% to 44% as some previous ‘No’s have changed to ‘Yes’s.
The algorithm is still missing some visited pubs, having now found another nine ‘No’s that should be ‘Yes’s...
Bucking Fastard
06-12-2018, 11:28
I think this is a great idea,and I look forward to future tweaks.
When in an area that I'm not familiar ,I may well be tempted to visit a pub that appears on this list to see what all the fuss is about.
It's all a bit of fun,but with many very knowledgeable reviewers on PuG it's a list that should have a lot more credibility that the ones we normally comment on.
ROBCamra
06-12-2018, 14:46
Seems to be a glitch in the picking up of the pubs I've reviewed/visited e.g. Rat & Ratchet in Huddersfield. I've reviewed it, but it wasn't showing as visited or the Lymestone Vaults in N-U-L. I've ticked them on the extract now though.:cheers:
Seems to be a glitch in the picking up of the pubs I've reviewed/visited e.g. Rat & Ratchet in Huddersfield. I've reviewed it, but it wasn't showing as visited or the Lymestone Vaults in N-U-L. I've ticked them on the extract now though.:cheers:
Yeah three for me as well, says No but I have visited and scored;
The Hope (https://www.pubsgalore.co.uk/pubs/35464/)
The White Star (https://www.pubsgalore.co.uk/pubs/64903/)
The Garland (https://www.pubsgalore.co.uk/pubs/36046/)
No obvious theme linking them... :confused:
I need to rerun the script I had a few years back to mark pubs that have been reviewed as visited.
Thought it was an ongoing thing, clearly not, can't find the script. :muppet:
Aqualung
06-12-2018, 21:42
No obvious theme linking them... :confused:
Other than the obvious one from Little Britain "The computer says no".
Could the script for this list be amended to update the visited flag for each user when it's run? It might mean that it ran for a week or more making it unviable. I'd like to see the result of changing the minimum reviews to 3,4 and 5. I think over 5 would be too much but I would also like to see what everyone else thinks.
trainman
07-12-2018, 17:41
I presume the search has been re-scripted? I have visited exactly half from the current listing, 50 for sure, maybe a couple of spares that escape me...
Aqualung
07-12-2018, 17:56
I presume the search has been re-scripted? I have visited exactly half from the current listing, 50 for sure, maybe a couple of spares that escape me...
It looks that way to me as well.
It looks that way to me as well.
Looks like it, my unvisited three are now visited. Well done Snow White!
Now, if SW could strip out leading 'the' prefixes so we could have a true alpha sort on Pub Name... :)
Aqualung
01-02-2019, 11:19
I've posted on this thread to bring it forward so the score based list can be compared to the Top 10 list.
Now, if SW could strip out leading 'the' prefixes so we could have a true alpha sort on Pub Name... :)
And stop 'Forresters' duplicating with Foresters', 'Cricketers' with 'Cricketers Arms', 'Holly Bush' and 'Hollybush' and 'Blue Bell'/'Bluebell'- at least they're the first 4 in the top 255 most popular.
sheffield hatter
01-02-2019, 17:30
'Blue Bell'/'Bluebell'
I went through all of those one by one a few weeks ago and changed the names of anything with a blue bell on the pub sign to Blue Bell, and made sure that anything called Bluebell had A (BLOODY) FLOWER ON THE SIGN! Failing that, I went by what the pub called itself on its website or Facebook page, so those should all be accurate now.
I went through all of those one by one a few weeks ago and changed the names of anything with a blue bell on the pub sign to Blue Bell, and made sure that anything called Bluebell had A (BLOODY) FLOWER ON THE SIGN! Failing that, I went by what the pub called itself on its website or Facebook page, so those should all be accurate now.
I admire your dedication!
I went through all of those one by one a few weeks ago and changed the names of anything with a blue bell on the pub sign to Blue Bell, and made sure that anything called Bluebell had A (BLOODY) FLOWER ON THE SIGN! Failing that, I went by what the pub called itself on its website or Facebook page, so those should all be accurate now.
Like me properly capitalising all the Prince of Wales etc places!
Aqualung
01-02-2019, 19:18
I went through all of those one by one a few weeks ago and changed the names of anything with a blue bell on the pub sign to Blue Bell, and made sure that anything called Bluebell had A (BLOODY) FLOWER ON THE SIGN! Failing that, I went by what the pub called itself on its website or Facebook page, so those should all be accurate now.
The most confusing thing on the other list is that there are two Astons one near Nantwich and one (I assume) the better known one in Brum.
Powered by vBulletin® Version 4.1.11 Copyright © 2024 vBulletin Solutions, Inc. All rights reserved.